Gradient Descent Take 2 (new code)

First of all, thanks a lot to everyone for your help yesterday.

I rewrote my code so that it uses arrays of arrays instead of data objects (I chose arrays of arrays instead of rectangular arrays to more easily take entire rows from the matrix, any comments on which is better?) and I only abstracted away the initial data read and matrix multiplication. The code should be much easier to read now :)

To summarize the problems I was seeing yesterday, my program appeared to be reducing errors, but unless I use a very small value for alpha, my weights explode to infinity. Also, the resulting predictions were way off. This is the equation I am trying to implement:

http://i.imgur.com/qVRfN.png

My understanding of the equation:

http://i.imgur.com/CH9qw.jpg

Here is my new code:

 static void Main(string[] args) { // 0 Baseline // 1 Cylinders // 2 Displacement // 3 HorsePower // 4 Weight // 5 Acceleration // 6 ModelYear var theta = new double[] { 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, }; var x = Importer.GetX(); // double[][] representing features var y = Importer.GetY(); // double[] representing mpg var alpha = 0.00000000001; var iterations = 1000; var rows = x.Length; var featureCount = theta.Length; for (int i = 0; i < iterations; i++) { // i loops through each theta var errorSum = new double[featureCount]; int k; for (int j = 0; j < rows; j++) { // j loops through each row in our data var errorFactor = y[j] - GetH(x[j], theta); for (k = 0; k < featureCount; k++) { // k loops through each column (feature) of our data errorSum[k] += errorFactor * x[j][k]; } } for (k = 0; k < featureCount; k++) { // k loops through each column (feature) of our data theta[k] += errorSum[k] * alpha; } } } static double GetH(double[] row, double[] theta) { return Matrix.Matrix.Multiply(row, theta); } 

Thank you all for your help, I feel so close to the answer!