Checking if gradient descent is working

This is a continuation on my post on how to run gradient descent using a quadratic fitting function.

One way to check if gradient descent is working is to compute the cost function and see if it is going down given theta values that have been being updated. The way I did this was to check during gradient descent every 1000 iterations. I computed the cost function, and if it is going down, then that means gradient descent is converging. Here is the code I put in the iteration loop:

 counter = counter + 1;
  if(counter % 1000 == 0)
        {
            for(j = 0; j<m; j++)
            {
                h0 = h0 + Math.pow(((theta0 + (x[j]*x[j]*theta1)) - y[j]), 2.0);
            }
            h0 = h0 / (2.0 * m);
            System.out.println("Cost at " + counter + " is " + h0);
        }

The first thing I did was have a variable that increments by one every iteration. I now notice that I could simply use the iteration variable that is in the loop but whatever. The if statement says that if counter is divisible by 1000, then do the following code. This is because I have 30000 iterations so I want to check every 1000 iterations that it is going down.

The code below computes this function:

 

 The code that can perform this is as follows:

for(j = 0; j<m; j++)
            {
                h0 = h0 + Math.pow(((theta0 + (x[j]*x[j]*theta1)) - y[j]), 2.0);
            }
            h0 = h0 / (2.0 * m);

As you can see, it simply calculating the summation of the squared error where h(x) is equal to:

 theta0 + (x[j]*x[j]*theta1)

 and then dividing by 2*m

If this number goes down, that means that gradient descent is converging.