Simple Gradient Descent in Visual Basic 6 Part 2

Let's look again at gradient descent in VB6. This is a continuation of that post, so if you haven't already read that, click here.

For this post, we will look at the effects of outliers in the data set. Try adding this to Form_Load:

List1.AddItem ("100")
List1.AddItem("301")

*Don't forget to change the variable m, x and y to 17 and change alpha = 0.001 or else overflow errors*

Here is a look at the output:

Close to the original. Let's try adding an extreme case. Add this to Form_Load and change the variable m (and x and y) to equal 18:

List1.AddItem ("1000")

List2.AddItem ("3001")

Now notice that this follows the same rule: output data is three times input plus one. But try running it and it will error. What's the culprit? A couple of things:

The error is actually in alpha and in iterators. alpha has to be set to:

alpha = 0.00001

and iterators has to be changed to Long and increased:

Dim iterations As Long
iterations = 188000 'set iterations high

Now run it. On my machine, it goes pretty slow but eventually converges to close to the original thetas. When using non-fixed data and encountering errors, alpha and iterations can cause all sorts of errors. So if you are getting errors, try looking at alpha and the iterations first before thinking there is a bug in the actual implementation of gradient descent.