Simple Gradient Descent in Visual Basic 6 Tutorial

Welcome to a tutorial on how to implement a simple version of gradient descent in Visual Basic 6. This tutorial won't cover the theory of how gradient descent works. It is simply to have a simple version of the algorithm working. It will be quick and dirty. If you don't know the basics of gradient descent, you may not be able to follow along.

Let's get started. Open up Visual Basic (I am using version 6) and start a new project. Add two Lists, a Label, and a Command button. Your form should look like this:

What we will be doing is populating the lists with training data and then when the button is clicked, gradient descent will run on the data. List1 will be our input training data and List2 will be our output training data. We will be using data that mimics the function f(x) = 3x + 1. This is just to make it a bit more interesting. We will be computing theta0 and theta1 which will give us a line that "fits" the data. If we are successful, theta0 will be close to 1 and theta1 will be close to 3. Double click on the form to get to Form_Load. Add the following code:

Private Sub Form_Load()
List1.AddItem ("1")
List1.AddItem ("2")
List1.AddItem ("3")
List1.AddItem ("4")
List1.AddItem ("5")
List1.AddItem ("6")
List1.AddItem ("7")
List1.AddItem ("8")
List1.AddItem ("9")
List1.AddItem ("10")
List1.AddItem ("11")
List1.AddItem ("12")
List1.AddItem ("13")
List1.AddItem ("14")
List1.AddItem ("15")
List1.AddItem ("16")


List2.AddItem ("4.1")
List2.AddItem ("7.3")
List2.AddItem ("10.2")
List2.AddItem ("13.1")
List2.AddItem ("16.3")
List2.AddItem ("19.1")
List2.AddItem ("21.9")
List2.AddItem ("25")
List2.AddItem ("28")
List2.AddItem ("31")
List2.AddItem ("34")
List2.AddItem ("37")
List2.AddItem ("40")
List2.AddItem ("43")
List2.AddItem ("46")
List2.AddItem ("49")


Label1.Caption = "Click button to run gradient descent"
Command1.Caption = "Click here"
Form1.Caption = "Gradient Descent"
End Sub

Notice that the output for each value in List1 is about three times the value plus one. When we run gradient descent, we want it to spit out values for theta0 and theta1 which will give us a line that looks a lot like 3x + 1. Double click on Command1 button and enter the following code:

Private Sub Command1_Click()
Dim i, j, m As Integer
Dim x(16) As Double
Dim y(16) As Double
Dim theta0, temp0, theta1, temp1 As Double

theta0 = 0 'initialize all variables to 0
theta1 = 0
temp0 = 0
temp1 = 0

m = 16 'number of samples

For i = 0 To List1.listCount - 1 'populate arrays with data
    x(i) = CDbl(List1.List(i))
    y(i) = CDbl(List2.List(i))
Next

Dim iterations As Integer
iterations = 18000 'set iterations high

Dim alpha As Double
alpha = 0.01 'set learning rate

Dim h0, h1 As Long
h0 = 0
h1 = 0

For i = 0 To iterations
    h0 = 0
    h1 = 0
   
    For j = 0 To m
    h0 = h0 + ((theta0 + x(j) * theta1) - y(j))
    h1 = h1 + ((theta0 + x(j) * theta1) - y(j)) * x(j)
    Next j
   
    temp0 = theta0 - (alpha * h0) / CDbl(m)
    temp1 = theta1 - (alpha * h1) / CDbl(m)
    theta0 = temp0
    theta1 = temp1
Next i

Label1.Caption = CStr(theta1) & "x + " & CStr(theta0)

End Sub

That's it. Run the program and click the button and you should get output that looks like this:

As you can see, the output function is close to 3x+1 which is what our data set suggests the function to be. By "fixing" the data set, we can see how gradient descent works. In the program, try modifying the input and output training data to see what functions get spit out. Note that the learning rate alpha can cause the program to overflow. If using other data, check that first if it results in an overflow.