Logistic regression in Java with plotting

This tutorial will show how to implement an easy example of logistic regression in Java. It assumes that JavaPlot is installed. See my other tutorial (link at bottom of page) or google how to import Javaplot (it is trivial).

The major change being made here from linear regression is the hypothesis function uses the sigmoid function.

Here is the data set we will be using:

 double x1[] = {0.0, 0.5, 1.0, 1.5, 2.0, 0.1, 3.0, 3.1, 3.5, 3.2, 2.5, 2.8};   
 double x2[] = {0.0, 1.0, 1.1, 0.5, 0.3, 2.0, 3.0, 2.3, 1.5, 2.2, 3.6, 2.8};   
 double y[]  = {0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0};

Here is a graph of the data set. We are trying to determine the boundary line, which is a line that will "split" the two groups in half.

The slight change we make will be too call the sigmoid function during gradient descent. We take the proper partial derivatives and step through m amount of examples. Here is the relevant code:

for(i = 0; i < iterations; i++)
            {
         
                h0 = 0.0;
                h1 = 0.0;
                h2 = 0.0;
               
                for(j = 0; j<m; j++)
                {
                    h0 = h0 + (sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2) - y[j]);
                    h1 = h1 + (sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2) - y[j])*x1[j];
                    h2 = h2 + (sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2) - y[j])*x2[j];
                }    
        temp0 = theta0 - (alpha*h0)/(double)m;
        temp1 = theta1 - (alpha*h1)/(double)m;
        temp2 = theta2 - (alpha*h2)/(double)m;
        theta0 = temp0;
        theta1 = temp1;
        theta2 = temp2;

}

Note that the equation we get is in two variables, x1 and x2. We will find the line by setting each to 0 and finding the intercepts. From there, we use the equation for a line to determine our boundary line. This is done in order to plot the line.

The method sigmoid(double x) is pretty straightforward. The function is rather complex but the explanation for it isn't hard to understand. See the machine learning course notes for coursera for the explanation. The code is:

private static double sigmoid(double x)
    {
        return 1 / (1 + Math.exp(-1*x));
    }
   

The resulting function is two variable as I said before. First, we need to set x2 = 0 and then x1 = 0. When those are the cases, we are left with bother theta1*x1 = theta0 and theta2*x2 = theta0. The code is as follows:

double point1 = -1 * theta0 / theta1; //x intercept

double point2 = -1 * theta0 / theta2; //y intercept

Now using these points, we can find a line using the equation for a line which is:

y - y1 = m(x-x1)

Our points are (0, point2) and (point1, 0)

Subbing in the values and isolating y we get the function for the line:

String outputFunction = String.valueOf(String.valueOf((-1*point2)/point1) + "*x+" + String.valueOf(point2));

The slope m is negative because theta0 has to be brought to the other side. Working it out on paper might make it clearer.

Here is a plot of the resulting line:

A good fit

Here is the full code. I added a plot of the cost function for the first 1000 iterations to check that it is going down. JavaPlot must be added for the plot to work. I discussed that in my post here

import com.panayotis.gnuplot.JavaPlot;
import com.panayotis.gnuplot.style.PlotStyle;
import com.panayotis.gnuplot.style.Style;

public class GradDescent {

    public static void main(String[] args) {
   
       
         double x1[] = {0.0, 0.5, 1.0, 1.5, 2.0, 0.1, 3.0, 3.1, 3.5, 3.2, 2.5, 2.8};   
         double x2[] = {0.0, 1.0, 1.1, 0.5, 0.3, 2.0, 3.0, 2.3, 1.5, 2.2, 3.6, 2.8};   
         double y[]  = {0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0};
         int counter = 0;
       
         JavaPlot p = new JavaPlot();
       
         /*number of examples*/
         int m = 12;
       
          /*thetas and temp thetas*/
            double theta0, temp0, theta1, temp1, theta2, temp2;
            theta0 = 0.0; temp0 = 0.0;
            theta1 = 0.0; temp1 = 0.0;
            theta2 = 0.0; temp2 = 0.0;
         
        /*# of iterations and learning rate*/
            int iterations = 2181800;
            float alpha = 0.009f;

            int j = 0;
            double h0 = 0.0;
            double h1 = 0.0;
            double h2 = 0.0;
           
            int i = 0;
            for(i = 0; i < iterations; i++)
            {
         
                h0 = 0.0;
                h1 = 0.0;
                h2 = 0.0;
               
                for(j = 0; j<m; j++)
                {
                    h0 = h0 + (sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2) - y[j]);
                    h1 = h1 + (sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2) - y[j])*x1[j];
                    h2 = h2 + (sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2) - y[j])*x2[j];
                }    
        temp0 = theta0 - (alpha*h0)/(double)m;
        temp1 = theta1 - (alpha*h1)/(double)m;
        temp2 = theta2 - (alpha*h2)/(double)m;
        theta0 = temp0;
        theta1 = temp1;
        theta2 = temp2;
       
        counter = counter + 1;
        if(counter < 1000)
        {
            for(j = 0; j<m; j++)
            {
                h0 = h0 + y[j]*Math.log(sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2)) + (1-y[j])*(1 - (sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta2)));                                                            //+ Math.pow(( sigmoid(theta0 + x1[j]*theta1 + x2[j]*theta1) - y[j]), 2.0);
            }
            h0 = (h0 / m) * -1;
            float[][] cost = {{(float)counter, (float) h0}};
            p.addPlot(cost);
       
            System.out.println("Cost at " + counter + " is " + h0);
           
        }
       
        }
         
       p.plot();
       System.out.println(theta2 + "x2 + " + theta1 + "x1 + " + theta0);
      

    }
   
   
    private static double sigmoid(double x)
    {
        return 1 / (1 + Math.exp(-1*x));
    }
   
    private static void testGradientDescent(float n, double theta0, double theta1, double theta2)
    {
         double x3[][] = {{0.0, 0.0}, {0.5, 1.0}, {1.0, 1.1}, {1.5, 0.5}, {2.0, 0.3}, {0.1, 2.0}};   
         double x4[][] = {{3.0, 3.0}, {3.1, 2.3},{3.5, 1.5}, {3.2, 2.2}, {2.5,3.6}, {2.8, 2.8}};  
       
        JavaPlot p = new JavaPlot();
        p.set("title", "'Gradient Descent'");
        p.set("xrange", "[0:4]");
        //p.addPlot(outputFunction);
       
        p.addPlot(x3);
        p.addPlot(x4);
       
        double point1 = -1 * theta0 / theta1;
        double point2 = -1 * theta0 / theta2;
       
        String outputFunction = String.valueOf(String.valueOf((-1*point2)/point1) + "*x+" + String.valueOf(point2));
        System.out.println("Plotting " + outputFunction);
       
        PlotStyle myPlotStyle = new PlotStyle();
        myPlotStyle.setStyle(Style.LINES);
        myPlotStyle.setLineWidth(2);
       
        double boundaryLine[][] = {{point1, 0},{0, point2}};
        //p.addPlot(boundaryLine);
        p.addPlot(outputFunction);
        p.plot();
       
    }

}

Checking if gradient descent is working

This is a continuation on my post on how to run gradient descent using a quadratic fitting function.

One way to check if gradient descent is working is to compute the cost function and see if it is going down given theta values that have been being updated. The way I did this was to check during gradient descent every 1000 iterations. I computed the cost function, and if it is going down, then that means gradient descent is converging. Here is the code I put in the iteration loop:

 counter = counter + 1;
  if(counter % 1000 == 0)
        {
            for(j = 0; j<m; j++)
            {
                h0 = h0 + Math.pow(((theta0 + (x[j]*x[j]*theta1)) - y[j]), 2.0);
            }
            h0 = h0 / (2.0 * m);
            System.out.println("Cost at " + counter + " is " + h0);
        }

The first thing I did was have a variable that increments by one every iteration. I now notice that I could simply use the iteration variable that is in the loop but whatever. The if statement says that if counter is divisible by 1000, then do the following code. This is because I have 30000 iterations so I want to check every 1000 iterations that it is going down.

The code below computes this function:

 

 The code that can perform this is as follows:

for(j = 0; j<m; j++)
            {
                h0 = h0 + Math.pow(((theta0 + (x[j]*x[j]*theta1)) - y[j]), 2.0);
            }
            h0 = h0 / (2.0 * m);

As you can see, it simply calculating the summation of the squared error where h(x) is equal to:

 theta0 + (x[j]*x[j]*theta1)

 and then dividing by 2*m

If this number goes down, that means that gradient descent is converging.

 

Improving the fit of gradient descent using better functions

This is a continuation of my previous post simple gradient descent example using Codeblocks-EP in C language

Here is an example of trying to fit a linear function "y = theta1*x + theta0" to a data set that mimics
y = x^2

The data set:

float x[] = {1,2,3,4,5,6,7,8,9,10};
float y[] = {1,4,9,16,25,36,49,64,81,100};

As you can see, the training set suggests that the data should follow y=x^2

However, when we plot it using a linear function, the plot is wildly inaccurate for most values of x. It is a terrible fit:

Not good. But now let's change our hypothesis function to a quadratic function and see if we get a better fit. The code as of now is currently:

for(j = 0; j<m; j++)
            {
                h0 = h0 + ((theta0 + x[j]*theta1) - y[j]);
                h1 = h1 + ((theta0 + x[j]*theta1) - y[j])*x[j];
            }

Here we are going through all of our training examples and updating them using a linear function. By adding another x[j] term, we can hopefully get a quadratic fit that more closely fits the data. So we will replace it with:

for(j = 0; j<m; j++)
            {
                h0 = h0 + ((theta0 + x[j]*x[j]*theta1) - y[j]);
                h1 = h1 + ((theta0 + x[j]*x[j]*theta1) - y[j])*x[j];
            }

Now let's see the plot and values for theta0 and theta1:

A much better fit. Note that the learning rate had to be changed as well as the iterations. We went from:

int iterations = 1000;

float alpha = 0.03;

to an increasre in iterations and a decrease in the learning rate (at this value, it fails to converge)

int iterations = 10000;

float alpha = 0.003;

Here is the full code:

#include "koolplot.h"

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char**argv) {

    /*training set1.2
    1.1
    2.4 2.2
    3.1 3.3
    4.4 4.3
    5.3 5.4
    6.3 6.7
    7.6 7.7
    8.7 8.6
    9.7 9.8*/
    float x[] = {1,2,3,4,5,6,7,8,9,10};
    float y[] = {1,4,9,16,25,36,49,64,81,100};

    /*number of examples*/
    int m = 10;

    /*thetas and temp thetas*/
        float theta0, temp0, theta1, temp1;
        theta0 = 0.0; temp0 = 0.0;
        theta1 = 0.0; temp1 = 0.0;

    /*iterations and learning rate HIGHLY VARIABLE*/
        int iterations = 1000;
        float alpha = 0.03;

        int j = 0;
        float h0 = 0.0;
        float h1 = 0.0;
        int i = 0;
        for(i = 0; i < iterations; i++)
        {

            h0 = 0.0;
        h1 = 0.0;
            for(j = 0; j<m; j++)
            {
                h0 = h0 + ((theta0 + x[j]*x[j]*theta1) - y[j]);
                h1 = h1 + ((theta0 + x[j]*x[j]*theta1) - y[j])*x[j];
            }

    //h0 = h0 / (float)m;
    //h1 = h1 / (float)m;
    temp0 = theta0 - (alpha*h0)/(float)m;
    temp1 = theta1 - (alpha*h1)/(float)m;
    theta0 = temp0;
    theta1 = temp1;
    }

    printf("Theta0: %f\n", theta0);
    printf("Theta1: %f\n", theta1);

    Plotdata xx(0.0, 10.0), yy = theta0 + xx*xx*theta1;
    plot(xx, yy);

    return 0;
}

Simple gradient descent example using Codeblocks-EP in C language

The intention of this tutorial is to show you how to set up Codeblocks-EP and use the plotting software called KoolPlot to run a simple gradient descent example and plot it. This is workable in Windows.

First, download CodeBlocks-EP at http://codeblocks.codecutter.org/

Note that this is a different program than the regular CodeBlocks but I am pretty sure it has the same functionality. It seems to be a version to learn from. Install it and open it.

Step 1: Create a new project

Step 2: Select "koolplot project" and click "Go"

Step 3: Select "Add Console" It is not the default so make sure that you select it and then hit next.

Step 4: Name your project "GradientDescent" and click next. On the next screen, click finish.

Step 5: Go to File -> New -> Empty file. Name it GradientDescent.c

Step 6: Right click and select edit and paste the following code:

#include "koolplot.h"

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char**argv) {

    /*training set1.2
    1.1
    2.4 2.2
    3.1 3.3
    4.4 4.3
    5.3 5.4
    6.3 6.7
    7.6 7.7
    8.7 8.6
    9.7 9.8*/
    float x[] = {1.2, 2.4, 3.1, 4.4, 5.3, 6.3, 7.6, 8.7, 9.7};
    float y[] = {1.1, 2.2, 3.3, 4.3, 5.4, 6.7, 7.7, 8.6, 9.8};

    /*number of examples*/
    int m = 9;

    /*thetas and temp thetas*/
        float theta0, temp0, theta1, temp1;
        theta0 = 0.0; temp0 = 0.0;
        theta1 = 0.0; temp1 = 0.0;

    /*iterations and learning rate HIGHLY VARIABLE*/
        int iterations = 1000;
        float alpha = 0.03;

        int j = 0;
        float h0 = 0.0;
        float h1 = 0.0;
        int i = 0;
        for(i = 0; i < iterations; i++)
        {

            h0 = 0.0;
        h1 = 0.0;
            for(j = 0; j<m; j++)
            {
                h0 = h0 + ((theta0 + x[j]*theta1) - y[j]);
                    h1 = h1 + ((theta0 + x[j]*theta1) - y[j])*x[j];
            }

    //h0 = h0 / (float)m;
    //h1 = h1 / (float)m;
     temp0 = theta0 - (alpha*h0)/(float)m;
     temp1 = theta1 - (alpha*h1)/(float)m;
     theta0 = temp0;
    theta1 = temp1;
    }

    printf("Theta0: %f\n", theta0);
    printf("Theta1: %f\n", theta1);

    Plotdata xx(0.0, 3.0), yy = theta0 + xx*theta1;
    plot(xx, yy);

    return 0;
}

 Step 7: Select Build -> Build and then Build -> Run

Lastly, if you followed all of these steps it should run. The command console will show the values of theta0 and theta1 and the plot with plot out "y = theta0 + x*theta1" Note that I had to use "xx" and "yy" at the end because these are the variables that koolplot is using to plot and x and y were already used to store our training data. If you click on the plot, you will be able to see the x and y values above. Here is what the output should look like:

And that's it.

 

Multi-feature gradient descent in java

public class GDescent {

    public static void main(String[] args) {
  
         float x1[] = {1f,1f,1f,1f,2f,2f,2f,3f,3f,3f};  
         float x2[] = {6f,6f,6f,6f,9f,9f,9f,13f,13f,13f};
         float y[] = {2f,2f,3f,4f,5f,5f,6f,5.5f,6f,7f};  
      
         /*number of examples*/
         int m = 10;
      
          /*thetas and temp thetas*/
            float theta0, temp0, theta1, temp1, theta2, temp2;
            theta0 = 0.0f; temp0 = 0.0f;
            theta1 = 0.0f; temp1 = 0.0f;
            theta2 = 0.0f; temp2 = 0.0f;
        
        /*# of iterations and learning rate*/
            int iterations = 3430000;
            float alpha = 0.003f;

            int j = 0;
            float h0 = 0.0f;
            float h1 = 0.0f;
            float h2 = 0.0f;
            int i = 0;
            for(i = 0; i < iterations; i++)
            {
        
                h0 = 0.0f;
                h1 = 0.0f;
                h2 = 0.0f;
                for(j = 0; j<m; j++)
                {
                    h0 = h0 + ((theta0 +  x1[j]*theta1 + x2[j]*theta2) - y[j]);
                    h1 = h1 + ((theta0 +  x1[j]*theta1 + x2[j]*theta2) - y[j])*x1[j];
                    h2 = h2 + ((theta0 +  x1[j]*theta1 + x2[j]*theta2) - y[j])*x2[j];
                }   
        temp0 = theta0 - (alpha*h0)/(float)m;
        temp1 = theta1 - (alpha*h1)/(float)m;
        temp2 = theta2 - (alpha*h2)/(float)m;
        theta0 = temp0;
        theta1 = temp1;
        theta2 = temp2;
        }
          
       System.out.println("" + theta2 + "x2 + " + theta1 + "x1 + " + theta0);
     
       testGradientDescent(2f, theta0, theta1, theta2);

    }
  
    private static void testGradientDescent(float n, float theta0, float theta1, float theta2)
    {
        float result = theta0 + (theta1*1) + (theta2*6);
        //float x[] = {1f,2f,3f,4f,5f,6f,7f,8f,9f,10f};  
        //float y[] = {2f,5f,10f,17f,26f,37f,50f,65f,82f,101f};
        //int marks[][]={{1,2},{2,5},{3,10},{4,17},{5,26},{6,37},{7,50},{8,65},{9,82},{10,101}};
      
        System.out.println("Result: " + result);
        //String outputFunction = String.valueOf(String.valueOf(theta0)  + "+" + String.valueOf(theta2) + "*x*x");
        //System.out.println("Plotting " + outputFunction);
        //JavaPlot p = new JavaPlot();
        //p.addPlot(marks);
      
      // // p.addPlot(outputFunction);
       // p.setPersist(true);
       // p.plot();
        //p.setPersist(true);
      
    }

}

Notice that the partial derivatives are calculated here:

h0 = h0 + ((theta0 +  x1[j]*theta1 + x2[j]*theta2) - y[j]);
h1 = h1 + ((theta0 +  x1[j]*theta1 + x2[j]*theta2) - y[j])*x1[j];
h2 = h2 + ((theta0 +  x1[j]*theta1 + x2[j]*theta2) - y[j])*x2[j];

Nibbler - Pacman/Snake game in Java+Libgdx+Universal Tween Engine

Prototype of the game (with terrible graphics) can be downloaded here

Source code can be downloaded here

Nibbler game tutorial - Java, libgdx, Universal Tween Engine Part 2

Welcome to part 2 of this tutorial. By the end of today we will have the basic game loop set up. First, double click on "DesktopLauncher.java" and replace the code with this:

package com.mygdx.nibbler.desktop;

import com.badlogic.gdx.backends.lwjgl.LwjglApplication;
import com.badlogic.gdx.backends.lwjgl.LwjglApplicationConfiguration;
import com.mygdx.nibbler.nibblergame;

public class DesktopLauncher {
    public static void main (String[] arg) {
        LwjglApplicationConfiguration config = new LwjglApplicationConfiguration();
        config.title = "Nibbler";
        config.width = 480;
        config.height = 640;
        new LwjglApplication(new nibblergame(), config);
    }
} 

All this is doing is changing the size of the window when we run the game on Windows. We want the window to be more of a "portrait" than a "landscape."

Now go back to the "core" directory and open "nibblergame.java" Replace all of the code with:

package com.mygdx.nibbler;
package com.mygdx.nibbler;


import com.badlogic.gdx.Game;
import com.badlogic.gdx.Gdx;

public class nibblergame extends Game {

    @Override
    public void create() {
        System.out.println("Game Created");
       
    }
   
}

This creates the game and now we will create a screen for it. Right click on "Nibbler-core" and select New -> Package. Name it com.mygdx.Screen

Add the following code:

package com.mygdx.Screen;
public class GameScreen implements Screen {
} 

Eclipse will show an error and it will say to import and then to add unimplemented functions. Do both. The resulting code should be:

package com.mygdx.Screen;

import com.badlogic.gdx.Screen;

public class GameScreen implements Screen {

    @Override
    public void show() {
        // TODO Auto-generated method stub
       
    }

    @Override
    public void render(float delta) {
        // TODO Auto-generated method stub
       
    }

    @Override
    public void resize(int width, int height) {
        // TODO Auto-generated method stub
       
    }

    @Override
    public void pause() {
        // TODO Auto-generated method stub
       
    }

    @Override
    public void resume() {
        // TODO Auto-generated method stub
       
    }

    @Override
    public void hide() {
        // TODO Auto-generated method stub
       
    }

    @Override
    public void dispose() {
        // TODO Auto-generated method stub
       
    }

}
 
Now return back to "nibblergame.java" and add this line:


setScreen(new GameScreen());

Remember to import "GameScreen" and you are done!

Nibbler game tutorial - Java, libgdx, Universal Tween Engine Part 1

Welcome to part one of the Nibbler game tutorial. Nibbler is a game that is a cross between snake and pacman. The point is to traverse a maze and collect "dots" without eating your own tail. It is based off of an older game which I believe was called Nibblet.

The first part of the tutorial will end with you having a game loop in java using libgdx set up. Using libgdx allows us to run the game on many platforms, including Windows, iOS, android and html.

The first thing you want to do is install Eclipse. There are several ways to do this. You will need certain other things installed along with Eclipse. The easiest way I have found is to download in order the applications listed under "Setting up Eclipse" right here

For more help on setting up the environment, check out kilobolt.com's tutorial on how to do this. Pay attention to where the android sdk is downloaded to, you will need to know the directory later.

Next you will want to set up libgdx. Head over to https://libgdx.badlogicgames.com/download.html and download (Download Setup App). Run gdx-setup.jar. This will give you a wizard that will generate the template we will be using. Fill in the values as such:

Click advanced and select Eclipse. Note that you will need to set the directory where the Android sdk is at. Select a new folder on the desktop named "Nibbler" and click generate.

Now open Eclipse and right click in the package explorer and choose Import. Select gradle - gradle project and click next. It will import and that's it.

You will notice that there are folders for all the platforms listed above. The code will be edited in "core." To run the sample program, expand Desktop, and go to src and right click DesktopLauncher.java and run as a java applet. You should get a red screen with a smiley face on it.
If you have gotten to this point, you are ready to go on to part 2.

Plotting gradient descent graphically on Windows

My previous example showed how to plot gradient descent using GNUPlot on Linux, but there was no workaround for running it on Windows. Here I will show you how using Mingw and Javaplot.

First, install mingw here (Make sure to keep the default install settings and directory)

Then get JavaPlot here Extract it somewhere you know where it will be.

Next open up a project in eclipse. Right click on the project and click Build Path -> Add External Archives. Find JavaPlot.jar and add it and then right click on your project again and click refresh. You should now be able to use JavaPlot. The source below is the same as the previous tutorial except that it plots using GNUPlot. Note that theta1 is not used. Your left side of the screen should look like this:

import com.panayotis.gnuplot.JavaPlot;

public class GradDescent {

    public static void main(String[] args) {
   
       
       
         float x[] = {1f,2f,3f,4f,5f,6f,7f,8f,9f,10f};   
         float y[] = {2f,5f,10f,17f,26f,37f,50f,65f,82f,101f};   
       
         /*number of examples*/
         int m = 10;
       
          /*thetas and temp thetas*/
            float theta0, temp0, theta1, temp1, theta2, temp2;
            theta0 = 0.0f; temp0 = 0.0f;
            theta1 = 0.0f; temp1 = 0.0f;
            theta2 = 0.0f; temp2 = 0.0f;
         
        /*# of iterations and learning rate*/
            int iterations = 3430000;
            float alpha = 0.003f;

            int j = 0;
            float h0 = 0.0f;
            float h1 = 0.0f;
            float h2 = 0.0f;
            int i = 0;
            for(i = 0; i < iterations; i++)
            {
         
                h0 = 0.0f;
                h1 = 0.0f;
                h2 = 0.0f;
                for(j = 0; j<m; j++)
                {
                    h0 = h0 + ((theta0 +  x[j]*x[j]*theta2) - y[j]);
                    h1 = h1 + ((theta0 +  x[j]*x[j]*theta2) - y[j])*x[j];
                    h2 = h2 + ((theta0 +  x[j]*x[j]*theta2) - y[j])*x[j];
                }    
        temp0 = theta0 - (alpha*h0)/(float)m;
        temp1 = theta1 - (alpha*h1)/(float)m;
        temp2 = theta2 - (alpha*h2)/(float)m;
        theta0 = temp0;
        theta1 = temp1;
        theta2 = temp2;
        }
           
       System.out.println("" + theta2 + "x^2 + " + theta1 + "x + " + theta0);
      
       testGradientDescent(2f, theta0, theta1, theta2);

    }
   
    private static void testGradientDescent(float n, float theta0, float theta1, float theta2)
    {
        float result = theta0 + (theta1*n) + (theta2*n*n);
        //float x[] = {1f,2f,3f,4f,5f,6f,7f,8f,9f,10f};   
        //float y[] = {2f,5f,10f,17f,26f,37f,50f,65f,82f,101f};
        int marks[][]={{1,2},{2,5},{3,10},{4,17},{5,26},{6,37},{7,50},{8,65},{9,82},{10,101}};
       
        System.out.println("Result: " + result);
        String outputFunction = String.valueOf(String.valueOf(theta0)  + "+" + String.valueOf(theta2) + "*x*x");
        System.out.println("Plotting " + outputFunction);
        JavaPlot p = new JavaPlot();
        p.addPlot(marks);
       
        p.addPlot(outputFunction);
        p.setPersist(true);
        p.plot();
        p.setPersist(true);
       
    }

}

At the end, I needed to convert our data set into a multi dimensional matrix for JavaPlot to plot the points with the curve.

Gradient Descent in Java fitting a quadratic function

Here we will use a modification to fit a quadratic line to our data set. The line will be of the form:

(theta0)x^2 + (theta1)x + (theta0)n

We will be using java and the source code is below. Simple make a class called GradDescent, paste the code and run.

public class GradDescent {

    public static void main(String[] args) {
   
         float x[] = {1f,2f,3f,4f,5f,6f,7f,8f,9f,10f};  
         float y[] = {2f,5f,10f,17f,26f,37f,50f,65f,82f,101f};  
      
         /*number of examples*/
         int m = 10;
        
          /*thetas and temp thetas*/
            float theta0, temp0, theta1, temp1, theta2, temp2;
            theta0 = 0.0f; temp0 = 0.0f;
            theta1 = 0.0f; temp1 = 0.0f;
            theta2 = 0.0f; temp2 = 0.0f;
        
        /*# of iterations and learning rate*/
            int iterations = 3430000;
            float alpha = 0.003f;

            int j = 0;
            float h0 = 0.0f;
            float h1 = 0.0f;
            float h2 = 0.0f;
            int i = 0;
            for(i = 0; i < iterations; i++)
            {
        
                h0 = 0.0f;
                h1 = 0.0f;
                h2 = 0.0f;
                for(j = 0; j<m; j++)
                {
                    h0 = h0 + ((theta0 + x[j]*theta1 +  x[j]*x[j]*theta2) - y[j]);
                    h1 = h1 + ((theta0 + x[j]*theta1 +  x[j]*x[j]*theta2) - y[j])*x[j];
                    h2 = h2 + ((theta0 + x[j]*theta1 +  x[j]*x[j]*theta2) - y[j])*x[j];
                }   
        temp0 = theta0 - (alpha*h0)/(float)m;
        temp1 = theta1 - (alpha*h1)/(float)m;
        temp2 = theta2 - (alpha*h2)/(float)m;
        theta0 = temp0;
        theta1 = temp1;
        theta2 = temp2;
        }
          
       System.out.println("" + theta2 + "x^2 + " + theta1 + "x + " + theta0);
     
       testGradientDescent(5f, theta0, theta1, theta2);

    }
   
    private static void testGradientDescent(float n, float theta0, float theta1, float theta2)
    {
        float result = theta0 + (theta1*n) + (theta2*n*n);
        System.out.println("Result: " + result);
    }

}

Running it with sample input shows that it fits quite nicely.

Gradient Descent in Visual Basic fitting a Quadtratic Function

In the last tutorial, I demonstrated gradient descent that fitted a linear function to the data. Now I will post an example of it fitting a quadratic function. The data set is as folllow (y = x^2 - 4) Note that the below is code for Form_Load (You will need one Command button, two listboxes, one Label, and one Picture for the code to run. Set the Picture width height and scalewidth and scaleheight to 5000)

List1.AddItem ("2")
List1.AddItem ("3")
List1.AddItem ("4")
List1.AddItem ("5")
List1.AddItem ("6")
List1.AddItem ("7")
List1.AddItem ("8")
List1.AddItem ("9")
List1.AddItem ("10")

List2.AddItem ("0")
List2.AddItem ("5")
List2.AddItem ("12")
List2.AddItem ("21")
List2.AddItem ("32")
List2.AddItem ("45")
List2.AddItem ("60")
List2.AddItem ("77")
List2.AddItem ("96")


Label1.Caption = "Click button to run gradient descent"
Command1.Caption = "Click here"
Form1.Caption = "Gradient Descent"

Notice how the data fits the function above. Hopefully gradient descent will pick up on this. The key change from the previous example is changing the minimization function.

Here is the code for Command1_Click (when clicked, will run and graph gradient descent)

Private Sub Command1_Click()
Dim i, j, m As Integer
Dim x(24) As Long
Dim y(24) As Long
Dim yIndex As Long
Dim theta0, temp0, theta1, temp1, theta2, temp2 As Long


theta0 = 0 'initialize all variables to 0
theta1 = 0
theta2 = 0
temp0 = 0
temp1 = 0
theta2 = 0

m = List1.ListCount - 1 'number of samples
yIndex = 50

For i = 0 To List1.ListCount - 1 'populate arrays with data
    x(i) = CLng(List1.List(i))
    y(i) = CLng(List2.List(i))
Next

Dim iterations As Long
iterations = 168000 'set iterations high

Dim alpha As Double
alpha = 0.001 'set learning rate

Dim h0, h1, h2 As Long
h0 = 0
h1 = 0

For i = 0 To iterations
    h0 = 0
    h1 = 0
  
    For j = 0 To m
    h0 = h0 + ((theta0 + x(j) * x(j) * theta1) - y(j))
    h1 = h1 + ((theta0 + x(j) * x(j) * theta1) - y(j)) * x(j)
    Next j
  
    temp0 = theta0 - (alpha * h0) / CDbl(m)
    temp1 = theta1 - (alpha * h1) / CDbl(m)
    theta0 = temp0
    theta1 = temp1
   
Next i

Label1.Caption = CStr(theta1) & "x^2 + " & CStr(theta0)

Dim xGrid As Long
Dim yGrid As Long
Dim c, d As Long
xGrid = Picture1.Width / yIndex
yGrid = Picture1.Height / yIndex
c = 0
d = 0
''Set up grid
For i = 0 To Picture1.Height
   If (c > xGrid) Then
        For j = Picture1.Width / 2 - xGrid / 2 To Picture1.Width / 2 + xGrid / 2
            Picture1.PSet (j, i)
        Next j
    c = 0
    End If
    c = c + 1
Next i

For i = 0 To Picture1.Width
    If (d > yGrid) Then
        For j = Picture1.Height / 2 - yGrid / 2 To Picture1.Height / 2 + yGrid / 2
            Picture1.PSet (i, j)
        Next j
    d = 0
    End If
    d = d + 1
Next i

For i = 0 To Picture1.Width
    Picture1.PSet (i, Picture1.Height / 2)
Next

For i = 0 To Picture1.Height
    Picture1.PSet (Picture1.Width / 2, i)
Next i


Picture1.DrawWidth = 2
Dim yPos, counter As Double
yPos = 0
counter = 0#
For i = 2500 To 5000
    yPos = (theta1 * counter) ^ 2 + (theta0 * xGrid / 100)
    Picture1.PSet (i, Picture1.Height / 2 - yPos * xGrid)
    counter = counter + 0.01
Next i

counter = 0#
For i = 2500 To 0 Step -1
    yPos = (theta1 * counter) ^ 2 + (theta0 * xGrid / 100)
    Picture1.PSet (i, Picture1.Height / 2 - yPos * xGrid)
    counter = counter + 0.01
Next i
End Sub

The part of interest is here:

For i = 0 To iterations
    h0 = 0
    h1 = 0
  
    For j = 0 To m
    h0 = h0 + ((theta0 + x(j) * x(j) * theta1) - y(j))
    h1 = h1 + ((theta0 + x(j) * x(j) * theta1) - y(j)) * x(j)
    Next j
  
    temp0 = theta0 - (alpha * h0) / CDbl(m)
    temp1 = theta1 - (alpha * h1) / CDbl(m)
    theta0 = temp0
    theta1 = temp1
   
Next i

Label1.Caption = CStr(theta1) & "x^2 + " & CStr(theta0)


h0 and h1 are set as the minimization of a quadratic function. This is what gives us a quadratic solution. It fits the data much better. The bottom part is complicated as I didn't use a graphing tool but instead wrote a simple graph myself. Playing around with the input data set can give interesting results.

Program/Post Per Day

I've been coding for over 15 years. Never professionally, although I have made some monies off of some of the stuff I've made. But I am a firm believer that practice makes perfect. In that spirit, I am going to program every day for at least 30 minutes. I have the time and it will hopefully improve my programming abilities.

The problem is what to code. I am not sure I want to take on a big project or continue small coding projects that I can post here. I think for now, since my schedule is not determined for the near future, I will take it as it comes.

Now I guess I am publicly accountable to this! Unless I come back and delete this post and no one will ever know..

Things I hope to accomplish by doing this:

• Become better at programming (duh)
• Decrease mental "fog"
• Be more prepared for school next fall/spring
• Possibly make interesting programs
• Improve confidence in all things computer related

So there it is. What about you?

Insertion Sort in Java

Insertion sort is a simple sorting algorithm that orders a list one element at a time. It goes through the list and picks out each element and then goes back through the list until it finds its place. Here is working code in Java:

public class Sort {

    public static void main(String[] args) {
        int[] x = {10,11,7,6,5,4,3,2,14,3,8,9,2,5,1};
       
        int[] selectionSortArray = selectionSort(x, x.length);
        for(int i : selectionSortArray)
            System.out.print(i + ",");
    }
   
    public static int[] selectionSort(int[] list, int length)
    {
        int[] listTemp = list;
        int size = length;
        int j = 0;
        for(int i = 1; i < length; i++)
        {
            j = i;
            while(j > 0 && listTemp[j-1] > listTemp[j])
            {
                int temp = listTemp[j-1];
                int temp2 = listTemp[j];
                listTemp[j-1] = temp2;
                listTemp[j] = temp;
                j--;
            }
        }
        return listTemp;
    }

}

Iris Wipe Effect tutorial using C, libgdx and Tween engine

Hello and welcome to the this tutorial. There won't be much explanation of the code but instead an explanation on how you can use the code to put an iris wipe effect in your game. If you don't know what an iris wipe is, here is a look at what it does:

The real version is smoother than this video of it.

This effect is similar to old cartoons. When I first wanted to add it to a game, I didn't even know what it was called. I believe I googled "old cartoon black hole" and found it.

This tutorial will assume you are using Eclipse with libgdx set up. It will also need the universal tween engine for libgdx which can be found here This is a great tool to use for all sorts of things and I suggest learning how to use it outside of this tutorial.

There are three classes that we will be using. They are called IrisWipe, IrisWipeManager, and IrisAccessor. IrisWipe is the easiest to understand so let's take a look at it first:

import com.badlogic.gdx.graphics.Color;

public class IrisWipe {

    private float rIn;
    private float centerX;
    private float centerY;
    private Color color; //color of iriswipe (mostly used for the alpha)
    private float timer;
   
    public IrisWipe()
    {
        rIn = 0f;
        centerX = 0f;
        centerY = 0f;
        color = new Color(0f,0f,0f,1f);
        timer = 0f;
    }
   
    public float getTimer() {
        return timer;
    }

    public void setTimer(float timer) {
        this.timer = timer;
    }

    public Color getColor() {
        return color;
    }

    public void setColor(Color color) {
        this.color = color;
    }

    public float getRIn()
    {
        return rIn;
    }
   
    public void setRIn(float rIn)
    {
        this.rIn = rIn;
    }
   
    public float getCenterX()
    {
        return centerX;
    }
   
    public void setCenterX(float centerX)
    {
        this.centerX = centerX;
    }
   
    public float getCenterY()
    {
        return centerY;
    }
   
    public void setCenterY(float centerY)
    {
        this.centerY = centerY;
    }
   
}

This class is used primarily to hold where the center of the wipe will be at. Simply create an instance of it and pass it to an instance of iriswipemanager for which the code is here:

import aurelienribon.tweenengine.BaseTween;
import aurelienribon.tweenengine.Timeline;
import aurelienribon.tweenengine.Tween;
import aurelienribon.tweenengine.TweenCallback;
import aurelienribon.tweenengine.TweenEquations;
import aurelienribon.tweenengine.TweenManager;
import aurelienribon.tweenengine.TweenPaths;

import com.badlogic.gdx.graphics.Color;
import com.kilobolt.gameobjects.IrisWipe;
import com.kilobolt.TweenAccessors.IrisWipeAccessor;
import com.kilobolt.TweenAccessors.Value;
import com.kilobolt.TweenAccessors.ValueAccessor;

public class IrisWipeManager {

    private IrisWipe iriswipe;
    private TweenManager irismanager;
    private boolean wiping; //true when wiping is moving
    private boolean wipingIdle; //true for wipes that have no OUT and just sit there
    private boolean start; //true when time to start game
    private float gameWidth, gameHeight;
   
    //private TweenManager iwmanager;
   
   
    public IrisWipeManager(IrisWipe iriswipe, float gameWidth, float gameHeight)
    {
        this.iriswipe = iriswipe;
        this.gameWidth = gameWidth;
        this.gameHeight = gameHeight;
        wiping = false; //set that no wipes happening to start
        wipingIdle = false;
        start = false; //true when game is to start
        Tween.registerAccessor(IrisWipe.class, new IrisWipeAccessor());
        irismanager = new TweenManager();
    }
   
    public void update(float delta)
    {
        if(wiping | wipingIdle)
        {
            irismanager.update(delta);
        }
    }

    //do iris wipe at a certain area
    //CENTERX AND CENTERY must be set BEFORE CALLING
    //diameter is how big the transparent hole will be
    //IRIS WIPE USAGE
    //position you want - (gameWidth or gameHeight) / 2f
    //iriswipe.setCenterX(x - (gameWidth / 2f));
    //iriswipe.setCenterY(y - (gameHeight / 2f));
    //iriswipemanager.iriswipeSpot(4f);
    public void iriswipeSpot(float diameter, boolean inANDout)
    {
        //kill any previous wipes
        if(irismanager.containsTarget(iriswipe))
        {
            System.out.println("PREVIOUS IRISWIPE KILLED in IWM");
            irismanager.killTarget(iriswipe);
        }

                //callback for no exand out
                TweenCallback cbNOOUT = new TweenCallback() {
                    @Override
                    public void onEvent(int arg0, BaseTween<?> arg1) {
                        //not set to false so that it still draws
                        wiping = false;
                        wipingIdle = true;
                    }
          
                };
       
       
                //shrink callback - expand out now
                TweenCallback cb = new TweenCallback() {
                    @Override
                    public void onEvent(int arg0, BaseTween<?> arg1) {
                       
                        //set wiping to false on final callback
                        TweenCallback cb = new TweenCallback() {
                            @Override
                            public void onEvent(int arg0, BaseTween<?> arg1) {
                                wiping = false;
                                wipingIdle = false;
                            }
                          
                           };
                       
                          
                        //expand wipe out
                        Timeline.createSequence()
                        .push(Tween.to(iriswipe, IrisWipeAccessor.RIN, 3.5f)
                            .target(710f)
                            .ease(TweenEquations.easeInOutCubic))
                            .setCallbackTriggers(TweenCallback.COMPLETE)
                            .setCallback(cb)
                        .start(irismanager);
                    } //this happens after every tween of word ends
                  
                   };
       
        //wipe in
        wiping = true; //wipe is moving
        wipingIdle = false; //wipe is not idling
        if(inANDout) //if it is to go in AND out, set a callback to do the out after in
        {
            Timeline.createSequence()
            .push(Tween.to(iriswipe, IrisWipeAccessor.RIN, 1.0f)
                .target(diameter)
                .ease(TweenEquations.easeOutQuad))
                .setCallbackTriggers(TweenCallback.COMPLETE)
                .setCallback(cb)
            .start(irismanager);
        }else//just have it go IN
        {
            Timeline.createSequence()
            .push(Tween.to(iriswipe, IrisWipeAccessor.RIN, 1.0f)
                .target(diameter)
                .ease(TweenEquations.easeOutQuad))
                .setCallbackTriggers(TweenCallback.COMPLETE)
                .setCallback(cbNOOUT)
            .start(irismanager);
        }
    }
   
   
   
    public void kill()
    {
        irismanager.killAll();
    }
   
    public boolean getWipingIdle() {
        return wipingIdle;
    }

    public void setWipingIdle(boolean wipingIdle) {
        this.wipingIdle = wipingIdle;
    }

    public boolean getWiping()
    {
        return wiping;
    }
   
    public void setWiping(boolean wiping)
    {
        this.wiping = wiping;
    }
   
    public boolean getStart()
    {
        return start;
    }
   
    public void setStart(boolean start)
    {
        this.start = start;
    }
   
}

Some notes about the usage of this class. You have to read the comments above irisWipeSpot() to see how to call it. This class also requires you to know the width and height of your game, which you should know. The parameter diameter that is passed to irisWipeSpot is how big you want the hole to be when it reverses direction. inANDout should be set to true. This can be called from anywhere. The real work occurs in the render method.

Computers apparently handle triangles a lot better than circles. For this reason, we will actually be using triangles. Lots and lots of them. What you are seeing in the gif above is many different triangles that are skinny enough so that it looks smooth like a circle. This part involves quite a bit of trig and the code looks quite ugly. That's the reason I am not going to go into it very much. If you like trig, feel free to dive into the code and have a look. Otherwise, just copy and paste and you'll be good to go. The following code needs to be in your render class. If you have gameRenderer.java or something like that, put the following above your constructor.

 private IrisWipe iriswipe;
    private IrisWipeManager iriswipemanager;
      private float rOut;
      private float rIn;
      private float iriswipecx;
      private float iriswipecy;
      private float iriswipex1;
      private float iriswipey1;
      private float iriswipex2;
      private float iriswipey2;
      private float iriswipex3;
      private float iriswipey3;
      private float iriswipex4;
      private float iriswipey4; 
      private Array<Vector2> iriswipeXY1, iriswipeXY2, iriswipeXY3;
      private Array<Color> iriswipecolor;
      private Value irisSwipeValue;
     private boolean irisswipe, irisswipeFirstTime;

That's a lot of variables. I told you to just copy and paste. Now in the constructor, put this:

         batcher = new SpriteBatch();
        // Attach batcher to camera
        batcher.setProjectionMatrix(cam.combined);
       
        shapeRenderer = new ShapeRenderer();
        shapeRenderer.setProjectionMatrix(cam.combined);
       
        rOut = gameHeight * 2f; //something big enough to cover whole screen
        rIn = 0f;//irisSwipeValue.getValue();
        iriswipecx = screenWidth / 2f;
        iriswipecy = gameHeight / 2f;
        irisswipe = true;
        irisswipeFirstTime = true;
        irisSwipeValue = new Value();
        irisSwipeValue.setValue(0f);
       
        iriswipex1 = 0f;;
        iriswipey1 = 0f;
        iriswipex2 = 0f;
        iriswipey2 = 0f;
        iriswipex3 = 0f;
        iriswipey3 = 0f;
        iriswipex4 = 0f;
        iriswipey4 = 0f;
       
        iriswipeXY1 = new Array<Vector2>();
        iriswipeXY2 = new Array<Vector2>();
        iriswipeXY3 = new Array<Vector2>();
        iriswipecolor = new Array<Color>();
        int iIris = 0;
       
        while(iIris <= 361)
        {
           
            iriswipeXY1.add(new Vector2((float)Math.cos(Math.toRadians((double)((iIris)))), (float)Math.sin(Math.toRadians((double)(iIris)))));
            iriswipeXY2.add(new Vector2((float)Math.cos(Math.toRadians((double)((iIris)))), (float)Math.sin(Math.toRadians((double)(iIris)))));
            iriswipeXY3.add(new Vector2((float)Math.cos(Math.toRadians((double)((iIris + 1f)))), (float)Math.sin(Math.toRadians((double)((iIris + 1f))))));
                     
            iIris++;
        }

If you already have a Spritebatch and/or shaperenderer you won't need the top part of the code. The mathy parts at the bottom are beyond what I will explain here. If you understand trig enough, you can get a sense of what these arrays are filling up with. Now in your game loop put:

drawIrisWipe();

And that's it! Wait, we need the code for the method above. This isn't going to look pretty:

 private void drawIrisWipe()
    {
        //if iriswipe is not moving and not idle either, dont draw
        if(!iriswipemanager.getWiping() & !iriswipemanager.getWipingIdle())
        {
            return;
        }
        shapeRenderer.begin(ShapeType.Filled);
        shapeRenderer.setColor(iriswipe.getColor());
        rIn =  iriswipe.getRIn();  //tweened value of rIn
       
        int degree = 0;
        while(degree < 360)
        {
            iriswipex1 = (rOut * iriswipeXY1.get(degree).x) + iriswipecx + iriswipe.getCenterX();
            iriswipey1 = (rOut * iriswipeXY1.get(degree).y) + iriswipecy + iriswipe.getCenterY();
            iriswipex2 = (rIn * iriswipeXY2.get(degree).x) + iriswipecx + iriswipe.getCenterX();
            iriswipey2 = (rIn * iriswipeXY2.get(degree).y) + iriswipecy + iriswipe.getCenterY();
            iriswipex3 = (rIn * iriswipeXY3.get(degree).x) + iriswipecx + iriswipe.getCenterX();
            iriswipey3 = (rIn * iriswipeXY3.get(degree).y) + iriswipecy + iriswipe.getCenterY();
            iriswipex4 = (rOut * iriswipeXY1.get(degree + 1).x) + iriswipecx + iriswipe.getCenterX();
            iriswipey4 = (rOut * iriswipeXY1.get(degree + 1).y) + iriswipecy + iriswipe.getCenterY();
           
                shapeRenderer.triangle(iriswipex1, iriswipey1, iriswipex2, iriswipey2, iriswipex3, iriswipey3);
                shapeRenderer.triangle(iriswipex1, iriswipey1, iriswipex4, iriswipey4, iriswipex3, iriswipey3);
           
           
            degree += 1;
        }
        shapeRenderer.end();
    }

I originally added this to a game I made, but I've added it now to my path finding program just as a demo. If you put in all the above code and call irismanager.update(delta) in your game loop, it should work fine. If you want to look at it in the path finding program, you can download that here

Hopefully this is helpful. There were no tutorials on how to make this effect online so if it is a little rough around the edges, fix it and tell me what you did! Comment if you need help