Neuroevolution simulation (Neural network and genetic algorithm)

m4l4

here we go, following up my previous post (https://forum.processing.org/two/discussion/23539/genetic-algorithm-driven-ecosystem#latest) i finally managed to write a fully functional multilayer perceptron to give my creatures their very own brain. it's a simple simulation, brains are randomly generated, creatures have 4 inputs, they triangulate the distance of the closest food with body and antennas, they can also perceive their health levels. creatures will die for starvation (change color from red to white) and overeating (from red to black), so over time they will learn that they need food to survive, how to seek it and to balance food consumption. outputs are: rotate right, rotate left, move forward, speed value. Click on a creature to display its brain.

the code is extensively commented, but for a better comprehension i suggest you to read it in the following order: Connection class, Neuron class, Layer class, Neural Network class, Eater/food class, World class, Main Tab (NeuroEvolution V3).

gitHub repo: https://github.com/m4l4/Neuroevolution-Simulation Hope you enjoy it, prepare for some code:

m4l4

/*
Neuroevolution simulation
 created by Marco Perozziello  22/10/2017
 https://github.com/m4l4/
 https://forum.processing.org/two/profile/39945/m4l4

 Credits to Daniel Shiffman from "The coding train" for his lessons
 5 moths ago i din't even knew how to draw a circle, he's done the rest.
 https://www.youtube.com/user/shiffman

 Credists to ScottC who created the Neural Net design that i've rearranged for my project.
 i'll refer to his blog for detailed explanation of his Neural Network
 http://arduinobasics.blogspot.com/p/arduinoprojects.html

 Also credits to the Processing community for helping me by answering all my noobish questions on the forum :)
 */

World world;

int time;
int generation;
int longestGeneration;
float oldestEater;
int bestGen;
int numE;
int numF;


void setup() {
  //  fullScreen();
  size(1280, 800);
  smooth();
  numE = 30;
  numF = 50;
  time = 0;
  generation = 0;
  longestGeneration = 0;
  oldestEater = 0;
  bestGen = 0;
  world = new World(numE, numF);
}


void draw() {
  background(255);
  if (world.eatersClones.size() < numE) {
    world.run();
    time += 1;
  } else {

    if (time > longestGeneration) longestGeneration = time;
    if (world.getEatersMaxFitness() > oldestEater) {
      oldestEater = world.getEatersMaxFitness();
      bestGen = generation;
    }

    time = 0;
    generation++;

    world.foods.clear();
    int r = world.eatersClones.get(0).r;
    for (int i=0; i < numF; i++) {                            
      world.foods.add(new Food(random(r+r/2, width-(r+r/2)), random(r+r/2, height-(r+r/2))));    //initialize food for the next gen
    }

    world.eaterSelection();
    world.eatersReproduction();
  }

  fill(0);                                                               // display some stats
  textSize(12);
  text("Generation #: " + (generation), 10, 18);
  text("Lifetime: " + (time), 10, 36);
  text("living Eaters: " + world.eaters.size(), 10, 54);
  text("Longest gen: " + longestGeneration, 10, 72);
  text("Oldest Eater: gen" +bestGen + " - " + oldestEater, 10, 90);


  if (mouseButton == LEFT) {
    for (int i = 0; i < world.eaters.size(); i++) {       // show brain structure and activity when you click on a creature
      world.eaters.get(i);
      if (mouseX >  world.eaters.get(i).position.x - world.eaters.get(i).r*2 && mouseX < world.eaters.get(i).position.x + world.eaters.get(i).r*2 && mouseY > world.eaters.get(i).position.y - world.eaters.get(i).r*2 && mouseY < world.eaters.get(i).position.y + world.eaters.get(i).r*2) {

        for (int j = 0; j < world.eaters.size(); j++) {
          world.eaters.get(j).displayBrain = false;
        }
        world.eaters.get(i).displayBrain = !world.eaters.get(i).displayBrain;
      }
    }
  }
}

m4l4

class Connection {
  float connEntry;           //raw input
  float weight;              //connection weight (-1 to 1)
  float connExit;            //processed output (raw * weight)

  Connection() {                       //This is the default constructor for an Connection
    randomiseWeight();
  }

  Connection(float tempWeight) {       //A custom weight for this Connection constructor
    setWeight(tempWeight);
  }

  void setWeight(float tempWeight) {   //Function to set the weight of this connection
    weight=tempWeight;
    weight = constrain(weight, -1, 1);
  }

  void randomiseWeight() {             //Function to randomise the weight of this connection
    setWeight(random(-1, 1));
  }

  float getWeight() {                   //function to get the weight of the connection
    return weight;
  }

  float calcConnExit(float tempInput) {  //function to process the input
    connEntry = tempInput;
    connExit = connEntry * weight;
    return connExit;
  }
}

m4l4

class Eater {

  NeuralNetwork NN = new NeuralNetwork();
  float[] myInputs = {};

  PVector position;
  PVector velocity;
  PVector leftAntennaPos;
  PVector rightAntennaPos;

  int r;                   //radius of the creature
  float antennaLength;     //lenght of the antennas
  float antennaRadius;     //radius of the antennas 
  float age;               //age in this simulation represent the fitness of the creatures.
  float health;            //health value
  float maxHealth;         //health cap
  float speed;             
  float maxSpeed;
  float theta;             //angle of rotation
  float rotationSpeed;     //speed of rotation
  float maxForce;          

  boolean stucked;
  boolean displayBrain;

  Eater(float x, float y, NeuralNetwork NN_) {

    NN = NN_;                  //to initialize the neural network you have to decide how many layer it will have and how many neurons each layer will have

    NN.addLayer(4, 5);         //first layer we add determines the number of input neurons (4 in this case) and the number of neurons of the first hidden layer (5)
    NN.addLayer(5, 4);         //every subsequent layer defines how many connection it has (5) and the number of neurons in the layer (4)
    //since the outputs of a layer become the inputs of the next one...
    //...the number of connections MUST equal the number of neurons of the previous layer
    //the last layer you add determines how many outputs neuron you will have (4 in the example)
    //the constructor i used will initialize a neural net made of: 4 inputs, 1 hidden layer (with 5 neurons), and 4 outputs
    //you can keep adding as many layers and neurons you want, just be sure to check connections or you'll get an error.

    position = new PVector(x, y);    
    velocity = new PVector(0, 0);
    leftAntennaPos = new PVector(0, 0);
    rightAntennaPos = new PVector(0, 0);

    r = 20;
    antennaLength = r;
    antennaRadius = r/4;
    age = 0;
    maxHealth = 200;
    health = maxHealth/2;  
    speed = 0;
    maxSpeed = 4;
    maxForce = 0.5;
    theta = 0;
    rotationSpeed = 2;
    stucked = false;
    displayBrain = false;
  }

  void run() {    
    wrapAround();
    seek();    

    float healthLoss = map(speed, 0, maxSpeed, 0, maxSpeed*0.1);    //healthLoss depends on actual speed, the faster we move the more we consume.
    if (healthLoss < 0.2) healthLoss = 0.2;                         
    health -= healthLoss;

    display();
    if (displayBrain) displayNeuralNet();
  }

  void seek() {
    float newFoodDistance = 10000;                                  // initialize the food distance to a vey high value to avoid getting stuck with disappeared targets
    PVector closestFood = new PVector (random(0, 1), random(0, 1));

    ArrayList<Food> foods = world.getFood();                        // get the distance of every plant in the world
    for (int j = foods.size()-1; j >= 0; j--) {                     // since plants will be removed when eaten, we have to check the arraylist backwards to avoid skipping some of them
      PVector foodPosition = foods.get(j).position;
      float foodDistance = PVector.dist(position, foodPosition);

      if (foodDistance < newFoodDistance) {                      
        newFoodDistance = foodDistance;
        closestFood = foodPosition;
      }
      if (foodDistance < r/2) {                  // if we reach a plant         
        foods.remove(j);                         // remove it from the world
        health += 35;                            // enjoy the meal
      }
    }
    myInputs = new float[0];                                                      //reset inputs array
    //we will tringulate the position of the closest food source by using 3 neurons
    float leftAntennaDistance = PVector.dist(leftAntennaPos, closestFood);        //calculate food distance from left antenna
    myInputs = (float[]) append(myInputs, leftAntennaDistance);                   //set first input neuron
    float rightAntennaDistance = PVector.dist(leftAntennaPos, closestFood);       //calculate food distance from right antenna
    myInputs = (float[]) append(myInputs, rightAntennaDistance);                  //set second input neuron
    float bodyDistance = PVector.dist(position, closestFood);                     //calculate food distance from the body
    myInputs = (float[]) append(myInputs, bodyDistance);                          //set the third input neuron

    float hunger = map(health, 0, maxHealth, -10, 10);                            //the forth one is the "hunger" neuron, it maps current health...
    myInputs = (float[]) append(myInputs, hunger);                                //and inputs a negative value if below "maxHealth/2" or a positive one if above it.
    //that way the creature can "feel" hunger and fullness

    NN.processInputsToOutputs(myInputs);                                          //once we've set the inputs we process them
    //some neurons will be used as binary actuators, if they output > than 0.0 we will take it as a "yes"

    if (NN.arrayOfOutputs[0] > 0.0) theta += radians(rotationSpeed);              //if first output is > 0 rotate right
    if (NN.arrayOfOutputs[1] > 0.0) theta -= radians(rotationSpeed);              //if the second output is > 0 rotate left

    speed = map(NN.arrayOfOutputs[2], -1, 1, 0, maxSpeed);                        //third output is the "Thruster potentiometer" it gets mapped to determine speed value

//    checkCollision();                                                             //check if we're about to collide with other eaters

    if (NN.arrayOfOutputs[3] > 0.0 && stucked == false) {                         //4rth output is the "Thruster" if it output > 0 we move forward based on speed and heading
      velocity.x = speed*cos(theta);                                              //this modify vel.x and vel.y
      velocity.y = speed*sin(theta);                                              //based ont the actual angle
      position.add(velocity);
    }
  }

  void checkCollision() {
    int touchedEaters = 0;
    for (int i=0; i < world.eaters.size(); i++) {                    //check eaters array
      float minDist = (r/2) + (world.eaters.get(i).r/2);             //calculate minimumdistance based on eaters radius
      PVector eaterPos = world.eaters.get(i).position.copy();        //get eater position
      PVector eaterVel = world.eaters.get(i).velocity.copy();        //get eater velocity
      PVector eaterFuturePos = PVector.add(eaterPos, eaterVel);      //forecast eater future position
      PVector futurePos = PVector.add(position, velocity);           //forecast your future position
      float d = PVector.dist(futurePos, eaterFuturePos);                   //check if your movement will exceed minDist

      if (d < minDist && d > 0.1) {     //keep track of "future" touched eaters till the end of the array
        touchedEaters ++;               //d > 0.1 is there to avoid considering ourselves "another" eater
      }
    }
    if (touchedEaters > 0) {    //if we will touch something
      stucked = true;        //get stucked
    } else {              
      stucked = false;
    }
  }

  boolean dead() {                                    // defines death conditions
    if (health < 0.0 || health > maxHealth) {         // creatures will die if their health gets to 0 or if it reaches maxHealth treshold (starving and overeating)
      return true;
    } else {
      return false;
    }
  }

  void borders() {                                            //function to make edge borders
    position.x=constrain(position.x, r+r/2, width-r+r/2);     //limit position vector to screen size
    position.y=constrain(position.y, r+r/2, height-r+r/2);
  }

  void wrapAround() {                                         //make edges continuos
    if (position.x < -r) position.x = width+r;                //wrap borders
    if (position.y < -r) position.y = height+r;
    if (position.x > width+r) position.x = -r;
    if (position.y > height+r) position.y = -r;
  }

  void display() {
    float colour = 0;

    stroke(0);
    if (health < maxHealth/2) {                                //color of the creature changes based on it's health value
      colour = map(health, 0, maxHealth/2, 255, 0);            //an healty creature will show a red color (healty is between 0 and maxHealth)
      fill(255, colour, colour);                               //if it starves the color will fade to white
    } else {
      colour = map(health, maxHealth/2, maxHealth, 255, 0);    //if it eats too much the color will darken to black
      fill(colour, 0, 0);
    }


    pushMatrix();
    translate(position.x, position.y);
    rotate(theta);    
    ellipseMode(CENTER);  

    line(0, 0, r, -r/2);                 //left antenna 
    ellipse(r, -r/2, r/3, r/3);
    leftAntennaPos.x = screenX(r, -r/2);
    leftAntennaPos.y = screenY(r, -r/2);

    line(0, 0, r, r/2);                  //right antenna
    ellipse(r, r/2, r/3, r/3);
    rightAntennaPos.x = screenX(r, r/2);
    rightAntennaPos.y = screenY(r, r/2);

    ellipse(0, 0, r, r);                 //body of the creature
    popMatrix();
  }

  void displayNeuralNet() {                  //Function to display brain structure and activity, it automatically adapt dimensions to brain structure
    //click on a creature to display it

    float brainBoxWidth = width/4;                                      
    float brainBoxHeight = height/4;

    strokeWeight(1);
    stroke(175);
    fill(255, 175);
    rect(width-brainBoxWidth, 0, brainBoxWidth, brainBoxHeight);

    float r = 0;                                                         //radius of the neurons
    int biggestLayer = NN.layers[0].neurons[0].getConnectionCount();     //find which layer has more neurons (initialize with input layer size)
    int networkDepth = NN.getLayerCount()+1;                             //how many layer there are  (+1 is for the input layer)

    for (int i = 0; i < NN.getLayerCount(); i++) {                        //find which layer has more neurons
      if (NN.layers[i].getNeuronCount() > biggestLayer) biggestLayer = NN.layers[i].getNeuronCount();
    }

    float maxYdistance = (brainBoxHeight/biggestLayer+1);                  //define how much space you will have vertically and horizontally
    float maxXdistance = (brainBoxWidth/networkDepth+1);

    if (maxYdistance > maxXdistance) r = maxYdistance/5;                   //the smaller one determine radius of the neurons
    else                             r = maxXdistance/4;

    float xOffset = (brainBoxWidth/(networkDepth+1));                      // distance between layers
    float xPosition = width-xOffset; 


    for (int i = NN.getLayerCount()-1; i >= 0; i--) {                      // starting from the last layer we draw the network backward
      // we do it so because of the way we store weights and bias
      float nextYposition = 0;                                           
      float nextYoffset = 0;
      int connections = 0;
      float yOffset = brainBoxHeight/(NN.layers[i].getNeuronCount()+1);   //distance between neurons of the same layer
      float yPosition = yOffset;                                          //y position of the neuron

      ArrayList<Float> tempWeights = new ArrayList<Float>();              //since the weights array contains biases we will need to remove them
      float[] layerWeights = NN.layers[i].getWeights();                   //removing object is easier with arraylist so we first get a copy of the layerWeights array

      for (int n = 0; n < layerWeights.length; n++) {                     //and we then clone it into an arraylist
        tempWeights.add(layerWeights[n]);
      }


      for (int j = 0; j < NN.layers[i].getNeuronCount(); j++) {                           //for every neuron of the layer...

        if (i > 0) {
          nextYoffset = brainBoxHeight/(NN.layers[i-1].getNeuronCount()+1);               //calculate yposition of the next layers neurons
          connections = NN.layers[i-1].getNeuronCount();                                  //check how many connection we have to make with the next layer
        } else {     
          nextYoffset = brainBoxHeight/(NN.layers[0].neurons[0].getConnectionCount()+1);  //if we are at the last layer (layer[0] since we are going backward)
          connections = NN.layers[0].neurons[0].getConnectionCount();                     //we use its connections to draw the input layer
        }

        nextYposition = nextYoffset;                                          

        for (int k = 0; k < connections; k++) {                      //finally we draw the connections

          float tempStrokeWeight = 0;                                //strokeweight and color depend on the connection weight
          if (tempWeights.get(0) < 0.0) {                            //if the connection has a negative weight
            tempStrokeWeight = (1+ tempWeights.get(0)*-1.5);         //we draw it as it was positive
            stroke(255, 0, 0);                                       //but we draw it in red
          } else {
            tempStrokeWeight = (1+ tempWeights.get(0)*1.5);
            stroke(0);                                               //positive connections are drawn in black
          }
          float nextXposition = xPosition - xOffset;                 //keep track of the x offset

          strokeWeight(tempStrokeWeight);                            //set strokeweight
          line(xPosition, yPosition, nextXposition, nextYposition);  //draw the connection

          nextYposition += nextYoffset;                              //point at the next neuron
          tempWeights.remove(0);                                     //remove the used weight from the arraylist
        }

        if (i == NN.getLayerCount()-1) {                             //output neurons change based on their activity
          if (NN.arrayOfOutputs[j] > 0.0) fill(0);                   //black = active
          else                            fill(255);                 //white = inactive
        } else {
          float tempColor = 0;                                         //neuron color depends on its bias
          if (tempWeights.get(0) > 0.0) {                              //if bias is positive color goes from white to black (0 to 1)
            tempColor = map(tempWeights.get(0), 0, 1, 255, 0);        
            fill(tempColor);
          } else {                                                       //if bias is negative color goes from white to red (0 to -1)
            tempColor = map(tempWeights.get(0), -1, 0, 0, 255);
            fill(255, tempColor, tempColor);
          }
        }
        strokeWeight(1);
        stroke(0);
        ellipseMode(CENTER);
        ellipse(xPosition, yPosition, r, r);                         //once every connections has been drawn, we draw the neuron
        yPosition += yOffset;                                        //we move down a neuron
        tempWeights.remove(0);                                       //we remove the neuron bias from the weights arraylist
      }
      tempWeights.clear();                                           //once we've finished drawing a layer we clear the weights array
      xPosition -= xOffset;                                          //and we advance to the next x position

      if (i == 0) {                                                                 //once we've finished with the layer 0
        xPosition = (width-brainBoxWidth)+xOffset;                                  //reset position X and y
        yOffset = brainBoxHeight/(NN.layers[0].neurons[0].getConnectionCount()+1);  //calculate yOffset based on the inputs num
        yPosition = yOffset;

        for (int m = 0; m < NN.layers[0].neurons[0].getConnectionCount(); m++) {    //draw input neurons with different color
          fill(255, 175, 0);                                             
          ellipse(xPosition, yPosition, r, r);
          yPosition += yOffset;
        }
      }
    }
  }
}

m4l4

class Food {                    //not much to say about that
  PVector position;

  Food(float x, float y) {
    position = new PVector(x, y);
  }

  void run() {
    display();
  }  

  void display() {
    stroke(0);
    fill(200, 200, 0);
    ellipse(position.x, position.y, 10, 10);
  }
}

m4l4

class Layer {
  Neuron[] neurons = {};

  float[] layerWeights = {};

  float[] layerInputs = {};
  float[] layerOutputs = {};

  Layer(int ConnectionNum, int NeuronNum) {
    for (int i = 0; i < NeuronNum; i++) {
      Neuron tempNeuron = new Neuron(ConnectionNum);                        //create neuron
      addNeuron(tempNeuron);                                                //add it to the neuron array
      addLayerOutputs();                                                    //add an output for every neuron int the layer
      for (int j = 0; j < tempNeuron.connectionWeights.length; j++) {        //for every neuron retrive weights and bias
        layerWeights = (float[]) append(layerWeights, tempNeuron.connectionWeights[j]);
      }
    }
  }

  void addNeuron(Neuron xNeuron) {                 //Function to add an input or output Neuron to this Layer
    neurons = (Neuron[]) append(neurons, xNeuron);
  }

  int getNeuronCount() {                            //Function to get the number of neurons in this layer
    return neurons.length;
  }

  float[] getWeights() {                             //Function to get Weights and biases of the whole layer
    return layerWeights;
  }

  void addLayerOutputs() {                          //Function to increment the size of the actualOUTPUTs array by one.
    layerOutputs = (float[]) expand(layerOutputs, (layerOutputs.length+1));
  }

  void setWeights(float[] tempWeights) {            //function to set weights and bias for every neuron of the layer
    layerWeights = new float[0];                    //first we clear the layerWeights array


    for (int i= 0; i < tempWeights.length; i++) {                                                         //for every value of the temporary array...

      for (int j = 0; j < getNeuronCount(); j++) {                                                        //for every neuron of the layer...   
        neurons[j].connectionWeights = new float[0];                                                      //we clear the neuron Weights array

        for (int k = 0; k < neurons[j].getConnectionCount(); k++) {                                       //for every connection of the neuron...
          neurons[j].connections[k].setWeight(tempWeights[i]);                                            //set the connection weights
          layerWeights = (float[]) append(layerWeights, tempWeights[i]);                                  //move the value into the layer weights array
          neurons[j].connectionWeights = (float[]) append(neurons[j].connectionWeights, tempWeights[i]);  //move the value into the neuron weights array
          i++;                                                                                            // "i" must advance since the value has been already set
        }
        neurons[j].setBias(tempWeights[i]);                                                               //once we've finished with the connection we set the neuron bias      
        layerWeights = (float[]) append(layerWeights, tempWeights[i]);                                    //and move value to the layer weights array
        neurons[j].connectionWeights = (float[]) append(neurons[j].connectionWeights, tempWeights[i]);    //aswell to the neuron weights array
        i++;                                                                                              //once again "i" must advance to avoid reusing values
      }
    }
  }

  float[] getWeigths() {
    return layerWeights;
  }

  void setInputs(float[] tempInputs) {              //set inputs for this layer
    layerInputs = tempInputs;
  }

  void processInputsToOutputs() {                   //process all the inputs to output for the neurons in this layer    
    int neuronCount = getNeuronCount();

    if (neuronCount > 0) {                                                //check if there are neurons to process inputs
      if (layerInputs.length!=neurons[0].getConnectionCount()) {          //check if num of inputs match num of neurons
        println("Error in Layer: processInputsToOutputs: The number of inputs do NOT match the number of Neuron connections in this layer");
        exit();
      } else {
        for (int i=0; i<neuronCount; i++) {                                //calculate layer outputs
          layerOutputs[i]=neurons[i].getNeuronOutput(layerInputs);
        }
      }
    } else {
      println("Error in Layer: processInputsToOutputs: There are no Neurons in this layer");
      exit();
    }
  }
}

m4l4

class NeuralNetwork {
  Layer[] layers = {};                  //array of layers
  float[] arrayOfInputs = {};           //inputs of the neural net
  float[] arrayOfOutputs = {};          //outputs of the neural net
  float[][] networkWeights = {};        // TO DO matrix of the NN weights and function to set it at once


  NeuralNetwork() {
  }

  void addLayer(int ConnectionNum, int NeuronNum) {            //Function to add a Layer to the Neural Network
    layers = (Layer[]) append(layers, new Layer(ConnectionNum, NeuronNum));
  }

  int getLayerCount() {                                        //Function to get the number of layers
    return layers.length;
  }

  void setInputs(float[] tempInputs) {                         //Function to set the inputs of the neural network
    arrayOfInputs = tempInputs;
  }

  void setLayerInputs(float[] tempInputs, int layerIndex) {    //Function to set the inputs of a specific layer
    if (layerIndex > getLayerCount()-1) {
      println("NN Error: setLayerInputs: layerIndex=" + layerIndex + " exceeded limits= " + (getLayerCount()-1));
    } else {
      layers[layerIndex].setInputs(tempInputs);
    }
  }

  void setOutputs(float[] tempOutputs) {                 //Function to set the outputs of the neural network
    arrayOfOutputs = tempOutputs;
  }

  float[] getOutputs() {                               //Function to get the outputs
    return arrayOfOutputs;
  }

  void processInputsToOutputs(float[] tempInputs) {    //function to process inputs to outputs using all the layers
    setInputs(tempInputs);

    if (getLayerCount() > 0) {      //make sure that the number ofinputs matches the neuron connections of the first layer
      if (arrayOfInputs.length != layers[0].neurons[0].getConnectionCount()) {
        println("NN Error: processInputsToOutputs: The number of inputs do NOT match the NN");
        exit();
      } else {                                              // number of inputs is fine
        for (int i = 0; i < getLayerCount(); i++) {         //set inputs for the layers
          if (i==0) {
            setLayerInputs(arrayOfInputs, i);               //first layer get inputs from the NN
          } else {
            setLayerInputs(layers[i-1].layerOutputs, i);    //other layer get inputs from the previous one
          }
          layers[i].processInputsToOutputs();               //once inputs have been set, convert them to outputs
        }
        setOutputs(layers[getLayerCount()-1].layerOutputs);
      }
    } else {
      println("Error: There are no layers in this Neural Network");
      exit();
    }
  }
}

m4l4

class Neuron {
  Connection[] connections={};              //array of connections
  float[] connectionWeights = {};           //array of connections weights

  float bias;
  float neuronInput;
  float neuronOutput;

  Neuron(int ConnectionNum) {                //constructor with random bias and connection weights
    randomiseBias();
    for (int i = 0; i < ConnectionNum; i++) {
      Connection conn = new Connection();
      addConnection(conn);
      float tempWeight = conn.getWeight();
      connectionWeights = (float[]) append(connectionWeights, tempWeight);  //populate the weights array
    }
    connectionWeights = (float[]) append(connectionWeights, bias);          //append the bias to the weights array
  }

  void addConnection(Connection conn) {                                     //Function to add a Connection to this neuron
    connections = (Connection[]) append(connections, conn);
  }

  int getConnectionCount() {                // Function to return the number of connections associated with this neuron.
    return connections.length;
  }

  void setBias(float tempBias) {            //Function to set the bias of this Neron
    bias = tempBias;
  }

  void randomiseBias() {                     //Function to randomise the bias of this Neuron
    setBias(random(-1, 1));
  }

  float getNeuronOutput(float[] connEntryValues) {            //Function to convert the inputValue to an outputValue
    if (connEntryValues.length!=getConnectionCount()) {       //Make sure that the number of connEntryValues matches the number of connections
      println("Neuron Error: getNeuronOutput() : Wrong number of connEntryValues");
      exit();
    }

    neuronInput = 0;                                                 // clear the previous input

    for (int i = 0; i < getConnectionCount(); i++) {                  // sum weighted inputs for all connections
      neuronInput += connections[i].calcConnExit(connEntryValues[i]);
    }

    neuronInput += bias;                           //add the bias
    neuronOutput = ActivateTanh(neuronInput);        //pass the sum into the activation function
    return neuronOutput;                           //return output
  }

  float ActivateTanh(float x) {                                    //Activation function
    float activatedValue = (2 / (1 + exp(-1 * (x*2)))) -1;         //TanH (hyperbolic tangent) returns -1 to 1
    return activatedValue;
  }
}

m4l4

class World {

  ArrayList<Food> foods;                   //plants array
  ArrayList<Eater> eaters;                 //creatures array
  ArrayList<Eater> eatersClones;           //clones array we will use to recover stats from dead creatures
  ArrayList<Eater> eatersMatingPool;       //selection pool we will use for reproduction

  float foodSpawnRate;                     
  float mutationRate;
  int r;

  World(int numE, int numF) {
    foodSpawnRate = 0.02;
    mutationRate = 0.01;
    foods = new ArrayList<Food>(); 
    eaters = new ArrayList<Eater>();
    eatersClones = new ArrayList<Eater>();
    eatersMatingPool = new ArrayList<Eater>();

    for (int i=0; i < numE; i++) {                                                        // populate the arraylist for eaters
      eaters.add(new Eater(random(0, width), random(0, height), new NeuralNetwork()));
    }

    r = eaters.get(0).r;
    for (int i=0; i < numF; i++) {                                                        // populate the arraylist for plants
      foods.add(new Food(random(r+r/2, width-(r+r/2)), random(r+r/2, height-(r+r/2))));   //to avoid generating out of reach food, we use creaure radius as reference
    }
  }

  void run() {                                       //since plants and creatures will be either added or removed during the simulation...
    for (int i = foods.size()-1; i >= 0; i--) {      //...we will check arrays backwards to avoid skipping elements.           
      Food f = foods.get(i);                                     
      f.run();
      if (foods.size() > numF*2) foods.remove(0);    // max plants num
    }

    for (int i = eaters.size()-1; i >= 0; i--) {                  
      Eater e = eaters.get(i);
      e.run();
      if (e.dead()) {
        eatersClones.add(e);                     //when a creature dies, we first move it int the clones array for later use
        eaters.remove(i);                        //than we remove it from the world
      } else {
        e.age ++;
      }
    }

    if (random(1) < foodSpawnRate) foods.add(new Food(random(r+r/2, width-(r+r/2)), random(r+r/2, height-(r+r/2))));    // plants will randomly spawn over time
  }

  void eaterSelection() {                             //function to prepare the mating pool for reproduction
    eatersMatingPool.clear();                         //first we clear the old mating pool
    float maxFitness = getEatersMaxFitness();         //we determine who's the best creature of the generation

    for (int i = 0; i < eatersClones.size(); i++) {                               //for every dead creature
      float fitnessNormal = map(eatersClones.get(i).age, 0, maxFitness, 0, 1);    //we normalize the fitness score between 0 and 1
      int n = (int) (fitnessNormal*100);                                          //multiply it by 100
      for (int j = 0; j < n; j++) {                                               //and add the clone to the pool as many times as it deserves.
        eatersMatingPool.add(eatersClones.get(i));                                //the better you are, the more chances you get to reproduce
      }
    }
  }

  void eatersReproduction() {                        //function to select parents and cross brain data
    float tempWeight;

    for (int i = 0; i < numE; i++) {                 
      eaters.add(new Eater(random(0, width), random(0, height), new NeuralNetwork())); //first we generate a random eater

      int m = int(random(eatersMatingPool.size()));          //choose two random parents from the mating pool
      int d = int(random(eatersMatingPool.size()));

      Eater mom = eatersMatingPool.get(m);                   //get them
      Eater dad = eatersMatingPool.get(d);

      for (int k = 0; k < mom.NN.getLayerCount(); k++) {     //for every layer of the brain...
        float[] momWeights = new float[0];                   //create 3 arrays to store weights and biases of the family
        float[] dadWeights = new float[0];
        float[] childWeights = new float[0];

        momWeights = mom.NN.layers[k].getWeigths();          //get mom and dad weights and biases
        dadWeights = dad.NN.layers[k].getWeigths();

        for (int j = 0; j < momWeights.length; j++) {                      //for every weights of the layer
          if (random(1) > 0.5)  tempWeight = momWeights[j];                //choose random between mom and dad
          else                  tempWeight = dadWeights[j];
          if (random(1) < mutationRate) tempWeight += random(-0.1, 0.1);   //apply chance of mutation
          tempWeight = constrain(tempWeight, -1, 1);                       //clamp new weight
          childWeights = (float[]) append(childWeights, tempWeight);       //add weight to the child weights and bias array
        }
        Eater e = eaters.get(i);                                           //take the random eater we've created at the start
        e.NN.layers[k].setWeights(childWeights);                           //set its brain with the new weights
      }
    }
    eatersClones.clear();                                                  //clear the clones array for the next generation
  }

  float getEatersMaxFitness() {                             //function to get the highest fitness score of the generation
    float record = 0;
    for (int i = 0; i < eatersClones.size(); i++) {
      if (eatersClones.get(i).age > record) {
        record = eatersClones.get(i).age;
      }
    }
    return record;
  }

  Eater getBestEater() {                                    //function to clone the best eater and store it across generations (actually unused)
    Eater bestEater = new Eater(0, 0, new NeuralNetwork());
    for (int i = 0; i < eatersClones.size(); i++) {
      if (eatersClones.get(i).age > bestEater.age) {
        bestEater = eatersClones.get(i);
      }
    }
    return bestEater;
  }

  ArrayList getFood() {
    return foods;
  }

  ArrayList getEater() {
    return eaters;
  }
} 

m4l4

ok, that should be it, with that configuration it takes something between 15-20 generations to see a coherent behavior, my record is 40k age on gen 19. Eaters seems to have some problems with the borders() function (limit their position to screen size), so i'm currently using wrapAround() (continuos screen) and working on a different set of inputs for the creatures. i'm working on adding sight, smell, hearing and whatsoever, (here's my line of sight project https://forum.processing.org/two/discussion/24691/solved-collision-and-obstacles-detection#latest) i'll be very interested in your opinions/suggestions. also if someone has a better understanding of neural networks (i'm a complete noob so if you have some insight about it, you're probably better than i am) i'd like to hear your opinions about my approach to the problem and the possible future developments (recurrent neural nets, backpropagation, fitness function etc.)

Chrisir

Could you try and upload your sketch to https://github.com ?

m4l4

i've never used github myself, i've got no account there

Chrisir

It was new to me too but it's a professional tool and makes downloading your sketch much easier than here.

m4l4

i hope it works https://github.com/m4l4/Neuroevolution-Simulation

Chrisir

It works! Well done!

jeremydouglass

Thank you for sharing this -- very interesting!

randomdude

Thank you so much for this code, it is extremely enlightening. I will go through it thoroughly these days, I’m sure it will make the process of making my own neural network much easier. Also expect a lot of questions from me in the coming days. To start: I haven’t examined your code, but you have used only a genetic algorithm to train the network right? No backpropagation?

randomdude

I have given your code a quick initial scan and I have another question: why did you use a tanh activation function? Why not just use a sigmoid one? I am not one to know for sure, but I don’t think your network requires enough gradient strength to require tanh over sigmoid.

randomdude

In line 32 of your Layer tab, is there a reason you're using expand() instead of append()? Why don't you use arraylists for everything data structure that changes, instead of arrays? It would save you a lot of hassle.