three dimensional neural network

dulce_a

I'm trying to take "The Nature of Code" chapter 10 example 4 and make it three dimensional, but I am having the hardest time! Does anyone have any suggestions as to how to make it easier or has anyone done it already?

TfGuy44

Start by posting the code you have so far. I don't have that book, so I'm at a complete loss as to where you're at and what help you need.

kn1c

like this example? openprocessing.org/sketch/11193

Chrisir

The book

book is online,

see

http://natureofcode.com/book/chapter-10-neural-networks/

look for Example 10.4 on that page

The idea

The code on openprocessing looks very good.

The idea of a neuronal network is a learning entity (e.g. recognizing hand writing). Much is said in the books chapter. I don't think that in terms of learning going 3D gains us much. It looks cool in 3D of course, when you display it. But for the learning I don't think it will bring us much...

The class Neuron

the code given in the example 10.4 is obviously building on the code in the previous examples

e.g. on the class Neuron

class Neuron {
  PVector location;

  Neuron(float x, float y) {
    location = new PVector(x, y);
  }

  void display() {
    stroke(0);
    fill(0);
    ellipse(location.x, location.y, 16, 16);
  }
}

Good for us: PVector can be 2D, but also 3D, so we can stick with PVector when going 3D now.

When you think of it in 3D, you need all three components x,y,z (and not only x,y).

Neuron(float x, float y, float z) {
    location = new PVector(x, y, z);
  }

Also ellipse doesn't work in 3D

so replace it with (box or sphere)

pushMatrix();
translate(x,y,z);
fill(244);
box(10);
popMatrix();

or so...

Other adjustements

for the other stuff also do similar adjustements

e.g. line (in display()) has a version in 3D (unlike ellipse!)

so this is easy:

from

line(a.location.x, a.location.y, b.location.x, b.location.y);

to (just use z as well)

line(a.location.x, a.location.y, a.location.z, b.location.x, b.location.y, b.location.z);

The function update()

also

  void update() {
    if (sending) {
      sender.x = lerp(sender.x, b.location.x, 0.1);
      sender.y = lerp(sender.y, b.location.y, 0.1);
      float d = PVector.dist(sender, b.location);
      if (d < 1) {
        b.feedforward(output);
        sending = false;
      }
    }
  }

becomes

  void update() {
    if (sending) {

      sender.x = lerp(sender.x, b.location.x, 0.1);
      sender.y = lerp(sender.y, b.location.y, 0.1);
      sender.z = lerp(sender.z, b.location.z, 0.1);

      float d = PVector.dist(sender, b.location);
      if (d < 1) {
        b.feedforward(output);
        sending = false;
      }
    }
  }

since PVector can just be 2D or 3D we don't need to change the occurance of sender or b

just when defining sender and b, think of defining z for both of them as well...

Greetings, Chrisir

PhiLho

Nothing to do with "Processing: installation, usage of the PDE", please read the To newcomers in this forum: read attentively these instructions. Thanks. (Moved.)

dulce_a

Thank you so much Chrisir!!!!! I'll work on it today and give an update later :)

Chrisir

I edited my post

please look at the openprocessing thing too, you can even rotate it with the mouse.

dulce_a

Hey Chrisir, so I made all the necessary changes, but an error keeps coming up saying that Component.x is not visible in my connections tab specifically under the translate in the display function. Any suggestions?

  void update() {
    if (sending) {
      // Use a simple interpolation
      sender.x = lerp(sender.x, b.location.x, 0.1);
      sender.y = lerp(sender.y, b.location.y, 0.1);
      sender.z = lerp(sender.z, b.location.z, 0.1);
      float d = PVector.dist(sender, b.location);
      // If we've reached the end
      if (d < 1) {
        // Pass along the output!
        b.feedforward(output);
        sending = false;
      }
    }
  }

  // Draw line and traveling circle
  void display() {
    stroke(0);
    strokeWeight(1+weight*4);
    line(a.location.x, a.location.y, a.location.z,b.location.x, b.location.y, b.location.z);

    if (sending) {
      pushMatrix();
      translate(x,y,z);
      fill(0);
      sphere(10);
      popMatrix();
      //strokeWeight(1);
      //ellipse(sender.x, sender.y, 16, 16);
    }
  }
}

Chrisir

my bad

I guess it's line 25

translate (sender.x, sender.y, sender.z);

since you're replacing

 //ellipse(sender.x, sender.y, 16, 16);

dulce_a

Thank you again Chrisir! I've gotten it to be three dimensional and I am beyond stoked! Your help has been great! THANK YOU! :)

dulce_a

One last question, Chrisir. Would adding neurons via mouse interaction be asking too much? I'm trying to set it up as follows:

void mousePressed(){
  if(mouseButton == RIGHT){
    Neuron.addSelf(new Neuron(x,y,z));
    network.connect(x,y,z);
    network.addNeuron(i);
  }
}

any suggestions? It doesn't seem to want to recognize the addSelf. I've imported the toxiclibs geometry but maybe I'm doing something wrong?

Thanks again!

Chrisir

hm...

as far as I understand the code in the chapter, we see

make a neuron

Neuron d = new Neuron(200,0); // or in 3D (200,0,-100)

Add the Neurons to the network.

network.addNeuron(d);

also a connection is made for that neuron

when you mouse click, you have to do the same stuff:

Neuron d = new Neuron(mouseX,mouseY,-100);  
  network.addNeuron(d);

do some connections

does that help ?

Chrisir

do you know about peasy cam?

it's easy and can be a great help to view your result from different angles. -

dulce_a

Yeah, I have Peasy Cam set up already, I'm overlaying this network on contour lines of Mexico City. I'll give the mouse thing a try after my next class. Thank you!

dulce_a

I'm struggling over here. So I'm trying to add the mouse interaction, but I can't seem to understand the best step to take. Here's the code:

import peasy.*;
import blobDetection.*;
import toxi.geom.*;


Network network;
PeasyCam dcam;
PImage img;

float levels = 45;
float factor = 1;
float elevation = 45;
float colorStart = 100;
float colorRange = 200;

BlobDetection[] theBlobDetection = new BlobDetection[int(levels)];

void setup() {
  size(800, 800, P3D); 
  dcam = new PeasyCam(this,100);
  colorMode(HSB, 360, 50, 50);
  img = loadImage("C:\\Users\\Dulce\\Desktop\\Studio\\A2\\mcity.gif"); 

  //COMPUTING BLOBS WITH DIFFERENT THRESHOLDS
  for (int i = 0; i<levels; i++) {
    theBlobDetection[i] = new BlobDetection(img.width, img.height);
    theBlobDetection[i].setThreshold(i/levels);
    theBlobDetection[i].computeBlobs(img.pixels);
  }

  // Create the Network object
  network = new Network(0, 0, 18);

  // Create a bunch of Neurons
  Neuron a = new Neuron(-2, 0, 1);
  Neuron b = new Neuron(2, 0, 1);
  Neuron c = new Neuron(0, 2, 1);
  Neuron d = new Neuron(mousePressed); 
  Neuron e = new Neuron(15, 0, -5);
  Neuron f = new Neuron(25, 0, -5);
  Neuron g = new Neuron(40, 5, -5);
  Neuron h = new Neuron(60, 3, 10);

  // Connect them
  network.connect(a, b,1);
  network.connect(b, c,random(1));
  network.connect(b, d,random(1));
  network.connect(c, e,random(1));
  network.connect(d, e,random(1));
  network.connect(e, f,random(1));
  network.connect(f, h, random(1));
  network.connect(g, h, 1);

  // Add them to the Network
  network.addNeuron(a);
  network.addNeuron(b);
  network.addNeuron(c);
  network.addNeuron(d);
  network.addNeuron(e);
  network.addNeuron(f);
  network.addNeuron(g);
  network.addNeuron(h);
}

void draw() {
  background(255);
  // Update and display the Network
  network.update();
  network.display();
  mousePressed();

  // Every 30 frames feed in an input
  if (frameCount % 30 == 0) {
    network.feedforward(random(1));
  }

   stroke(0);
  strokeWeight(1);
  noFill();

  translate(-img.width*factor/2, -img.height*factor/2);

  for (int i = 0; i < levels; i++) {
    translate(0, 0, elevation/levels);
    drawContours(i);
  }
}

void mousePressed(){
if(mouseButton == RIGHT){
   Neuron.addSelf(new Neuron(mouseX, mouseY,0));
  }
}


void drawContours(int i){
  Blob b;
  EdgeVertex eA, eB;
  for(int n = 0; n<theBlobDetection[i].getBlobNb(); n++){
    b = theBlobDetection[i].getBlob(n);
    if(b!=null){
      stroke((i/levels*colorRange)+colorStart,50,50);

      for(int m = 0; m<b.getEdgeNb(); m++){
        eA = b.getEdgeVertexA(m);
        eB = b.getEdgeVertexB(m);
        if(eA != null && eB != null)
        line(
          eA.x*img.width*factor, eA.y*img.height*factor, 
          eB.x*img.width*factor, eB.y*img.height*factor
            );
      }
    }
  }
}

Chrisir

line 38 won't work

delete line 70

replace line 91

with

// Create new neuron
Neuron d = new Neuron(mouseX,mouseY,-100); 
network.connect(b, d,random(1));
// Add it to the Network
network.addNeuron(d);

edited - you need to define b - see post below

the process is the same as in setup() but only for one node

Later

later you need an idea for a better z-pos (instead of -100) - e.g. have a zPos stored that you change with pg up/pg dn (and that is displayed as a kind of 3D cursor in space)

also now we always connect to node b. You'll need an idea how to replace b.

E.g.

• connect to the last node that has been clicked before the current right click or

• connect to the nearest node or to the third closest node or so.

Greetings, Chrisir

Chrisir

for anyone who is interested

EDITED ++++++++++++++++++++++

             new program version 

this version uses no libs, no images

but comes with a 3D cursor

positioning left/right and top/down

• use mouse

• use wasd keys

positioning depth

• mouse wheel

• use mouse x-pos while dragging (press and hold left mouse and move mouse left and right)

• use r and f keys

• use + and - keys

the new nodes are connected to two random nodes

/*

 this version uses no libs, no images

 but comes with a 3D cursor

 positioning left/right and top/down :

 * use mouse

 * use wasd keys

 positioning depth : 

 * mouse wheel

 * use mouse x-pos while dragging (press and hold left mouse and move mouse left and right)

 * use r and f keys

 * use + and - keys

 the new nodes are connected to two random nodes


 */

// import peasy.*;
// import blobDetection.*;
// import toxi.geom.*;


Network network;

// PeasyCam dcam;

PImage img;

boolean crosshair = true;

float levels = 45;
float factor = 1;
float elevation = 45;
float colorStart = 100;
float colorRange = 200;

float theX, theY, theZ; 

//BlobDetection[] theBlobDetection = new BlobDetection[int(levels)];

void setup() {
  size(800, 800, P3D);

  smooth(); 
  //dcam = new PeasyCam(this, 100);
  colorMode(HSB, 360, 50, 50);
  // img = loadImage("C:\\Users\\Dulce\\Desktop\\Studio\\A2\\mcity.gif"); 

  //COMPUTING BLOBS WITH DIFFERENT THRESHOLDS
  for (int i = 0; i<levels; i++) {
    //    theBlobDetection[i] = new BlobDetection(img.width, img.height);
    //    theBlobDetection[i].setThreshold(i/levels);
    //    theBlobDetection[i].computeBlobs(img.pixels);
  }

  // Create the Network object
  network = new Network(0, 0, 18);

  // Create a bunch of Neurons
  Neuron a = new Neuron(-2, 0, 1);
  Neuron b = new Neuron(2, 0, 1);
  Neuron c = new Neuron(0, 12, 1);
  Neuron d = new Neuron(2, -13, 1); 
  Neuron e = new Neuron(-15, 0, -5);
  Neuron f = new Neuron(25, 0, -5);
  Neuron g = new Neuron(40, 15, -5);
  Neuron h = new Neuron(60, 61, -110);

  // Connect them
  network.connect(a, b, 1);
  network.connect(b, c, random(1));
  network.connect(b, d, random(1));
  network.connect(c, e, random(1));
  network.connect(d, e, random(1));
  network.connect(e, f, random(1));
  network.connect(f, h, random(1));
  network.connect(g, h, 1);

  // Add them to the Network
  network.addNeuron(a);
  network.addNeuron(b);
  network.addNeuron(c);
  network.addNeuron(d);
  network.addNeuron(e);
  network.addNeuron(f);
  network.addNeuron(g);
  network.addNeuron(h);
  //
  theX=mouseX;
  theY=mouseY;
  theZ=-50;
  //
}

void draw() {
  background(255);

  lights();

  //camera 
  camera(width/2.0, height/2.0, 210, 
  width/2.0, height/2.0, -1000, 
  0, 1, 0); 

  // Update and display the Network
  //   network.update();
  translate(width/2, height/2);
  network.display();

  show3DCursor(); 

  // Every 30 frames feed in an input
  if (frameCount % 30 == 0) {
    //  network.feedforward(random(1));
  }

  stroke(0);
  strokeWeight(1);
  noFill();

  //  translate(-img.width*factor/2, -img.height*factor/2);
  // translate(width/2, height/2);
  //  for (int i = 0; i < levels; i++) {
  //    //translate(width/2, height/2);
  //    translate(0, 0, elevation/levels);
  //    drawContours(i);
  //  }

  // display some text as 2D
  showTextInHUD();
} // func draw() 

// -----------------------------------------------------------------
// mouse 

void mousePressed() {
  if (mouseButton == RIGHT) {
    // create new neuron
    createNewNeuron();
  }
  else if (mouseButton == LEFT) {
    //
    // do nothing
  }
  else {
    //  should not be reached
  }
  //
} // func 

void mouseMoved() {
  if (!keyPressed) {
    theX = map (mouseX, 0, width, -130, 130);
    theY = map (mouseY, 0, height, -130, 130);
    // theZ not changed here
  }
  else 
  {
    //    colorMode(RGB, 255);
    //    color clickedAt = get(mouseX, mouseY);
    //    if (red(clickedAt) > 240) {
    //      println ("red");
    //      theZ-=4;
    //    }
    //    else if (green(clickedAt) > 240) {
    //      println ("green");
    //      theZ+=4;
    //    } // else if 
    //    colorMode(HSB, 360, 50, 50);
  } // else
} // func 

void mouseDragged() {
  //theZ
  theZ= map (mouseX, width, 0, -130, 130);
  theY = map (mouseY, 0, height, -130, 130);
}

void mouseWheel(MouseEvent event) {
  float e = event.getAmount();
  // println(e);
  theZ += e*4;
}

//--------------------------------------------------------------
// keyboard 

void keyPressed() {
  // this implements wasd + rf steering for our 3D cursor (the sphere)
  switch (key) {
  case 'w':
    theY--;
    break;
  case 's':
    theY++;
    break;
  case 'a':
    theX--;
    break;
  case 'd':
    theX++;
    break;
    // f and plus
  case 'f':
  case '+': 
    theZ++;
    break;
    // r and minus 
  case 'r':    
  case '-': 
    theZ--;
    break;    
    // ENTER 
  case ENTER:
  case RETURN:
    // create new neuron
    createNewNeuron();
    break;
  case ' ':
    crosshair = !crosshair;
    break;
  default:
    // do nothing
    break;
  } // switch
} // func 

//--------------------------------------------------------------
// other

void show3DCursor() {

  // show 3D cursor 

    if (crosshair) {
    // 3D crosshair 
    final int len = 15; 
    colorMode(RGB, 255); // standard for color 

    stroke(244, 2, 2); // red   
    line (theX, theY, theZ-len, theX, theY, theZ+len);
    stroke(0, 0, 255);
    line (theX, theY-len, theZ, theX, theY+len, theZ);
    stroke(2, 244, 2);
    line (theX-len, theY, theZ, theX+len, theY, theZ);

    pushMatrix();
    translate(theX, theY, theZ+len);
    fill(244, 2, 2);
    noStroke();
    sphere(1);
    popMatrix();

    colorMode(HSB, 360, 50, 50);
  }
  else 
  {
    // sphere 
    //colorMode(RGB, 255); // standard for color 
    pushMatrix();
    translate(theX, theY, theZ);
    fill(244, 2, 2);
    fill(255);
    noStroke();
    sphere(10);
    //  translate( +5, -10, -20 );
    //  fill(244, 2, 2);
    //  sphere(4);
    //  translate( 0, 0, 40 );
    //  fill(2, 244, 2);
    //  sphere(4);
    popMatrix();
    // colorMode(HSB, 360, 50, 50); //
  } // else
} // func 

void showTextInHUD() {
  // A small 2D HUD for text
  camera();
  hint(DISABLE_DEPTH_TEST);
  noLights();
  textMode(MODEL);
  fill(0);
  text("Use mouse and mouse wheel (or wasd and rf or +-) for"+
    " a 3D cursor (the crosshair) - right click to place node", 20, 20);
  String text1 = "xyz:"+int(theX)+","+int(theY)+","+int(theZ);  
  text (text1, width-130, 20);
} // func 

void createNewNeuron() {
  // Create new neuron
  Neuron oldNeuron1;
  oldNeuron1 = network.neurons.get(3);
  oldNeuron1 = network.neurons.get( int( random(network.neurons.size()) ));

  Neuron oldNeuron2;
  oldNeuron2 = network.neurons.get( int( random(network.neurons.size()) ));

  Neuron newNeuron = new Neuron(theX, theY, theZ);

  network.connect(oldNeuron1, newNeuron, random(1));
  network.connect(newNeuron, oldNeuron2, random(1));
  // Add it to the Network
  network.addNeuron(newNeuron);
}

void drawContours(int i) {
  //  Blob b;
  //  EdgeVertex eA, eB;
  //  for (int n = 0; n<theBlobDetection[i].getBlobNb(); n++) {
  //    b = theBlobDetection[i].getBlob(n);
  //    if (b!=null) {
  //      stroke((i/levels*colorRange)+colorStart, 50, 50);
  //
  //      for (int m = 0; m<b.getEdgeNb(); m++) {
  //        eA = b.getEdgeVertexA(m);
  //        eB = b.getEdgeVertexB(m);
  //        if (eA != null && eB != null)
  //          line(
  //          eA.x*img.width*factor, eA.y*img.height*factor, 
  //          eB.x*img.width*factor, eB.y*img.height*factor
  //            );
  //      }
  //    }
  //  }
}

// ====================================================

class Network {

  // A Network is a list of neurons.
  ArrayList<Neuron> neurons;
  PVector location;

  Network(float x, float y, float z) {
    location = new PVector(x, y, z);
    neurons = new ArrayList<Neuron>();
  }

  // We can add an neuron to the network.
  void addNeuron(Neuron n) {
    neurons.add(n);
  }

  // connect it
  void connect(Neuron a, Neuron b, float weight1) {
    Connection c = new Connection(a, b, weight1);
    a.addConnection(c);
  }

  //We can draw the entire network.
  void display() {
    pushMatrix();
    translate(location.x, location.y);
    for (Neuron n : neurons) {
      n.display();
    }
    popMatrix();
  }

  // The Connection is active with data traveling from a to b.

  //  void feedforward(float val) {
  //    output = val*weight;
  //    sender = a.location.get();
  //    sending = true;
  //  }

  //  void update() {
  //    if (sending) {
  //      sender.x = lerp(sender.x, b.location.x, 0.1);
  //      sender.y = lerp(sender.y, b.location.y, 0.1);
  //
  //
  //      //How far are we from neuron b?
  //
  //      float d = PVector.dist(sender, b.location);
  //
  //
  //      //If we’re close enough (within one pixel) pass on the output. Turn off sending.
  //
  //      if (d < 1) {
  //        b.feedforward(output);
  //        sending = false;
  //      }
  //    }
  //  }
} // class

// ===============================================================

class Neuron {
  // pos
  PVector location;

  // The neuron stores its connections.
  ArrayList<Connection> connections;

  Neuron(float x, float y, float z) {
    location = new PVector(x, y, z);
    connections=new ArrayList();
  }

  void display() {
    stroke(0);
    fill(0);
    // instead of 
    // ellipse(location.x, location.y, 16, 16);
    // we use 
    pushMatrix();
    translate(location.x, location.y, location.z);
    fill(244);
    box(10);
    // sphere (10);
    popMatrix();
    for (Connection c : connections) {
      c.display();
    }
  }

  void addConnection(Connection c) {
    connections.add(c);
  }
}

// ======================================================

class Connection {

  // A connection is between two neurons.
  Neuron a;
  Neuron b;

  // A connection has a weight.
  float weight;

  boolean sending = false;
  PVector sender;
  float output;

  Connection(Neuron from, Neuron to, float w) {
    weight = w;
    a = from;
    b = to;
  }

  // A connection is drawn as a line.
  void display() {
    stroke(0);
    strokeWeight(weight*4);
    line(a.location.x, a.location.y, a.location.z, 
    b.location.x, b.location.y, b.location.z);
    strokeWeight(1);
  }

  // The Connection is active with data traveling from a to b.
  void feedforward(float val) {
    output = val*weight;
    sender = a.location.get();
    sending = true;
  }
} // class
//

dulce_a

Thank you Chrisir!

One final question/thought. Is it possible to make the neurons add themselves autonomously based on some sort of summation? I mean I figure there is a way, I guess I don't understand the math too well...

Chrisir

here...

/*

 this version uses no libs, no images

 but comes with a 3D cursor

 positioning left/right and top/down :

 * use mouse

 * use wasd keys

 positioning depth : 

 * mouse wheel

 * use mouse x-pos while dragging (press and hold left mouse and move mouse left and right)

 * use r and f keys

 * use + and - keys

 the new nodes are connected to two random nodes


 */

// import peasy.*;
// import blobDetection.*;
// import toxi.geom.*;


Network network;

// PeasyCam dcam;

//PImage img;

boolean crosshair = true;
boolean auto = true; 

float levels = 45;
float factor = 1;
float elevation = 45;
float colorStart = 100;
float colorRange = 200;

float theX, theY, theZ; 

//BlobDetection[] theBlobDetection = new BlobDetection[int(levels)];

void setup() {
  size(800, 800, P3D);

  smooth(); 
  //dcam = new PeasyCam(this, 100);
  //  colorMode(HSB, 360, 50, 50);
  // img = loadImage("C:\\Users\\Dulce\\Desktop\\Studio\\A2\\mcity.gif"); 

  //COMPUTING BLOBS WITH DIFFERENT THRESHOLDS
  // for (int i = 0; i<levels; i++) {
  //    theBlobDetection[i] = new BlobDetection(img.width, img.height);
  //    theBlobDetection[i].setThreshold(i/levels);
  //    theBlobDetection[i].computeBlobs(img.pixels);
  // }

  // Create the Network object
  network = new Network(0, 0, 18);

  // Create a bunch of Neurons
  Neuron a = new Neuron(-2, 0, 1);
  Neuron b = new Neuron(2, 0, 1);
  Neuron c = new Neuron(0, 12, 1);
  Neuron d = new Neuron(2, -13, 1); 
  Neuron e = new Neuron(-15, 0, -5);
  Neuron f = new Neuron(25, 0, -5);
  Neuron g = new Neuron(40, 15, -5);
  Neuron h = new Neuron(60, 61, -110);

  // Connect them
  network.connect(a, b, 1);
  network.connect(b, c, random(1));
  network.connect(b, d, random(1));
  network.connect(c, e, random(1));
  network.connect(d, e, random(1));
  network.connect(e, f, random(1));
  network.connect(f, h, random(1));
  network.connect(g, h, 1);

  // Add them to the Network
  network.addNeuron(a);
  network.addNeuron(b);
  network.addNeuron(c);
  network.addNeuron(d);
  network.addNeuron(e);
  network.addNeuron(f);
  network.addNeuron(g);
  network.addNeuron(h);
  //
  theX=mouseX;
  theY=mouseY;
  theZ=-50;
  //
}

void draw() {
  background(255);

  lights();

  //camera 
  camera(width/2.0, height/2.0, 210, 
  width/2.0, height/2.0, -1000, 
  0, 1, 0); 

  // Update and display the Network
  //   network.update();
  translate(width/2, height/2);
  network.display();

  show3DCursor(); 

  // Every 30 frames feed in an input
  // if (frameCount % 30 == 0) {
  //  network.feedforward(random(1));
  // }

  if (auto && (random(1000)>870)) {

    theX = random(-130, 130);
    theY = random(-130, 130);
    theZ = random(-320, 0);

    createNewNeuron();
  }

  stroke(0);
  strokeWeight(1);
  noFill();

  //  translate(-img.width*factor/2, -img.height*factor/2);
  // translate(width/2, height/2);
  //  for (int i = 0; i < levels; i++) {
  //    //translate(width/2, height/2);
  //    translate(0, 0, elevation/levels);
  //    drawContours(i);
  //  }

  // display some text as 2D
  showTextInHUD();
} // func draw() 

// -----------------------------------------------------------------
// mouse 

void mousePressed() {
  if (mouseButton == RIGHT) {
    // create new neuron
    createNewNeuron();
  }
  else if (mouseButton == LEFT) {
    //
    // do nothing
  }
  else {
    //  should not be reached
  }
  //
} // func 

void mouseMoved() {
  if (!keyPressed) {
    theX = map (mouseX, 0, width, -130, 130);
    theY = map (mouseY, 0, height, -130, 130);
    // theZ not changed here
  }
  else
  {
    //    colorMode(RGB, 255);
    //    color clickedAt = get(mouseX, mouseY);
    //    if (red(clickedAt) > 240) {
    //      println ("red");
    //      theZ-=4;
    //    }
    //    else if (green(clickedAt) > 240) {
    //      println ("green");
    //      theZ+=4;
    //    } // else if 
    //    colorMode(HSB, 360, 50, 50);
  } // else
} // func 

void mouseDragged() {
  //theZ
  theZ = map (mouseX, width, 0, -130, 130);
  theY = map (mouseY, 0, height, -130, 130);
}

void mouseWheel(MouseEvent event) {
  float e = event.getAmount();
  // println(e);
  theZ += e*4;
}

//--------------------------------------------------------------
// keyboard 

void keyPressed() {
  // this implements wasd + rf steering for our 3D cursor (the sphere)
  switch (key) {
  case 'w':
    theY--;
    break;
  case 's':
    theY++;
    break;
  case 'a':
    theX--;
    break;
  case 'd':
    theX++;
    break;
    // f and plus
  case 'f':
  case '+': 
    theZ++;
    break;
    // r and minus 
  case 'r':    
  case '-': 
    theZ--;
    break;    
    // ENTER 
  case ENTER:
  case RETURN:
    // create new neuron
    createNewNeuron();
    break;
  case ' ':
    crosshair = !crosshair;
    break;
  default:
    // do nothing
    break;
  } // switch
} // func 

//--------------------------------------------------------------
// other

void show3DCursor() {

  // show 3D cursor 
  //
  if (crosshair) {
    // 3D crosshair 
    final int len = 15; 
    //colorMode(RGB, 255); // standard for color 

    stroke(244, 2, 2); // red   
    line (theX, theY, theZ-len, theX, theY, theZ+len);
    stroke(0, 0, 255);
    line (theX, theY-len, theZ, theX, theY+len, theZ);
    stroke(2, 244, 2);
    line (theX-len, theY, theZ, theX+len, theY, theZ);

    pushMatrix();
    translate(theX, theY, theZ+len);
    fill(244, 2, 2);
    noStroke();
    sphere(1);
    popMatrix();

    //colorMode(HSB, 360, 50, 50);
  }
  else
  {
    // sphere 
    //colorMode(RGB, 255); // standard for color 
    pushMatrix();
    translate(theX, theY, theZ);
    fill(244, 2, 2);
    fill(255);
    noStroke();
    sphere(10);
    //  translate( +5, -10, -20 );
    //  fill(244, 2, 2);
    //  sphere(4);
    //  translate( 0, 0, 40 );
    //  fill(2, 244, 2);
    //  sphere(4);
    popMatrix();
    // colorMode(HSB, 360, 50, 50); //
  } // else
} // func 

void showTextInHUD() {
  // A small 2D HUD for text
  camera();
  hint(DISABLE_DEPTH_TEST);
  noLights();
  textMode(MODEL);
  fill(0);
  text("Use mouse and mouse wheel (or wasd and rf or +-) for"+
    " a 3D cursor (the crosshair) - right click to place node", 20, 20);
  String text1 = "xyz:"+int(theX)+","+int(theY)+","+int(theZ);  
  text (text1, width-130, 20);
} // func 

void createNewNeuron() {
  // Create new neuron
  Neuron oldNeuron1;
  oldNeuron1 = network.neurons.get(3);
  oldNeuron1 = network.neurons.get( int( random(network.neurons.size()) ));

  Neuron oldNeuron2;
  oldNeuron2 = network.neurons.get( int( random(network.neurons.size()) ));

  Neuron newNeuron = new Neuron(theX, theY, theZ);

  network.connect(oldNeuron1, newNeuron, random(1));
  network.connect(newNeuron, oldNeuron2, random(1));
  // Add it to the Network
  network.addNeuron(newNeuron);
}

void drawContours(int i) {
  //  Blob b;
  //  EdgeVertex eA, eB;
  //  for (int n = 0; n<theBlobDetection[i].getBlobNb(); n++) {
  //    b = theBlobDetection[i].getBlob(n);
  //    if (b!=null) {
  //      stroke((i/levels*colorRange)+colorStart, 50, 50);
  //
  //      for (int m = 0; m<b.getEdgeNb(); m++) {
  //        eA = b.getEdgeVertexA(m);
  //        eB = b.getEdgeVertexB(m);
  //        if (eA != null && eB != null)
  //          line(
  //          eA.x*img.width*factor, eA.y*img.height*factor, 
  //          eB.x*img.width*factor, eB.y*img.height*factor
  //            );
  //      }
  //    }
  //  }
}

// ====================================================

class Network {

  // A Network is a list of neurons.
  ArrayList<Neuron> neurons;
  PVector location;

  Network(float x, float y, float z) {
    location = new PVector(x, y, z);
    neurons = new ArrayList<Neuron>();
  }

  // We can add an neuron to the network.
  void addNeuron(Neuron n) {
    neurons.add(n);
  }

  // connect it
  void connect(Neuron a, Neuron b, float weight1) {
    Connection c = new Connection(a, b, weight1);
    a.addConnection(c);
  }

  //We can draw the entire network.
  void display() {
    //    pushMatrix();
    //  translate(location.x, location.y);
    for (Neuron n : neurons) {
      n.display();
    }
    // popMatrix();
  }

  // The Connection is active with data traveling from a to b.

  //  void feedforward(float val) {
  //    output = val*weight;
  //    sender = a.location.get();
  //    sending = true;
  //  }

  //  void update() {
  //    if (sending) {
  //      sender.x = lerp(sender.x, b.location.x, 0.1);
  //      sender.y = lerp(sender.y, b.location.y, 0.1);
  //
  //
  //      //How far are we from neuron b?
  //
  //      float d = PVector.dist(sender, b.location);
  //
  //
  //      //If we’re close enough (within one pixel) pass on the output. Turn off sending.
  //
  //      if (d < 1) {
  //        b.feedforward(output);
  //        sending = false;
  //      }
  //    }
  //  }
} // class

// ===============================================================

class Neuron {
  // pos
  PVector location;
  color col = color(random(255), random(255), random(255));

  // The neuron stores its connections.
  ArrayList<Connection> connections;

  Neuron(float x, float y, float z) {
    location = new PVector(x, y, z);
    connections=new ArrayList();
  }

  void display() {
    //    stroke(0);
    //    fill(0);

    for (Connection c : connections) {
      c.display();
    }

    // instead of 
    // ellipse(location.x, location.y, 16, 16);
    // we use 


    pushMatrix();
    translate(location.x, location.y, location.z);
    fill(col);
    ///noStroke();
    stroke(111);
    box(10);
    // sphere (10);
    popMatrix();
    //
  }

  void addConnection(Connection c) {
    connections.add(c);
  }
}

// ======================================================

class Connection {

  // A connection is between two neurons.
  Neuron a;
  Neuron b;

  // A connection has a weight.
  float weight;

  boolean sending = false;
  PVector sender;
  float output;

  Connection(Neuron from, Neuron to, float w) {
    weight = w;
    a = from;
    b = to;
  }

  // A connection is drawn as a line.
  void display() {
    stroke(244, 2, 2);
    strokeWeight(weight);
    line(a.location.x, a.location.y, a.location.z, 
    b.location.x, b.location.y, b.location.z);
    // weight, color (244, 0, 0));
    strokeWeight(1);
  }

  void drawLine(float x1, float y1, float z1, 
  float x2, float y2, float z2, 
  float weight, color strokeColour)
    // drawLine was programmed by James Carruthers
    // see http://processing.org/discourse/yabb2/YaBB.pl?num=1262458611/0#9
  {
    PVector p1 = new PVector(x1, y1, z1);
    PVector p2 = new PVector(x2, y2, z2);
    PVector v1 = new PVector(x2-x1, y2-y1, z2-z1);
    float rho = sqrt(pow(v1.x, 2)+pow(v1.y, 2)+pow(v1.z, 2));
    float phi = acos(v1.z/rho);
    float the = atan2(v1.y, v1.x);
    v1.mult(0.5);
    pushMatrix();
    translate(x1, y1, z1);
    translate(v1.x, v1.y, v1.z);
    rotateZ(the);
    rotateY(phi);
    noStroke();
    fill(strokeColour);
    box(weight, weight, p1.dist(p2)*1.2);
    popMatrix();
  }

  // The Connection is active with data traveling from a to b.
  void feedforward(float val) {
    output = val*weight;
    sender = a.location.get();
    sending = true;
  }
} // class
//

dulce_a

Hey Chrisir, so I took your code and put it with my stuff, the only problem is that when I click the mouse, the node gets placed very far away from where the mouse is actually clicked. Any idea how that can be fixed?

import peasy.*;
import blobDetection.*;
import toxi.geom.*;


Network network;
PeasyCam dcam;
PImage img;

float levels = 45;
float factor = 1;
float elevation = 45;
float colorStart = 10;
float colorRange = 150;
float theX, theY, theZ;

BlobDetection[] theBlobDetection = new BlobDetection[int(levels)];

void setup() {
  size(1000, 1000, P3D); 
  smooth();
  dcam = new PeasyCam(this, 200);
  colorMode(RGB, 180);
  img = loadImage("C:\\Users\\Dulce\\Desktop\\Studio\\A2\\mcity.gif"); 

  //COMPUTING BLOBS WITH DIFFERENT THRESHOLDS
  for (int i = 0; i<levels; i++) {
    theBlobDetection[i] = new BlobDetection(img.width, img.height);
    theBlobDetection[i].setThreshold(i/levels);
    theBlobDetection[i].computeBlobs(img.pixels);
  }

  // Create the Network object
  network = new Network(0, 0, 18);

  // Create a bunch of Neurons
  Neuron a = new Neuron(-2, 0, 1);
  Neuron b = new Neuron(2, 0, 1);
  Neuron c = new Neuron(0, 2, 1);
  //Neuron d = new Neuron(6, -7.5, 0); 
  //Neuron e = new Neuron(15, 0, -5);
  //Neuron f = new Neuron(25, 0, -5);
  //Neuron g = new Neuron(40, 5, -5);
  //Neuron h = new Neuron(60, 3, 10);

  // Connect them
  network.connect(a, b,1);
  network.connect(b, c,random(1));
  //network.connect(b, d,random(1));
  //network.connect(c, e,random(1));
  //network.connect(d, e,random(1));
  //network.connect(e, f,random(1));
  //network.connect(f, h, random(1));
  //network.connect(g, h, 1);

  // Add them to the Network
  network.addNeuron(a);
  network.addNeuron(b);
  network.addNeuron(c);
  //network.addNeuron(d);
  //network.addNeuron(e);
  //network.addNeuron(f);
  //network.addNeuron(g);
  //network.addNeuron(h);
}

void draw() {
  background(180);
  // Update and display the Network
  network.update();
  network.display();

  // Every 30 frames feed in an input
  if (frameCount % 30 == 0) {
    network.feedforward(random(1));
  }

   stroke(255);
  strokeWeight(1);
  noFill();

  translate(-img.width*factor/2, -img.height*factor/2);

  for (int i = 0; i < levels; i++) {
    translate(0, 0, elevation/levels);
    drawContours(i);
  }

  theX = mouseX;
  theY = mouseY;
  theZ = 2;

}



void drawContours(int i){
  Blob b;
  EdgeVertex eA, eB;
  for(int n = 0; n<theBlobDetection[i].getBlobNb(); n++){
    b = theBlobDetection[i].getBlob(n);
    if(b!=null){
      stroke((i/levels*colorRange)+colorStart,50,50);

      for(int m = 0; m<b.getEdgeNb(); m++){
        eA = b.getEdgeVertexA(m);
        eB = b.getEdgeVertexB(m);
        if(eA != null && eB != null)
        line(
          eA.x*img.width*factor, eA.y*img.height*factor, 
          eB.x*img.width*factor, eB.y*img.height*factor
            );
      }
    }
  }
}

void createNewNeuron(){
    pushMatrix();
  translate(theX, theY, theZ);
  fill(148, 131, 155);
  sphere(.25);
  popMatrix();

  Neuron oldNeuron1;
  oldNeuron1 = network.neurons.get(1);
  oldNeuron1 = network.neurons.get(int(random(network.neurons.size())));

  Neuron oldNeuron2;
  oldNeuron2 = network.neurons.get(int(random(network.neurons.size())));

  Neuron newNeuron = new Neuron(theX, theY, theZ);

  network.connect(oldNeuron1, newNeuron, theZ);
  network.connect(newNeuron, oldNeuron2, theZ); 


}

void mousePressed(){
  if(mouseButton == RIGHT){
    createNewNeuron();
  }
}

// ======================================================

class Connection {
  // Connection is from Neuron A to B
  Neuron a;
  Neuron b;

  // Connection has a weight
  float weight;

  // Variables to track the animation
  boolean sending = false;
  PVector sender;

  // Need to store the output for when its time to pass along
  float output = 0;

  Connection(Neuron from, Neuron to, float w) {
    weight = w;
    a = from;
    b = to;
  }


  // The Connection is active
  void feedforward(float val) {
    output = val*weight;        // Compute output
    sender = a.location.get();  // Start animation at Neuron A
    sending = true;             // Turn on sending
  }

  // Update traveling sender
  void update() {
    if (sending) {
      // Use a simple interpolation
      sender.x = lerp(sender.x, b.location.x, 0.1);
      sender.y = lerp(sender.y, b.location.y, 0.1);
      sender.z = lerp(sender.z, b.location.z, 0.1);
      float d = PVector.dist(sender, b.location);
      // If we've reached the end
      if (d < 1) {
        // Pass along the output!
        b.feedforward(output);
        sending = false;
      }
    }
  }

  // Draw line and traveling circle
  void display() {
    stroke(35);
    strokeWeight(weight*.5);
    line(a.location.x, a.location.y, a.location.z,b.location.x, b.location.y, b.location.z);

    if (sending) {
      pushMatrix();
      translate(sender.x,sender.y,sender.z);
      noFill();
      sphere(.25);
      popMatrix();
      //strokeWeight(1);
      //ellipse(sender.x, sender.y, 16, 16);
    }
  }
}

// ======================================================


class Network {

  // The Network has a list of neurons
  ArrayList<Neuron> neurons;

  float sum = 0;

  // The Network now keeps a duplicate list of all Connection objects.
  // This makes it easier to draw everything in this class
  ArrayList<Connection> connections;
  PVector location;

  Network(float x, float y, float z) {
    location = new PVector(x, y, z);
    neurons = new ArrayList<Neuron>();
    connections = new ArrayList<Connection>();
  }

  // We can add a Neuron
  void addNeuron(Neuron n) {
      neurons.add(n);
  }

  // We can connection two Neurons
  void connect(Neuron a, Neuron b, float weight) {
    Connection c = new Connection(a, b, weight);
    a.addConnection(c);
    // Also add the Connection here
    connections.add(c);
  } 

  // Sending an input to the first Neuron
  // We should do something better to track multiple inputs
  void feedforward(float input) {
    Neuron start = neurons.get(0);
    start.feedforward(input);
  }

  // Update the animation
  void update() {
    for (Connection c : connections) {
      c.update();
    }
  }

  // Draw everything
  void display() {
    pushMatrix();
    translate(location.x, location.y, location.z);
    for (Neuron n : neurons) {
      n.display();
    }

    for (Connection c : connections) {
      c.display();
    }
    popMatrix();
  }
}


// ======================================================

class Neuron {
  // Neuron has a location
  PVector location;

  // Neuron has a list of connections
  ArrayList<Connection> connections;

  // We now track the inputs and sum them
  float sum = 0;

  // The Neuron's size can be animated
  float r = 32;

  Neuron(float x, float y, float z) {
    location = new PVector(x, y, z);
    connections = new ArrayList<Connection>();
  }

  // Add a Connection
  void addConnection(Connection c) {
    connections.add(c);
  } 

  // Receive an input
  void feedforward(float input) {
    // Accumulate it
    sum += input;
    // Activate it?
    if (sum > 1) {
      fire();
      sum = 0;  // Reset the sum to 0 if it fires
    }
  }

  // The Neuron fires
  void fire() {
    r = 64;   // It suddenly is bigger

    // We send the output through all connections
    for (Connection c : connections) {
      c.feedforward(sum);
    }
  }

  // Draw it as a circle
  void display() {
    pushMatrix();
    //stroke(0);
    //strokeWeight(1);
    // Brightness is mapped to sum
    translate(location.x, location.y, location.z);
    //float b = map(sum, 0, 50, 175, 0);
    fill(148, 131, 155);
    sphere(.25);
    popMatrix();

    //ellipse(location.x, location.y, r, r);

    // Size shrinks down back to original dimensions
    r = lerp(r, 32, 0.1);
  }
}

Chrisir

I don't know

because it is in 3D space there is of course a certain gap where your mouse is and where the point is in 3D - that's why I invented the 3D cursor.

dulce_a

Alright, well thank you anyway.