Night:Code

Eone

V proud to share this first project for my masters. I wrote a piece of code that motion tracks webcam movement, converts it to MIDI data and then sends it out to Ableton to create a soundscape.

I present to you, Night:Code

Eone

Let me know if you want to see the code!

Chrisir

Code plase

;-)

Eone

        import gab.opencv.*;
        import processing.video.*;
        import ipcapture.*;
        import org.opencv.core.Rect;
        import themidibus.*; //Import the library


        OpenCV opencv;
        IPCapture cam;
        MidiBus midi; 

        int lastMillis;

        int [][] timestamps = new int [16][128];


        void setup() {
          size(800, 450);
          for (int ch = 0; ch < 16; ch++) {
            for (int i = 0; i < 128; i++) {
              timestamps[ch][i] = -1;
            }
          }

          cam = new IPCapture(this, "http://65.152.238.150:80/mjpg/video.mjpg?resolution=800x450", "", "");
          cam.start();
          opencv = new OpenCV(this, 800, 450);

          opencv.startBackgroundSubtraction(5, 3, 0.5);
          MidiBus.list();
          midi = new MidiBus(this, -1, "Bus 1");
        }

        void sendNote(int ch, int n) {
          if (timestamps[ch][n] == -1) {
            midi.sendNoteOn(ch, n, 100);
            timestamps[ch][n] = millis();
          }
        }

        void draw() {

          //check for overdue notes
          for (int ch = 0; ch < 16; ch++) {
            for (int i = 0; i < 128; i++) {
              if (timestamps[ch][i]  > -1) {
                if (millis()>timestamps[ch][i] +400) {
                  midi.sendNoteOff(ch, i, 100);
                  timestamps[ch][i] = -1;
                }
              }
            }
          }

          if (cam.isAvailable()) {
            cam.read();
            image(cam, 0, 0);  
            opencv.loadImage(cam);
            opencv.updateBackground();

            opencv.dilate();
            opencv.erode();

        //LINE COLOUR

            noFill();
            stroke(0, 0);

            strokeWeight(1);

            int rx = 580; 
            int ry = 15;
            int rw = 80;
            int rh = 70;
            rect(rx, ry, rw, rh);

            int rx2 = 40; 
            int ry2 = 140;
            int rw2 = 70;
            int rh2 = 120;
            rect(rx2, ry2, rw2, rh2);


            int rx3 = 500; 
            int ry3 = 190;
            int rw3 = 80;
            int rh3 = 40;
            rect(rx3, ry3, rw3, rh3);


            int rx4 = 200; 
            int ry4 = 50;
            int rw4 = 75;
            int rh4 = 150;
            rect(rx4, ry4, rw4, rh4);

            int rx401 = 275; 
            int ry401= 50;
            int rw401 = 75;
            int rh401 = 150;
            rect(rx401, ry401, rw401, rh401);


            int rx5 = 50; 
            int ry5 = 280;
            int rw5 = 80;
            int rh5 = 80;
            rect(rx5, ry5, rw5, rh5);

            int rx501 = 50; 
            int ry501 = 280;
            int rw501 = 80;
            int rh501 = 80;
            rect(rx501, ry501, rw501, rh501);

            int rx6 = 490; 
            int ry6 = 300;
            int rw6 = 100;
            int rh6 = 40;
            rect(rx6, ry6, rw6, rh6);

            int rx7 = 310; 
            int ry7 = 300;
            int rw7 = 100;
            int rh7 = 140;
            rect(rx7, ry7, rw7, rh7);


            int rx8 = 50; 
            int ry8 = 380;
            int rw8 = 90;
            int rh8 = 70;
            rect(rx8, ry8, rw8, rh8);


            int rx9 = 600; 
            int ry9 = 240;
            int rw9 = 25;
            int rh9 = 20;
            rect(rx9, ry9, rw9, rh9);

            int rx901 = 600; 
            int ry901 = 240;
            int rw901 = 25;
            int rh901 = 20;
            rect(rx901, ry901, rw901, rh901);


            int rx10 = 700; 
            int ry10 = 250;
            int rw10 = 35;
            int rh10 = 15;
            rect(rx10, ry10, rw10, rh10);

            int rx101 = 700; 
            int ry101 = 250;
            int rw101 = 90;
            int rh101 = 35;
            rect(rx101, ry101, rw101, rh101);


            int rx11 = 675; 
            int ry11 = 300;
            int rw11 = 55;
            int rh11 = 50;
            rect(rx11, ry11, rw11, rh11);


            int rx12 = 500; 
            int ry12 = 40;
            int rw12 = 45;
            int rh12 = 80;
            rect(rx12, ry12, rw12, rh12);

            int rx13 = 450; 
            int ry13 = 190;
            int rw13 = 45;
            int rh13 = 15;
            rect(rx13, ry13, rw13, rh13);

            int rx14 = 390; 
            int ry14 = 85;
            int rw14 = 45;
            int rh14 = 15;
            rect(rx14, ry14, rw14, rh14);


            int rx141 = 390; 
            int ry141 = 85;
            int rw141 = 45;
            int rh141 = 15;
            rect(rx141, ry141, rw141, rh141);


            int rx15 = 390; 
            int ry15 = 150;
            int rw15 = 45;
            int rh15 = 45;
            rect(rx15, ry15, rw15, rh15);


            int rx16 = 700; 
            int ry16 = 100;
            int rw16 = 100;
            int rh16 = 100;
            rect(rx16, ry16, rw16, rh16);


            int rx17 = 500; 
            int ry17 = 250;
            int rw17 = 100;
            int rh17 = 40;
            rect(rx17, ry17, rw17, rh17);


            int rx18 = 525; 
            int ry18 = 350;
            int rw18 = 20;
            int rh18 = 20;
            rect(rx18, ry18, rw18, rh18);


            int rx181 = 545; 
            int ry181 = 370;
            int rw181 = 20;
            int rh181 = 20;
            rect(rx181, ry181, rw181, rh181);

            stroke(255, 1, 1);
            strokeWeight(1);
            for (Contour contour : opencv.findContours()) {


              println(contour.getBoundingBox().x);
              println(contour.getBoundingBox().y);
              int x = contour.getBoundingBox().x;
              int y = contour.getBoundingBox().y;

              contour.draw();



              if (( x > rx && x < rx + rw) && (y > ry && y < ry + rh)) {
                sendNote(0, 60);
              }

              if (( x > rx2 && x < rx2 + rw2) && (y > ry2 && y < ry2 + rh2)) {
                sendNote(0, 61);
              }

              if (( x > rx3 && x < rx3 + rw3) && (y > ry3 && y < ry3 + rh3)) {
                sendNote(0, 62);
              }

              if (( x > rx4 && x < rx4 + rw4) && (y > ry4 && y < ry4 + rh4)) {
                sendNote(1, 60);
              }

              if (( x > rx401 && x < rx401 + rw401) && (y > ry401 && y < ry401 + rh401)) {
                sendNote(1, 51);
              }


              if (( x > rx5 && x < rx5 + rw5) && (y > ry5 && y < ry5 + rh5)) {
                sendNote(1, 61);
              }

              if (( x > rx501 && x < rx501 + rw501) && (y > ry501 && y < ry501 + rh501)) {
                sendNote(0, 51);
              }

              if (( x > rx6 && x < rx6 + rw6) && (y > ry6 && y < ry6 + rh6)) {
                sendNote(1, 62);
              }

              if (( x > rx7 && x < rx7 + rw7) && (y > ry7 && y < ry7 + rh7)) {
                sendNote(1, 63);
              }

              if (( x > rx8 && x < rx8 + rw8) && (y > ry8 && y < ry8 + rh8)) {
                sendNote(1, 64);
              }

              if (( x > rx9 && x < rx9 + rw9) && (y > ry9 && y < ry9 + rh9)) {
                sendNote(1, 52);
              }

                    if (( x > rx901 && x < rx901 + rw901) && (y > ry901 && y < ry901 + rh901)) {
                sendNote(1, 65);
              }

              if (( x > rx10 && x < rx10 + rw10) && (y > ry10 && y < ry10 + rh10)) {
                sendNote(1, 66);
              }

              if (( x > rx101 && x < rx101 + rw101) && (y > ry101 && y < ry101 + rh101)) {
                sendNote(1, 53);
              }


              if (( x > rx11 && x < rx11 + rw11) && (y > ry11 && y < ry11 + rh11)) {
                sendNote(1, 67);
              }

              if (( x > rx12 && x < rx12 + rw12) && (y > ry12 && y < ry12 + rh12)) {
                sendNote(1, 68);
              }

              if (( x > rx13 && x < rx13 + rw13) && (y > ry13 && y < ry13 + rh13)) {
                sendNote(1, 69);
              }

              if (( x > rx14 && x < rx14 + rw14) && (y > ry14 && y < ry14 + rh14)) {
                sendNote(1, 70);
              }

              if (( x > rx141 && x < rx141 + rw141) && (y > ry141 && y < ry141 + rh141)) {
                sendNote(1, 54);
              }

              if (( x > rx15 && x < rx15 + rw15) && (y > ry15 && y < ry15 + rh15)) {
                sendNote(0, 71);
              }

              if (( x > rx16 && x < rx16 + rw16) && (y > ry16 && y < ry16 + rh16)) {
                sendNote(1, 55);
              }

              if (( x > rx17 && x < rx17 + rw17) && (y > ry17 && y < ry17 + rh17)) {
                sendNote(0, 72);
              }

              if (( x > rx18 && x < rx18 + rw18) && (y > ry18 && y < ry18 + rh18)) {
                sendNote(1, 72);
              }

                    if (( x > rx181 && x < rx181 + rw181) && (y > ry181 && y < ry181 + rh181)) {
                sendNote(1, 73);
              }

            }
          }
        }

Eone

Ffs, Why does the code button never work for me?!

Chrisir

Go back

Edit your post with the gear

Ctrl-a and ctrl-o

kfrajer

Make sure there is an empty line above and below your code block. It makes a big difference.

Kf

Eone

Ayyye, done. Thank you. The empty line was the trick that worked for me :)

jeremydouglass

Congratulations on completing the project. Looks / sounds great!

One suggestion for making the code more readable and easier for you to work with in the future: consider compressing your code by using a for loop over repeating bits and feeding in different values.

For example, lines 71-227 are really the same five lines over and over again looping over an array of numbers, so you could that into ~7 lines plus separate rectangle data. Ditto lines 243-338.

Eone

Cool, thank you @jeremydouglass I did wonder if a for loop might be easier but it was getting close to the end of my deadline and I needed to get my work in. Also, this is my first piece of code ever written so I'm still a noob to all this!

jeremydouglass

For example (this is a simple way, without classes), you can create an arraylist in the header to store all your rectangles:

• https://processing.org/reference/ArrayList.html

like this:

ArrayList<int[]> rectangles = new ArrayList();

..and then, in setup, you can save a list of four numbers for each rectangle:

rectangles.add(new int[]{580,  15,  80,  70});
rectangles.add(new int[]{ 40, 140,  70, 120});
rectangles.add(new int[]{500, 190,  80,  40});

...and then, in draw, replace all of your code that looks like this:

    int rx = 580; 
    int ry = 15;
    int rw = 80;
    int rh = 70;
    rect(rx, ry, rw, rh);

...with just one loop that draws a rectangle for each item in your list:

for (int i = 0; i < rectangles.size(); i++) {
  int[] r = rectangles.get(i);
  rect(r[0], r[1], r[2], r[3]);
}

Now say you want to bind note values to that. Update your data in setup so that it adds the values at the end of each rectangle data line:

rectangles.add(new int[]{580,  15,  80,  70,  0, 60});
rectangles.add(new int[]{ 40, 140,  70, 120,  0, 61});
rectangles.add(new int[]{500, 190,  80,  40,  0, 62});

Now in draw replace all your code that looks like this:

  if (( x > rx && x < rx + rw) && (y > ry && y < ry + rh)) {
    sendNote(0, 60);
  }

...with just one loop that sends the listed note variables for each item, e.g. (untested):

for (int i = 0; i < rectangles.size(); i++) {
  int[] r = rectangles.get(i);
  if (( x > r[0] && x < r[0] + r[2]) && (y > r[1] && y < r[1] + r[3])) {
    sendNote(r[4], r[5]);
  }
}

Eone

Holy sh*t, that's great! It would have made it so much more compact. I would have liked to make it so that the movement only turned red when it moved into one of my rectangles but was time bound. I'd like to keep it up and carry on programming. Not just learn it once for the course and then leave it.

Eone

@jeremydouglass