Issues with color tracking in Processing (Raspbian)

Denez

Hi !

I'm trying to run a sketch on my raspberry Pi 3 B that worked perfectly on my laptop. Now on Raspbian it doesn't want to work at all, I always get an out of bounds exception... I've replaced the Video library by the GLVideo one (recommended with RPi) but it doesn't seem to be better. What I want to do is to have a simple color tracking (copied from a Daniel Shiffman's example) on a video and when the x and y position of that color is at a certain distance from a point on the screen, it sends a 1 to an Arduino Uno blablabla... "ArrayIndexOutOfBoundsException : 56960"

import gohai.glvideo.*;
import processing.serial.*;


GLMovie movie; 
color trackColor; 
float hand = 0; 

Serial myPort;


void setup() {

  size(640, 359, P2D); 

  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 9600);

  movie = new GLMovie(this, "boulez.mov");
  movie.loop();

  trackColor = color(-5595749);
}

void draw() {
  background(100);

  if (movie.available()) {
    movie.read();
  }

  imageMode(CENTER);
  image(movie, width/2, height/2, width, height);


  float worldRecord = 500; 

  int closestX = 0;
  int closestY = 0;

  //Code from Daniel Shiffman's color tracking example with few ajustments

  for (int x = 0; x < movie.width - (movie.width/2.5); x ++ ) {
    for (int y = height/4; y < movie.height - (movie.height/4); y ++ ) {

      int loc = x + y*movie.width;
      // What is current color
      color currentColor = movie.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      float r2 = red(trackColor);
      float g2 = green(trackColor);
      float b2 = blue(trackColor);

      float d = dist(r1, g1, b1, r2, g2, b2);

      if (d < worldRecord) {
        worldRecord = d;
        closestX = x;
        closestY = y;
      }
    }
  }


  if (worldRecord < 5) { 
    fill(trackColor);
    strokeWeight(4.0);
    stroke(255, 0, 0);
    ellipse(closestX, closestY, 16, 16);
  }

  hand = dist(width/2, height, closestX, closestY); 

  noStroke();
  if (hand > 250) {
    myPort.write('1');
  } else {
    myPort.write('0');
  }
}

Thanks for your help !

gohai

@Denez Which version of Processing are you testing this with?

koogs

Is the first height in line 44 meant to be movie.height?

koogs

I'd also move lines 52-54 outside the loop - no need to recalculate them add they don't change.

There are also faster methods of extracting the channels from a pixel using bitwise methods.

koogs

(I'd also get rid of the dist () because the implicit sqrt is quite expensive)

Denez

@gohai The last one, I dowloaded it yesterday ! Thanks for your awesome work btw !

Thanks @koogs, you think it has to do with ram or processor ? Yep it's a height, here to constrain the color tracking to be just on a certain part of the video. For now it doesn't change anything I get the same error !

gohai

@Denez And the version of the GL Video library is also the latest? (1.1)

What do you get when you do println(movie.width + " " + movie.height)? Could you upload the mov file for me somewhere to try and debug?

Denez

@gohai Yes I'm afraid. With the println I got "0 0" instead of the "640 359" expected... Maybe the video doesn't have the time to load ?

Here's a link for the original code I wrote that worked on my laptop with the video in the data folder :

https://dropbox.com/sh/m7uq5u1abi3dxmd/AABio66Qd9h8SazZ8OrIVp18a?dl=0

Denez

Oh and I forgot to mention that I have no error (out of bounds exception) when I use the original video library except that the video won't display...

I increased the memory of the gpu just in case but nothing happened.

I think it has nothing to do with the loading of the video, because I tried not to use variables, hardcode everything but same issue always.

Playing with the "if (movie.available()) {movie.read();}" gives me strange results, maybe it can be a clue...

Edit : I finally got my sketch to work by changing the codec of the video (to theora). Now it works fine with the original video library on the raspberry, unfortunately not with the GL Video one that would probably give me better frame rate.

gohai

@Denez You need to call movie.loadPixels() after each movie.read(). Only then the pixels array will be populated. Hope this helped.