opencv and gscapture object

treebykoo..

in Contributed Library Questions • 3 years ago

Hello,

Is it possible to have a GSCapture object and a OpenCV object in the same program? I have two separate programs, one that tracks a face with OpenCV and one that tracks a color with GSCapture but when I merge them together I get errors all over the place...as I slowly combine them the first error I get is when I add video = new GSCapture(this, width, height, 30);.....i get NullPointerException at image( opencv.image(), 0, 0 );

is there any way around this or is it because I'm using two different libraries to access my webcam?

Thanks for any info!

Replies(8)

bjordal

Re: opencv and gscapture object

3 years ago

The NullPointerExceptionis thrown because you haven't initialized the opencv object (or the initialization failed ? :s ).

So if you haven't initialized opencv, that is calling opencv= new something(argumentss);, you should do so. If you have initialized and are still getting this error, i think you need to post some more code.

treebykoo..

Re: Re: opencv and gscapture object

3 years ago

Thanks again for your help! I definately have the object initilized. Sorry the code below is such a mess!

So if I have the image( opencv.image(), 0, 0 ); in I get NullPointerException error...so I comment it out because I don't actually need it anyway....
stilll the image doesn't come up for the face recognition...any ideas? i get a small error on start up that is "Error while starting capture : device 0" and then it makes me choose my webcam that's hooked up...making me think that there's a conflict between the two video libraries I have....

thanks so much again for all your help today...

import codeanticode.gsvideo.*;
import ddf.minim.*;
import hypermedia.video.*;
import java.awt.Rectangle;
Minim sound;
AudioPlayer groove;
WaveformRenderer waveform;
AudioSample[] snip = new AudioSample[14]; //i can choose between 15 audio snippets

GSCapture video;
PImage[] mymask = new PImage[5];
OpenCV opencv;
float xxx;
float yyy;

boolean myswitch1=true;
boolean myswitch2=true;
boolean myswitch3=true;
boolean myswitch4=true;
boolean grooveswitch = false;
color trackColor = color(255,0,0); //red and green make yellow
int cellSize = 20; //pixel size. change for different sized pixels.
int cols, rows;

void setup() {
size(640, 480, P2D); // Change size to 320 x 240 if too slow at 640 x 480
frameRate(30);
video = new GSCapture(this, width, height, 30);
    textureMode(NORMALIZED);
cols = width / cellSize;
rows = height / cellSize;
smooth();

sound = new Minim(this);
for (int i = 1; i < snip.length; i++) {
    snip[i] = sound.loadSample("samp" + i + ".wav");
}

groove = sound.loadFile("samp15.wav",1411);
waveform = new WaveformRenderer();
groove.addListener(waveform);
opencv = new OpenCV(this);
    opencv.capture( width, height );
    opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT);    // load the FRONTALFACE description file
    xxx = 100;
    yyy = 100;
    for (int i = 1; i < mymask.length; i++) {
      mymask[i] = loadImage("face"+i+".png");
    }

}

void draw() {
//background(0,0,255); //can have a blue background instead of video
        opencv.read();
    //image( opencv.image(), 0, 0 );   ///////////////////////////******throws NullPointerException error here if left in
       Rectangle[] faces = opencv.detect();
stroke(0);
float one = width*.25;
float two = width*.5;
float three = (width*.75)-1;
    line(one, 0, one ,height);
    line(two, 0, two, height);
    line(three, 0, three, height);

if (video.available()) {
   video.read();

   //image(video, 0, 0, width, height); // leave in for testing

   // Begin loop for columns //DS Code for mirror with pixels. mirror so it looks right on the screen
    for (int i = 0; i < cols; i++) {
      // Begin loop for rows
      for (int j = 0; j < rows; j++) {

        // pixelize for "style"
        int x = i*cellSize;
        int y = j*cellSize;
        int loc = (video.width - x - 1) + y*video.width; // Reversing x to mirror the image

        float r = red(video.pixels[loc]);
        float g = green(video.pixels[loc]);
        float b = blue(video.pixels[loc]);
        // Make a new color with an alpha component
        color c = color(r, g, b, 75);

        // Code for drawing a single rect
        // Using translate in order for rotation to work properly
        pushMatrix();
        translate(x+cellSize/2, y+cellSize/2);
        // Rotation formula based on brightness

        rectMode(CENTER);
        fill(c);
        noStroke();
        // Rects are larger than the cell for some overlap
        rect(0, 0, cellSize, cellSize);
        popMatrix();
      }
    }///end DS code



   int closestX = 0; // X-coordinate of the closest in color video pixel
   int closestY = 0; // Y-coordinate of the closest in color video pixel
   float worldRecord = 500; //we set this to be abritrarily large, once program runs, the first pixel it scans will be set to this value
   // Search for the closest in color pixel: For each row of pixels in the video image and
   // for each pixel in the yth row, compute each pixel's index in the video
   video.loadPixels();

   int index = 0;
   for (int y = 0; y < video.height; y++) {
     for (int x = 0; x < video.width; x++) {
       // Get the color stored in the pixel
       color currentColor = video.pixels[index];

           float r1 = red(currentColor);
           float g1 = green(currentColor);
           float b1 = blue(currentColor);
           float r2 = red(trackColor);
           float g2 = green(trackColor);
           float b2 = blue(trackColor);
           float d = dist(r1,g1,b1,r2,g2,b2);

       // If that value is closer in color value than any previous, then store the
       // color proximity of that pixel, as well as its (x,y) location
       if (d < worldRecord) {
       worldRecord = d;
       worldRecord = worldRecord-10; //change the 10 number for different accuracy
       closestX = x;
       closestY = y;
     }
       index++;
     }
   }

////face recognition/////// ////////
   for( int i=0; i<faces.length; i++ ) {

       float j_f = random(1,5);
       int j = int(j_f);
       beginShape();
       texture(mymask[j]);
       vertex(faces[i].x,faces[i].y,0,0);
       vertex(faces[i].x+faces[i].width,faces[i].y,1,0);
       vertex(faces[i].x+faces[i].width,faces[i].y+faces[i].height,1,1);
       vertex(faces[i].x,faces[i].y+faces[i].height,0,1);
       endShape(CLOSE);

    }




// Draw a yellow circle at the brightest pixel

   int flipx = video.width - closestX - 1;
   fill(255, 204, 0, 128);
   noStroke();
   ellipse(flipx, closestY, 30, 30);

//logic to get the yellow circle to play a unique note once during entry in to each chamber

if (groove.isPlaying()) {
grooveswitch = false;
}

   if ((closestX < one)&&(closestX>0)&&(grooveswitch = true)) {
        if (myswitch1) {
         println("FIRST CHAMBER");
         //snip[15].trigger();
         groove.loop(1);
         myswitch1=false;
         myswitch2=true;
         myswitch3=true;
         myswitch4=true;
     }
}

if ((closestX > one)&&(closestX<two)) {
      if (myswitch2) {
        println("SECOND CHAMBER");
        snip[2].trigger();
        myswitch1=true;
        myswitch2=false;
        myswitch3=true;
        myswitch4=true;
      }
   }

if ((closestX>two)&&(closestX<three)) {
      if (myswitch3) {
        println("THIRD CHAMBER");
        snip[3].trigger();
        myswitch1=true;
        myswitch2=true;
        myswitch3=false;
        myswitch4=true;
      }
   }

if ((closestX>three)&&(closestX<width)) {
      if (myswitch4) {
       println("FOURTH CHAMBER");
       snip[4].trigger();
       myswitch1=true;
       myswitch2=true;
       myswitch3=true;
       myswitch4=false;
      }
   }


} //end if videoavailable statment

} //end void draw

//minim stop stuff
void stop() {
   for (int i = 1; i < snip.length; i++) {
   snip[i].close();
   }
    sound.stop();
    super.stop();
}

bjordal

Re: opencv and gscapture object

3 years ago

hmm i have no idea what's wrong then... like i said before i don't have a webcam so i can't try your code but as far as i can tell a NullPointerException shouldn't be thrown, unless the constructor fails and doesn't throw an error...

edit: it doesn't have to be the constructor that fails, can also be the capture() or read()

but yeah, two libraries using the webcam sounds like something that will cause errors...

treebykoo..

Re: opencv and gscapture object

3 years ago

Thanks for your help.

So does anyone out there know if you can use two different video libraries?

Or does anyone have any ideas how I could track both a color and a face and do different things with each? I read here that OpenCV doesn't support two objects at the moment either...

bjordal

Re: opencv and gscapture object

3 years ago

Hold on a second if i understand your code correctly (if im not it is because im very tired) all you are using GSCapture for is getting the current image of the camera and then applying your own algorithms to the current image...

If this is the case then you don't actually need GSCapture because you can use opencv.image() instead. I'm guessing you've just overlooked this but in case you haven't, all GSCapture appears to do is extend the PImage class and directly reading the captured video into a PImage. But since opencv.image() returns the current video as a PImage you can just use this instead!

So basically in the beginning of draw you would do something like:

PImage currentVid = opencv.image();

and then just do your color tracking on currentVid.

but hey, maybe im just super tired :)

treebykoo..

Re: opencv and gscapture object

3 years ago

Yup that did it! You're a genius! Thanks again for all your help!

bjordal

Re: Re: opencv and gscapture object

3 years ago

haha, no problem :)

andres

Re: opencv and gscapture object

3 years ago

Hi guys, just want to mention that it is possible to combine gsvideo capture with opencv face detection. See the code below, where the opencv image buffer has to be explicitly initialised with opencv.allocate(640,480). Then copy the pixels from gscapture to opencv with opencv.copy(cam).

Even though OpenCV has its own capture mechanism, integrating with GSVideo could be useful in some circumstance where the capture device is supported by gstreamer but not by opencv.

import hypermedia.video.*;
import java.awt.Rectangle;
import codeanticode.gsvideo.*;

OpenCV opencv;
GSCapture cam;

// contrast/brightness values
int contrast_value    = 0;
int brightness_value = 0;

void setup() {
    size( 640, 480 );

    cam = new GSCapture(this, 640, 480);

    opencv = new OpenCV( this );
    opencv.allocate(640,480);
    opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT ); // load detection description, here-> front face detection : "haarcascade_frontalface_alt.xml"
}

void captureEvent(GSCapture c) {
c.read();
}

public void stop() {
    opencv.stop();
    super.stop();
}

void draw() {
    opencv.copy(cam);

    opencv.convert( GRAY );
    opencv.contrast( contrast_value );
    opencv.brightness( brightness_value );

    // proceed detection
    Rectangle[] faces = opencv.detect( 1.2, 2, OpenCV.HAAR_DO_CANNY_PRUNING, 40, 40 );

    // display the image
    image(cam, 0, 0 );

    // draw face area(s)
    noFill();
    stroke(255,0,0);
    for( int i=0; i<faces.length; i++ ) {
        rect( faces[i].x, faces[i].y, faces[i].width, faces[i].height );
    }
}

void mouseDragged() {
    contrast_value   = (int) map( mouseX, 0, width, -128, 128 );
    brightness_value = (int) map( mouseY, 0, width, -128, 128 );
}

Top Reply