How to use Processing 3 and the video capture library on Raspberry PI ?

orgicus · November 2015

Hi,

I am trying to use a USB webcam with Processing 3 on a Raspberry PI 2.

This is using the preinstalled version from the Processing raspian disturbution(processing-3.0.1-linux-raspbian.zip, 2015-09-24-raspbian-jessie.img).

Using the Contribution Manager I've installed the video library, but I am getting an error using an USB webcam:

No such Gstreamer factory: v4l2src

I can see the camera showing up at /dev/video0 and after installing gstreamer1.0 and gstreamer1.0-tools I can open the camera:

gst-launch-1.0 v4l2src ! videoconvert ! ximagesink

Using v4l2-ctl --list-formats-ext yelds:

ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUV 4:2:2 (YUYV)
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x120
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 176x144
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x176
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 352x288
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 432x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 544x288
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 752x416
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x448
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x600
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 864x480
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x544
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x720
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1024x576
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1184x656
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1280x960
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'MJPG' (compressed)
    Name        : MJPEG
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x120
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 176x144
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x176
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 352x288
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 432x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 544x288
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 752x416
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x448
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x600
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 864x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x544
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x720
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1024x576
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1184x656
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1280x960
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.040s (25.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.200s (5.000 fps)

How can I access the USB camera from Processing on Raspian ?

Thank you, George

orgicus · November 2015

Gottfried Haider's version of the video library works and Andres was very helpful in explaining the easiest way to tweak the library.

Details here

matthieu · December 2015

I tried all it's written but without results. Could you write what you did step by step.

Thanks a lot

orgicus · December 2015

Hi @matthieu,

Here are the steps I've used:

Download Gottfried's p5-arm video library
Unzip the contents in /home/sketchbook/libraries (assuming you're using the Raspian Jessie with Processing preinstalled and with the default settings: sketchbook folder in ~/sketchbook). You should have a /home/sketchbook/libraries/video folder now.
Use the library in Processing.

There are few caveats though since this is an experimental version:

The constructor doesn't allow any options (like resolution/framerate/etc.). There is a single argument: the sketch. Here's an example:

Capture cam = new Capture(this); cam.start();
The default stream is hardcoded to 640x480 and it's YUV, not RGB so the colours will look strange.

In my case I needed to do some basic frame differencing and needed to use grayscale (which is what the Y channel of YUV provides).

In terms of resizing, I noticed PImage's resize() function is kinda slow, but drawing a smaller image into a frame buffer (PGraphics) works pretty fast:

import processing.video.*;

Capture cam;

//resized frame
PGraphics frame;
//resized dimenions
int cw = 40;
int ch = 30;

void setup(){

  size(100,100);
  noSmooth();

  cam = new Capture(this);
  cam.start();

  frame = createGraphics(cw,ch);

}
void draw(){
  //display image on screen
  image(frame,0,0);
}
void captureEvent(Capture c){
  c.read();
  if (frame != null){
    //draw resized image to buffer -> faster than PImage.resize()
    frame.beginDraw();
    frame.image(c,0,0,width,height);
    frame.endDraw();
  }
}

I haven't had a chance to setup the video library project in eclipse to tinker with the size and colour space issues yet.

keng · February 2016

Hi guy, I follow orgicus's step and it work, newbies can see this clip to guide. thanks orgicus for guide step and Gottfried's library.

https://youtu.be/ma9xj0zxrts

Bonem · October 2016

@Gohai

I found this and it works!

kucky · February 2017

Hi, I need a video capture for my GUI. Can I use this GoHai-Library under Processing 3 and windows (both 64bit)? GSVideo doesn't works.

  if (cameras.length == 0)
    {
    println("There are no cameras available for capture.");
    exit();
    } else {
    println("Available cameras:");
    for (int i = 0; i < cameras.length; i++) {
      println(cameras[i]);   <--------- OK
    }
    cam = new GSCapture(this, 640, 480, cameras[0]);   <--- Error
    cam.start();

Thanks for help, I´m not an expert

P.S. My IDE is Netbeans 8.2

orgicus · February 2017

@kucky I'd start a separate thread on GSVideo with Processing3 on Windows 64bit (as this thread is focusing on Raspberry PI only).

Thank you

kucky · February 2017

OK, thanks

nelson · November 2017

hello, please How to use Processing 3 in ipcapture?

nelson · November 2017

I have this code does not work

import ipcapture.*;

IPCapture cam;

void setup() {
  size(640,480);

  cam = new IPCapture(this, "http://" + "192.168.1.101/index.html", "admin", "Muitas"); 
  cam.start();
}

void draw() {
  if (cam.isAvailable()) {
    cam.read();
    image(cam,0,0);
  }
}

void keyPressed() {
  if (key == ' ') {
    if (cam.isAlive()) cam.stop();
    else cam.start();
  }
}

gohai · November 2017

@nelson This does not have anything to do with the Video library or GL Video library on the Raspberry Pi. Are you even running on a Raspberry Pi?

Howdy, Stranger!

Categories

In this Discussion

How to use Processing 3 and the video capture library on Raspberry PI ?

Best Answer

Answers