I am working on a project that uses 3 webcams that when looked at, will play a video. I am thinking of the screens as entities that need to be acknowledged before they communicate with someone.

Everything was going dandy until I ran into two hiccups.

One is that opencv.loadImage(camright); seems to the culprit of this error "width(0) and height (0) cannot be <=0" which doesn't make any sense to me because there are opencv.loadImage(camleft); and opencv.loadImage(camcenter); before it and they don't seem to be returning the same issue.

The second hiccup is that I am trying to use keystone so that I can projection map these videos to hanging plexi... I just can't seem to figure out how to get the videos onto the keystone 'mesh'. Did that make sense?

Anyways, I am very green (taking my first class) and any help would be appreciated immensely. Especially since this project is due next week...

Below is my code... (I really couldn't get the whole code thing to work- I tried- I am sorry-if you want to help me with that too that'd be nice)

//LIBRARIES 
import deadpixel.keystone.*;
import gab.opencv.*;
import processing.video.*;
import java.awt.*; 
import java.awt.Rectangle;

//keystone stuff 
Keystone ks;
CornerPinSurface surfaceleft;
CornerPinSurface surfacecenter;
CornerPinSurface surfaceright;
PGraphics offscreenleft;
PGraphics offscreencenter;
PGraphics offscreenright;

// movie object to play and pause later
//there will be three videos playing... 
Movie myMovieleft;
Movie myMoviecenter;
Movie myMovieright;

OpenCV opencv;

//https://processing.org/discourse/beta/num_1221233526.html
//https://forum.processing.org/two/discussion/5960/capturing-feeds-from-multiple-webcams
Capture camleft;
Capture camcenter;
Capture camright;

String[] captureDevices;  

void setup() {
  //this will println listing the webcams you need to but the number on the left in the [] 
  //to make them work 
  printArray(Capture.list());
  background(0);
  size(2640, 1080, P3D); //this should be large enough to house all the videos 
  opencv = new OpenCV(this, 160, 120);

  //keystone stuff 
  ks = new Keystone(this);
  //this can change 
  surfaceleft = ks.createCornerPinSurface(400, 300, 20);
  surfacecenter = ks.createCornerPinSurface(400, 300, 20);
  surfaceright = ks.createCornerPinSurface(400, 300, 20);
  // We need an offscreen buffer to draw the surface we
  // want projected
  // note that we're matching the resolution of the
  // CornerPinSurface.
  // (The offscreen buffer can be P2D or P3D)
  //P3D is telling processing to be in 3D mode
  //the number is 400, 300 is related to eachother they must be the same 
  offscreenleft = createGraphics(400, 300, P3D);
  offscreencenter = createGraphics(400, 300, P3D);
  offscreenright = createGraphics(400, 300, P3D);

  //webcam stuff 
  //this is turning the webcam on and to run
  //the numbers correlate to the println list 
  camleft = new Capture(this, Capture.list()[3] ); //LEFT CAM IS LOGITECH HD 
  //WEBCAM C310 
  camleft.start();

  camcenter = new Capture(this, Capture.list()[79] ); //CENTER CAM IS LOGITECH HD
  //WEBCAM C310-1 
  camcenter.start();

  camright = new Capture(this, Capture.list()[155] ); //RIGHT CAM IS LOGITECH HD 
  //WEBCAM C310-2
  camright.start();

  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);

  // movie stuff 
  // load video
  myMovieleft = new Movie(this, "testvideo.mp4");

  // need to play, pause, loop 
  myMovieleft.play();
  myMovieleft.pause();
  myMovieleft.loop();

  myMoviecenter = new Movie(this, "testvideo-1.mp4");

  // need to play, pause, loop 
  myMoviecenter.play();
  myMoviecenter.pause();
  myMoviecenter.loop();

  myMovieright = new Movie(this, "testvideo-2.mp4");

  // need to play, pause, loop 
  myMovieright.play();
  myMovieright.pause();
  myMovieright.loop();
}
void captureEvent(Capture cam) {
  cam.read();
}

void draw() {

  // open cv detect faces
  opencv.loadImage(camleft);

  // load in faces as rectangles
  Rectangle[] facesleft = opencv.detect();

  // are there faces?
  if (facesleft.length > 0) {
    // sees a face!
    myMovieleft.play();
  } else {
    // no face
    myMovieleft.pause();
  }

  // play video
  if (myMovieleft.available()) {
    myMovieleft.read();
    //offscreen.image
    image(myMovieleft, 0, 540);
  }

  opencv.loadImage(camcenter);

  // load in faces as rectangles
  Rectangle[] facescenter = opencv.detect();

  // are there faces?
  if (facescenter.length > 0) {
    // sees a face!
    myMoviecenter.play();
  } else {
    // no face
    myMoviecenter.pause();
  }

  // play video
  if (myMoviecenter.available()) {
    myMoviecenter.read();
    image(myMoviecenter, 780, height/2);
  }

// THERE IS SOMETHING WRONG WITH THIS opencv.loadImage(camright);

//RIGHT CAM
  // load in faces as rectangles
  Rectangle[] facesright = opencv.detect();

  // are there faces?
  if (facesright.length > 0) {
    // sees a face!
    myMovieright.play();
  } else {
    // no face
    myMovieright.pause();
  }

  // play video
  if (myMovieright.available()) {
    myMovieright.read();
    image(myMovieright, 1560, height/2);

    //keystone stuff
    surfaceleft.render(offscreenleft);
    surfacecenter.render(offscreencenter);
    surfaceright.render(offscreenright);
  }
}
//the save and load function for keystone 
void keyPressed() {
  switch(key) {
  case 'c':
    // enter/leave calibration mode, where surfaces can be warped 
    // and moved
    ks.toggleCalibration();
    break;

  case 'l':
    // loads the saved layout
    ks.load();
    break;

  case 's':
    // saves the layout
    ks.save();
    break;
  }
}
`

I'm working on java project that uses processing to draw objects to for projection onto a surface that is not dead on square with the projector. I'm trying to skew the canvas / processing output so the entire output can be adjusted to result in it's original ratio when it hit's its surface.

There doesn't seem to be a transform function to modify the canvas shape and size() only takes x and y. Using camera() adjustments in 2D resulted in harshly aliased(?) shapes and really doesn't handle x AND y corrections very well. (Picture placing the projector tilted up and to the rotated to the right in relation to the projection surface, for example.)

Gestalt had the ability to do this, but, upgrading the source seems to be turning into be a bigger headache than just creating a new solution.

It seems to me from following the gestalt code, I'm trying to skew the canvas that processing is drawing ON before processing gets it's hands on it, so the rest of the drawing and images display coordinates is happening ignorant of the skew.

Any clues would be grand as I've research the heck out of this and little is out there about skewing processing's overall output into it's framebuffer.

Thanks!

import deadpixel.keystone.*;
String[] texts = 
  {
  "#peace, this is what I've wanted from SF --", 
  "market street is always so colorful, the", 
  "Opened Graffiti request via android at 990 Market St", 
  "the Opened Street or Sidewalk Cleaning request", 
  "via android at 1000 Market St", 
  "#sfc, join our lonely little squad tonight,", 
  "the operation nerds gathering, the new False Gods,", 
  "in for some #bigdatatalk, and to answer your email", 
  "#streetstyle #streetphotography #sf #walk #voyager:", 
  "I met this beautiful soul at the #BlueLamp a few months back --", 
  "just posted a photo,", 
  "so impressed,", 
  "I'm suffering through the widthdrawl..."};
String[] mentions =
  { 
  "@melematique + @mistry_chintan", 
  "@JoCrra", 
  "@SF311 Reports", 
  "@SF311 Reports", 
  "@SF311 Reports", 
  "@totallyclubbin", 
  "@calitalieh + @catsynth", 
  "@agentv", 
  "@JoCrra", 
  "@Delany_Rene", 
  "@Delany_Rene", 
  "@WillKirkpatrick", 
  "@hchiu13", 
};
DisplayBox dp1; //single
DisplayBox[] dps; //multiple
PFont font;
PFont fontS;
Keystone ks;
CornerPinSurface surface;
PGraphics offscreen;
float[] widthVariable;
float angle = 0;
float textFade = 0;
float tf = 0;
int counter = 0;
float rCounter = 0;
float tCounter = 0;
int switchVariable = 0;
void setup() {
  font = loadFont("c.vlw");
  fontS = loadFont("b.vlw");
  //fullScreen(P3D, 1);
  size(1325, 750, P3D);
  ks = new Keystone(this);
  surface = ks.createCornerPinSurface(1325, 750, 20);
  offscreen = createGraphics(1325, 750, P3D);
  //dp1 = new DisplayBox(0, texts[0], 0, height/2, 0);
  dps = new DisplayBox[texts.length];
  for (int i=0; i<texts.length; i++) {
    dps[i] = new DisplayBox(i, texts[i], 10, 100+(i*50), 0);
  }
  widthVariable = new float[texts.length];
}
void draw() {
  rCounter+=.05;
  tCounter+=5;
  //setting text widths;
  for (int i=0; i<texts.length; i++) {
    widthVariable[i]=offscreen.textWidth(texts[i])+20;
  }
  PVector surfaceMouse = surface.getTransformedMouse();
  offscreen.beginDraw();
  offscreen.background(0);
  offscreen.textFont(font, 42);
  offscreen.fill(255);

  //(*.) try to figure out the flickering
  //stroke, rect, text, rotate, translate;
  //remember the transparency effects;
  for (int i=0; i<texts.length; i++) {
    dps[i].DRT(255, 5, 0, 255, 255, 255, 0, rCounter, 0, 0, 0, 0);
  }

  offscreen.endDraw();
  background(0);
  surface.render(offscreen);
  //for text-fading;
  textFade = sin(angle);
  angle += 0.25;
  tf = map(textFade, -1, 1, 255/2, 255);
}

void keyPressed() {
  switch(key) {
  case 'c':
    // enter/leave calibration mode, where surfaces can be warped 
    // and moved
    ks.toggleCalibration();
    break;

  case 'l':
    // loads the saved layout
    ks.load();
    break;

  case 's':
    // saves the layout
    ks.save();
    break;
  }
}

class DisplayBox {
  //STRING DATA;
  int sentenceW; //sentenceWidth;
  String sentence; //the string;
  //POSITION DATA;
  float xPos; //x pos
  float yPos; //y pos
  float zPos; //z pos
  //COLORS;
  float sW; //stroke weight;
  float rSC; //stroke color;
  float rST; //stroke transparency;
  float rFC; //fill color;
  float rFT; //fill transparency;
  float tF; //text fill;
  float tT; //text transparency;
  //TRANSLATIONS;
  float transX; //move x
  float transY; //move y
  float transZ; //move z
  //ROTATION DATA;
  float ogX; //origin x
  float ogY; //origin y
  float ogZ; //origin z
  DisplayBox(int sWI, String s, float xP, float yP, float zP) {
    sentenceW = sWI;
    sentence = s;
    xPos = xP;
    yPos = yP;
    zPos = zP;
  }
  //colors, rotations, translations
  void DRT(float rSC, float sW, float rFC, float rFT, 
  float tF, float tT, float rX, float rY, 
  float rZ, float transX, float transY, float transZ) {
    ogX = xPos + widthVariable[sentenceW]/2;
    ogY = yPos - 15;
    ogZ = zPos;
    offscreen.pushMatrix();
    offscreen.translate(transX, transY, transZ);
    offscreen.translate(ogX, ogY, ogZ);
    offscreen.rotateX(rX);
    offscreen.rotateY(rY);
    offscreen.rotateZ(rZ);
    offscreen.translate(-ogX, -ogY, -ogZ);
    offscreen.stroke(rSC);
    offscreen.strokeWeight(sW);
    offscreen.fill(rFC, rFT);
    offscreen.rect(xPos, yPos-42, widthVariable[sentenceW], 50);
    offscreen.fill(tF, tT);
    offscreen.text(sentence, xPos+5, yPos, zPos);
    offscreen.popMatrix();
  }
}

Here is my code:

/* Import Keystone Framework */
import deadpixel.keystone.*;
/* Processing Video Framework */
import processing.video.*;


Keystone ks;
CornerPinSurface surface;
CornerPinSurface surface2;
CornerPinSurface surface3;
CornerPinSurface surface4;
CornerPinSurface surface5;
CornerPinSurface surface6;

PGraphics offscreen;
PGraphics offscreen2;
PGraphics offscreen3;
PGraphics offscreen4;
PGraphics offscreen5;
PGraphics offscreen6;

Movie myMovie;
Movie myMovie2;
Movie myMovie3;
Movie myMovie4;
Movie myMovie5;
Movie myMovie6;

void setup() {
  size(1200, 600, P3D);

  ks = new Keystone(this);
  surface = ks.createCornerPinSurface(320, 240, 20); 
  offscreen = createGraphics(320, 240, P3D);
  surface2 = ks.createCornerPinSurface(320, 240, 20); 
  offscreen2 = createGraphics(320, 240, P3D);
  surface3 = ks.createCornerPinSurface(320, 240, 20); 
  offscreen3 = createGraphics(320, 240, P3D);
  surface4 = ks.createCornerPinSurface(320, 240, 20); 
  offscreen4 = createGraphics(320, 240, P3D);
  surface5 = ks.createCornerPinSurface(320, 240, 20); 
  offscreen5 = createGraphics(320, 240, P3D);
  surface6 = ks.createCornerPinSurface(320, 240, 20); 
  offscreen6 = createGraphics(320, 240, P3D);


  myMovie = new Movie(this, "mask1.mov");
  //myMovie.loop();
  myMovie.pause();

  myMovie2 = new Movie(this, "mask2.mov");
  //myMovie2.loop();
  myMovie2.pause();

  myMovie3 = new Movie(this, "mask3.mov");
  //myMovie3.loop();
  myMovie3.pause();

  myMovie4 = new Movie(this, "mask4.mov");
  //myMovie4.loop();
  myMovie4.pause();

  myMovie5 = new Movie(this, "mask5.mov");
 // myMovie5.loop();
  myMovie5.pause();

  myMovie6 = new Movie(this, "mask6.mov");
  //myMovie6.loop();
  myMovie6.pause();

}


void draw() {

  PVector surfaceMouse = surface.getTransformedMouse();

  offscreen.beginDraw();
  offscreen.background(255);
  offscreen.fill(0, 255, 0);
  offscreen.image(myMovie,0,0,320,240);
  offscreen.endDraw();

  offscreen2.beginDraw();
  offscreen2.background(255);
  offscreen2.fill(0, 255, 0);
  offscreen2.image(myMovie2,0,0,320,240);
  offscreen2.endDraw();

  offscreen3.beginDraw();
  offscreen3.background(255);
  offscreen3.fill(0, 255, 0);
  offscreen3.image(myMovie3,0,0,320,240);
  offscreen3.endDraw();

  offscreen4.beginDraw();
  offscreen4.background(255);
  offscreen4.fill(0, 255, 0);
  offscreen4.image(myMovie4,0,0,320,240);
  offscreen4.endDraw();

  offscreen5.beginDraw();
  offscreen5.background(255);
  offscreen5.fill(0, 255, 0);
  offscreen5.image(myMovie5,0,0,320,240);
  offscreen5.endDraw();

  offscreen6.beginDraw();
  offscreen6.background(255);
  offscreen6.fill(0, 255, 0);
  offscreen6.image(myMovie6,0,0,320,240);
  offscreen6.endDraw();  

  background(0); 
  surface.render(offscreen);
  surface2.render(offscreen2);
  surface3.render(offscreen3);
  surface4.render(offscreen4);
  surface5.render(offscreen5);
  surface6.render(offscreen6);
}

void keyPressed() {
  switch(key) {
  case 'c':
    // enter/leave calibration mode, where surfaces can be warped 
    // and moved
    ks.toggleCalibration();
    break;

  case 'l':
    // loads the saved layout
    ks.load();
    break;

  case 's':
    // saves the layout
    ks.save();
    break;
  case 'p':
    myMovie.play();
    myMovie2.play();
    myMovie3.play();
    myMovie4.play();
    myMovie5.play();
    myMovie6.play();
    break;
  }
}

void movieEvent(Movie m) {
  m.read();
}