Converting explode example to p5js

chrisgoc

OS: Windows 10 Browsers: Chrome Version 48.0.2564.103 m (64-bit), Edge 25.10586.0.0 Graphics Card: NVidia GTX 860M

Hello,

I'm trying to convert the explode example from Processing's Image Processing Topic to p5js. The original example is

/* @pjs preload="eames.jpg"; */

/**
 * Explode 
 * by Daniel Shiffman. 
 * 
 * Mouse horizontal location controls breaking apart of image and 
 * Maps pixels from a 2D image into 3D space. Pixel brightness controls 
 * translation along z axis. 
 */

PImage img;       // The source image
int cellsize = 2; // Dimensions of each cell in the grid
int columns, rows;   // Number of columns and rows in our system

void setup() {
  size(640, 360, P3D); 
  img = loadImage("eames.jpg");  // Load the image
  columns = img.width / cellsize;  // Calculate # of columns
  rows = img.height / cellsize;  // Calculate # of rows
}

void draw() {
  background(0);
  // Begin loop for columns
  for ( int i = 0; i < columns; i++) {
    // Begin loop for rows
    for ( int j = 0; j < rows; j++) {
      int x = i*cellsize + cellsize/2;  // x position
      int y = j*cellsize + cellsize/2;  // y position
      int loc = x + y*img.width;  // Pixel array location
      color c = img.pixels[loc];  // Grab the color
      // Calculate a z position as a function of mouseX and pixel brightness
     float z = (mouseX / float(width)) * brightness(img.pixels[loc]) - 20.0;
      // Translate to the location, set fill and stroke, and draw the rect
      pushMatrix();
      translate(x + 200, y + 100, z);
      fill(c, 204);
      noStroke();
      rectMode(CENTER);
      rect(0, 0, cellsize, cellsize);
      popMatrix();
    }
  }
}

I'm dealing with a couple of issues. One is that the p5js sketch is very slow compared to the above sketch in Processing.js. The second is that I can't grab a color using var c = pixels[loc] and use it in the brightness method. If I leave out brightness from calculating var z for translation in z and use a numerical scaling value, the sketch runs but very slowly.

var cellSize = 2; // Dimensions of each cell in the grid
var columns, rows;  // Number of columns and rows in our system

function preload() {
img = loadImage("assets/eames.jpg");  // Preload the image
}

function setup() {
createCanvas(640, 360, WEBGL);
columns = img.width / cellSize; // Calculate # of columns
rows = img.height / cellSize; // Calculate # of rows
}

function draw() {
background(0);
// Begin loop for columns
for(var i = 0; i < columns; i++) {
    // Begin loop for rows
    for(var j = 0; j < rows; j++) {
    var x = i * cellSize;  // x position
    var y = j * cellSize;  // y position
    img.loadPixels();
    var loc = x + y * img.width;  // Pixel array location
    var t = img.get(x, y, cellSize, cellSize); // texture for plane 
    var c = pixels[loc]; // Grab the color
    // Calculate a z position as a function of mouseX and pixel brightness        
    var z = mouseX / width * brightness(c - 20); // <-- doesn't work
    // Translate to the location, set fill and stroke, and draw the rect
    push();
    translate(x, y, z);
    texture(t);
    plane(cellSize, cellSize);
    pop();
    }
  }
}

I'd like to add this example to the p5js examples so please let me know if you have any insight on how to speed up the sketch or how to get the color from img.pixels[].

Thanks, Chris!

Screenshot of working sketch without brightness scaling Super slow! (< 1fps)

GoToLoop

I've got an explode example from:
https://forum.Processing.org/two/discussion/comment/57206/#Comment_57206

Hosted here: http://p5js.SketchPad.cc/sp/pad/view/ro.C7yI5n-rAMHkBH/latest

Perhaps it can help you having some idea. Perhaps later I'll try to peruse your code. 8-|

chrisgoc

Thanks. I can't get the last three commits of the sketch you referenced to run either in Chrome or Edge (WEBGL is working though, I tested other sketchpads).

GoToLoop

It runs on my SlimJet (Chrome-based). But I have to click WAIT button that shows up. It's painfully slow!
However, both Firefox-based browsers and even IE11 manage to run it reasonably comfortable.

chrisgoc

Okay, when I hit Cancel on the WEBGL? popup it worked. But it has similar lag to the sketch I'm trying to port. If I use img.get(x, y) inside the array loops my sketch will work, it's just too slow. It seems like maybe the p5js 3d library is just not optimized enough because the original sketch runs reasonably well in Processing.js. Thanks for your help, I'm not sure if there's much that can be done from inside the sketch. Even when I moved sampling pixels from image into setup, it didn't help.

GoToLoop

• In my own example, get() is used inside setup(). Using it once sketch hits draw() is suicide! :-SS
• Your for loop got lotsa redundancy too! Why loadPixels() all the time?
• IMO, you don't need translate(), push() & pop() in order to get coords right.
• I also fear that texture() can prove to be too heavy for p5js? :-??

chrisgoc

I have a variant here where I address the issues in bullets 1, 2.

var cellSize = 4; // Dimensions of each cell in the grid
var columns, rows;  // Number of columns and rows in our system
var x, y, t, particleScaling, xdir;

function preload() {
img = loadImage("assets/eames.jpg");  // Preload the image
}

function setup() {
createCanvas(640, 360, WEBGL);
columns = img.width / cellSize; // Calculate # of columns
rows = img.height / cellSize; // Calculate # of rows
img.loadPixels();
x = [];
y = [];
xdir = [];  
t = [];
particleScaling = [];
for(var i=0; i<columns; i++) {
    x[i] = [];
    y[i] = [];
    xdir[i] = [];    
    t[i] = [];
    particleScaling[i] = [];
    for(var j=0; j<rows; j++) {
    x[i][j] = i * cellSize;  // x position
    y[i][j] = j * cellSize;  // y position
    xdir[i][j] = random(-1, 1);
    t[i][j] = img.get(x[i][j], y[i][j], cellSize, cellSize);  // Grab the color
    particleScaling[i][j] = brightness(img.get(x[i][j], y[i][j]));
    }
  }
}

function draw() {
background(0);
// Begin loop for columns
translate(0, -100, 250);
for(var i = 0; i < columns; i++) {
    // Begin loop for rows
    for(var j = 0; j < rows; j++) {
    var x = i* cellSize;
    var y = j * cellSize;
    // Calculate a z position as a function of mouseX and pixel brightness        
    var z = mouseX / width * particleScaling[i][j];
    // Translate to the location, set fill and stroke, and draw the rect
    push();
    translate(x + 200, y + 100, z);
    texture(t[i][j]);
    plane(cellSize, cellSize);
    pop();
    }
  }
}

FPS is maybe 10 - 15. Point 3 - I know there are ways you can do this all in 2D. I was trying to make it work in 3D, hence the 3D translation. Is there another approach in 3D? Point 4 - [Edit] Actually was able to get brightness value on line 30 so I guess this is resolved?

I'm not sure if this is answered or not. The speed is still unacceptable for animations but I did implement your suggestions and got a performance increase, so thanks for your time thus far.

GoToLoop

Your last attempt is much better. Everything is pre-calculated! :-bd

The reason I'm using planes is because rect is undefined with WEBGL.

I woulda never guessed that 2D primitives like rect(), ellipse(), etc. weren't allowed in WEBGL! @-)
It greatly diverts from all Processing modes & spin-offs, where 3D is an extension of 2D rather than something completely different! 8-}

chrisgoc

The error line is coming from p5js.

the if statement above is failing?

GoToLoop

• In Processing, the base for every graphic is PImage.
• Then it is followed by PGraphics, which inherits from PImage.
• Now each PGraphics renderer got their own particular implementation.
• But nevertheless, they all inherit from the basics, like rect(), ellipse(), etc.

chrisgoc

Looks like there is an open issue for this:

RFC: Geometry3D refactor #1056

indefinit opened this issue on Oct 31, 2015

"With a Geometry3D refactor, both 3D and 2D geometry primitives would be created similarly, each in "retainedMode". Because the Geometry3D class would consequently contain both 3D and "2D" primitives, we'd rename the class to simply "p5.Geometry", and 3d_primitives.js would become primitives.js."

GoToLoop

• p5.js' graph base is p5.Element, which is followed by an almost empty p5.Renderer.
• Then there are 2 sub-classes: p5.Renderer2D & p5.Renderer3D.
• Seems like they both differ greatly 1 from the other, unfortunately.
• To make matters worse, createGraphics() returns a wrapper p5.Graphics object.
• Which in turn got either a p5.Renderer2D or a p5.Renderer3D in its _renderer property.
• Even worse, p5.Image, even though it has everything a p5.Element got, doesn't inherit or it is inherited by n1!
• Therefore a p5.Renderer doesn't necessarily got all p5.Image's members. X(

chrisgoc

Thanks for your help. Hopefully this will be helpful for people who run into issues related to this.

GoToLoop

With a Geometry3D refactor, both 3D and 2D geometry primitives would be created similarly, ...

I wish they would look up how Processing, or even Processing.js, did the inheritance chain rather than coming up w/ their bug 1!

Because the Geometry3D class would consequently contain both 3D and "2D" primitives,

Why create 1 class more! Why not follow the Processing Java Mode wisdom? ~O)
Just correct the inheritance chain to the following natural flow:

1. p5.Element
2. p5.Image
3. p5.Renderer
4. p5.Renderer2D & p5.Renderer3D

chrisgoc

You should comment on the issue

GoToLoop

I can't! I'm personally banned there! :-&

chrisgoc

:((