The limits of remotely loading and manipulating images: discussion and advice

newmiracle

I've been working on a few little projects and demos and things that have similar goals.

I wanted to grab images from a URL and manipulate them in processing.js without having to pre-load them. For example, the functionality would have a user selecting whatever images they wanted via URL, then applying an image mask to that image. The key being dynamically loading whatever images you could find on the internet.

Trying things out, searching on forums, and asking questions seems to only offer up traditional advice for PJS: "Preload your images!". Obviously, if you want to let a user choose an image URL dynamically, this won't apply. However, this isn't strictly necessary in PJS. You can, in fact, load an image via URL by passing a valid URL string to loadImage() or requestImage(). Getting this working made me think, "Great! Now I can put a mask on it!" Not so fast.

Through a process of trial and error, it would seem that accessing the pixel array of a remote image is not possible (if the javascript console in my browser is correct). So, not only can you not mask a remote image, you can't take that image data and 'clone' it to a 'local' version.

Then I thought I would be clever. Reading through Pomax's guide about interfacing Javascript in your HTML document with the Javascript generated by your Processing sketches, I thought I could do an end-run around this limitation. First up: load an image in javascript and then pass it to a pjs function. Didn't work.

What else? Well, then I analyzed the image in Javascript pixel by pixel. Then my javascript would write each pixel's values to a string, and when done I would pass that string to the PJS sketch, and in the PJS sketch "decode" the string and write the values to a pixel array. I'm reading the pixels, passing the pixels, but ultimately for whatever reason that has not worked either.

I'm not super experienced, so I'm not sure if this is simply making a mistake or if I have somehow hit a limit to PJS. Its just surprising to me that an image can be loaded and displayed, but accessing its pixel array isn't possible. So, I'm convinced that I'm just doing something wrong, but then I've seen some error reports that may indicate that there are limits regarding this.

processing-js.lighthouseapp.com/projects/41284/tickets/1776-pimagemask-should-spit-out-a-useful-error-if-the-image-is-remote

I'm thinking that there might be some more mojo to be done with the @pjs directives or perhaps doing some kind of in-line javascript with processing, but I haven't seen a simple implementation or something like that and I haven't had the time yet to go that deep into researching how that would work.

So... is this a real limit to PJS? Or am I just doing it wrong? Any thoughts or advice appreciated.

Pomax

As the processing.org reference explains, the extremely simple solution if you don't want to preload/need dynamic images is to load the image using requestImage, and then simply periodically check the dimensions for your image. If the width/height of the image is no longer 0, it has been loaded in and you can start using it in earnest in your sketch.

http://www.processing.org/reference/requestImage_.html

Also note that asking questions in the Processing.js channel and then leaving makes it really hard to answer those questions.

newmiracle

Sorry, I had hung around before but it looked like everyone was afk for a while so I figured the message board would be better.

Again, maybe I'm implementing this wrong, but I've used requestImage, verified that the width/height is greater than zero, and still not been able to copy the pixels to a new PImage/PGraphics object.

newmiracle

Ok, I'm still looking at this, hoping there is a way to do it.

So, for example, look at the Alphamask example for the JS mode. I'm trying to modify it minimally in order to have the base functionality of what I am describing. What I'm trying to do: load an image URL using requestImage. Verify the image is loaded. Apply mask (created from a PGraphics).

PImage img;

PImage maskPatternImg;
PGraphics maskPattern;

void setup() {
  size(640, 360);

  img = requestImage("http://"+"www.photo-dictionary.com/photofiles/list/3022/4047rose_leaves.jpg");

}

void draw() {
  background(0, 0, 153);


  if(img.width>0){
  maskSet();

  img.mask(maskPatternImg);
  imageMode(CENTER);


  image(img, width/2, height/2);
  }

}

void maskSet(){      

      maskPattern = createGraphics(img.width,img.height);

      maskPattern.width=img.width;
      maskPattern.height=img.height;      

      maskPattern.background(0);

      maskPattern.beginDraw();

      maskPattern.stroke(255);
      maskPattern.fill(255);
      maskPattern.ellipse(width/2, height/2, 300, 300);

      maskPattern.endDraw();

      maskPatternImg = maskPattern.get();      
}

This works in Java, but not JS. I've moved this around, tried to break apart each piece to see if I'm doing things right. But I haven't gotten it yet.

blyk

So I found something in a old forum ... this might interest you http://forum.processing.org/one/topic/snippet-crazy-fast-image-loading.html

blindfish

@blyk: necromancy is all well and good, but this corpse isn't going to be revived with a link to what looks very much like a Java only solution :/