loading image from URL lags in JS,

newmiracle

Ok, I'm working more on my image manipulation stuff in PJS. Having some trouble loading images. The following code won't work correctly in JS mode, but will work fine in normal Processing.

In order for this to run, you'll need 3 local images: black.jpg, leaves.jpg, and moonwalk.jpg. black.jpg is an all black image. Not sure I even need this, but that isn't important now. Leaves.jpg and moonwalk.jpg can be whatever you like.

(I know this isn't the best way to implement all this. But I boiled everything down to it's base parts in order to try and figure out what was going on. Pardon the amateur implementation.)

/* @pjs preload="black.jpg, leaves.jpg, moonwalk.jpg"; 
 */

/*

*/

PGraphics maskPattern;
PGraphics savedimg;
PImage img1;
PImage img2;
PImage maskPatternImg;

int displayImageX;
int displayImageY;

float scaleFactor;
float centerScaledAdjustedXpos;
float centerScaledAdjustedYpos;


boolean isImg1Loaded;

void setup() {
  size(800,800,JAVA2D);
  img1 = loadImage("moonwalk.jpg");

  maskPattern = createGraphics(img1.width,img1.height);
  savedimg = createGraphics(img1.width,img1.height);
  img2 = loadImage("leaves.jpg");

  scaleFactor=1;

  displayImageX = width/2;
  displayImageY = height/2;

  centerScaledAdjustedXpos=0;
  centerScaledAdjustedYpos=0;


  maskPatternImg = loadImage("black.jpg");
  maskPatternImg.width=maskPattern.width;
  maskPatternImg.height=maskPattern.height;  
}



void draw() {

  pushMatrix();
    scale(scaleFactor);

    background(0);
    imageMode(CENTER);

    image(img2,displayImageX/scaleFactor,displayImageY/scaleFactor);

    if (mousePressed == true) {
      maskPaint(); 
      }

    if (img1.width == maskPatternImg.width && img1.height == maskPatternImg.height){ 
      //checking to make sure the mask size vs. image size is valid

        img1.mask(maskPatternImg);
        image(img1,displayImageX/scaleFactor,displayImageY/scaleFactor);

      }
    //else{println("mask isn't the same size");}

  popMatrix();

imageLoadTest();
}

void imageLoadTest(){
 if(img1.width>0){isImg1Loaded = true; }
 if(img1.width<1){isImg1Loaded = false; }

 // println("isImg1Loaded is " + isImg1Loaded);
}


void maskPaint(){

  centerScaledAdjustedXpos = ((mouseX-displayImageX)/scaleFactor+(maskPattern.width/2));
  centerScaledAdjustedYpos = ((mouseY-displayImageY)/scaleFactor+(maskPattern.height/2));


  maskPattern.beginDraw();

  maskPattern.stroke(255);
  maskPattern.fill(255);
  maskPattern.ellipse(centerScaledAdjustedXpos, centerScaledAdjustedYpos, 50, 50);

  maskPattern.endDraw(); 

  maskPatternImg = maskPattern.get();
}


void urlLoad(String imageURL){

  println("img1 width check 1= " + img1.height);
  img1 = loadImage(imageURL);
  println("img1 width check 2= " + img1.height);

}

void imageOpen(String url){  

 if (url == null) {
    println("url is null");
  } else {
    println("User loaded " + url);


    //img1 = loadImage(url);

    urlLoad(url);

    println("img1 width check 3= " + img1.height);

    if( img1.width != 0){ 
      //An interesting note: 
      //using "if( isImg1Loaded == true){" will pass as "true" even when img1.height=0
      //Not sure why. That shouldn't happen.

      println("img1 width check 4= " + img1.height);


      maskPattern = createGraphics(img1.width,img1.height);

      maskPatternImg.width=maskPattern.width;
      maskPatternImg.height=maskPattern.height;

      maskPatternImg.width=img1.width;
      maskPatternImg.height=img1.height;

      savedimg = createGraphics(img1.width,img1.height);

      maskPattern.background(0);
      }

    }

}




void keyPressed() { 

   if(key == 's')
    {
      savedimg.beginDraw();
        savedimg.image(img2,0,0);
        img1.mask(maskPattern);
        savedimg.image(img1,0,0);
      savedimg.endDraw();

    savedimg.save("final_image.jpg");
    println("Saved ");
    }


  if(key == ' '){     
    maskPattern.background(0);
    maskPatternImg = maskPattern.get();
  }  


  if (key == '-') {
    println("minus"); 
    scaleFactor=scaleFactor-.01;    
  }
  if (key == '=') {
    println("plus");    
    scaleFactor=scaleFactor+.01;
  }
  if(key == 'u'){
  //println("Letter U pressed");
  imageOpen("http://" + "www.photo-dictionary.com/photofiles/list/3022/4047rose_leaves.jpg");
  }
}

Now, I know what you're thinking. "You have to pre-load images!" Except you don't. Loading an image from a URL is totally possible. In the urlLoad function, change the "img1 = loadImage(imageURL)" to img2. That will load just fine in JS mode, after a brief moment.

My issue is that I need to wait until img1 is fully loaded in order to apply the masking functionality to it. However, try as I might, I can't seem to get it to work. It works just fine in normal processing, but not JS. I know I probably just need to find the right way to wait for loading, but for whatever reason, what I've been trying hasn't been working.

How can I get this to work?

Any help is appreciated.

chrisjmears

Have you tried using requestImage() coupled with @pjs preload directives?. I think requestImage does it async so no significant delay

newmiracle

I've used requestImage() in a few different places, but I'm not sure if I'm implementing it right. @pjs won't do (I think), because I'm looking for dynamic loading capability. I want the user to define an image URL, then load it. Preloading like that requires knowing the filename beforehand and defining it at the beginning of the sketch, correct?

And again, I don't have to use @pjs and pre-load with the other image that isn't being processed through my masking implementation. It comes up, it just takes a second. I don't mind waiting, I just want to know when it is fully loaded in order to then apply the mask.

I had set up an if() check that looked at the image's width. If it is zero, it isn't loaded. I was trying to say, "don't apply the mask until the image width is greater than zero (aka loaded)". I thought that would do it, but it would just wind up stalling the sketch and crashing. It never got to the 'loaded' condition. I thought it would work like the other, plain image: gone for a moment and then loaded and then appears. But for whatever reason it just never loads. Might be in how I'm structuring things. Back to the drawing board.

GoToLoop

I've tweaked a program I've got here to use requestImage() and hosted it online:

http://OpenProcessing.org/sketch/139280

From this thread:

http://forum.Processing.org/two/discussion/3439/how-to-do-flip-canvas-horizontal-to-an-image-and-save#Item_21

newmiracle

Sorry, but it doesn't work.

Line 120 goes back to a function urlLoad(). In that function on line 105, you'll see where I'm loading an image. I've changed that to requestImage(). Program doesn't work.

That function itself is vestigial, from abstracting things and working with previous issues. If you take that out of the mix, and do the requestImage() on line 120 directly instead- still won't work.

I just don't have the experience to know what is exactly going on. But from what I'm thinking, I'm just not doing the "load>veryify loading>then apply effects" chain correctly.

Under line 120, I've done the proper implementation of the requestImage() check found on the processingjs site

processingjs.org/reference/requestImage_/

if (img1.width == 0){
     println("Not yet loaded");
    } 
    if( img1.width == -1){
     println("Error");
    }
    else{ 
//... on to the rest of the program

Doesn't work. I've done if (img1.width == 0) return, like in your example, and that doesn't work either.

And again, I can't implement a solution that is loading the URL explicitly in the setup() function because ultimately, the program will allow to load images dynamically by inputting whatever img URL the user specifies.

I just need to know when the image is fully loaded, thats all. Unless maybe I'm messing things up with mask() and how that whole functionality works?

Thanks for the help. I'll continue to play around. Any further advice or insight as to why things are happening the way they are would be greatly appreciated.

newmiracle

I've been working on this and looking at the developer console in Chrome and Firefox to debug things. I'm thinking that using masks on external images in processing js just might not work.

processing-js.lighthouseapp.com/projects/41284/tickets/1776-pimagemask-should-spit-out-a-useful-error-if-the-image-is-remote

Is that what this error report is saying?

GoToLoop

I got "TypeError: ea.data is undefined" when using img.mask(img); in the online example! :-&

P.S.: When an image is loaded locally, mask() works!
Perhaps if we stamp a remotely loaded image in a PGraphics and crop it back as another PImage, we end up w/ a naturalized 1,
capable of applying mask() and other functions? :-??

newmiracle

Well, I've tried that approach. Ruling things out, breaking the functionality down to very explicit step-by-step instructions, the closest I got was an error in the browser console that says "Uncaught Image is loaded remotely. Cannot get pixels."

Yes, I've waited until the image is loaded completely (at least I think...but I'm pretty positive). As far as I can tell, Processing won't let me load/get pixels of any kind for an image that has been loaded remotely. Which confuses me... because it will display an image loaded remotely just fine. But just the image. Trying to load it and put things into a PGraphics/PImage first and then trying some image manipulation at that point won't work.

Its like you can't trick it from being remote to local.

I feel like I'm a little out of my scope here, so I'm not entirely sure what to think. But it seems to me like Processing.js simply cannot apply a mask to an image loaded from a remote source. Or get pixels from a remote image, for that matter.

Can we establish some kind of consensus for people who are asking this question in the future? And is it possible for Processing.js people to try and implement this?

As a follow-up, I was thinking a system where in the Javascript in the HTML "passed" an image reference to Processing.js. I've been able to do this with strings, thanks to the Pomax guide. But I haven't seen anything similar for images, yet. Anyone know if that is possible?

CheekyBastard

Did anybody figure this out? I'm also getting "Uncaught Image is loaded remotely. Cannot get pixels." in the Chrome developer console :/

CheekyBastard

Also, I'm pretty sure the images not loading is not the issue (see screenshot below). And the error only comes up when you click.

GoToLoop

Sorry @CheekyBastard, found none! X_X
Any remotely loaded PImage doesn't have a corresponding pixels[] property! 8-|
And any PGraphics it touches, corrupts and invalidates it as well! :-SS

I still have hope that it gotta be a way to turn such remote PImage objects into regular 1s.
If it can be displayed, why can't we get its pixels[]?

Perhaps we should ask @fjenett why?!
https://github.com/fjenett/javascript-mode-processing

Or @jeresig:
https://github.com/processing-js/processing-js/

ZylerTeller

@GoToLoop Isn't the problem to do with XSS? I think in the documentation for loadImages it says that loading images from a remote server is a security issue.

"The filename parameter can also be a URL to a file found online. For security reasons, a Processing sketch found online can only download files from the same server from which it came. ".

GoToLoop

For security reasons, a Processing sketch found online can only download files from the same server from which it came.

Processing.JS framework can load & display images from other servers.
Problem is, as I've already mentioned, they don't have pixels[] and remove pixels[] from any PGraphics which they were displayed at forever. That's what I call "corruption"! :-O