Processing 1.0 - Processing Discourse - Importing & moving images

Ladle

Importing & moving images
Jan 27^th, 2010, 11:24am

Im using a sketch which uses loadImage() to load an array of 15 images into an array, then draws them to the screen. I want to be able to move the images about etc, but at the moment it's far too slow, as I think I'm trying to load all of the images each loop through inside draw(). Is this lagging inescapable? or would using pixels[] or copy() give faster performance as a rule? This is the section of code from inside draw():
Code:

 for(int i=0; i<2; i++){
     translate(0, -i*i, -200);
    if(imageLinkArray[i] != null){
    imagesArray[i] = loadImage(imageLinkArray[i], "jpg");
    tint(255-(i*15));  
    image(imagesArray[i], 0, 0, imagesArray[i].width, imagesArray[i].height);
     }
   }

koogy God Member Offline Posts: 788	Re: Importing & moving images Reply #1 - Jan 27^th, 2010, 1:29pm yes, loadImage inside draw() is terribly slow, don't do it. load them all in setup and then you'll just need the tint() / image() within draw()

Ladle Junior Member Offline Posts: 91	Re: Importing & moving images Reply #2 - Jan 27^th, 2010, 1:48pm I'd love to but i'm getting the images at regular intervals from an api, so the images change every now and again, but I want to be able to run the applet on the web, so I cant save() the images then load them locally. Is there a good way I can copy the images into memory, then display them without such delay?

blindfish

Re: Importing & moving images
Reply #3 - Jan 27^th, 2010, 2:27pm

By "getting the images at regular intervals" do you mean they get updated at regular intervals or just that that's what you're doing? If they get updated at regular intervals you know to only call loadImage at the same regular interval. If not then does the API include some kind of notification that an image has updated? If so you can use this to trigger a loadImage(). At the very least use a timer to reduce the rate at which images are loaded within draw: I'm sure your users aren't going to notice if, for example, there's a 1 second delay on image updating... Seems a better option than trying to loadImage 60 times a second!

Where is the what if the what is in why?

koogy

Re: Importing & moving images
Reply #4 - Jan 27^th, 2010, 2:52pm

who is setting you this problem? we see it a lot... webcam shots from adelaide usually.

what blindfish said, basically. only load an image when you know it's changed. or you can cycle through them but wait 10 seconds or so between each request - webcam pictures won't change any more frequently than that anyway.

something like this in draw():

int now = millis();
if ((now - lastTime) > 10000) {
// 10 seconds have passed
lastTime = now;
// load picture and bump counter
imagesArray[picture] = loadImage(imageLinkArray[picture], "jpg");
picture = (picture + 1) % total_pictures;
}

where lastTime and picture are global variables, initialised to 0

Ladle

Re: Importing & moving images
Reply #5 - Jan 28^th, 2010, 6:37am

Well this is the code I have at the moment (minus the xml/html request blocks). The images are coming from a site where users upload new images at any time, so I am just going for a regularly checking sketch to make sure its quite up to date. I can set the timer here to give longer intervals, but possibly I've structured it wrongly..

As for the question, I'm just trying to learn processing on my own, so no-one's set it! webcams from adelaid Shocked

Quote:

void setup() {
  size(400,600, P3D);
  background(0);
  startTime = millis();
}

void draw(){

  XMLRequest xreq = new XMLRequest(this,"http://cagd.leedsmet.ac.uk/feeds/groupWork.php?gid=g3000002");
    HTMLRequest req = new HTMLRequest(this,"http://cagd.leedsmet.ac.uk/feeds/groupWork.php?gid=g3000002");

  int now = millis();//start counter for updates.
  if (now - startTime > 5000) { // Every 5 seconds, make a new request
    req.makeRequest();
    xreq.makeRequest();
    println("Updating..");
    startTime = now;
    println(authorArray[0]+" posted the "+typeArray[0]+" called "+titleArray[0]+" on "+dateArray[0]+", see it here: "+imageLinkArray[0]);

  background(0);

pushMatrix();
translate(100, height*0.6, 0);
   for(int i=0; i<authorArray.length; i++){
     translate(0, -i*i, -200);
    if(imageLinkArray[i] != null){
    imagesArray[i] = loadImage(imageLinkArray[i], "jpg");
    tint(255-(i*15));
    image(imagesArray[i], 0, 0, imagesArray[i].width, imagesArray[i].height);

           stroke(255);
    line(imagesArray[i].width+10, imagesArray[i].height, 0, width, height/2, 200);

  //println(image1.width + " width");
     }
   }
   popMatrix();


//  imagesArray[3] = loadImage(imageLinkArray[3], "jpg");
//  imagesArray[3].loadPixels();

  }
}//end of draw.

koogy

Re: Importing & moving images
Reply #6 - Jan 28^th, 2010, 8:02am

this bit

XMLRequest xreq = new XMLRequest(this,"http://cagd.leedsmet.ac.uk/feeds/groupWork.php?gid=g3000002");
HTMLRequest req = new HTMLRequest(this,"http://cagd.leedsmet.ac.uk/feeds/groupWork.php?gid=g3000002");

is obviously setup - it's using constant values so they won't change with every frame. make the xreg and reg global (outside setup()) and initialise them inside setup().

haven't run the rest of it (it's not all there!) but you have to remember that draw runs as fast as it possibly can. if nothing has changed from one frame to the next (and it doesn't appear to in your case) then you're just wasting cycles.

change the frameRate to something a lot lower (like 1 or 2) and put a check in there somewhere so that draw() returns before the background() call if nothing has changed (no new picture)

even hitting those two http requests every 5 seconds looks like massive overkill...

Ladle

Re: Importing & moving images
Reply #7 - Jan 28^th, 2010, 10:07am

koogy wrote on Jan 28^th, 2010, 8:02am:

is obviously setup - it's using constant values so they won't change with every frame. make the xreg and reg global (outside setup()) and initialise them inside setup().
haven't run the rest of it (it's not all there!)

Ok, I've sorted out those requests into the setup, I had thought they should be there before, but I'd forgotten to initialise them so I was getting some very nasty errors!

I keep going round in circles here though I think, I want to keep the high framerate, as ideally the images will update/check for updates every 30 secs, but I want to draw the current images into their 3D stack frequently, so I can add some controls for spinning them around/zooming in etc.

This is the code with the requests, so it should work, loading 15 images into a stack, but it only actually redraws the stack once every 30secs. I cant seem to just pick out the 4 lines of 'tint(), image(), fill(), text(). Sad

Code:

import simpleML.*;
PImage image1;
PImage image2;
ArrayList images;
PFont font;
int startTime; 
int zoom;
int imagePositionCount;
int textPositionCount;
int linkPositionCount;
int videoPositionCount;
String authorString;
String pubDateString;
String workTitleString;
String linkString;
String[] authorArray = new String[15];
String[] dateArray = new String[15];
String[] titleArray = new String[15];
String[] imageLinkArray = new String[15];
PImage[] imagesArray = new PImage[15];
String[] typeArray = new String[15];
XMLRequest xreq; 
HTMLRequest req;

void setup() {
  size(800,400, P3D);
  background(0);
  xreq = new XMLRequest(this,"http://cagd.leedsmet.ac.uk/feeds/groupWork.php?gid=g3000002");
  req = new HTMLRequest(this,"http://cagd.leedsmet.ac.uk/feeds/groupWork.php?gid=g3000002");
  req.makeRequest();
  xreq.makeRequest();
  startTime = millis();
  font = createFont("Arial", 15);
  textFont(font);
}

void draw(){
  int now = millis();//start counter for updates.
  if (now - startTime > 30000) { // Every 30 seconds, make a new request
    req.makeRequest();
    xreq.makeRequest();
    println("Updating..");
    startTime = now;
    println(authorArray[0]+" posted the "+typeArray[0]+" called "+titleArray[0]+" on "+dateArray[0]+", see it here: "+imageLinkArray[0]);
    background(0);

    pushMatrix();
    for(int i=1; i<authorArray.length; i++){
      if(typeArray[i] == "image"){
        imagePositionCount = imagePositionCount+1;
        println(imagePositionCount+"images in top 15");
        translate(100, 0, -100);
      }else if(typeArray[i] == "text"){
        textPositionCount = textPositionCount+1;
        translate(100, 0, -100);
      }else if(typeArray[i] == "video"){
        videoPositionCount = videoPositionCount+1;
        translate(100, 0, -100);
      }else if(typeArray[i] == "link"){
        linkPositionCount = linkPositionCount+1;
        translate(100, 0, -100);
      }
      translate(0, 0, zoom);
      if(imageLinkArray[i] != null){
        imagesArray[i] = loadImage(imageLinkArray[i], "jpg");
        tint(255-(i*15));  
        image(imagesArray[i], 0, 0, 100, 100);
        fill(255);
        text(authorArray[i], 10, 0);
      }
    }
    popMatrix();
  }
  imagePositionCount = 0;

  if(mousePressed){
    zoom = zoom-1;
    println(zoom);
  }
}//end of draw.



void netEvent(XMLRequest ml) {
  for(int i=0; i<15;i++){
    String[] mostRecentAuthorsAll = ml.getElementArray("author");
    authorString = mostRecentAuthorsAll[i];
    int startAuthor = authorString.indexOf("(");
    int endAuthor = authorString.indexOf(")", startAuthor);
    String author = authorString.substring(startAuthor+1, endAuthor);
    authorArray[i] = author;

    String[] mostRecentPostDateAll = ml.getElementArray("pubDate");
    pubDateString = mostRecentPostDateAll[i];
    int startPubDate = pubDateString.indexOf(",");
    int endPubDate = pubDateString.indexOf(":", startPubDate);
    String pubDate = pubDateString.substring(startPubDate+2, endPubDate-3);
    dateArray[i] = pubDate;

    String[] mostRecentPostTitle = ml.getElementArray("title");
    titleArray[i] = mostRecentPostTitle[i+2];
  }
}

void netEvent(HTMLRequest ml) {// this currently gets 2 results. Need to put this into a loop(?) to get it working for all the items in the feed.
  String feedHtml = ml.readRawSource();//gets the html from the feed.
  String[] CDATABLOCKS = split(feedHtml, "<item>");//splits the feed into item chunks. - the first chunk is the 'header'.
  for(int i=1; i<CDATABLOCKS.length; i++){//I have set i=1 to make it skip past the 'header' in slot[0].
    int startGetType = CDATABLOCKS[i].indexOf("posted");
    int endGetType = CDATABLOCKS[i].indexOf("<", startGetType);
    String typeAll = CDATABLOCKS[i].substring(startGetType+7, endGetType-1);

    if((typeAll.length()) == 8){
      println("it's an image!");
      typeArray[i-1] = "image";
    }else if((typeAll.length() == 6)){
      println("it's a link!");
      typeArray[i-1] = "link";
    }else if((typeAll.length() == 14)){
      println("it's a moving image");
      typeArray[i-1] = "video";
    }else if((typeAll.length() == 15) || (typeAll.length() == 14)){
      println("it must be a bit of text?..");
      typeArray[i-1] = "text";
    }
    int startGetCDATA = CDATABLOCKS[i].indexOf("CDATA[");
    int endGetCDATA = CDATABLOCKS[i].indexOf("]]",startGetCDATA);
    String CDATA = CDATABLOCKS[i].substring(startGetCDATA, endGetCDATA);

    int startGetLink = CDATA.indexOf("=");
    int endGetLink = CDATA.indexOf(">", startGetLink);
    String gotLink = CDATA.substring(startGetLink+2, endGetLink-1);
    if(gotLink.indexOf("h") == 0){
      imageLinkArray[i-1] = gotLink;
    }
    else{
      imageLinkArray[i-1] = null;
    }
  }
}

koogy

Re: Importing & moving images
Reply #8 - Jan 28^th, 2010, 12:42pm

ok, looking at that now (when i've installed xml library...)

in the meantime, try splitting the code that downloads the data every 30 seconds into a separate method that's called first thing in draw() and leave the drawing code where it is. this might be easier than working out what the if goes around.

ie

draw() {
checkWeb();
for (...) {
image(...);
}
}

void checkWeb() {
now = millis();
...
}

have also noticed that you're downloading the same file twice, once using xml request and once with html. kids today with their unlimited bandwidth...

Ladle

Re: Importing & moving images
Reply #9 - Jan 28^th, 2010, 1:13pm

Hehe thanks, you're really helping me out! - Yes I know it seems a bit strange to get the same source twice, but some of the bits of info i need are inside a CDATA block in the xml file, so im having to get the full html source also and parse it out from there.

I'll get onto those changes now, hopefully it's looking a bit more manageable anyway, been looking at it for a loong time today!

koogy

Re: Importing & moving images
Reply #10 - Jan 28^th, 2010, 2:04pm

this is a bit fiddly.

int startTime = 0;

at the top rather than setting it to millis in setup(). this'll force the update code to run the first time through to pick up the initial load of data.

unfortunately the xml requests don't stop the code so draw() keeps executing before you've got data, leading to NullPointerExceptions. so i added null pointer checks around the

image(imagesArray[i], 0, 0, 100, 100);
and
text(authorArray[i], 10, 0);

the way i have it currently reloads all 15 images every update (10 seconds) because you don't know how many more new entries you've just got. unfortunately this takes a while, long enough that the next update starts almost immediately after the last one finishes. it should keep track and just load the new images.

output looks a bit odd too, but then i'm not sure what it's meant to look like 8)

koogy

Re: Importing & moving images
Reply #11 - Jan 28^th, 2010, 3:50pm

changes so far

at the top:
int startTime = Integer.MIN_NUMBER; // force initial load

new method:
Code:


void checkForUpdate() {
    println("Update");
    int now = millis();//start counter for updates.
    if (now - startTime > 15000) { // update every 30 seconds
	req.makeRequest();
	xreq.makeRequest();
	println("Updating..");
	startTime = now;
	// reload all images
	for (int i = 0 ; i < 15 ; i++) {
	  if (imageLinkArray[i] != null){
  println("Loading new image: " + imageLinkArray[i]);
	    imagesArray[i] = loadImage(imageLinkArray[i], "jpg");
	  }
	}
    } else {
	println("Updating Nothing");
    }
  }

new draw:
Code:


void draw(){
  checkForUpdate();
  background(0);
  pushMatrix();
  imagePositionCount = 0;
  for(int i = 1; i < authorArray.length; i++){
    // get item from array
    Item item = (Item)items.get(i);
    if (item.type == "image") {
	imagePositionCount++;
	println(imagePositionCount+" images in top 15");
	translate(100, 0, -100);
    } else if(item.type == "text") {
	textPositionCount = textPositionCount+1;
	translate(100, 0, -100);
    } else if(item.type == "video") {
	videoPositionCount = videoPositionCount+1;
	translate(100, 0, -100);
    } else if(item.type == "link") {
	linkPositionCount = linkPositionCount+1;
	translate(100, 0, -100);
    }
    translate(0, 0, zoom);
    if (item.imageLink != null) {
	tint(255-(i*15));
	if (item.img != null) {
	  image(item.img, 0, 0, 100, 100);
	}
	fill(255);
	if (item.author != null) {
	  text(item.author, 10, 0);
	}
    }
  }
  popMatrix();

  if(mousePressed){
    zoom = zoom-1;
    println(zoom);
  }
}//end of draw.

and i've also moved
String[] mostRecentAuthorsAll = ml.getElementArray("author");
String[] mostRecentPostDateAll = ml.getElementArray("pubDate");
String[] mostRecentPostTitle = ml.getElementArray("title");
outside the loop in netEvent(XMLRequest ml) as they only need doing once.

images aren't being drawn until the *second* time through. not quite sure why. text looks dreadful, probably my fonts.

was toying with making the items a Class and having an ArrayList of them so i could add only the new items to the top and push everything else down easily. was a bit too invasive though. that and it's a quarter to midnight here... where's the 'sleepy' emoticon? 8)

PhiLho YaBB Moderator Offline Posts: 4190 Near Paris (France)	Re: Importing & moving images Reply #12 - Jan 29^th, 2010, 2:16am koogy wrote on Jan 27^th, 2010, 1:29pm: loadImage inside draw() is terribly slow Unless it is carefully made conditional to be done only every n seconds or every n frames.

Ladle

Re: Importing & moving images
Reply #13 - Jan 29^th, 2010, 8:32am

koogy wrote on Jan 28^th, 2010, 3:50pm:

changes so far...

Wow. Thanks! - I'm just going to tie up another bit of work then i'll get into this lot and see what you've suggested. I know, im not sure what the score is with the text, characters always seem to be horrible quality, and sometimes look like the top third of the letters has been chopped off:/ I assume createFont() would be too slow inside draw() - I had always assumed the text was nasty because it may have been translated in 3D..

koogy

Re: Importing & moving images
Reply #14 - Jan 29^th, 2010, 9:03am

my text is unreadable but it looks it's being drawn as single characters with the background of the one overwriting the foreground of the others. i tried making everything larger to see if that would help but it didn't.

oh, have thought of an easy way of only loading new images - check that the image array item is null first and only load if it is...

oh, and generally if you have this kind of thing:
String[] authorArray = new String[15];
String[] dateArray = new String[15];
String[] titleArray = new String[15];
String[] imageLinkArray = new String[15];
PImage[] imagesArray = new PImage[15];
String[] typeArray = new String[15];
it's time to start thinking about using a class...