RaspberryPi Processing 3 Performance Issues for Loading images on runtime

Bhargav

I am using processing 3.2.1 on RasbperryPi 3

In one of my programs I am loading three images on runtime from set of images in a folder. The images are changing runtime. Raspberrypi is fast enough to run the code and atleast load three images of each less then 150kb and of resolution 1260x760.

But my output becomes extremely slow while loading these images together. Loading one image runs smooth, two slower and three super slow. I have no guess about why is processing not able to run fast when Raspberypi is capable to running the code. The same code runs with perfect speed on my laptop in windows.

Part of the code is as under :

  //Load Images
  imgA = loadImage(path+ displayuserid +"/"+ displayobjectid +"/" + (imageno)%36 +  ".jpg");
  imgB = loadImage(path+ displayuserid +"/"+ displayobjectid +"/" + (imageno+1)%36 +  ".jpg");
  imgC = loadImage(path+ displayuserid +"/"+ displayobjectid +"/" + (imageno+2)%36 +  ".jpg");     


    //Display Images
    imgA.resize(0,250);
    image(imgA, width/2 - imgA.width/2, height/2 + height/2 - imgA.height);      


     pushMatrix();
     translate(width/2,height/2);
     rotate(90*TWO_PI/360);
     imgB.resize(0,250);
     image(imgB, -imgB.width/2, height/2-imgB.height);
     popMatrix();

     pushMatrix();
     translate(width/2,height/2);
     rotate(-90*TWO_PI/360);
     imgC.resize(0,250);
     image(imgC, -imgC.width/2, height/2-imgC.height);
     popMatrix();  

koogs

Is that all in draw ()?

Never load images in draw.

And ditch the resize () - why resize the images every single frame?

Bhargav

@Koogs yes its all in draw().......

if I don't load in draw then how do I load images from dynamic path at run time except draw?

and resize is required because source images are big and display area needs to accommodate three images in small size of area....this can be resolve by saving images in multiple resolution while saving...but any alternative for this also?

I've tried Without resize also the output is still slow...so i think resize's role in reducing performance might be negligible

gohai

Try doing it like this:

PImage imgA;

void setup() {
  imgA = loadImage("...");
  imgA.resize(0, 250);
}

void draw() {
  image(imgA, 0, 0);
}

Bhargav

@gohai thanks for the response but in my code imgA has a dynamic path ...based on user's need differrent image will be loaded and again there are 3 images together.....in that case how do I go about it?

kfrajer

Only load it once when the path is available. The key point is to reduce the number of times you load the image in the case you do it in draw. For example, lets assume you load the image after a mouse action:

PImage imgA;
boolean loadImgNow=false;

void setup() {
  size(400,600);  
}

void draw() {
  if(loadImgNow==true){
    imgA = loadImage("...");
    loadImgNow=false;
    imgA.resize(0, 250);
  }
  image(imgA, 0, 0);
}

void mouseReleased(){
  //... steps to obtain the image: read it from a file or get it from the user
  loadImgNow=true;
}

This is just a possible way to do things. I hope it helps,

Kf

Bhargav

@kfrajer that'll be a good technique to speedup execution for processes other then image loading. But for my program loading image is the key activity. Multiple images are loaded and displayed on screen dynamically. And the core what i think is the processor on Raspberrypi3 is capable enough to execute it but it is still working slowly....and I am not able to get if the issue is with image loading function, processing performance on pi or just on logic level?

koogs

That's pointless though. Do it all in setup.

Gohai's code is obviously an example with one image. You need to convert it to handle your 36 images.

GoToLoop

• If it's affordable (enough RAM or time) to load all resources within setup(), do so! L-)
• Also if a PImage needs resize(), do so inside setup() too!
• Loading resources once draw() starts is slow, unless we separate a thread() to deal with it.

Bhargav

@koogs @GoToLoops @gohai I understand that solution and I was using the same until I had limited number and fix set of images.

to elaborate mMy program has evolved to basically be a system which is running on Raspberry-Pi. While running it is capturing images, storing it loading images from recently captured ones. Total images can be in range of hundreds but from that on screen I need to load only 3 images which are also changing every second. So declaration in setup doesn't work as all image doesn't exist in begining.

jeremydouglass

@Bhargav I think you may not be understanding the key issue here.

By putting loadImage in draw you are loading the same image over and over 60 times a second and 60 new times the next second and then 60 times again, and again, and again, and again. Unless you are changing which image you load 60 times a second, this is... not smart.

• You want to load a new image each time the mouse is clicked? Load each new image with mouseClicked().
• You want to load a new image each second? Create a separate function that is called once a second to load new images.
• You want to load an image each time the use presses a key? Load new image with keyPressed().
• etc. etc.

This has nothing to do with you using a lot of images or loading often. Even a program that loads 100 new images each second should not put loadImage in draw. If it did, it would load images 0-99 from the hard disk, then load those same images 59 more times in a row for no reason -- then on the next second it would load images 100-199 from the hard disk, then it would reload those identical images again 59 more times in a row for no reason, etc.

Bhargav

@jeremydouglass thanks for the elaborating this in details...i got the point...I will try loading images outside draw function and share the results here soon...

jeremydouglass

Good luck! Do share your code -- happy to help if you run into further problems.

Bhargav

This one shows images for a 3 side hologram from set of 10 images. It changes set based on number of key pressed and shows next image within set when space is pressed. This one works good! It takes hault/stops for almost 5 seconds everytime while switching to another set, I don't know how to fasten that...

@jeremydouglass here's the code..Used suggestions given by all of you. Thanks a lot!

@koogs @GoToLoops @gohai @kfrajer this is what i am trying to do....and dynamically load more then 100 set each with 10= images! Current code has 2 sets with 10 miages...Any tricks for more optimization?

int numFrames = 10;  
PImage[] images = new PImage[numFrames];
PImage imgA, imgB, imgC;
int i=0;
String status="ready";

void setup() {
  size(1920, 960);
  loadSet(0);
} 

void draw() { 


  imgA = images[i];
  imgB = images[(i+3)%numFrames];
  imgC = images[(i+8)%numFrames];

  //Display Images
  //imgA.resize(0,250);
  image(imgA, width/2 - imgA.width/2, height/2 + height/2 - imgA.height);         

  //top
  pushMatrix();
  translate(width/2, height/2);
  rotate(90*TWO_PI/360);
  //imgB.resize(0,250);
  image(imgB, -imgB.width/2, height/2-imgB.height);
  popMatrix(); 

  //Left   
  pushMatrix();
  translate(width/2, height/2);
  rotate(-90*TWO_PI/360);
  //imgC.resize(0,250);
  image(imgC, -imgC.width/2, height/2-imgC.height);
  popMatrix();
}


void loadSet(int set)
{

  if (set==0)
  {   


    images[0]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/0/0.jpg");
    images[0].resize(0, 250);
    images[1]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/0/1.jpg");
    images[1].resize(0, 250);
    images[2]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/0/2.jpg");
    images[2].resize(0, 250);
  //same for 3,4,5,6,7,8....
    images[9]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/0/9.jpg");
    images[9].resize(0, 250);
  } else if (set==1)
  {   


    images[0]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/1/0.jpg");
    images[0].resize(0, 250);
    images[1]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/1/1.jpg");
    images[1].resize(0, 250);
    images[2]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/1/2.jpg");
    images[2].resize(0, 250);
     //same for 3,4,5,6,7,8....
    images[9]  = loadImage("/home/pi/sketchbook/Optimized_Display/data/1/9.jpg");
    images[9].resize(0, 250);
  }
}

void keyPressed() {

  if (key=='0')
    loadSet(0);
  else if (key=='1')
    loadSet(1);
  else if (key==' ')
  {
    i++;          
    if (i==numFrames)
    { 
      i=0;
    }
  }
}

jeremydouglass

@Bhargav -- some suggestions for refactoring your code to make it easier to work with multiple sets.

1. create a global array String[] imageSets which contains a list of directories -- one for each set
2. loadSet() should take three arguments: an index to imageSets (0,1,2), and a resize width / height. It should then load all the images in that directory and resize them.
3. keyPressed should check to see if the key pressed is a number less than the length of imageSets. If it is, it should call loadSet().

Re:optimization. How do you plan on loading your 100 sets? Probably not with keyboard buttons? With next / previous arrows, or on a click? If you know the order in which sets will load, or if the sets are going to load at random, you could pre-load the next set and then swap them as a way of increasing performance.

Of course, you could also just preload all 1000 images during setup, but that will probably require a large amount of RAM.

If you are working from a fixed image set and you are always shrinking them down to the same size then you should also consider transforming your images to thumbnail size (e.g. x250) once before the program runs, saving those images, and running your sketch on the thumbnails. Right now you are wasting a ton of processing power loading large images and resizing them over and over -- just use saved thumbnails.

Bhargav

@jeremydouglass

Thanks again for the detailed response.

After trying multiple things I am reaching to following conclusions :

[a] All image sets are dynamically created. So once the device with R's pi starts there might be few 50 - 100 sets. One part of the code (which is not here) will keep making such sets and the code above will display recently captured set. So in this scenario at any point of time it is not fixed that how many sets will be there! And the total RAM remains constant only 1 GB. So in this scenario pre-loading all will not work because RAM is limited and sets are also created run time only and they are many (seriously I cant decide quantity they can be 10 - 100 - 500 or range!)

[b] I can only and I have to load one set at a time which i want to display next. And this set (id) will be decided run time. So I can only load all images in one set together. as the example i have tried. And loading images will be taken care only when required and out of draw method.

[c] After incorporating all above things I can display images within one set properly. but every time I change the set it is taking sometime almost 8 to 10 seconds to load new images! Which I believe might be the speed of loading images while runtier on processing running on raspberry pi.

Some more Possibilities :

[a] While one set is being displayed means while processing is already running draw function and displaying images, How can i run a parallel chunk of code which can possibly load next probable set of images? I don't know how this can happen in processing? Means loading more resources while some other code is already running.

[b] thrugh some offline discussions I got to know that may be in this case processing is not able to use full RAM and GPU memories of Raspberry-Pi. I can see 30-60% use of processor while I run my entire code, and the current code is just part of it. I dont know how to change such processing - raspbeerrypi memory access issues.

koogs

If memory and CPU is a problem then resize the images before loading them, don't do it as part of the load process (as Jeremy says).

I've seen jcache used to store images in situations like this, it uses a mix of memory and disk to store more images than memory alone can handle. And does it better than anything you can probably think up yourself.

koogs

Previous thread: https://forum.processing.org/one/topic/getting-multiple-images-to-queue-and-load.html