Copying and Resizing PImages: get() in Processing 2.1

Ignotus

Has there been any change to the behavior of PImage.get() in Processing 2.1? I have been working with 2.0x and copying and resizing images thusly:

fitImg = img.get();
fitImg.resize(scaledWidth, scaledHeight);
if (null != myFrame) myFrame.setSize(scaledWidth, scaledHeight + 48);

In 2.1, the image was displaying as a blank (white) display. I corrected the problem by replacing img.get() with:

fitImg = copyImagePixels(img);

where copyImagePixels is:

public PImage copyImagePixels(PImage image) {
    int h = image.height;
    int w = image.width;
    int i = 0;
    PImage newImage = createImage(w, h, ARGB);
    newImage.loadPixels();
    for (i = 0; i < image.pixels.length; i++) {
        newImage.pixels[i] = image.pixels[i];
    }
    newImage.updatePixels();
    return newImage;
}

I'm developing in Eclipse, but the problem also occurs in Processing 2.1, too. It does not occur in Processing 2.0x.

Why would get() no longer behave the same way? Has the way that alpha channels are copied changed? Or is it something else?

Puzzled, and wondering how much of my code has depended on the previous behavior of get() and may have to be changed.

cheers,

-- Paul

Poersch

Don't know about get()'s behavior in the Eclipse environment, but the following code works perfectly in the Processing environment. Plus the getCopy() function is faster than Processing's get():

PImage img, fitImg;

void setup() {

    img = loadImage("h t t p://upload.wikimedia.org/wikipedia/commons/9/98/Information_magnifier_icon.png".replace(" ", ""));

    // Works
    //fitImg = img.get();
    //fitImg.resize(width, height);

    // Works (image is aliased)
    //fitImg = createImage(width, height, ARGB);
    //fitImg.copy(img, 0, 0, img.width, img.height, 0, 0, fitImg.width, fitImg.height);

    // Works and is in fact faster than Processing's get() function (because of less overhead)
    fitImg = getCopy(img);
    fitImg.resize(width, height);

    image(fitImg, 0, 0);

}

public PImage getCopy(PImage image) {
    PImage newImage = createImage(image.width, image.height, image.format);
    newImage.loadPixels();
    System.arraycopy(image.pixels, 0, newImage.pixels, 0, image.pixels.length);
    newImage.updatePixels();
    return newImage;
}

Ignotus

Thanks, but your solution is pretty much exactly what I am doing. It clearly has less overhead than get() (https://github.com/processing/processing/blob/master/core/src/processing/core/PImage.java), so that is useful. Maybe the System.arraycopy call will be somewhat optimized compared with my loop, maybe not. In any case, it leaves the fundamental question of how get() behaves in 2.1 unanswered. I'll work up an app that demonstrates the problem. more precisely.

Poersch

Java's arraycopy() is native and therefore implemented in platform-dependent code (mostly C, C++ or an assembly language) which reduces it's overhead to a minimum. Additionally, instead of copying every single array element, it copys whole memory areas. So, yes, it is faster - roughly twice as fast - depending on your platform.

Yes, a more detailed example would be great. :)

Edit: Does get() work in your Processing 2.1 environment (it works for me)? Just to eliminate JRE/OS based problems.

GoToLoop

Processing canonized System.arraycopy() as arrayCopy() FYI. ;;)
http://processing.org/reference/arrayCopy_.html

Ignotus

Okay, here is a test application that demonstrates the problem. It resizes an image to fit the screen. If you use PImage.get in Processing 2.1, the resized image is blank. In 2.0.1 it works. If you use arrayCopy, everything works in both versions. Eclipse behaves the same way, depending on which version of processing.core I am using.

At the very least, there's an inconsistency in the behavior of PImage.get between Processing 2.0.1 (old copy I have) and Processing 2.1. I haven't tested in 1.5.1, but assume it will behave as in 2.0.1.

The app gives you a file dialog to open an image. Once open, you can toggle resizing the window to fit the screen with an 'F' keypress. 'O' will open another file. 'G' will toggle between using PImage.get and arrayCopy, and revela the problem.

Obviously, there are all sorts of good reasons to use arrayCopy, as comments have pointed out. OTOH, inconsistencies of this sort worry me, in case they point other weirdness ("bugs" rather than "features").

Thanks for the comments. I'd be real interested to hear if my problems are replicated.

The code style munges my JavaDoc comments with HTML markup, but I'll leave them in as they may be useful.

import java.awt.Container;
import java.awt.Frame;
import java.io.File;

// Test for possible bug in PImage.get method.
// by Paul Hertz, 2014
// http://paulhertz.net/

/** the primary image to display and glitch */
PImage img;
/** a version of the image scaled to fit the screen dimensions */
PImage fitImg;
/** true if image should fit screen, otherwise false */
boolean isFitToScreen = false;
/** maximum width for the display window */
int maxWindowWidth;
/** maximum height for the display window */
int maxWindowHeight;
/** width of the image when scaled to fit the display window */
int scaledWidth;
/** height of the image when scaled to fit the display window */
int scaledHeight;
/** reference to the frame (display window) */
Frame myFrame;
/** current width of the frame (display window) */
int frameWidth;
/** current height of the frame (display window) */
int frameHeight;
/** the selected file */
File displayFile;
/** toggle use of get() or of copyImagePixels() */
boolean isUseGet = true;
/** display verbose messages if verbose == true */
boolean verbose = false;



public void setup() {
  println("Display: "+ displayWidth +", "+ displayHeight);
  size(640, 480);
  smooth();
  // max window width is the screen width
  maxWindowWidth = displayWidth;
  // leave window height some room for title bar, etc.
  maxWindowHeight = displayHeight - 56;
  // image to display
  img = createImage(width, height, ARGB);
  chooseFile();
  // Processing initializes the frame and hands it to you in the "frame" field.
  // Eclipse does things differently. Use findFrame method to get the frame in Eclipse.
  myFrame = findFrame();
  myFrame.setResizable(true);
  // the first time around, window won't be resized, a reload should resize it
  if (null != displayFile) {
    loadFile();
    if (isFitToScreen) fitPixels(true);
  }
  printHelp();
}


/**
 * @return   Frame where Processing draws, useful method in Eclipse
 */
public Frame findFrame() {
  Container f = this.getParent();
  while (!(f instanceof Frame) && f!=null)
    f = f.getParent();
  return (Frame) f;
}


public void printHelp() {
  println("Test for possible bug in PImage.get method.");
  println("Press 'F' to toggle image size to screen.");
  println("Press 'G' to toggle use of Pimage.get or arrayCopy.");
  println("Press 'O' to open a new file.");
}


public void draw() {
  if (isFitToScreen) {
    image(fitImg, 0, 0);
  }
  else {
    background(255);
    image(img, 0, 0);
  }
}


public void keyPressed() {
  if (key == 'f' || key == 'F') {
    fitPixels(!isFitToScreen);
  }
  else if (key == 'o' || key == 'O') {
    chooseFile();
  }
  else if (key == 'g' || key == 'G') {
    isUseGet = !isUseGet;
    if (isUseGet) println("Using PImage.get method.");
    else println("Using arrayCopy method");
  }
}


/**
 * Copy an image pixel by pixel, test code to work around problems in Processing 2.1 with PImage.get.
 * @param image   image to copy
 * @return        a copy of the image submitted
 */
public PImage copyImagePixels(PImage image) {
  int h = image.height;
  int w = image.width;
  PImage newImage = createImage(w, h, ARGB);
  newImage.loadPixels();
  arrayCopy(image.pixels, newImage.pixels);
  newImage.updatePixels();
  return newImage;
}


/**
 * Fits images that are too big for the screen to the screen, or displays as much of a large image 
 * as fits the screen if every pixel is displayed. There is still some goofiness in getting the whole
 * image to display--bottom edge gets hidden by the window. It would be good to have a scrolling window.
 * 
 * @param fitToScreen   true if image should be fit to screen, false if every pixel should displayed
 */
public void fitPixels(boolean fitToScreen) {
  if (fitToScreen) {
    fitImg = createImage(img.width, img.height, ARGB);
    scaledWidth = fitImg.width;
    scaledHeight = fitImg.height;
    // TODO this is where the fun begins
    if (isUseGet) {
      fitImg.loadPixels();
      fitImg = img.get();
      fitImg.updatePixels();
    }
    else {
      fitImg = copyImagePixels(img);
    }
    // We calculate the relative proportions of window and image. 
    // ratio of the window height to the window width
    float windowRatio = maxWindowHeight/(float)maxWindowWidth;
    // ratio of the image height to the image width
    float imageRatio = fitImg.height/(float)fitImg.width;
    if (verbose) {
      println("maxWindowWidth "+ maxWindowWidth +", maxWindowHeight "+ maxWindowHeight +", screen ratio "+ windowRatio);
      println("image width "+ fitImg.width +", image height "+ fitImg.height +", image ratio "+ imageRatio);
    }
    if (imageRatio > windowRatio) {
      // image is proportionally taller than the display window, 
      // so scale image height to fit the window height
      scaledHeight = maxWindowHeight;
      // and scale image width by window height divided by image height
      scaledWidth = Math.round(fitImg.width * (maxWindowHeight / (float)fitImg.height));
    }
    else {
      // image is proportionally equal to or wider than the display window, 
      // so scale image width to fit the window width
      scaledWidth = maxWindowWidth;
      // and scale image height by window width divided by image width
      scaledHeight = Math.round(fitImg.height * (maxWindowWidth / (float)fitImg.width));
    }
    fitImg.resize(scaledWidth, scaledHeight);
    // 48 pixels compensate for window decorations such as bar, etc.
    if (null != myFrame) myFrame.setSize(scaledWidth, scaledHeight + 48);
  }
  else {
    scaledWidth = img.width;
    scaledHeight = img.height;
    if (null != myFrame) {
      frameWidth = scaledWidth <= maxWindowWidth ? scaledWidth : maxWindowWidth;
      frameHeight = scaledHeight <= maxWindowHeight ? scaledHeight : maxWindowHeight;
      myFrame.setSize(frameWidth, frameHeight + 38);
    }
  }
  if (verbose) println("scaledWidth = "+ scaledWidth +", scaledHeight = "+ scaledHeight +", frameWidth = "+ frameWidth +", frameHeight = "+ frameHeight);
  isFitToScreen = fitToScreen;
}


/**
 * @return   true if a file reference was successfully returned from the file dialogue, false otherwise
 */
public void chooseFile() {
  selectInput("Choose an image file.", "displayFileSelected");
}

public void displayFileSelected(File selectedFile) {
  File oldFile = displayFile;
  if (null != selectedFile && oldFile != selectedFile) {
    noLoop();
    displayFile = selectedFile;
    loadFile();
    if (isFitToScreen) fitPixels(true);
    loop();
  }
  else {
    println("No file was selected");
  }
}


/**
 * loads a file into variable img.
 */
public void loadFile() {
  println("\nselected file "+ displayFile.getAbsolutePath());
  img = loadImage(displayFile.getAbsolutePath());
  fitPixels(isFitToScreen);
  println("image width "+ img.width +", image height "+ img.height);
}

amnon

In this thread, I see a lot of resize() calls and a lot of problems with transparency, except when an ARGB PImage is created. I believe you may be experiencing the same issue as filed here: https://github.com/processing/processing/issues/2228

For an analysis of the problem see the pull request I have made, which also contains a proposed fix for this issue, here: https://github.com/processing/processing/pull/2312

Ignotus

Thanks, amnon! That makes a lot of sense. I'm glad to see you've proposed a fix.

Ignotus

And in fact, if I write to the image alpha right after resizing the image, the problem goes away. That is, however, just a workaround.

public int[] rgbComponents(int argb) {
  int[] comp = new int[3];
  comp[0] = (argb >> 16) & 0xFF;  // Faster way of getting red(argb)
  comp[1] = (argb >> 8) & 0xFF;   // Faster way of getting green(argb)
  comp[2] = argb & 0xFF;          // Faster way of getting blue(argb)
  return comp;
}

public PImage loadImageAlpha(PImage image, int alpha) {
  int i = 0;
  image.loadPixels();
  for (i = 0; i < image.pixels.length; i++) {
    int[] rgb = rgbComponents(image.pixels[i]);
    image.pixels[i] = alpha << 24 | rgb[0] << 16 | rgb[1] << 8 | rgb[2];
  }
  image.updatePixels();
  return image;
}

// ...

fitImg.resize(scaledWidth, scaledHeight);
loadImageAlpha(fitImg, 255);

Of course, I can speed the remedy up a little:

 image.pixels[i] = alpha << 24 | image.pixels[i];