I'm writing a library for Processing with the intent of replacing Flash for some museum exhibits (most of of existing exhibits, using Flash, have very poor reliability). Basically, its a simplified GUI intended for controlling the behavior of screen elements for touchscreen kiosk setups. I'm not having any sort of problem with the library itself, though, but when the library tries to render its elements to the screen, I get very poor, choppy refresh rates. I've even written up my own image composting code to try to speed things up... that helped, but only marginally.
Let me describe what I am currently doing to render images to the screen: I have a global, static PImage object that is initialized to the size of my screen area (for the display I'm prototyping it's 1366x768). I also have a global list of top level container objects which hold all other screen elements (which can also include nested container objects). Each screen element designates a PImage that they will present to the screen at any given time.
The PImages for screen elements are loaded once, in the constructor, where there is an immediate loadPixels() call. The static PImage "screen" is encapsulated by a BBCanvas class that I wrote, which contains a reference to the "screen" paired up with a bounding box defined by the container object that invokes it. So each frame, when my library draws, it:
1) Discards the old PImage "screen," generating a new one from the PApplet object.
2) Invokes, in sequence, the top-level container objects to draw.
3) Each container object, when told to draw, creates a BBCanvas object for its screen area.
4) The container invokes a method to draw a background PImage on the BBCanvas, then tells each of its contained objects to draw.
5) The contained object does any sort of prep work to select a PImage to draw, then invokes the draw method on the BBCanvas of its container.
6) Once all objects have had a chance to draw, the "screen" PImage is drawn to the actual screen via PApplet.
The heavy lifting is done in a draw(BBObject, PImage) method I wrote for BBCanvas, so this is where I expect the slowdown is happening:
void draw(BBObject object, PImage image) { if (object == null || image == null) return; if (!object.canDisplay()) return; if (image.width < 1 || image.height < 1) return;
// Bind drawing surface to usable surface of canvas // Return if the image we want to display can't appear on the screen image if (x1 < 0) x1 = 0; else if (x1 >= screen.width) return; if (y1 < 0) y1 = 0; else if (y1 >= screen.height) return; if (x2 < 0) return; else if (x2 >= screen.width) x2 = screen.width - 1; if (y2 < 0) return; else if (y2 >= screen.height) y2 = screen.height - 1;
// Paint each pixel of the bound drawing surface. int[] pixelSrc = image.pixels; int[] pixelDest = screen.pixels; for (int j = y1; j <= y2; j++) for (int i = x1; i <= x2; i++) { // Calculate the source pixel coordinates float xSrc = (float)(i - object.x - cornerX) / (float)(object.width - 1) * (float)(image.width - 1); float ySrc = (float)(j - object.y - cornerY) / (float)(object.height - 1) * (float)(image.height - 1); // Grab the source pixel color data int colorSrc = pixelSrc[(Math.round(ySrc) * image.width) + Math.round(xSrc)]; // Calculate effective alpha value of the pixel float alphaSrc = (float)((colorSrc >> 24) & 0xFF) * (float)(object.alpha) / 65025.0f * alpha; float alphaDest = 1.0f - alphaSrc; // Grab the destination pixel to blend with int colorDest = pixelDest[(j * screen.width) + i]; // Now set the color of the actual pixel pixelDest[(j * screen.width) + i] = 255 << 24 | (int)(((float)((colorSrc >> 16) & 0xFF) * alphaSrc) + (((float)((colorDest >> 16) & 0xFF) * alphaDest))) << 16 | (int)(((float)((colorSrc >> 8) & 0xFF) * alphaSrc) + (((float)((colorDest >> 8) & 0xFF) * alphaDest))) << 8 | (int)(((float)(colorSrc & 0xFF) * alphaSrc) + (((float)(colorDest & 0xFF) * alphaDest))); } }
So basically what I'm doing, is going pixel by pixel in the bounding rectangle of what's supposed to appear on screen, grabbing the pixel data that's already on the "screen" image, blending it with the appropriate pixel from my source image (taking into account both the alpha of the pixel itself, and an alpha value you can set on each object to fade the whole thing) and then writing the result back to the "screen" image.
Where is my program slowing down? It works, but in my test program I have a container object that fades into view inside a backdrop container object, and the animation is very choppy. (I used the Ani library to animate the alpha variable on the object.) I had assumed that only ever blasting one PImage per frame would be fast, like at least 12 fps would be nice, but that doesn't seem to be the case. Is it just that I'm trying to do too many per-pixel calculations on too big a screen area (1366x768) and there just isn't any good way around that? It's hard to imagine that the sheer math is really that bad for a modern multi-core processor, but I'm at a loss as to what's bogging down.
