Hello,
I'm writing a library for Processing with the intent of replacing Flash for some museum exhibits (most of of existing exhibits, using Flash, have very poor reliability). Basically, its a simplified GUI intended for controlling the behavior of screen elements for touchscreen kiosk setups. I'm not having any sort of problem with the library itself, though, but when the library tries to render its elements to the screen, I get very poor, choppy refresh rates. I've even written up my own image composting code to try to speed things up... that helped, but only marginally.
Let me describe what I am currently doing to render images to the screen: I have a global, static PImage object that is initialized to the size of my screen area (for the display I'm prototyping it's 1366x768). I also have a global list of top level container objects which hold all other screen elements (which can also include nested container objects). Each screen element designates a PImage that they will present to the screen at any given time.
The PImages for screen elements are loaded once, in the constructor, where there is an immediate loadPixels() call. The static PImage "screen" is encapsulated by a BBCanvas class that I wrote, which contains a reference to the "screen" paired up with a bounding box defined by the container object that invokes it. So each frame, when my library draws, it:
1) Discards the old PImage "screen," generating a new one from the PApplet object.
2) Invokes, in sequence, the top-level container objects to draw.
3) Each container object, when told to draw, creates a BBCanvas object for its screen area.
4) The container invokes a method to draw a background PImage on the BBCanvas, then tells each of its contained objects to draw.
5) The contained object does any sort of prep work to select a PImage to draw, then invokes the draw method on the BBCanvas of its container.
6) Once all objects have had a chance to draw, the "screen" PImage is drawn to the actual screen via PApplet.
The heavy lifting is done in a draw(BBObject, PImage) method I wrote for BBCanvas, so this is where I expect the slowdown is happening:
Where is my program slowing down? It works, but in my test program I have a container object that fades into view inside a backdrop container object, and the animation is very choppy. (I used the Ani library to animate the alpha variable on the object.) I had assumed that only ever blasting one PImage per frame would be fast, like at least 12 fps would be nice, but that doesn't seem to be the case. Is it just that I'm trying to do too many per-pixel calculations on too big a screen area (1366x768) and there just isn't any good way around that? It's hard to imagine that the sheer math is really that bad for a modern multi-core processor, but I'm at a loss as to what's bogging down.
I'm writing a library for Processing with the intent of replacing Flash for some museum exhibits (most of of existing exhibits, using Flash, have very poor reliability). Basically, its a simplified GUI intended for controlling the behavior of screen elements for touchscreen kiosk setups. I'm not having any sort of problem with the library itself, though, but when the library tries to render its elements to the screen, I get very poor, choppy refresh rates. I've even written up my own image composting code to try to speed things up... that helped, but only marginally.
Let me describe what I am currently doing to render images to the screen: I have a global, static PImage object that is initialized to the size of my screen area (for the display I'm prototyping it's 1366x768). I also have a global list of top level container objects which hold all other screen elements (which can also include nested container objects). Each screen element designates a PImage that they will present to the screen at any given time.
The PImages for screen elements are loaded once, in the constructor, where there is an immediate loadPixels() call. The static PImage "screen" is encapsulated by a BBCanvas class that I wrote, which contains a reference to the "screen" paired up with a bounding box defined by the container object that invokes it. So each frame, when my library draws, it:
1) Discards the old PImage "screen," generating a new one from the PApplet object.
2) Invokes, in sequence, the top-level container objects to draw.
3) Each container object, when told to draw, creates a BBCanvas object for its screen area.
4) The container invokes a method to draw a background PImage on the BBCanvas, then tells each of its contained objects to draw.
5) The contained object does any sort of prep work to select a PImage to draw, then invokes the draw method on the BBCanvas of its container.
6) Once all objects have had a chance to draw, the "screen" PImage is drawn to the actual screen via PApplet.
The heavy lifting is done in a draw(BBObject, PImage) method I wrote for BBCanvas, so this is where I expect the slowdown is happening:
- void draw(BBObject object, PImage image)
{
if (object == null || image == null)
return;
if (!object.canDisplay())
return;
if (image.width < 1 || image.height < 1)
return;
// Drawing surface bounding coordinates
int x1, y1, x2, y2;
x1 = object.x;
if (x1 < 0)
x1 = cornerX;
else
x1 += cornerX;
y1 = object.y;
if (y1 < 0)
y1 = cornerY;
else
y1 += cornerY;
x2 = object.x + object.width - 1;
if (x2 > (width - 1))
x2 = cornerX + width - 1;
else
x2 += cornerX;
y2 = object.y + object.height - 1;
if (y2 > (height - 1))
y2 = cornerY + height - 1;
else
y2 += cornerY;
// Bind drawing surface to usable surface of canvas
// Return if the image we want to display can't appear on the screen image
if (x1 < 0)
x1 = 0;
else if (x1 >= screen.width)
return;
if (y1 < 0)
y1 = 0;
else if (y1 >= screen.height)
return;
if (x2 < 0)
return;
else if (x2 >= screen.width)
x2 = screen.width - 1;
if (y2 < 0)
return;
else if (y2 >= screen.height)
y2 = screen.height - 1;
// Paint each pixel of the bound drawing surface.
int[] pixelSrc = image.pixels;
int[] pixelDest = screen.pixels;
for (int j = y1; j <= y2; j++)
for (int i = x1; i <= x2; i++)
{
// Calculate the source pixel coordinates
float xSrc = (float)(i - object.x - cornerX) / (float)(object.width - 1) * (float)(image.width - 1);
float ySrc = (float)(j - object.y - cornerY) / (float)(object.height - 1) * (float)(image.height - 1);
// Grab the source pixel color data
int colorSrc = pixelSrc[(Math.round(ySrc) * image.width) + Math.round(xSrc)];
// Calculate effective alpha value of the pixel
float alphaSrc = (float)((colorSrc >> 24) & 0xFF) * (float)(object.alpha) / 65025.0f * alpha;
float alphaDest = 1.0f - alphaSrc;
// Grab the destination pixel to blend with
int colorDest = pixelDest[(j * screen.width) + i];
// Now set the color of the actual pixel
pixelDest[(j * screen.width) + i] =
255 << 24 |
(int)(((float)((colorSrc >> 16) & 0xFF) * alphaSrc) + (((float)((colorDest >> 16) & 0xFF) * alphaDest))) << 16 |
(int)(((float)((colorSrc >> 8) & 0xFF) * alphaSrc) + (((float)((colorDest >> 8) & 0xFF) * alphaDest))) << 8 |
(int)(((float)(colorSrc & 0xFF) * alphaSrc) + (((float)(colorDest & 0xFF) * alphaDest)));
}
}
Where is my program slowing down? It works, but in my test program I have a container object that fades into view inside a backdrop container object, and the animation is very choppy. (I used the Ani library to animate the alpha variable on the object.) I had assumed that only ever blasting one PImage per frame would be fast, like at least 12 fps would be nice, but that doesn't seem to be the case. Is it just that I'm trying to do too many per-pixel calculations on too big a screen area (1366x768) and there just isn't any good way around that? It's hard to imagine that the sheer math is really that bad for a modern multi-core processor, but I'm at a loss as to what's bogging down.
1