Processing 2 sketch not using the graphics card for rendering....

GoToLoop

Re: Processing 2 sketch not using the graphics card for rendering....

6 months ago

Perhaps when using Processing 2+ versions, use either JAVA2D or OPENGL graphics engines.

Since P2D & P3D are Processing 1.5.1- exclusive!

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

6 months ago

Really? Then I guess the reference page for size() (the one that says "This reference is for processing 2.0+." at the top) is really utterly misleading, since it clearly states that:

" The renderer parameter selects which rendering engine to use. For example, if you will be drawing 3D shapes, use P3D . In addition to the default renderer, other renderers are:

P2D (Processing 2D): A renderer that supports two-dimensional drawing.

P3D (Processing 3D): 3D graphics renderer that makes use of OpenGL-compatible graphics hardware.

PDF : The PDF renderer draws 2D graphics directly to an Acrobat PDF file. This produces excellent results when you need vector shapes for high-resolution output or printing. You must first use Import Library → PDF to make use of the library. More information can be found in the PDF library reference."

I also clearly recall having to use the JAVA2D and OPENGL in previous processing 1.5+ sketches to get them to use opengl, so I think you might have this the wrong way round, GoToLoop.

(I actually tried setting the rendered to OPENGL and, while it didn't fall over, it also still did not use the graphics card hardware at all).

GoToLoop

Re: Processing 2 sketch not using the graphics card for rendering....

6 months ago

I confess I'm not following beta development. I'm waiting a little longer for it to become stable!

Just seen size() reference link and verified its latest update date:

Updated on February 19, 2013 07:58:51pm EST

It doesn't mention which is the default renderer. But most probably is JAVA2D as always has been!

Since I'm pretty sure original P2D & P3D engines were axed out for good,

1 or both of them are actually either JAVA2D or OPENGL! That's what I've meant by "v1.5.1 exclusive"!

I guess we both need to w8 for some1 w/ more updated news on how's beta is going on by now!

akiersky

Re: Processing 2 sketch not using the graphics card for rendering....

6 months ago

Hi Nick,

Performance in opengl is going to depend a lot on both your code and your graphics card. of course using opengl for drawing is only going to improve the drawing of your sketch, if you have lots of reading and writing to memory it's going to slow your sketch down a lot.

println's in the draw loop (nested loops are especially bad) can significantly tax the processor and slow the rest of the sketch down.

got some code and specs to help diagnose?

-ak

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

6 months ago

Hi akiersky,

Its not performance I'm worried about per se, its the fact that the sketch isn't using the graphics card at all! Once we have it using opengl properly, then we'll worry about performance....

The codebase for the sketch is pretty chunky so I'll have hack out a core bit for a test tomorrow and stick it up here...

For specs... well, we've tried it on a few machines but the main one we're using is:

PowerColor HD 6870 2GB GDDR5 Eyefinity 6 Edition

AMD FX-6 6100 Black Edition 6 Core 3.3Ghz Socket AM3+ 8MB L3 Cache

Corsair 8GB DDR3 1600MHz XMS3

Asus M5A99X EVO 990X Socket AM3+

Microsoft Windows 7 Home Premium 64 Bit

60GB SSD

64bit java 1.7

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

OK, here is some test code. The code below starts by drawing a black background to which it adds red dots at random location until 100,000 dots is reached (then it stops adding dots). each dot is actually 9 pixels in size:

[[0, 1, 2]

[3, 4, 5]

[6, 7, 8]]

with pixels 0, 2, 6, & 8 remaining the background colour, pixel 4 a bright colour (red) and pixels 1, 3, 5 & 7 a darker version of the same colour.

To draw the dots incrementally, as fast as possible, it creates a set of 100,000 random screen coordinated during setup() then, each frame, it updates a master reference array (called master_pixels) with just new dots to draw that frame. This array already contains all the previously drawn dots so we're only altering a few hundred elements each frame. Then it copies the whole of the master_pixels array to the pixels buffer (using a loadPixels() and updatePixels() call).

The settings from lines 14-18 can be used to test the performance.

import java.util.Arrays;
import java.util.Collections;
import java.util.List;
Integer n_points_done, j, n_frame_av;
float frame_time, frame_end, n_points_2_draw, av_frame_time;
List<Integer> coords;
color[] master_pixels;
Boolean loadedpix;
int[] points_info;
String display_fr;
String renderer = "P2D";
int x = 1000;
int y = 1000;
int max_points = 100000;
float points_ps = 2000;
void setup() {
if (renderer=="P2D") {
size(x,y,P2D);
} else if (renderer=="P3D") {
size(x,y,P3D);
} else if (renderer=="OPENGL") {
size(x,y,OPENGL);
} else {
exit();
}
// setup coordinate space
Integer[] tcoords = new Integer[(width/3)*(height/3)];
int x = 0;
int y = 0;
for (int i=0; i<tcoords.length; i++) {
tcoords[i] = (y*width)+x;
x=x+3;
if (x>(width-3)) {
x=0;
y=y+3;
}
}
coords = Arrays.asList(tcoords);
Collections.shuffle(coords);
//starting settings
frame_time = 0.017;
av_frame_time = 0.017;
n_points_2_draw = 0;
n_points_done=0;
loadedpix=false;
j=0;
display_fr = "";
n_frame_av = 1;
}
void draw() {
background(0);
if (frameCount>1) {
//println(frameCount);
points_info = display_RandomPoints(frame_time);
//println(points_info);
if (j==n_frame_av) {
String display_renderer = "Current Renderer: "+renderer;
String display_fps = "Frame rate: "+nfc(n_frame_av/av_frame_time, 1)+" frames/s (frame time: "+nfc(av_frame_time/n_frame_av, 3)+"s)";
String display_points_info0 = "Number of points to be drawn: "+str(max_points);
String display_points_info1 = "Points added this frame: "+str(points_info[0]);
String display_points_info2 = "Total points drawn so far: "+str(points_info[1]);
display_fr = display_renderer+"\n"+display_fps+"\n"+display_points_info0+"\n"+display_points_info1+"\n"+display_points_info2;
av_frame_time = frame_time;
j=0;
//println(display_fr);
//exit();
} else {
av_frame_time = av_frame_time + frame_time;
}
j++;
textSize(20);
textAlign(LEFT,TOP);
fill(255);
text(display_fr, 10, 10);
}
frame_time = (millis()-frame_end)/1000;
frame_end = millis();
}
int[] display_RandomPoints(float frametime) {
// check and peg at the max number of things that can be displayed.
if (max_points > coords.size()) {
max_points = coords.size();
}
int nDrawingPoints = 0;
// if there are still points that haven't been added yet, work out how many more to add
if (n_points_done<max_points) {
// work out how many hits to draw in this frame
float new_points = (points_ps*frametime);
n_points_2_draw = n_points_2_draw + (points_ps*frametime);
//println("n_points_2_draw: " + n_points_2_draw);
// if there are points to draw, make sure its an int and update how many have been drawn and how many remain
if (n_points_2_draw>1) {
nDrawingPoints = int(n_points_2_draw);
//println("nDrawingPoints: " + nDrawingPoints);
//println("n_points_done: " + n_points_done);
color col = color(255,0,0);
color col2 = color(red(col)*0.7, green(col)*0.7, blue(col)*0.7);
for (int i=n_points_done; i<n_points_done+nDrawingPoints; i++) {
int k = coords.get(i);
if (!loadedpix) {
loadPixels();
//println("fixing mp....");
master_pixels = new color[pixels.length];
arrayCopy(pixels, master_pixels);
updatePixels();
loadedpix = true;
}
// add anit-aliased point to the master_pixel array
master_pixels[k+1] = col2;
master_pixels[k+width] = col2;
master_pixels[k+width+2] = col2;
master_pixels[k+(2*width)+1] = col2;
master_pixels[k+width+1] = col;
//println("[[" + master_pixels[k] + "," + master_pixels[k+1] + "," + master_pixels[k+2] + "]");
//println("[" + master_pixels[k+width] + "," + master_pixels[k+1+width] + "," + master_pixels[k+2+width] + "]");
//println("[" + master_pixels[k+(2*width)] + "," + master_pixels[k+(2*width)+1] + "," + master_pixels[k+(2*width)+2] + "]]");
}
n_points_2_draw = n_points_2_draw-nDrawingPoints;
n_points_done = n_points_done+nDrawingPoints;
if (n_points_done>=max_points) {
n_points_done = max_points;
n_points_2_draw=0;
}
}
}
// draw the points updating the pixels array directly here is much faster
// than loading images or setting pixel colours with set().
loadPixels();
arrayCopy(master_pixels, pixels);
updatePixels();
int[] dat = {nDrawingPoints, n_points_done};
return(dat);
}

As far as I understand it, this should be pretty-much the fastest way of doing this in processing. I've tried using a 3x3 image/texture and drawing lots of them but that is considerably worse performance wise. Editing the pixels array in place from scratch each time is about the same speed but is less clean (IMHO).

On my test machine here with Processing 2.0b8 and 64-bit Java 1.7.0_03 with windows 7:

with the screen set to 1,000 x 1,000 I get about 35-40 fps regardless of the renderer, with nothing at all registering on the graphics card activity monitor and one of the 6 CPUS basically at 100%.
with the screen set to 2,560 x 1,600 I get about 10 fps regardless of the renderer, with nothing at all registering on the graphics card activity monitor and one of the 6 CPUS basically at 100% (see screenshots below).

Eventually I want to run this on a screen 7,680 x 4,800 with up to ~10M points (not 100,000!), and I'd like to have a usable frame-rate, but that just isn't going to happen if I can't get some of this off the CPU and onto the GPU.

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Interestingly, changing the number of dots to be drawn, or how quickly to draw them, doesn't impact on the frame-rate at all. Only the screen size (and hence the size of the pixels array) seems to have a significant impact, suggesting that its probably the method of accessing the pixels array directly that slowing things down... maybe?

PhiLho

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

I am not a specialist of OpenGL, but my understanding is that it is comfortable drawing triangles and thus various shapes, but not so much at manipulating pixels.
The loadPixels() operation is totally Processing-specific, where it has to capture the content of the graphics card to fill an array of ints, so that's a CPU operation. Such operation is very efficient in the old P2D mode, which was acting on such array of pixels, a bit less in JAVA2D, and much less in OpenGL, where you must transfer data to the GPU.
To exploit the power of the GPU, perhaps having a small triangle (or square / two triangles) for each point might be more efficient, if smartly using the OpenGL primitives and such. Here, I hit the limits of my knowledge of OpenGL, I fear...

Out of curiosity, what display device do you use, to have to manage such resolution in real time?

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Hi PhiLho,

Thanks for the advice, I'll try replacing the pixels with small triangles, but I'll be surprised if it makes a considerable difference. I think what I might need to do is use a Pixel Buffer Ob ject (see http://www.songho.ca/opengl/gl_pbo.html) but I have no idea how to implement this.

At the moment we just have a large monitor (2560x1600), but we're planning on putting this sketch on a 4x4 video wall with 33M pixels (in a few weeks time!!). ho hum...

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Here is a version of the same test script but using PhiLho's suggestion of drawing lots of small triangles rather than modifying the pixels array directly. As I suspected, its nowhere near as fast. Whats worse, it is also suffers as the number of triangles increases.

On my test rig I get ~18fps at the start but beyyond about 20-25k triangles it slows to ~5fps and falling.

import java.util.Arrays;
import java.util.Collections;
import java.util.List;
Integer n_points_done, j, n_frame_av;
float frame_time, frame_end, av_frame_time;
ArrayList<int[]> coords;
int[] points_info;
String display_fr;
String renderer = "OPENGL";
int x = 1000;
int y = 1000;
int max_points = 100000;
float points_ps = 2000;
PShape tri;
void setup() {
if (renderer=="P2D") {
size(x,y,P2D);
} else if (renderer=="P3D") {
size(x,y,P3D);
} else if (renderer=="OPENGL") {
size(x,y,OPENGL);
} else {
exit();
}
// setup coordinate space
int x = 0;
int y = 0;
coords = new ArrayList();
for (int i=0; i<((width/3)*(height/3)); i++) {
//tcoords[i] = (y*width)+x;
int[] thiscoord = {x,y};
coords.add(thiscoord);
x=x+3;
if (x>(width-3)) {
x=0;
y=y+3;
}
}
//coords = Arrays.asList(tcoords);
//coords = tcoords;
Collections.shuffle(coords);
//starting settings
frame_time = 0.017;
av_frame_time = 0.017;
n_points_done=0;
j=0;
display_fr = "";
n_frame_av = 1;
fill(0, 0, 255);
noStroke();
tri = createShape(TRIANGLE, 0, 0, 0, 3, 3, 3);
}
void draw() {
background(0);
if (frameCount>1) {
//println(frameCount);
points_info = display_RandomPoints(frame_time);
//println(points_info);
if (j==n_frame_av) {
String display_renderer = "Current Renderer: "+renderer;
String display_fps = "Frame rate: "+nfc(n_frame_av/av_frame_time, 1)+" frames/s (frame time: "+nfc(av_frame_time/n_frame_av, 3)+"s)";
String display_points_info0 = "Number of points to be drawn: "+str(max_points);
String display_points_info1 = "Points added this frame: "+str(points_info[0]);
String display_points_info2 = "Total points drawn so far: "+str(points_info[1]);
display_fr = display_renderer+"\n"+display_fps+"\n"+display_points_info0+"\n"+display_points_info1+"\n"+display_points_info2;
av_frame_time = frame_time;
j=0;
//println(display_fr);
//exit();
} else {
av_frame_time = av_frame_time + frame_time;
}
j++;
textSize(20);
textAlign(LEFT,TOP);
fill(255);
text(display_fr, 10, 10);
}
frame_time = (millis()-frame_end)/1000;
frame_end = millis();
}
int[] display_RandomPoints(float frametime) {
// check and peg at the max number of things that can be displayed.
if (max_points > coords.size()) {
max_points = coords.size();
}
int new_points=0;
// if there are still points that haven't been added yet, work out how many more to add
if (n_points_done<max_points) {
// work out how many hits to draw in this frame
new_points = int(points_ps*frametime);
n_points_done = n_points_done+new_points;
if (n_points_done>=max_points) {
n_points_done = max_points;
}
}
for (int i=0; i<=n_points_done; i++) {
int[] k = coords.get(i);
shape(tri, k[0], k[1]);
}
int[] dat = {new_points, n_points_done};
return(dat);
}

rbrauer

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Try the following maybe. Also, if you can set a "profile" on your graphics card control panel for Processing's java.exe and disable any power saving stuff it might help.

PShape test;

void setup() {
  size(1280,720,P3D);
  noSmooth();
  noFill();
  stroke(255);
  background(0);
  
  test = createShape();
  test.beginShape(POINTS);
  test.vertex(-1,0);
  test.vertex(1,0);
  test.vertex(0,-1);
  test.vertex(0,1);
  test.vertex(0,0);
  test.endShape();
}

void draw() {
  if(frameCount%60==0)println(frameRate);
  int count = (int)random(50,200);
  for(int a=0; a<count; a++) {
    shape(test, random(width), random(height));
  }
}

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Thanks for the reply rbrauer, it does indeed work very quickly! There are a few problems with it though that mean it doesn't solve the problem I'm trying to solve...

1) while the test code I posted doesn't have anything except dots on it, the actual sketch is more complex and has other things overlayed and moving on top of the dots. This means we have to call background() each time, which means we need to draw all the dots, every frame.

2) I don't want the dots to overlap, so I need to make sure once a position is used, no other dot chooses that position (hence why I was pregenerating the random co-ordinate set).

point 2 is not really a problem, but point 1 is. adding background(0); and incrementing the number of points drawn each frame reduces the framerate exactly like the 'triangles' example above.

I'm wondering, actually, if it might be quicker for me to draw the points on an offscreen graphics object rather than the draw window and then just draw that object as an image each time; kind of a 'layer' effect. That way maybe I can avoid using background() on it?

rbrauer

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

I'm wondering, actually, if it might be quicker for me to draw the points on an offscreen graphics object rather than the draw window and then just draw that object as an image each time; kind of a 'layer' effect. That way maybe I can avoid using background() on it?

That is exactly what I would suggest. Use a separate PGraphics object (call it "pg" for example) to draw the points into.
pg.shape(test);
then
image(pg, 0,0,width,height);
//draw other things

I also wanted to mention, by the way, P3D and OPENGL refer to the same thing in the betas. P2D also uses OpenGL and JAVA2D is also still available.

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Thanks rbrauer, I'll give that a try. I guess it might even give me multi-threading as well since the separate PGraphics object will be being drawn with its own thread?

I'll let you know how it goes ;)

GoToLoop

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

P3D and OPENGL refer to the same thing in the betas. P2D also uses OpenGL and JAVA2D is also still available.

That's what I was trying to say since the beginning!!!

It's either JAVA2D or OPENGL renderers for Processing beta!

Although I'm suspicious that P2D might use OPENGL in a more 2D way?!

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Initial tests on doing this by drawing to a separate PGraphics object look very promising. I'll test it later and then let you know for sure...

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

OK, so, the results are in and they say... its a considerable improvement but not getting us all the way there. For the full sketch, drawing the background random points to a separate PGraphics object as shapes improves the frame-rate at 2,560 x 1,600 from an unusable 7fps to 30fps. Yey!

So, its not 60fps, but its OK. Scaling this up though to a much larger screen still causes problems. The target screen size of 7,680 x 4,800 is rendered at about 7-10 fps, so we're still not there. The bottleneck is still that the GPU is doing essentially nothing. The sketch is hammering on one CPU, without the graphics card even being touched it would seem, regardless of which renderer is being used.

Anyone any more ideas for how to speed thing up/force the graphics card to do the work? Anyone have any luck using Pixel Buffer Objects with processing?

GoToLoop

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

What about multi-threading?

http://forum.processing.org/topic/csv-files-read-from-specific-url#25080000002086871

forum.processing.org/topic/help-with-updating-thread-when-key-is-pressed

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Yeah, I'm looking into that now as a last resort. I'd have tried it sooner but it looks hard and complicated and still doesn't really get around the the fact that I have an uber-graphics-card in this machine that I'm not really utilizing (which seems like a waste). I really don't get why the hard work here is not being done by the card...

rbrauer

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

Also, if you can set a "profile" on your graphics card control panel for Processing's java.exe and disable any power saving stuff it might help.

Did you look into this at all? If there are settings for vsync in there you can try turning that off too. I think you wrote that you had AMD cards, but these kinds of things were causing quite a bit of trouble on my particular NVidia card and I thought it might be something to at least look at with your setup.

I've done PBOs for my capture setup, but that was just for readback. Haven't tried doing writes with them. The article in my recording thread seems really good. This: http://www.songho.ca/opengl/gl_pbo.html will get you started at least if you think you have to go that route.

nickschur..

Re: Processing 2 sketch not using the graphics card for rendering....

5 months ago

I've given up on the multithreading bit for the moment, I can't persuade it to work well. I've been looking at PBOs, but I don't seem to be having much luck - any help with making the PBOs work would be greatly appreciated- see https://forum.processing.org/topic/using-opengl-pixel-buffer-objects-and-a-weird-1282-error