Processing 1.0 - Processing Discourse - Processing vs. openframeworks speed test

Giles

Processing vs. openframeworks speed test
Feb 12^th, 2010, 11:15am

I've been writing some programs in Processing which are really pushing my machine, and I kept hearing about how much faster openframeworks is, but I found it hard to get an idea (from hearsay) about just how much faster it is. So I converted one of my simple programs to openframeworks and checked the framerates.

The programs I used are here: http://pastebin.com/f5c8cecb

My computer specs: AMD 3800+ (dual core 2.0 gHz), 2 gigs ram (533 MHz I think). Not super fast, and as I don't really know how to do threading in Java, I can only get Processing to work on one of the cores (as evidenced by maximum of 50% CPU usage). I noticed that the OF program was using both cores (85% CPU time). It's nice that the C++ program just "knows" how to use the full CPU power (I've no idea why).

The results:

Processing: 10 fps
openframeworks: 42 fps

From a pretty basic test, this is quite an impressive difference. OF is much harder to get working, though. If someone on the OF forums hadn't helped me I never would have got the program working....

Anyway, Processing is still king for simplicity and rapid exploration of ideas. OF is faster. I guess you pay quite a lot in performance for the non-platform-specific nature of Java (I'm no expert on these matters, so feel free to enlighten me if I've got things wrong).

V Senior Member Offline Posts: 430 portugal	Re: Processing vs. openframeworks speed test Reply #1 - Feb 12^th, 2010, 12:15pm two things. you are comparing a software renderer against a hardware renderer. Enable OPENGL for processing before comparison. Then make another test. Render with direct calls to JOGL and compare again (this approach should be much closer to what OF does). Post back your new values.
	Victor Martins \| www.pixelnerve.com/v

Giles Senior Member Offline Posts: 252	Re: Processing vs. openframeworks speed test Reply #2 - Feb 12^th, 2010, 12:21pm Great.... if I can get closer to openframeworks performance with Processing I'll be very happy. How do I do what you suggest - do I have to import a special library? How do I render with direct calls to JOGL? Thanks.

Giles Senior Member Offline Posts: 252	Re: Processing vs. openframeworks speed test Reply #3 - Feb 12^th, 2010, 12:30pm Enabled opengl - by import processing.opengl.*; and size(500, 500, OPENGL). Framerate dropped to 3.7fps. I do have a fairly fast graphics card - NVidia 9600GT - so something isn't working right here....

watz

Re: Processing vs. openframeworks speed test
Reply #4 - Feb 12^th, 2010, 2:37pm

I get similar results:

- JAVA2D: 19.7 fps
- P3D: 11.6 fps
- OPENGL: 6.4 fps

Looking in PGraphicsJava2D.java gives the following for ellipse():

Code:

protected void ellipseImpl(float x, float y, float w, float h) {
    ellipse.setFrame(x, y, w, h);
    drawShape(ellipse);
  }

I'm guessing that's much faster than the beginShape() / endShape() code used for PGraphics3D.

marius watz // unlekker // code & form // Generator.x

subpixel

Re: Processing vs. openframeworks speed test
Reply #5 - Feb 13^th, 2010, 2:27am

Machine: Pentium-M 2GHz ("Centrino notebook") about 5 years old, 1GB Ram, 128MB ATI Mobility Radeon 2700.

OS: Windows XP, SP3

Processing 1.0.9:
- default renderer: ~13.3fps
- OPENGL renderer: ~3.5fps
OF: maybe next time.

I wonder why my machine's Processing result is faster than yours. Perhaps the graphics card makes the difference.

Comment on using "direct calls to JOGL": doing this breaks the usability of Processing (you can't really mix it with regular Processing drawing commands), and is really more like straight Java programming in the PDE.

-spxl

Processing: subpixels.com, OpenProcessing.org | VJ portfolio: SPXL.TV

memo

Re: Processing vs. openframeworks speed test
Reply #6 - Feb 13^th, 2010, 7:55am

I thought I'd give this a shot, my results on 3.06Ghz Macbook Pro with 9600GT (I'm using 50K particles cos otherwise I get 300fps+ in OF!)

openframeworks : 40 fps
Processing / JAVA2D : 1.8 fps
Processing / P2D : 17.5 fps
Processing / P3D : 2.7 fps
Processing / OPENGL : 1.21 fps

Since a circle has a resolution - and maybe these don't match between OF and processing, I thought I'd try rectangles since we know in all cases it will have just 4 corners (hopefully!). 50000 rectangles:

openframeworks : 65 fps
Processing / JAVA2D : 11.7 fps
Processing / P2D : 42.7 fps
Processing / P3D : 11.1 fps
Processing / OPENGL : 2.8 fps

Of course both apps can be optimized quite a bit too. Just using point-sprites and a vertex-array for the whole system would improve performance a lot, and probably the difference between processing and openframeworks would then decrease too. But for an 'out-of-the-box' performance comparison this is a good test I guess.

On a side note, unfortunately openframeworks doesn't automatically use multiple threads - unless you are using an addon that has multithreading built in (e.g. parsing incoming OSC messages). In this simple example I can't see what multi-threading would happen automatically.

peterkirn

Re: Processing vs. openframeworks speed test
Reply #7 - Feb 13^th, 2010, 8:17am

Well, I can't say I'm surprised by the results, but a few comments.

1. Can you guys list which JRE you're running in? This can make a difference.

2. Let's recall that in a real-world application, both oF and Processing would benefit in this case from the use of point sprites in place of ellipses. Because Processing is so slow drawing in the way you're describing, it's naturally going to benefit even more.

3. There are various optimizations possible in Java that we're skipping over here (things like substituting bit primitives for ints, etc.)

4. No one seems to be testing GLGraphics, which is pretty much the way forward as far as Processing OPENGL rendering.

One thing that struck me as odd was that people have been talking about using openFrameworks to achieve speed through GPU-native code -- which is something Java can do reasonably easily, too, and which assumes you're executing code on the GPU anyway. Wrapping OpenGL in JNI does accumulate some overhead, so it'd be interesting to do a speed test there, too, but on modern machines this margin shouldn't be as massive as the ones you're finding here with, say, the software renderer.

peterkirn

Re: Processing vs. openframeworks speed test
Reply #8 - Feb 13^th, 2010, 8:20am

Also, did I read that wrong, or did an old AMD get 6x the performance of a brand-new MacBook Pro?

JRE versions ... please?

(I see you just mentioned optimization, Memo... of course, I think it is a little unrealistic to expect that the only thing people will do is run huge particle systems, let alone huge particle systems without optimization, but yes, it is still a meaningful benchmark of several aspects of the performance of the platforms.)

peterkirn

Re: Processing vs. openframeworks speed test
Reply #9 - Feb 13^th, 2010, 8:25am

Oh, lastly, I have to disagree that using JOGL "directly" has to defeat usability.

Look at what GLGraphics is doing. It's really an attempt to solve exactly this problem.
http://users.design.ucla.edu/~acolubri/processing/glgraphics/home/examples.html

Secondly, if there are JOGL calls people are tending to use, those can be wrapped into libraries so that you can retain readable code.

Lastly, for a different platform altogether, check out what jReality is doing with the JOGL renderer. They're doing incredible, performance-taxing stuff ... and they have a scenegraph, which Processing lacks. (OpenFrameworks, too, though I gather some people have tried using OpenScenegraph with OF?)

http://www3.math.tu-berlin.de/jreality/

(I'm doing more with jReality, so it may be possible to pull some components and make a library for Processing, too.)

Faster/slower always makes me shiver a little bit, because it takes out all of the other elements of what you're actually trying to do and what might help you pick the best tool for the job. I'm looking at Processing.js for certain tasks, now, for instance, because load time matters more than baseline performance once you've started. And whatever the JS people say, a lot of that stuff is quite a bit slower even in the new implementations in Chrome, Safari, and Firefox.

That's not to say it doesn't make sense to look at performance, though, and pick and choose, and now that you can set up Eclipse with OF and Processing and Processing.js side-by-side... Wink

Giles

Re: Processing vs. openframeworks speed test
Reply #10 - Feb 13^th, 2010, 9:42am

Thanks for your replies - I know my test was a kind of naive one - not really optimising the code in either language. I hoped to raise some debate and maybe learn how to make things run a bit faster in Processing - as I don't necessarily want to move my projects to OF - there's heaps of libraries I really enjoy using in Processing/Java.

I'm a bit gutted that my computer is slower than a 5 year old laptop Sad

I did just overclock it (up to 2.3 GHz) and got up to 12.5fps...but still slower. The Macbook was running 5x the number of particles, so definitely way faster than the old AMD.

Memo said:

Quote:

On a side note, unfortunately openframeworks doesn't automatically use multiple threads - unless you are using an addon that has multithreading built in (e.g. parsing incoming OSC messages). In this simple example I can't see what multi-threading would happen automatically.

Do you usually have to use multiple threads to get code running on more than one core of a multicore processor? Because my OF program was definitely hitting 85% CPU time on my dual core machine which means it must have been running on both cores (the Processing sketch hits 50% and sits there).

Giles Senior Member Offline Posts: 252	Re: Processing vs. openframeworks speed test Reply #11 - Feb 13^th, 2010, 10:17am If anyone wants to post an faster version of the Processing sketch (however you want to do it), I'd love to see it. I'm keen to learn some more efficient ways....

th0m

Re: Processing vs. openframeworks speed test
Reply #12 - Feb 13^th, 2010, 10:34am

I'm just reading this, and I want to agree with some of the first repliers that the OpenGL should be much better. It might be worth trying the latest JOGL, or making sure you have the latest binary proprietary drivers. I think with OF being closer to the metal so to speak, it is masking the fact there might be something wrong with your JOGL. It could also be that OF is turning off the sync by default and your JOGL isn't? I am not sure, although I have experienced situations where the 2d renderer is quicker myself, although it sure shouldn't be.

subpixel

Re: Processing vs. openframeworks speed test
Reply #13 - Feb 13^th, 2010, 1:31pm

peterkirn wrote on Feb 13^th, 2010, 8:25am:

Oh, lastly, I have to disagree that using JOGL "directly" has to defeat usability.

Perhaps it doesn't always, but my feeling is that it is likely to.

From the OpenGL reference page:
Quote:

Using the OpenGL Renderer - advanced users only!

Generally, this should be only considered a last-ditch way to get at a particular GL-specific feature. It makes your code completely incompatible with everything else (such as future versions of Processing, or other 3D renderers) and will confuse the hell outta people when you post your code to the web. Again, if this works for you great, but if not, we aren't responsible and we will not field questions about it on the board. If you want to write code for Java and JOGL, then find yourself a good Java IDE and get JOGL installed, Processing is probably not the way to do it because that's the kind of confusing mess we're trying to insulate you from.

To get access to the GL object, use the following:

GL gl = ((PGraphicsOpenGL)g).gl;

This will give you JOGL's GL object that talks directly to OpenGL and will let you make OpenGL calls directly.

Again, this is not supported, in the sense that if you get weird problems with a GL-based sketch that uses this method, you're on your own.

Because most of the work in drawing is handled by Processing, most OpenGL calls will not work, because most things won't be set up properly. Therefore, using the GL object may only be useful for tweaking obscure OpenGL parameters.

Another option for talking directly to OpenGL in release 0116 and later is to use the beginGL() and endGL() methods. These were added against my better judgement, and handle setting up the camera matrices (though still no lighting or other parameters). Many features still will not work, but it opens up lots of control for geometry. All the above caveats about "don't cry when it breaks" apply.

I seem to recall fiddling with a sketch a little while ago that had direct JOGL calls in it, and something just wasn't working in combination with Processing functions. I can't remember what it was now, but might have a look for it later.

Basically, I think once you start hitting JOGL calls for performance, you're probably ready to bust out of the Processing (sand-)box.

-spxl

Processing: subpixels.com, OpenProcessing.org | VJ portfolio: SPXL.TV

davbol

Re: Processing vs. openframeworks speed test
Reply #14 - Feb 13^th, 2010, 4:49pm

i haven't run the code, but know that ellipse() emits varying detail depending on the diameter, probably around ~15 triangles per ellipse with the diameter you're using.

i'd suggest replacing ellipse() with rect() or even just a triangle() to first get a handle on what's affecting performance.

Does framerate go up significantly with simpler geometry? If so, then... does it then drop again if you increase the screen size of that geometry? (giving a clue to either either geometry or fillrate as the bottleneck, tho prolly not fill rate if OF performs ok)

Then, how does OF's ellipse drawing method work internally? Like, might it perhaps use a predefined display list, just transform as necessary? That's a relatively simple trick to accomplish, even in processing, and might speed things up greatly, but will skew the test between platforms.

If you can make sure the two platforms are using the same "types" of calls it might be more obvious what the performance difference means. (that is, if OF does use a display list (et al) then no surprise at all it beats processing's immediate mode, there'd be no mystery here)

Or, does moving the fill() statement outside the particle loop affect things much? So instead of 10000 fill(0) statements per frame you'd have just 1. Less calls is always better, especially with the JNI overhead, but also sometimes certain combinations of OpenGL calls causes state-switches that JOGL may handle poorly.

hth, am curious to follow your results

[edit: oops, just noticed that memo above already did the rect() instead of elllipse() test, my bad, ignore]

[edit: here's a display list version list for comparison: http://pastebin.com/m49b3d5d2 ]

« Last Edit: Feb 14^th, 2010, 6:51am by davbol »