limits of processing

akiersky

in Integration and Hardware • 2 years ago

Hi all,

I have been asked by a client to produce a video wall. The final installed piece will be 8 monitors mounted in vertically, in a row, driven by one powerful computer. We have a computer with two video cards that can output the 8 displays and handle the cms system we have planned. Our concept involves a particle system working as the background, and various news and rss feeds displaying text, images and videos at determined times across the 8 screens.

We have chosen to explore Processing as a possible solution for driving the visuals for its ability to output very large sketches and ability to process a large amount of data, however, with our limited experience with the language, I'm not sure that it is the right solution for this project.

Has anyone had experience with projects like this? examples, or at least knowledge that our goals can be achieved?

Thanks!
ak

Replies(10)

biodesign

Re: limits of processing

2 years ago

The limits of Processing are the limits of Java and eventually of OpenGL. Processing just ease the use of Java with a simplified syntax, tons of libraries and a great community of users and developers ; )

There are some libraries to span a sketch on multiple windows (Most Pixel Ever), to play with RSS (romefeeder), for particle systems (anar, Physics) and a lot more.

We have made a multitouch table from multiple LCD displays in under a week, building the table, programming a cheap video tracking from scratch and using a fluid simulation library (MSAFluids) with Supershapes flocking in it. The only limits I saw were time and programming skills ; )

If you use Processing as is, there is some limitations though.

Processing only support Java 1.4. This is not a big deal unless you want to use some advance syntax or some libraries that need it.

You cannot currently use 1.5 syntax in the Processing Development Environment. This means no generics, templates, enum, varargs, foreach, and the rest. If you want to use Java 1.5 (or later), take core.jar and develop your project with another Java IDE [...]

Processing also extends an applet and it limits the use of AWT and Swing. For example, it can be a bit tedious to do a layout without the AWT or Swing layout managers. You have to position all your graphic elements by hand.

You cannot use most of the AWT or Swing (which is built on the AWT), because it will interfere with the graphics model.

The OpenGL support is not complete. You can use the OpenGL context directly, but:

This will give you JOGL's GL object that talks directly to OpenGL and will let you make OpenGL calls directly.

Again, this is not supported, in the sense that if you get weird problems with a GL-based sketch that uses this method, you're on your own.

Processing is using a display loop more than an event-driven style of programming. It's not easy to bind events on-the-fly to any elements (like you can do in Javascript or ActionScript for example).

The IDE is also very basic and unusable for big projects.

All those limitations are made to simplify the use of Java for very fast prototyping, easy deploying as a web applet, converting to Processing.js, etc. But when you want production code, you need to use it in a different way. For example, you can use it as a Java library in Eclipse (see here) using the Java version you want, opening your applet fullscreen, etc. For OpenGL, you can use some libraries instead of the internal one (ex.: GLGraphics, anar), etc.

If you are a Java/OpenGL programmer that want to use all the great Processing libraries to simplify your work, I think you can manage this kind of project easily. But if you want a framework that can do very powerful production code out-of-the-box, but with a more steep learning curve, there is OpenFrameworks (as cited by hapticdata) and Cinder that are C++ based.

But depending on your needs (number of particles, display resolution, video resolution), this kind of project is not necesarily has computer-intensive as it sounds. For example, according to your description, I don't see any use for a 3D graphics framework. Maybe you could use NodeBox for OpenGL, Pyglet, Summon or SLUT that are Python based, a little less powerful but very easy to learn. And you can benefit from all the Python libraries. But if you want, for example, over 1 billion particles interacting in realtime, you have to do it on the GPU, no matter if it's from Java, Python or C++.

So the question is: what are your programming skills, the languages you already know and the level of involvement you are ready to put in learning a new language?

hapticdata

Re: limits of processing

2 years ago

Hello there!

Sounds like a fun project. I have done some similar work and can tell you that Processing will work for this, but not without obstacles. I dont know enough about your development skills to say whether or not it is the correct answer.

If you are skilled Java Developers I would say to do it in Processing, otherwise, I would say a better option would be OpenFrameworks. OpenFrameworks is a C++ framework and it is very simple to make an application span multiple monitors, its one line of code.

I use Processing a lot more than I use OpenFrameworks, but your project taps into 2 things that Processing is a little weaker at. The main one is GPU acceleration. If you are going to be powering a particle system over 8 monitors with one computer it would be best to throw as much of that load as possible onto your GPU (instead of CPU), Processing's current implementation of OpenGL is not very good at that (my understanding is that is quickly changing). Secondly, it is a little more complicated to get Processing to span multiple monitors at fullscreen. The suggested Most Pixels Ever is a server/client solution intended for a single display for each computer, so your setup would involve 8 computers using that library. The best way to do this is to write pure Java in an IDE like Eclipse, and incorporate p5 as a Frame. This will work, but things can get sticky, this is straight from Processing's source code (probably written by Fry):

Processing on multiple displays

I was asked about Processing with multiple displays, and for lack of a better place to document it, things will go here.

You can address both screens by making a window the width of both, and the height of the maximum of both screens. In this case, do not use present mode, because that's exclusive to one screen. Basically it'll give you a PApplet that spans both screens. If using one half to control and the other half for graphics, you'd just have to put the 'live' stuff on one half of the canvas, the control stuff on the other. This works better in windows because on the mac we can't get rid of the menu bar unless it's running in present mode.

For more control, you need to write straight java code that uses p5. You can create two windows, that are shown on two separate screens, that have their own PApplet. this is just one of the tradeoffs of one of the things that we don't support in p5 from within the environment itself (we must draw the line somewhere), because of how messy it would get to start talking about multiple screens. It's also not that tough to do by hand w/ some Java code.

If you decide on using Processing, this extra framework may be of help to you, this guy has figured out the GPU acceleration issue: http://code.google.com/p/creativecomputing/

And here is a file in his trunk using it with Processing

Best of luck!

- Kyle Phillips

akiersky

Re: limits of processing

2 years ago

Thanks for the great responses guys!

To be quite honest, I am just getting started with Processing and Java, but have a strong base in actionscript. I would be using flash to accomplish this, but the stage dimensions are greater than flash will suppport. I have used Processing a handful of times to solve various smaller projects, and felt like the transition was fairly painless.

That said, I have looked into openframeworks as an option, and it looks very promising, but I would like to stick to semi-familiar code for the timeframe we have to work with. it seems like, even with a bit of objective-C, C++ would be a bit much to learn right now.

I have created a few working models of the parts of our project, the one thing I am having the most trouble with is the displaying of video. It seems like the video class built in is slow, i have tried a few other libraries for video such as GSVideo, jmcvideo, pogg, , both with mixed results. GSVideo seemed to get the best results, but I was only able to get 5-10fps with an HD video (1080p, h264, aac audio). It seems like it should be able to do better. Ive tried streaming the video with similar results, at best about 15fps. Maybe the compression type?

Thanks for the input, Ill keep you posted!

ak

cedrickie..

Re: limits of processing

2 years ago

just want to add the update of GSVideo and the life in a day project :
http://codeanticode.wordpress.com/2011/03/15/gsvideo-0-8/

so i guess its definitely possible if you look at the youtube video.

akiersky

Update on the project

1 years ago

Hi All,

Just to give an update, the monitor wall went off without a hitch! we ended up using a single beefy computer, with 10 outputs, to eight 60" sharp monitors ( http://www.sharpusa.com/ForBusiness/PresentationProducts/ProfessionalLCDMonitors/PNV601.aspx) mounted in portrait in a single row, to create a 20' x 5' video wall.

Processing worked great for the project, the end display was 6144px X 1366px, with a single, chromeless sketch.

All in all the whole app was able to get 90-120fps using only about 5% of our 3GHz processor and under 1GB of ram.

Good times,
ak

Jesse Sco..

Re: Update on the project

1 year ago

Interested to know what computer you used (and Graphics Card) ?

I'm looking at a similar project right now, and am researching machines...

andres

Re: limits of processing

1 years ago

it sounds very impressive! what video library did you end up using?

akiersky

Re: limits of processing

1 years ago

I tried quite a few of them, but in the end, GSVideo & GLGraphics had the 1-2 combo that worked the best all around. plus the added benefit of using GLGraphcis elsewhere in my code

Thanks for the support!
ak

jeff_g

Re: limits of processing

1 years ago

You have a picture or video of it? Would love to see it. :)

akiersky

Re: limits of processing

1 years ago

Here's a picture from this article, http://www.mlive.com/business/west-michigan/index.ssf/2011/11/steelcases_new_workcafe_redesi.html:

Top Reply