Video of a digital painting I've done with a program I wrote in processing.

shawnlau

A couple of years ago, I wrote a paint program that worked fairly well with smallish/medium images ( about 1200 x 1200 pixels. I'm a digital artists and work much larger than that, usually 4800 x 6000 pixels to print 16 x 20 inches. A few weeks ago, I began optimizing the program so it could handle large images.

https://forum.processing.org/two/discussion/24570/optimizing-garbage-collection-in-processing-question-s https://forum.processing.org/two/discussion/24613/multidimensional-arrays-and-java

Its working pretty well now. I still need to add the ability to load or create different brush heads. Right now its confined to a simple round brush. In writing the program, I used it to create a painting to test it and to feel what was needed. I'm posting a link to a time lapse video of the painting and you will notice new features emerge as the video goes on. Thats because I'd be painting and realize the program needs something, like a thumbnail view. So I'd stop painting and put that in there. Then I'd find pulling up the color picker to lightly adjust the color was too cumbersome, so I added an easier way to adjust it on the fly.

Here's a video of a painting totally created in a program I wrote in processing 2.2.1. I also use the codeanticode tablet and G4P libraries for that version of Processing. The painting does contain artistic nudity, but it is quite innocent.

jeremydouglass

Beautiful process video -- thank you for sharing it. Watching the feature set evolve was also very interesting.

How did you implement the floating popups -- like the color picker, and the thumbnail preview view. Were those done with a Processing GUI library, or did you use something else?

shawnlau

Thanks! The pop ups are G4P (quark's place) Windows. The G4P library has a color picker, but I wrote my own specific to how I wanted it to work.

I added some different brush heads so I can now create more texture in the painting. The problem I'm having is designing some type of undo system. Right now it has none. I had one in place where it would keep track of the max and min x and the max and min y during the time the mouse was pressed and released. It would then save that rectangle from a hidden version not affected by the draw, and then pasted the drawn on rectangle to hidden version. It works pretty well for small areas, but for painting very large areas its clunky. I'm not too concerned about it. I only used undo for minor mistakes. Now I just paint over them :)

Here's a process vid with the new brush heads. There is much more texture.

jeremydouglass

Re: undo -- is your data currently handled in layers, or is it all one canvas?

If it is all one canvas then a very simple undo would simply use saveFrame() / loadFrame() -- e.g. every 60 seconds (or every time you switch tools, or every time you hit "save" etc.) the sketch uses saveFrame to store a go-back state, up to a certain number of saves (a buffer of 10 mins, or whatever). When you hit undo, you are just loading a previous frame onto the canvas.

Or perhaps the save/load time was already bad for even small rectangles, which was what you meant by "clunky"?

shawnlau

What you see me drawing on is actually a huge image- 6000 x 4800 pixels. When you see the entire image, its because the real image is reduced to fit on the screen or to be whatever size the current zoom is. If fact I had to design it to only do that reduction when I wouldn't be doing any actual painting (e.g., when zooming or dragging). During painting, the only areas of the screen that are redrawn are the small areas that have changed or had the ellipse move over them.

The actual drawing or pixel manipulation happens on the huge image. If I was to save the frame that is displayed, I'd be saving a very degraded copy of the real image. I have to save the part of the real image ( the image that is not reduced or enlarged) that has changed.

In my first paint program, it could only handle smallish images ( maybe 1200 x 1200 pixels). With that, I could get way with saving the whole off screen (1200 x 1200) image. But these images are much too large to do that. I can think of only two ways to do it and each has its drawbacks.

I can save only the portion that has changed during a painting action by keeping track of the rectangle that was affected(minX,minY,maxX,maxY) as an undo record. The record holds the x,y, w,h of the rectangle and the pixel values. Its a basically a delta. The problem, even that can lead to huge amounts off memory being used and performance problems. Just imaging a long painting event between pressing and releasing the mouse. The rectangle could be half or even the entire image. Most times, it would be small, but zoomed out and working on the whole image, they are more likely to be over 25% of the huge image.

The second way would save a huge amount of memory and save the undo extremely quickly. The idea there would be to sort of vectorize the painting events. Say we start with a clean canvas. I save that clean canvas and from then on only save the parameters of the paint action. I save the brush size, the brush color, the pen pressure etc. I save everything that goes into the method that does the actual drawing or manipulation of pixels. And its quite easy to replay these into the method that does the drawing. This way uses very little memory, and saves the actions very quickly. But to undo something, you have to start with the clean canvas, and replay each action up to the point you want to undo. So if you have 600 undo actions saved you have to through those 600 to get to the last thing you did.

So I have a clean canvas. I draw line 1, draw line 2, draw line 3, draw line 4, line 5, line 6 oops! I want to undo it so I have to get the saved clean canvas, redraw line 1,2,3,4 and 5 just to undo line 6.

jeremydouglass

Ah, a massive canvas, of course -- I see what you mean! Implementing everything using "vectorized" events would be labor intensive. I believe (?) that programs e.g. Photoshop use a combination of an event stack and checkpoints to shorten re-computation of that stack. Also, some events are discrete and some are reversible -- e.g. mirroring the canvas or rotating it by 180 degrees are specifically reversible events that don't require recomputing the stack behind them.

One note: I don't think that an event model would require a clean canvas to start. You could load an image as step one and build an event stack on top of it, because if you can replay events then you could replay them on top of any arbitrary surface with deterministic results on that surface (assuming that all actions are deterministic).

shawnlau

Do you think programs like photoshop might save some of theses canvas states to disk ( I know photoshop uses something called a scratch disk)? Maybe go through a hundred draw actions, then save that canvas state to disk? Playing back 100 paint actions ( or flip actions etc.) might not be that bad. But saving that canvas state to disk ( a huge image) would probably be noticed.

With that model, there would be a definite number ( a set number) of actions to replay. The only noticeable delay from loading a canvas state would be undoing every 100 or so paint actions. I could live with that. Its extremely rare to have to undo that much.Its saving the canvas states that bothers me as that would probably happen during actual painting and cause a noticeable stutter.

jeremydouglass

One thought -- lazy-saving during periods of low interaction -- essentially set a last-click time on mouseReleased, and if current time is x greater than that (like 10 seconds?) and you haven't already checkpointed then auto-checkpoint. For more aggressive sleep detection you could also tie it to mouse movement -- if e.g. mouseX/mouseY stay the same for x seconds.

shawnlau

That's a good idea. And change the cursor while its saving. Chances are if 10 seconds pass, I'm thinking and not drawing. And if I go to draw, I'll see the cursor showing me its busy. Its not like being caught in mid stroke or something.

Rammschnev

Most impressive. Really demonstrates what's cool about the Processing crew. Quite beautiful. Thanks for the show.

shawnlau

Thanks, glad you liked it!