How can I incorporate my drawings into a Processing sketch without using an image verbatim?

General Discussion Other

milespeyt..

How can I incorporate my drawings into a Processing sketch without using an image verbatim?

in General Discussion • Other • 1 year ago

I've been using Processing to create art for some time now. I'm attracted to purely generative works, but have come to feel that it's difficult to imbue a personal style in an algorithm in the way that is intuitive (for me) with drawing. I'd still like to work with code because it allows for infinite variations on an idea, and for unpredictable, organic behaviors. In contrast, a bitmap is fairly static.

So my question is, how can I use Processing to interpret images of drawings that I have made? What I'm having trouble doing is moving from an image file to a series of vectors, coordinates, etc. that I can manipulate.

If the drawing is black on white paper, and I boost the contrast in Photoshop, would it be difficult to code something that can estimate the positions of lines on a drawing, and store this data in an array? I don't know that much about edge detection, or even if this is the right approach.

Any help would be appreciated, and please let me know if I can clarify anything.

Thanks

Replies(6)

jbum

Re: How can I incorporate my drawings into a Processing sketch without using an image verbatim?

1 year ago

One idea might be to scan some of your drawings in a vector program, and convert them to vector format. Then use a library like geomerative to read the vector data and use that as the basis for generative art.

A data structure / technique you might find of particular interest is Markov Chains. These provide a simple way to emulate the statistical character of a body of data. It's commonly used in music composition software, for example, to imitate various musical styles, by generating music which has the statistical character of a particular body of music samples. Another common use of Markov Chains is to emulate the prose style of a particular author (generating prose with Markov chains is a good way to learn the technique, since it's fairly simple). Google "markov chain text" or "markov chain prose" to find various examples.

Let's say you have a a body of text. You scan the text and discover that for the letter A, 30% of the time it's followed by T, 23.4% of the time it's followed by S, etc. You create a table, for each letter, of the probability of the letter being followed by another letter. Then you can use those statistics with the random number generator to produce text with the same properties. If you do the same with two-letters (AB has a X% chance of being followed by C), you get a better result, although it takes up more storage (instead of 26 letters, you have to store 26x26 letter pairs). This is called a 2nd order markov chain. As the order of the markov chain increases, it's resemblance to the original source text increases. 2nd and 3rd order chains produce very nice, but still random results.

The same technique works with music - you store tables for particular pitches followed by other pitches. And the same technique can work with vector art (although the programming will certainly be more complicated - you'll have to find a discrete set of ways of characterizing the strokes in your vector drawings, so you can build tables.

But the first step is getting your art into vector format.

asimes

Re: How can I incorporate my drawings into a Processing sketch without using an image verbatim?

1 year ago

jbum, could you describe this for drawings a little more? That sounds really interesting and I can imagine how this would work for text or sound but I have a hard time visualizing how it would work for drawings. Would you come up with some kind of aesthetic for drawing by storing series of strokes? If this is true would you take a new stroke and compare it to previous series of strokes and generate a series of strokes that should follow? I can't think of a way to make something generative like this that wouldn't be easily predictable.

jbum

Re: How can I incorporate my drawings into a Processing sketch without using an image verbatim?

1 year ago

I should start by admitting that I haven't actually *tried this* for vector or raster images (I have used Markov Chains a few times for text and music, as long ago as the mid-80s). But I think the idea is feasible, and could be really interesting. For sure it's more challenging, for the same reasons that graphics programming is harder than text processing.

Basically the challenge is coming up with a discrete vocabulary for describing drawings. I think you'd have to use a larger "alphabet" or "lexicon" than you use for text or music, but you don't want the alphabet to get too large, because the matrix will be too sparse (and then each symbol will only have one following symbol). You want the number of symbols to be small enough so that the average number of choices is >= 2, ideally. You want to avoid long strings of symbols with 1 choice, which do nothing more than parrot the source material.

There are tons of ways I can imagine to set this up - some raster, some vector.

For a vector based one - all pen strokes can be deconstructed into a series of end-to-end bezier curves. You could treat each set of 4 coordinates (for bezierVertex) as source for a symbol, and quantize them into, perhaps 100 symbols, based on such factors as the slope of the line connecting the endpoints, the distance of the end points, and the amount of curvature.

This adds up quickly though, 16 slopes x 5 lengths x 4 curvature characterizations = 320 "symbols". Might want to reduce the number of "slopes" to 8, and the curvatures to 3. 8*5*3 seems workable. You'd want to load in a lot of drawing data to generate enough symbols to make a dense chain.

In order to reproduce the subtleties of the original material, you can attach, to each symbol, all the "variants" or "instances" of that symbol you've encountered in the source data. So a bezier identified as A-B-C might have 30 or 40 real strokes which are representative of the A-B-C signature. Whenever the markov chain says to produce an A-B-C, you randomly use one of those stored strokes (at the current drawing offset), rather than simply playing back a quantized stroke.

By "quantized" I mean taking something with a lot of values, and reducing it to just a few characteristic values. For example the numbers between 1 and 50 could be quantized to 5 values (10,20,30,40,50). This is a common operation in music software, used to even out beats and such.

Idea: It might be interesting to look at the kinds of things that handwriting analysts look at... just a thought...

A raster based system, based on scanned illustrations would be easier to make (and I suspect if you google it, you'll find some) - you could do it in a reduced color space (e.g. 6 bit RGB) to reduce the number of symbols, and analyse RGB pixel pairings. So each pixel maps to a 6-bit number with 2 bits for R,G,B each (giving you 64 unique color values). Then you treat the pixels like letters in a text-based markov chain.

* * *

I should also mention that, Markov Chains aside, the Geomerative library is really a great tool for taking SVG or other vector data, and tweaking it. The are other ways it could be used in the spirit that the OP intended...

milespeyt..

Re: How can I incorporate my drawings into a Processing sketch without using an image verbatim?

1 year ago

Wow, thank you so much for the detailed response jbum.

It seems like the Geomerative library is a worthy first step, and I'll be playing with it in the next few days after converting some drawings to .svg's in Illustrator.

I've researched Markov chains per your suggestion -- I'm excited about their potential. Like you said, I'll have to reduce the drawings to discrete symbols, but the possibility is certainly there.

I'll update this thread when I've made progress on the project.

EDIT: Hm, it seems like Geomerative's RShape does not work with the new version of Processing. I get an error referring to processing.xml, which has been changed to XMLElement I think. Is there an updated library or alternative to Geomerative?

ricardmp

Re: How can I incorporate my drawings into a Processing sketch without using an image verbatim?

1 year ago

Hi, the new version of Geomerative should be compatible:

http://www.ricardmarxer.com/geomerative/geomerative.zip

By the way, kudos to Florian Jenett for sending me the patches to update the lib for the new version of Processing.

hellochar

Re: How can I incorporate my drawings into a Processing sketch without using an image verbatim?

1 year ago

There are also services out there to convert image formats into SVGs (e.g. http://image.online-convert.com/convert-to-svg), which you could then parse and grab data from.

Top Reply