We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpPrograms › How to use image data for motion
Page Index Toggle Pages: 1
How to use image data for motion? (Read 3107 times)
How to use image data for motion?
Mar 1st, 2010, 7:27am
 
Hello, I need help, basically I need to understand how the process of moving particles following a noise heightmap works.

So I can draw a noise heightmap, throw a bunch of particles and the particles will move to the white areas.

Links, tutorials, algorithms, logics anything will much appreciated

Cheers
rS
Re: How to use image data for motion?
Reply #1 - Mar 1st, 2010, 9:02am
 
I made this a while back, on a whim. It sounds like it might be a good starting point for you...

Quote:
int WIDTH = 200;
int HEIGHT = 200;
int ENTITIES_COUNT = 400;

class Entity {
  int myX;
  int myY;
  int myMood;
  color myColor;
  
  Entity(){
    myX = int(random( width ));
    myY = int(random( height ));
    myMood = 1;
    myColor = color( 0,random(255),random(255) );
  }

  void simulate(){
    if( get(myX, myY) != color(0) ){
      myMood=7;
    } 
    else if( myMood > 1) {
      myMood--;
    } 
    else {
      myMood = 1; 
    }
    if( get(myX, myY) == color(255,0,0) ){
      myX++;
    } 
    if( get(myX, myY) == color(128,0,0) ){
      myX--;
    } 
    
    strokeWeight( myMood );
    stroke(myColor);
    point( myX, myY );
    myX+=int(random(3));
    myX--;
    myY+=int(random(3));
    myY--;
  }
}

Entity[] Entities;

void setup() {
  size( WIDTH, HEIGHT );
  Entities = new Entity[ENTITIES_COUNT];
  for( int i = 0; i< Entities.length; i++) {
    Entities[i] = new Entity();
  }
}

void draw() {
  background(0);
  noStroke();
  fill(255,0,0);
  rect( 20, 20, 80, 30 );
  fill(128,0,0);
  rect( 20, 70, 80, 30 );
  for( int i = 0; i< Entities.length; i++) {
    Entities[i].simulate();
  }
}

Re: How to use image data for motion?
Reply #2 - Mar 2nd, 2010, 1:23am
 
Thanks for the sample code.

Now I need to work out the following (I post this logic in another post but nobody answer)
Quote:
The process is as following.

1->Generate Perlin in black and white

2->Generate text, place in center

3->Imagine the whiteness to be a height value. Darker == lower, ligher == higher.

4->Convert the bitmap into a slope / normal map. Follow this process :
   1.Convolution with a 3x3 kernel, y-1 and y+1 as 1,div by 2 bias 127, copy to red channel
   2.Convolution with a 3x3 kernel x-1 and x+1 as 1, div by 2, bias 127, copy to blue channel

Now red and blue are converted into slope values for every pixel. Use this slope value to accelerate or decelerate your particles.

Use a linked list with small datatypes for your particles. Read out from the slope map to move them, add some elastics...et voila.

A position on your normal / slope map has a color. This color encodes x in red and y in green. If you read the green value, you divide it's value by 128 and -1 one that value.

That will give you a value between -1 and +1. Depending on the speed you want to add to the particles, you multiply that value and add it to the velocity of the particle. Same goes for the red channel.


I am having problems understanding how to proceed in step 4, any ideas?

Cheers
rS
Re: How to use image data for motion?
Reply #3 - Mar 2nd, 2010, 8:46am
 
Hope to make this more clear, this is what I am trying to work out

http://en.wikipedia.org/wiki/Image_gradient

It looks like the sobel operator may help to build my kernel, I will post some code if I get this working

Cheers
rS
Re: How to use image data for motion?
Reply #4 - Mar 4th, 2010, 2:41am
 
Ok guys I am almost there but got stuck, from this link http://en.wikipedia.org/wiki/Image_gradient they said I can use the Sobel Operator to get the gradient values, so I did.

I manage to get the same results using the same image like they show here http://en.wikipedia.org/wiki/Sobel_operator, so far so good.

From the algorithm I post previously, they said Quote:
1.Convolution with a 3x3 kernel, y-1 and y+1 as 1,div by 2 bias 127, copy to red channel
2.Convolution with a 3x3 kernel x-1 and x+1 as 1, div by 2, bias 127, copy to blue channel


So I did (I think)
Code:
for (int x = 1; x < width - 1; x++) { // Skip left and right edges
   for (int y = 1; y < height - 1; y++) { // Skip top and bottom edges
     
     float cX = convolution(x, y, kernelX, noiseImg);
     float cY = convolution(x, y, kernelY, noiseImg);
     cX = (cX / 2.0) - 127.0;
     cY = (cY / 2.0) - 127.0;
     
     // gradient vector
     PVector c = new PVector(cX, cY);
     
     // For this pixel in the new image, set the gray value
     // based on the sum from the kernel
     sobelImg.pixels[y * width + x] = color(c.mag());
   }
 }


Here is the important bit
The gradient image looks right, but if I print the results at a specific mouse location, I get negative values, and that is not right, because if using the sobel operator I get a gradient vector, and use this gradient vector to move the particles, they will always go to the top-left


Here is the sketch, please note that I haven't done any optimization
Code:
import toxi.math.noise.*;

// sobel operator kernels
// http://en.wikipedia.org/wiki/Sobel_operator
float[][] kernelX = { {  1,  0, -1  },
                     {  2,  0, -2  },
                     {  1,  0, -1  } };

float[][] kernelY = { {  1,  2,  1  },
                     {  0,  0,  0  },
                     { -1, -2, -1  } };

float NS = 0.01; // noise scale (try from 0.005 to 0.5)
float noiseOffset = 100.0;

PImage noiseImg, sobelImg;


void setup() {
 size(300, 225, P3D);
}


void draw() {
 // generate noise canvas
 noiseImg = createImage(width, height, RGB);
 for (int i = 0; i < width; i++) {
   for (int j = 0; j < height; j++) {
     float noiseVal = 0.0;
     // for noise motion add noiseOffset
     //noiseVal = (float)SimplexNoise.noise(i * NS + noiseOffset, j * NS + noiseOffset, frameCount * 0.02);
     // for static noise
     noiseVal = (float)SimplexNoise.noise(i * NS, j * NS, frameCount * 0.02);
     int c = (int)(noiseVal * 127 + 128);
     
     noiseImg.set(i, j, c << 16 | c << 8 | c | 0xff000000);
   }
 }
 // render noise image
 //image(noiseImg, 0, 0);
 noiseImg.loadPixels();
 
 // Create an opaque image of the same size as the original
 sobelImg = createImage(width, height, RGB);
 
 // Loop through every pixel in the image.
 for (int x = 1; x < width - 1; x++) { // Skip left and right edges
   for (int y = 1; y < height - 1; y++) { // Skip top and bottom edges
     
     float cX = convolution(x, y, kernelX, noiseImg);
     float cY = convolution(x, y, kernelY, noiseImg);
     cX = (cX / 2.0) - 127.0;
     cY = (cY / 2.0) - 127.0;
     
     // gradient vector
     PVector c = new PVector(cX, cY);
     
     // For this pixel in the new image, set the gray value
     // based on the sum from the kernel
     sobelImg.pixels[y * width + x] = color(c.mag());
   }
 }
 sobelImg.updatePixels();
 // render sobel resulting image
 image(sobelImg, 0, 0);
 
 if (mousePressed) {
   float cX = convolution((int)mouseX, (int)mouseY, kernelX, noiseImg);
   float cY = convolution((int)mouseX, (int)mouseY, kernelY, noiseImg);
   cX = (cX / 2.0) - 127.0;
   cY = (cY / 2.0) - 127.0;
   
   PVector c = new PVector(cX, cY);
   //c.normalize();
   println(c);
 }
 
 //println("fps: " + frameRate);
}


float convolution(int x, int y, float[][] kernel, PImage img) {
 float sum = 0.0; // Kernel sum for this pixel
 
 for (int kx = -1; kx <= 1; kx++) {
   for (int ky = -1; ky <= 1; ky++) {
     // Calculate the adjacent pixel for this kernel point
     int pos = (x + kx) + (y + ky) * width;
     // Image is grayscale, red/green/blue are identical
     //float val = red(img.pixels[pos]);
     float val = img.pixels[pos] >> 16 & 0xFF;
     // Multiply adjacent pixels based on the kernel values
     sum += kernel[ky + 1][kx + 1] * val;
   }
 }
 
 return sum;
}


Hope that makes any sense, and any help at this point will be much appreciated

Cheers
rS
Re: How to use image data for motion?
Reply #5 - Mar 4th, 2010, 9:49am
 
Where did you get this from..

nardove wrote on Mar 4th, 2010, 2:41am:
1.Convolution with a 3x3 kernel, y-1 and y+1 as 1,div by 2 bias 127, copy to red channel
2.Convolution with a 3x3 kernel x-1 and x+1 as 1, div by 2, bias 127, copy to blue channel


The number you get from convolving with each kernel will be somewhere around zero (ie might be positive or negative).

Since you can't have 'negative' color values, if you want to visually represent the result, you need to shift everything up into the positives. Mid-grey is (about) 127, so adding 127 will give values centred around 127. You are subtracting 127, hence all your results are going to show negative.

To use these values (rather than display them), you are probably going to want to have them as the 'raw' values, so not shifted to centre around 127, but remain centred around 0. If you save a pixel buffer with modified values, to get them back to raw you subtract the offset and multiply by 2. There is, of course, a small rounding error in this due to the previous division.

Consider:

Code:
float cX = convolution(x, y, kernelX, noiseImg);
float cY = convolution(x, y, kernelY, noiseImg);

// gradient vector; use this for accelerating object
PVector c = new PVector(cX, cY);
     
// Ramp up the differences so we can see them! x20 seems nice
// Also, subtract instead off add to get "light from top-left"
// visual effect (only because most people are used to things
// being lit from above)
cX = 127 - (cX * 20.0);
cY = 127 - (cY * 20.0);

// Display as red/green image for horiz/vert gradient
sobelImg.pixels[y * width + x] = color(cX, cY, 0);


-spxl
Re: How to use image data for motion?
Reply #6 - Mar 6th, 2010, 5:59am
 
That works, subpixel save the day! I have a particle and its always falling into the dark areas, just what I was looking for.

I need to finish the particle class, and do some optimization, at the moment a 640x480px sketch runs at 18fps and 1 particle! not good.

Will post the source once I finish

Many thanks
rS
Re: How to use image data for motion?
Reply #7 - Mar 7th, 2010, 4:30am
 
Suggestion: there is no need to apply the Sobel operator to the entire image if you do not intend to display the "Sobel image"; you only need to calculate it at the coordinates you are interested in (that is, at the particle coordinates).
Re: How to use image data for motion?
Reply #8 - Mar 7th, 2010, 5:08am
 
Further suggestions:

In my tests, using P3D is about 4fps slower than using P2D or JAVA2D.

I used Processing's noise(a,b,c) method instead of toxi's library, so can't comment on toxi's library specifically, but can say that computing the noise image in every frame is what is killing your frame rate.

a) Generate your noise map at a lower resolution

b) Generate the noise map less often (either partly generate on each frame, or generate the noise map in a separate thread)

I suggest trying both, combined.

-spxl
Re: How to use image data for motion?
Reply #9 - Mar 7th, 2010, 8:35am
 
Sample sketch on OpenProcessing:

spxlImageGradient

Optimisation for mode == 5 is incomplete; the intention is to replace the "per pixel" inline code with code that references data/calculations from the previous pixel going along each row; only 3 noiseImg.pixel[] values should need to be accessed, not 8 (or 9 x 2, as in the original!).

-spxl
Re: How to use image data for motion?
Reply #10 - Mar 8th, 2010, 7:21am
 
Hi subpixel, thanks a lot for the sample code, its very helpful.

Now after spending some time over the weekend trying different parameters, and optimizing all the for loops, I figure that the problem with the frame rate is trying to draw and calculate the noise for the entire canvas on every frame, that is a killer, and for my needs I can do with a transition between a set of black and white images just fine, and calculate the gradient for every particle using their location vector, so its is all good. The noise was just for testing.

Now play time!

Cheers
rS

Re: How to use image data for motion?
Reply #11 - Mar 8th, 2010, 9:05am
 
Note my earlier comment, "computing the noise image in every frame is what is killing your frame rate."

Even so, gaining an overall 35%+ framerate improvement by optimising the Sobel operator isn't too shabby!

-spxl
Page Index Toggle Pages: 1