Processing 1.0 - Processing Discourse

Huskee

Average RGB to LEDs
Feb 24^th, 2008, 9:40pm

Hi,

first off I really like what people are doing with Processing !! I'm very interested in visual arts and actually that is how I found your web site.

my project..:

Similar to the DigitalColor Meter by Apple I am trying to write Code that will analyze a specified area of the screen and calculate the percentage of Red, Green and Blue pixels within.

Actually DigitalColor Meter does exactly what I'm trying to do..

Depending on the colors and brightness of the pixels (and the overall Brightness of the screen) RGB values will then (hopefully some day) be output to an Arduino and used to control RGB LED's - 'Ambilight', you guessed it.

I've only started with Processing last week and am having a bit of a hard time figuring out how to use Arrays to do the job. Still working hard on my Processing vocabulary too.. I have no experience at programming..

.. but, the Percentage should be easily calculated like this:

AllRedValuesAddedUp (divided by) 255 * theNumberOfPixelsScanned

My Problem is very simple actually.. How to I add up the red values?

Any Help is greatly appreciated!
and sorry for beeing such a noob

antiplastik

Re: Average RGB to LEDs
Reply #1 - Feb 24^th, 2008, 10:39pm

By a specified area of the screen, I hope you mean a specified area of your sketch window? Have a look at the reference, and you'll get details on all needed methods to retrieve the RGB pixel values:
http://processing.org/reference/index.html

Basically, the idea is to call loadPixels() and then iterate the pixels[] array.

By the way, there is already a method to get the average brightness of one pixel :
http://processing.org/reference/brightness_.html

Huskee

Re: Average RGB to LEDs
Reply #2 - Feb 24^th, 2008, 10:56pm

Thanks for the quick response.
I don't think I'm coming across right.

The question is basically: How do you add up all elements (values) of an array and then divide that number by the number of elements in the Array..?

and no I actually meant the entire screen not just the sketch... that's a different problem though.
(Might be possible with a screen snapshot? though this probably eats up a lot of processing power..)

here's what I've managed to put together so far.. (I know it's ugly and doesn't do what I want .. yet!)

/**
* AverageColorCalculator RGB
* by the Noob: Huskee
*
* once finished this is supposed to look at the entire screen, or a defined portion
* extract the RGB color and brightness of each pixel or a defined number of pixels.
* calculate the average (percentage) of red, green, and blue on the screen and output
* this result to an Arduino for RGB LED lighting. got it? yes ... Ambilight like.
*
*any help greatly appreciated!!
*/

PImage img;

int cellSize = 20;
int cols, rows;
int numPixels;

void setup(){
img = loadImage("hotrod.gif");
size(img.width, img.height);
colorMode(RGB, 255,255,255,100);

cols = width / cellSize;
rows = height / cellSize;

numPixels = img.width/cellSize * img.height/cellSize;

}

void mousePressed(){
loop();
}

void draw (){
for (int i = 0; i < cols ; i++) {
for (int j = 0; j < rows ; j++) {

int x = i*cellSize;
int y = j*cellSize;
int loc = x + y*width;

float r = red(img.pixels[loc]);
float g = green(img.pixels[loc]);
float b = blue(img.pixels[loc]);

color c = color(r,g,b,100);

rectMode(CORNER) ;
fill(c);
noStroke();
rect(x,y,cellSize,cellSize);

println( " - RED: " + r + " GREEN: " + g + " BLUE: " + b );
if(i == cols);
noLoop();
}
}
}

(sorry didn't know how to add code nicely)

antiplastik

Re: Average RGB to LEDs
Reply #3 - Feb 24^th, 2008, 11:52pm

Quote:

and no I actually meant the entire screen not just the sketch... that's a different problem though.

yes. you'd better focus on that later ;-)

Quote:

The question is basically: How do you add up all elements (values) of an array and then divide that number by the number of elements in the Array..?

well, I would simply answer : iterate the array and sum its content using a variable, then divide the result by the array's length.

Code:

float avgRed = 0;
float avgGreen = 0;
float avgBlue = 0;
for (int i = 0; i < cols ; i++) {
  for (int j = 0; j < rows ; j++) {
    avgRed += red(img.pixels[loc]);
    avgGreen += green(img.pixels[loc]);
    avgBlue += blue(img.pixels[loc]);
  }
}
avgRed = avgRed/(cols*rows);
avgGreen = avgGreen/(cols*rows);
avgBlue = avgBlue/(cols*rows);

Quote:

(sorry didn't know how to add code nicely)

in Processing Menu : Tools > Copy for Discourse

« Last Edit: Feb 25^th, 2008, 12:41pm by antiplastik »

Huskee

Re: Average RGB to LEDs
Reply #4 - Feb 25^th, 2008, 12:24am

Thank you soo much,

I actually figured out pretty much the same just a few minutes ago. Kept me up til now .. couldn't sleep... must solve simple problem..(I live in germany and I have to get up at 5 am tomorrow).

took me almost three days to realize that what I was looking for was really just :

totalred += r;

adding this to the above code answers all my questions ... for now,
finally haha

damn.. so easy - thank you for your help!! I will be back for more, I promise!

Huskee

Re: Average RGB to LEDs
Reply #5 - Mar 5^th, 2008, 7:23am

Okay now this is what I have fixed up so far. It' still very messy and slow so any advice is welcome. the large square at the top left represents the final color resulting from the lerp function.

Robin

Quote:

PImage img;

int cellSize = 4;
int cols, rows;
int numPix;
int avgRed, avgGreen, avgBlue;
int brgtnss;
int factor = 40;
int f_red,f_green,f_blue;

void setup(){
img = loadImage("24.jpg"); // loads the Image
size(img.width, img.height);
colorMode(RGB, 255,255,255,100); // setting colormode

cols = width / cellSize;
rows = height / cellSize;

numPix = rows * cols;
}

void mousePressed(){
loop();
}

void draw (){
image(img,0,0);
for (int i = 0; i < cols ; i++) {
for (int j = 0; j < rows ; j++) {

int x = i*cellSize;
int y = j*cellSize;
int loc = x + y*width;

float r = red(img.pixels[loc]);
float g = green(img.pixels[loc]);
float b = blue(img.pixels[loc]);

float PixelBrightness = brightness(img.pixels[loc]);
brgtnss += PixelBrightness;

avgRed += r;
avgGreen += g;
avgBlue += b;

if(i == cols);
noLoop();
}
}
avgRed = avgRed/numPix;
avgGreen = avgGreen/numPix;
avgBlue = avgBlue/numPix;

brgtnss = brgtnss/(factor*numPix);

// calculates the dominant color and returns a factor between 0.? and 1
if ((avgRed >= avgGreen)&&(avgRed >= avgBlue)) {
f_red = 1;
f_green = avgGreen/avgRed;
f_blue = avgBlue/avgRed;
} else if ((avgGreen >= avgRed)&&(avgGreen >= avgBlue)) {
f_red = avgRed/avgGreen;
f_green = 1;
f_blue = avgBlue/avgGreen;
} else if ((avgBlue >= avgRed)&&(avgBlue >= avgGreen)) {
f_red = avgRed/avgBlue;
f_green = avgGreen/avgBlue;
f_blue = 1;
}

color a = color(avgRed,avgGreen,avgBlue,100); // simple average calculation

float c_red = brgtnss*avgRed;
float c_green = brgtnss*avgGreen;
float c_blue = brgtnss*avgBlue;
color b = color(c_red,c_green,c_blue,100); // brightness augmented color

float c_r = f_red*255;
float c_g = f_green*255;
float c_b = f_blue*255;

// dominant color is boosted to 100 percent
float d_red = brgtnss*c_r;
float d_green = brgtnss*c_g;
float d_blue = brgtnss*c_b;
color c = color(d_red,d_green,d_blue,100);

// dominant color (66%) and
// brightness augmented color (33%) calcualtions are put together.
color d = lerpColor(b,c,0.33);
float lerp_r = red(d);
float lerp_g = green(d);
float lerp_b = blue(d);

float m = millis();

println(brgtnss + " " + lerp_r + " : " + lerp_g + " : " + lerp_b + " | " + m + " msecs since start");

fill(a);
stroke(0);
rect (10,10,100,60);
fill(b);
rect (10,80,50,20);
fill(c);
rect (60,80,50,20);

fill(d);
rect (10,110,100,100);
}

acapulco

Re: Average RGB to LEDs
Reply #6 - Mar 5^th, 2008, 7:51am

I haven't executed your code, but one thing pops up: You are creating a new variable in each frame of the program.

What I mean is, when you do float number=5; for example, you are saying "create a new variable of type float, called number, and assing the value 5 to it".

If you do this inside the draw() loop, you will create a new variable over and over again. This might be causing performance problems, and if its not, I believe it's always a good idea to declare the variables at the top of the file (for example float number1,number2; int number3,number4;) and then just assign the value whenever needed (number1=1.55; number3=0);

Huskee

Re: Average RGB to LEDs
Reply #7 - Mar 5^th, 2008, 9:11am

thanks, completely makes sense what you're saying. I changed the code to the following which did make it faster:

if you do execute the code please give me some feedback on it!
much appreciated

Robin

Quote:

PImage img;

int cellSize = 4;
int cols, rows;
int numPix;
int avgRed, avgGreen, avgBlue;
int brgtnss;
int factor = 40;
int f_red,f_green,f_blue;
float r,g,b,c_red,c_green,c_blue,c_r,c_g,c_b;
float d_red,d_green,d_blue,lerp_r,lerp_g,lerp_b;

void setup(){
img = loadImage("24.jpg"); // loads the Image
size(img.width, img.height);
colorMode(RGB, 255,255,255,100); // setting colormode

cols = width / cellSize;
rows = height / cellSize;

numPix = rows * cols;
}

void mousePressed(){
loop();
}

void draw (){
image(img,0,0);
for (int i = 0; i < cols ; i++) {
for (int j = 0; j < rows ; j++) {

int x = i*cellSize;
int y = j*cellSize;
int loc = x + y*width;

r = red(img.pixels[loc]);
g = green(img.pixels[loc]);
b = blue(img.pixels[loc]);

float PixelBrightness = brightness(img.pixels[loc]);
brgtnss += PixelBrightness;

avgRed += r;
avgGreen += g;
avgBlue += b;

if(i == cols);
noLoop();
}
}
avgRed = avgRed/numPix;
avgGreen = avgGreen/numPix;
avgBlue = avgBlue/numPix;

brgtnss = brgtnss/(factor*numPix);

// calculates the dominant color and returns a factor between 0.? and 1
if ((avgRed >= avgGreen)&&(avgRed >= avgBlue)) {
f_red = 1;
f_green = avgGreen/avgRed;
f_blue = avgBlue/avgRed;
} else if ((avgGreen >= avgRed)&&(avgGreen >= avgBlue)) {
f_red = avgRed/avgGreen;
f_green = 1;
f_blue = avgBlue/avgGreen;
} else if ((avgBlue >= avgRed)&&(avgBlue >= avgGreen)) {
f_red = avgRed/avgBlue;
f_green = avgGreen/avgBlue;
f_blue = 1;
}

color a = color(avgRed,avgGreen,avgBlue,100); // simple average calculation

c_red = brgtnss*avgRed;
c_green = brgtnss*avgGreen;
c_blue = brgtnss*avgBlue;
color b = color(c_red,c_green,c_blue,100); // brightness augmented color

c_r = f_red*255;
c_g = f_green*255;
c_b = f_blue*255;

// dominant color is boosted to 100 percent
d_red = brgtnss*c_r;
d_green = brgtnss*c_g;
d_blue = brgtnss*c_b;
color c = color(d_red,d_green,d_blue,100);

// dominant color (66%) and
// brightness augmented color (33%) calcualtions are put together.
color d = lerpColor(b,c,0.33);
lerp_r = red(d);
lerp_g = green(d);
lerp_b = blue(d);

float m = millis();

println(brgtnss + " " + lerp_r + " : " + lerp_g + " : " + lerp_b + " | " + m + " msecs since start");

fill(a);
stroke(0);
rect (10,10,100,60);
fill(b);
rect (10,80,50,20);
fill(c);
rect (60,80,50,20);

fill(d);
rect (10,110,100,100);
}

acapulco Junior Member Offline Posts: 63	Re: Average RGB to LEDs Reply #8 - Mar 5^th, 2008, 9:20am Where exactly are you putting your image? in the data folder inside the sketch folder?

acapulco

Re: Average RGB to LEDs
Reply #9 - Mar 5^th, 2008, 9:25am

Nevermind my previous post.

Ok, I've executed the code.

What are your perfomance metrics? What I mean is, how do you notice the perfomance is low? I just opened a 1.7 MB image and it the output says
Code:


3   255.0 : 170.0 : 170.0 | 2893.0 msecs since start

What does that mean? Can you be more specific on what would your target perfomance be?

You are still declaring some variables inside the draw() method, like float m, color a,b,c,d (by the way "b" is declared twice with different types, which is not wrong but troublesome. It makes your code harder to read, so try to pick unique variable names). So you might want to check that out.

Also, in the nested loop where you run through the image, you do "x = i*cellSize;" inside the second for loop, meaning that you calculate x for every pixel, even when you just need to calculate for every row. You need to put that one only when "i" changes. I.e. pur it inside the first loop, but outside the second one, like this:

for (int i = 0; i < cols ; i++) {
x=i*cellSize;
for (int j = 0; j < rows ; j++) {
y=j*cellSize;
//other code
}
}

Even when you won't make any perceived differece with this change, little things like this can add up for really better perfomance. And even if it doesn't, it still makes your code cleaner and easier to read.

Huskee

Re: Average RGB to LEDs
Reply #10 - Mar 5^th, 2008, 1:36pm

Ok thanks for the advice! much appreciated!

What I am aiming for is a code that will calculate the "average" RGB and output the values to an Arduino Board. This will be controlling RGB LED's lighting up in the average desktop color. a kind of Ambilight.

THe entire process should not take much longer than half a second and a new Image should eventually be scanned every second (faster would be better of course!).

It is supposed to work for any app (meaning not depending on which app is running - e.g. iTunes visualizer, VLC, Photo Slideshow).

I am hoping that the java function Robot will make the screenshots

the sketch window is invisible in the final script - output is a Red Green and Blue Value between 0 and 255 about every second - or faster.

acapulco

Re: Average RGB to LEDs
Reply #11 - Mar 5^th, 2008, 6:44pm

Ok. I think I get the idea.

Well, as far as I can tell, the thing that impacts performance the most is actually running through the image, so I believe the time will mostly depend on the resolution.

Measure the speed of your program with different resolution images, say 320x240, 640x480, 800x600, 1024x768 and see what happens.

Obviously bigger images will take longer as you need to process more pixels, so, my advice would be to either use fixed sized images or maybe process less pixels with bigger images, at the cost of "average resolution".

What I mean is, say you get an acceptable performance using images of size 640x480. Then if you get an image of size 1280x960 (ie 4 times more pixels), then you evaluate not every (X,Y) location, but every other location, so you end up processing 4 times less pixels, which is the same as processing one full 640x480 image.

Did I explain myself correctly? Cause it seems a little confusing. What I mean is, just process the same number of pixels every time, regardless of image size. The drawback being that for bigger images, you will *proportionally* process less pixels, that's why I said you get less "resolution" on your average calculation.

Huskee

Re: Average RGB to LEDs
Reply #12 - Mar 5^th, 2008, 11:59pm

I agree with you and yes you explained yourself very comprehensively.

I have experimented with the cellSize integer thinking it would speed up the process which of course is wrong. A cell Size of about 5 (every fifth pixel) results in a color close to scanning all pixels. THis means that scanning every fourth or fifth pixel should be sufficient for my purpose.

I'm not so sure how I could express it in the code, that only every fifth pixel be processed.. Any ideas?

I will try to figure something out and post the result (if I find any) to this thread.

Robin

acapulco Junior Member Offline Posts: 63	Re: Average RGB to LEDs Reply #13 - Mar 6^th, 2008, 2:55am Just do something like if(j%5==0){ processPixel(); } Such that you get (0,0) is processed, (0,1)...(0,4) are not, then (0,5) is, then.... (1,0) processed...(1,5) processed etc. EDIT: "%" is the modulus operator. It takes the remainder of a division. Generally used to "loop around" fixed-length lists of things.

acapulco Junior Member Offline Posts: 63	Re: Average RGB to LEDs Reply #14 - Mar 6^th, 2008, 2:58am Or maybe you could divide the image into smaller "tiles", and then just randomly pick X number of pixels from each tile. Say you divide the image into 32 tiles, then take 10 random pixels from each tile. Maybe that will go faster, although I'm not sure how precise this method can be.