Averaging a few seconds of a video feed pixel

scottreinhard

I would like to display a video pixel's average over a couple of seconds. What would be the best method to.

To make the question simple, let's say I want to get the color of the center pixel of a video feed and fill a rectangle the size of the window to this particular color. To mitigate the flickering of this colored rectangle which will naturally occur from the video, I would like to average, say, the color of the last 60 frames of the video and always have the rectangle displaying that averaged color.

What would be an approach to making this happen?

jeremydouglass

Is this realtime (while the video is playing live) or not? Can you pre-process the video(s) when the sketch starts, or do you not know what video you will be showing / are you dealing with a live stream?

Do you always know ahead of time how long you will be sampling (always 60 frames)?

By average do you need the mean, median, or mode? From your description, it sounds like mean or median would be fine.

1. A formal way uses an array of 60 pixels, then compute the average. You can keep changing which array index you update each frame based on an index variable that climbs to 60 and resets -- or, if you are feeling fancy, you can use a first-in-first-out queue data structure.

2. OR a very simple and fast way (but inflexible) is to save your average value in one pixel, then average it with the new pixel at a weight of 59:1 (or whatever).

3. A Processing-esque way of writing approach 2 is to use lerpColor on your old and new pixel, with an amt setting of 1/60. This will "smooth" sudden color changes. It won't give the exact same results as a straight mean, but it sounds like that doesn't matter.

scottreinhard

Thank you for your answer.

A few followups: Live video. Safe to assume that I will know how many frames I'd be sampling. I'm referring to the mean (add the values and divide by number of values). Doesn't have to be 60 frames, would probably experiment a bit to find the right balance of shift in color vs. being to jumpy.

Although not a beginner, arrays are not my strong suit. Could you point me to a straightforward method of averaging the values of an array?

Many thanks.

jeremydouglass

Here is a very simple example sketch that demonstrates simple color smoothing using lerpColor (rather than an array) just to play with the concept.

// Averaging Color
// https: //forum.processing.org/two/discussion/18894/averaging-a-few-seconds-of-a-video-feed-pixel
// 2016-12-20

color pxSmooth;
float sensitivity = 6.0/64;
void setup(){
  size(200,200);
  pxSmooth = get(width/2,height/2);
  noStroke();
  frameRate(8);
}
void draw(){
  background(255);
  // simulate a flickery video
  fill(random(255),random(255),random(255),random(255));
  rect(0,0,20,20);

  // smooth color
  pxSmooth = lerpColor(pxSmooth, get(10,10), sensitivity);
  // show smoothed color
  fill(pxSmooth);
  rect(20,20,width-20,height-20);
}

You may want to test with sampling averaged pixel areas from live video. Here the fake input is purely random, so if you turn the sensitivity down too much or the sample rate up too much you start to get a perfectly gray output (because the random values cancel). It could be improved by using a hue-based color space, doing a random walk, and wrapping around values.

You can create an array of colors using either an array or an ArrayList:

color[] colorArray = new color[60];

color[] colorArray = { 
  color(20, 134, 204),
  color(61, 118, 153), 
  color(0, 255, 216)
};

or:

ArrayList<Integer> colorList = new ArrayList<Integer>();
colorList.add(color(20, 134, 204));
colorList.add(color(61, 118, 153));
colorList.add(color(0, 255, 216));

(Under the hood Processing stores these colors as simple ints using a special format that keeps the RGB channels in the int bits.)

See:

• https://processing.org/reference/arrayaccess.html
• https://processing.org/reference/Array.html
• https://processing.org/reference/ArrayList.html