How to make movie from real-time sensor data with lower fps-rate look more natural?

peter_h

I have a problem that is maybe impossible to solve, but I thought I’ll give it a try, maybe I did overlook something.

I want to make a video capture from my project that works with a pulse sensor (using Arduino). It shows a wave form of your heartrate from the sensor data in real time. The video will be made out of frame captures using saveFrame(). Of course, this slows down the fps of the sketch pretty much and this is exactly the problem: while getting the data in real time, Processing draws the waveform really slow, so naturally it looks like my heartrate is extremely high when I convert the frames into video. I could lower the fps in the output when converting the video, but this results in frame rates like 8 fps to make it look natural, but the movement wouldn’t be smooth at all. Another flaw is that the waveform is more dense, because there is more input data per second, but that wouldn’t hurt too much if I could get a slower animation.

The main issue is that I can’t make the movement slower, because in the sketch (which I adapted from a sketch from Joel Murphy who made the PulseSensor: http://pulsesensor.myshopify.com/) the waveform moves by first adding the incoming data to the last item of an array (RawY[]) and then shifting it one „pixel“ to the left, like RawY[i+1] (see code below) in the loop that draws the waveform. Since I have to use integer numbers I cannot just shift it a „half pixel“, if that even makes sense.

I'd appreciate any ideas how I could solve this problem. Screen capture wouldn't help either, because in my project there is much stuff going on besides the heartbeat measurement, so it would be slow even without the saveFrame() actions. I don't know much about video making, so maybe there is a workaround to make it look smooth even at lower fps?

I'll show only the important parts of the code:

This is the setup:

int PulseWindowWidth = 490;
int PulseWindowHeight = 512; 

  RawY = new int[PulseWindowWidth];          // initialize raw pulse waveform array
  ScaledY = new int[PulseWindowWidth];       // initialize scaled pulse waveform array
  zoom = 0.75; //0.75   

Serial events from Arduino (baud rate is 115200, if that helps somehow):

    if (inData.charAt(0) == 'S') {          // leading 'S' for sensor data
      inData = inData.substring(1);        // cut off the leading 'S'
      Sensor = int(inData);                // convert the string to usable int
    }

Draw loop:

// DRAW THE PULSE WAVEFORM
  // prepare pulse data points    
  RawY[RawY.length-1] = (1023 - Sensor) - 212;   // place the new raw datapoint at the end of the array
  zoom = scaleBar.getPos();                      // get current waveform scale value
  offset = map(zoom,0.5,1,150,0);                // calculate the offset needed at this scale
  for (int i = 0; i < RawY.length-1; i++) {      // move the pulse waveform by
    RawY[i] = RawY[i+1];                         // shifting all raw datapoints one pixel left
    float dummy = RawY[i] * zoom + offset;       // adjust the raw data to the selected scale
    ScaledY[i] = constrain(int(dummy),44,556);   // transfer the raw data array to the scaled array
  }
  stroke(250,0,0);                               // red is a good color for the pulse waveform
  noFill();
  beginShape();                                  // using beginShape() renders fast
  for (int x = 1; x < ScaledY.length-1; x++) {    
    vertex(x+10, ScaledY[x]);                    //draw a line connecting the data points
  }
  endShape();

GoToLoop

I wonder if you should just accumulate all of the input data in a IntList: :-?

final IntList inputs = new IntList();
if (inData.charAt(0) == 'S')  inputs.append(int(inData.substring(1)));

Once finished save inputs to disk. Then have another sketch which will render frames based on that input data.
This way, rendering & saveFrame() speeds aren't an issue anymore! *-:)

• https://processing.org/reference/IntList.html

peter_h

Thanks, that's a great idea! I'll try that and report back.

peter_h

Thanks to your solution it works fine now. :)

Compare this:

To this (both are screenshots from the video results):

In my final sketch I have to isolate the recording of the pulse to not let the other stuff drag down the bpm.

arduinosmith

Hello peter_h,

I'm sitting here with a very similar problem and was wondering if you could post your code or at least snippets from it. I've been working with arduino for a while, but am very new to processing. This seems like a great solution and I would appreciate any help. Thanks a lot, your project looks cool!