Reduce delay in writing data to a graph

ferret4king

I am trying to continuously show data as a graph in real-time. The data is from a sensor connected to an Arduino. After Processing has run for some time (usually about 1:30 min) there is a significant delay between the change in value and what is being drawn on screen. As the program runs for longer i see multiple spikes in the graph above and below the "actual" data read from the sensor. This is the Processing code:

import processing.serial.*;

Serial myPort;
//String dataRead;
String[] dataOutput = {};

int xPos = 1;    // horizontal graph position
// variables to draw a continuous line
int lastxPos = 1;
int lastHeight = 0;

void setup()
{
  //set window size
  size(800,600);

  //print list of available serial ports
  println(Serial.list());

  myPort = new Serial(this, Serial.list()[0], 9600);
  myPort.bufferUntil('\n');

  background(0);
}

void draw()
{
  // nothing else happens here but must be included! 
}

void serialEvent(Serial myPort)
{
  // get ASCII string
  String inString = myPort.readStringUntil('\n');
  dataOutput = append(dataOutput, inString);
  saveStrings("new_data.txt", dataOutput);

  if(inString != null)
  {
    inString = trim(inString);                 // trim whitespace
    float inByte = float(inString);            // convert to a number
    inByte = map(inByte, 0, 5, 0, height);  // map to screen height

    // Draw a line from last inByte to new one
    stroke(255,255,255);
    strokeWeight(1);
    line(lastxPos, lastHeight, xPos, height - inByte);
    lastxPos = xPos;
    lastHeight = int(height - inByte);

    // return to beginning of frame once boundary has been reached
    if(xPos >= width)
    {
      xPos = 0;
      lastxPos = 0;
      background(0);
    }
    else
    {
      xPos++;
    }
  }
}

This is the Arduino code that is detecting the value from the sensor:

int light;
int sensor = A0;

void setup()
{
  pinMode(sensor, INPUT);
  Serial.begin(9600);
}

void loop()
{
  light = analogRead(sensor);
  float val = (5.0/1024.0)*light;
  Serial.println(val);
}

I need to reduce the delay in Processing and remove the unwanted spikes as the data is being saved as a .txt file so that it can be used in Matlab for analysis.

Any ideas?

I am running Windows 7 on a core i3.

quark

The problem is probably caused because the arduino to producing data faster that Processing is consuming it.

Try putting a small delay inside the loop() method.

Alternatively you might try

void serialEvent(Serial myPort)
{
  // Keep get ASCII strings until none left
  if (null != (inString = myPort.readStringUntil('\n')))
  {
    dataOutput = append(dataOutput, inString);
    saveStrings("new_data.txt", dataOutput);
    inString = trim(inString);                 // trim whitespace
    float inByte = float(inString);            // convert to a number
    inByte = map(inByte, 0, 5, 0, height);  // map to screen height

      // Draw a line from last inByte to new one
    stroke(255, 255, 255);
    strokeWeight(1);
    line(lastxPos, lastHeight, xPos, height - inByte);
    lastxPos = xPos;
    lastHeight = int(height - inByte);

    // return to beginning of frame once boundary has been reached
    if (xPos >= width)
    {
      xPos = 0;
      lastxPos = 0;
      background(0);
    } else
    {
      xPos++;
    }
  }
}

ferret4king

I tried your amendment and the delay still occurs. I have added a 5ms delay to the Arduino sketch but can't add a greater delay as the detection needs to be very fast. I expect some sort of delay in Processing but as the program runs for longer the delay between change in value and what is shown on the graph becomes very large. The main issue is the random spikes that occur though. I have attached the image:

I am wondering if it is due to the delay between the read and written data in Processing.

Thanks for your help. Any further suggestions?

colouredmirrorball

I think it's a memory issue caused by line 35: dataOutput = append(dataOutput, inString);. What the append() function does is create a new array with one more element in it! The old array will eventually get erased by the garbage collector, but clutters memory in the mean time. Also, after a couple minutes, the dataOutput array will have a gigantic size. It's better to create an array with a length equal to width (the number of horizontal pixels) then fill it in with the incoming data, then reset the iterator when it reached width.

The remaining problem is writing the data to the file. What you're doing now is overwriting the text file with each new incoming data point. This is of course not ideal. I think what you need to use is createWriter, but that's something I haven't worked with before. I believe this allows you to add strings to the PrintWriter object which you can "flush" to the text file (for example at each reset).

ferret4king

That makes sense. I removed the append function and saw that the delay had been reduced and there were no longer any spikes even after running for several minutes. This suggests to me that the problem is caused by trying to save the data in an array as per your comment. However, i do not know how to implement your suggestion. I'm not really a programmer.

Basically i need to monitor the graph generated in real time but i also need to save that data to a file so it can easily be read in Matlab i.e a .txt file. The sensor could be running for a long time so is there a way of resetting the array after the .txt has been saved and then appending that .txt file once the "new" array has been filled? Any tips/ideas?

Thanks

colouredmirrorball

I think this works. It compiles but can't test. It writes a line at each data point, not sure if that's what you want. But I got rid of the array dataOutput in its entirety.

import processing.serial.*;

Serial myPort;
//String dataRead;
//String[] dataOutput;
PrintWriter writer;

int xPos = 0;    // horizontal graph position
// variables to draw a continuous line
int lastxPos = 1;
int lastHeight = 0;

void setup()
{
  //set window size
  size(800,600);

  //print list of available serial ports
  println(Serial.list());

  myPort = new Serial(this, Serial.list()[0], 9600);
  myPort.bufferUntil('\n');

  //dataOutput = new String[width];
  writer = createWriter("new_data.txt");

  background(0);
}

void draw()
{
  // nothing else happens here but must be included!
}

void serialEvent(Serial myPort)
{
  // get ASCII string
  String inString = myPort.readStringUntil('\n');

  //dataOutput = append(dataOutput, inString);
  //dataOutput[xPos] = inString;
  //saveStrings("new_data.txt", dataOutput);

  if(inString != null)
  {
    writer.println(inString);
    inString = trim(inString);                 // trim whitespace
    float inByte = float(inString);            // convert to a number
    inByte = map(inByte, 0, 5, 0, height);  // map to screen height

    // Draw a line from last inByte to new one
    stroke(255,255,255);
    strokeWeight(1);
    line(lastxPos, lastHeight, xPos, height - inByte);
    lastxPos = xPos;
    lastHeight = int(height - inByte);

    xPos++;
    // return to beginning of frame once boundary has been reached
    if(xPos >= width)
    {
      xPos = 0;
      lastxPos = 0;
      background(0);
    }

  }

}

void exit()
{
  writer.flush();
  writer.close();
}

ferret4king

Thanks for that mate. Had the script running for several minutes without trouble.