Writing to file goes slower and slower, what to do?

hejhej

Hi!

I ran into a problem when using the Serial library. I have an Arduino program that sends data through the serial transmitter. The arduino is connected to a Raspberry Pi where a processing program collects the data and prints them to a txt-file. The txt-file is overwritten every time there is new data from arduino. With the help of the serial library and Printwriter, this has worked well. Until now...

Now suddenly with each transmission, the writing to the file goes slower and slower. There are no error messages in the processing editor.

Any ideas?

Thanks in advance

Lord_of_the_Galaxy

How about writing to the file only when the program closes, storing the data until then. Or maybe use multithreading, though that would be difficult.

hejhej

I assume you mean just writing to the file one time? The purpose of the txt-file is for it to work as a database that is updated continuously. Here's my code:

import processing.serial.*;                    //Imports the serial library
int lf = 10;                                   //Linefeed in ASCII
String myString = null;                        //Empties the value of myString
String previousString = null;                  //The last string printed to the txtfile
Serial mySerial;                               //the serial port
PrintWriter output;                            //Sets the PrintWriter to output

/** the setup-method runs once*/
void setup() {
  // Stores the second serial-port in the portName string
  String portName = Serial.list()[1];
  // Opens the chosen serial-port, on the right baudrate, the same as on the Arduino
  mySerial = new Serial( this, portName, 9600);
  /** Throws out the first reading, in case the reading started
  in the middle of a string from the sender*/
  mySerial.clear();
  // Reads and stores the latest json string from the serial transmitter
  myString = mySerial.readStringUntil(lf);
  // Empty myString
  myString = null;
}

/** the draw-method runs until the program stops*/
void draw() { 
  // If the serial port is available, read the latest json string 
  if (mySerial.available()> 0) {
    myString = mySerial.readStringUntil(lf);

    /** Every time the returned json string have a new and valid value(not null) 
    a txt.file is created in the sketch directory, overwriting the previous one.
    */
    if (myString != null && myString != previousString) {
      // Set the most recently r
      previousString = myString;
      output = createWriter( "data.txt" );
      // the json string is printed into the new txt file
      output.println( myString );
      // Writes the remaining data to the file
      output.flush();
      // Finishes the file
      output.close();
    }
  }
}

Do you have any other suggestions? Thanks for your time : )

koogs

put some timing debug printlns around the actual writing.

and give us some idea of the data that's being written. it's not just a case that the data is growing with time, is it?

(and why are you overwriting the same file again and again?)

hejhej

As I mentioned I use the txt-file as a database. The values that I get in from arduino comes in the form of a json string {"in":x, "out":y}. Then I have a JavaScript that reads the values of the json-objects. I overwrite the file since the program will be running for a long time, and I don't want the file to be too large.

Lord_of_the_Galaxy

JavaScript? Where?

hejhej

The JavaScript is irrelevant to the problem. I figured out the slowdown was due to a memory leakage. The conditions for creating the file was always true, since I mistakenly used != comparison when I really should have used !string1.equals(string2). When I changed that, it solved the problem.

jeremydouglass

@hejhej -- thank you for sharing how you found the problem and your resolution.