append a text file

edited January 2017 in Arduino

Hello everyone, I know this has a lot of people talking about it, but I haven't found what I need. I checked a lot of this forum, learned a lot, since it is my first java experience. I have an arduino with sensor, that is sending via serial to the PC. I already figured out how to get the info using printWriter, but my problem is when I stop the program and run it again, it deletes the info in the old file and then adds the new data. is it possible to have the processing add to an existing file when you start it? This is my program

import processing.serial.*; 
Serial myPort;    // The serial port
PFont myFont;     // The display font
String inString;  // Input string from serial port
int lf = 10;      // ASCII linefeed 
String timestamp = "";
PrintWriter output;

void setup() { 
  output = createWriter("data.csv"); 
  size(400, 200); 
  myFont = loadFont("ArialMT-18.vlw"); 
  textFont(myFont, 18); 
  // List all the available serial ports: 
  printArray(Serial.list()); 

  myPort = new Serial(this, Serial.list()[1], 9600); 
  myPort.bufferUntil(lf);
} 

void serialEvent(Serial p) { 
  inString = p.readString();
  inString=trim(inString);
  timestamp=year()+"/"+month()+"/"+day()+" "+hour()+":"+minute()+":"+second()+"," ;
  output.write(timestamp + inString +"\n");
  output.flush();  // Writes the remaining data to the file
  if (output==null) {
    output.close();
  }
}

void draw() { 
  background(0); 
  text("received: " + inString, 10, 50);
  text("at " + timestamp, 10, 80);
} 

Answers

  • edited January 2017

    Thank you. I had to redo some processing for the message received because I was sending it with commas, (sending values of multiple sensors), so I had to do some work on the message received.

    Here it is if anyone needs it:

    import processing.serial.*; 
    Serial myPort;    // The serial port
    PFont myFont;     // The display font
    String inString;  // Input string from serial port
    int lf = 10;      // ASCII linefeed 
    String timestamp = "";
    Table table;
    String TrackDis;
    String comp;
    String inStringBuf;
    int safety;
    
    void setup() { 
      size(400, 200); 
      myFont = loadFont("ArialMT-18.vlw"); 
      textFont(myFont, 18); 
      // List all the available serial ports: 
      printArray(Serial.list()); 
    
      myPort = new Serial(this, Serial.list()[1], 9600); 
      myPort.bufferUntil(lf);
    }
    void serialEvent(Serial p) { 
      delay(10);
      inString = p.readString();
      safety=parseInt(inString);
      if (safety<=28) {
        inString=trim(inString);
        inStringBuf=inString;
        timestamp=year()+"/"+month()+"/"+day()+" "+hour()+":"+minute()+":"+second() ;
        table = loadTable("data.csv", "header");
        TableRow newRow = table.addRow();
        int k=0;
        int c=1;
        int j=0;
        for (int i=0; i<inString.length(); i++) {
          inStringBuf=inString;
          comp=inStringBuf.substring(i, i+1);
          if (comp.equals(",")==true) {
            inStringBuf=inString;
            j=i-1;
            TrackDis=inStringBuf.substring(k, j);
            newRow.setString("Timestamp", timestamp);
            newRow.setString("Col"+c, TrackDis);
            k=i+2;
            c++;
            saveTable(table, "data.csv");
          }
        }
      }
    }
    
    
    void draw() { 
      background(0); 
      text("received: " + inString, 10, 50);
      text("at " + timestamp, 10, 80);
    } 
    
  • edited January 2017
    • Loading & saving operations slow down everything! :-&
    • Doing both inside serialEvent() can even impact the speed that bytes are received! :-SS
    • Preferably we should load once inside setup() or settings(), before draw() starts.
    • And save when sketch exit() or whenever the user requests it. :-B
  • Or if you have the know-how, add the save to a queue that will save the file in a separate thread. The loading only needs to be done once even here.
    And an even better approach would be coupling it with you own version of saveTable (name it addRowToSave or something like that) which appends to the file instead of rewriting it every time.

  • Actually I was looking for something that only appends to the file and doesn't have to rewrite every time. Couldn't do it with my very little knowledge with java. This is my 3rd day working on java, ever.

    I don't have a lot of bytes received i have only 9 float numbers (4 digits each). And time is not a big issue for me. If someone can help in optimizing it, that would be good especially for people in the future.

    Is there some sort of a function other than draw than can loop? like "loop" in arduino

  • Anything inside a while(true){} block will loop indefinitely.

  • If you are not flooded with data then you can just store the data in memory and save it at the end as it mentioned in other posts. The idea is you shouldn't be making to many calls to file IO specially in serialEvent. If you save data in an ArrayList for example, then you can trigger to save data once you reach a fixed number of entries. You flush the data into your file, reset the array and keep collecting data. Or you can save the data every so often after so many milliseconds or after so many frames (if(frameCount%600==0){ SAVE DATA HERE NOW } which is roughly every 10 seconds). Lastly, you can trigger saving data after a key mo mouse event as you will see in one of the posts below.

    You could also create a a separate thread to handle the task. However I don't think is needed here.

    There is some code to append to a file that you can find under the following search: https://forum.processing.org/two/search?Search=filewriter

    Relevant posts:
    https://forum.processing.org/two/discussion/14595/simple-file-operations/p1
    https://forum.processing.org/two/discussion/comment/47934/#Comment_47934 https://forum.processing.org/two/discussion/561/easiest-way-to-append-to-a-file-in-processing

    From most of these examples, the file stream is open and close every time the data is saved.

    Lastly, you can google ways to append data to a file. Anything Java related can be used in processing as well.

    Kf

  • Actually, if you're really not flooded by data, then you can save in SerialEvent itself. The condition is just that there should be sufficient time in between consequent data inputs to save the data.

Sign In or Register to comment.