Splitting a Text file using CRLF and parsing file.

gennylk

Hi There, I have a text file, which has serialdata sent from a Arduino. The text file containing the data has a set of coordinates, (say x,y,z) which is stored in a continuous stream, but has CRLF breaks after a number of data pertaining to a timeframe. For example

1245

909 1105

1096 768

The data above is really x,y,z with following values (note : x takes two digits always, and first is left blank if x is less than 10)

(12,4,5)

(9,0,9), (11,0,5)

(10,9,6),(7,6,8)

I need to find the maximum and minimum of the x values in the file.

i've managed to read the data using following program

void setup()
{
  noLoop();
  size(100,100);
  background(153);
  int lf = 10;    // Linefeed in ASCII
}
void draw(){
String filename=dataPath("test.txt");
String[] data = loadStrings(filename); 
String[] lines = split(data,lf); //doesn't work
println(data);
}

can someone suggest how I can split the text into different strings by using splitdata (or some other method)- I cant seem to give LF to break the string.

Many Thanks, Genny

GoToLoop

AFAIK, loadStrings() already splits a whole text into lines as array slots using CR &/or LF as end of line characters!
BtW, CR = RETURN = '\r'. And LF = ENTER = '\n'.

gennylk

Thanks GoToLoop!.

Yes you were right. I thought it split-ted using space. Btw hit a problem parsing a line of text

void setup()
{
  noLoop();
  size(100, 100);
  background(153);
}
void draw() {
  String filename=dataPath("test.txt");
  String[] data = loadStrings(filename);
  for (int i =0; i <data.length; i++)
  {
    String  subString = data[i]; //OneLine
    println(subString);
    println(subString.length);
  }
}

I get an error saying Substring cannot be resolved (for calculating length of one line) Basically what im trying to do is to get the substring length (Say12 or 8) and divide by 4 which gives me the no. of coordinates in that line.

GoToLoop

http://processing.org/reference/String_length_.html

gennylk

Thank you GotoLoop!. I'll post my complete code to replace this post soon!

gennylk

Here is the working program :)

int maxX = 0;
int minX = 100;  
int maxY = 0;
int minY = 100;
int maxPoints = 0;
int minPoints = 100;
int y, points;
int x;
int frames=0;
int coordinatesInFrame=0;
void setup()
{
  noLoop();
  //size(100, 100);
  //background(153);
}
void draw() {
  for (int fileNo=1; fileNo <21; fileNo++) { //used to read multiple files with similar filenames (such as G1-1.txt) in a loop
    String filename=dataPath("G1-"+fileNo+".txt"); 
    String[] data = loadStrings(filename);
    frames=data.length;
    for (int i =0; i <frames; i++)
    {
      String subString = data[i]; //OneLine
      coordinatesInFrame=subString.length()/4;
      for (int j=0; j<coordinatesInFrame; j++) {
        x = (int) float(subString.substring(j*4, j*4+2));
        y = int(subString.substring(j*4+2, j*4+3));
        points = int(subString.substring(j*4+3, j*4+4));
        if (maxX<x) maxX=x; 
        else if (minX>x) minX=x; 
        if (maxY<y) maxY=y; 
        else if (minY>y) minY=y; 
        if (maxPoints<points) maxPoints=points; 
        else if (minPoints>points) minPoints=points;
      }
    }
    println("FileName : "+fileNo);
    println("No. of Coordinates in File : "+frames*coordinatesInFrame);
    println("Max X Coordinate : "+maxX+" \t Minimum X Coordinate : "+minX);
    println("Max Y Coordinate : "+maxY+" \t Minimum Y Coordinate : "+minY);
    println("Max Points : "+maxPoints+" \t Min Points :  "+minPoints);
  }
}

GoToLoop

@gennylk, I've made a more complex alternative version just for fun. (*)
Dunno whether it still works for your files though! :-SS

---

/**
 * Custom Split Parse (v2.05)
 * by GoToLoop (2014/Aug)
 *
 * forum.processing.org/two/discussion/6677/
 * splitting-a-text-file-using-crlf-and-parsing-file-
 */

import java.util.List;
import java.io.FilenameFilter;

static final FilenameFilter FILTER = new FilenameFilter() {
  static final String NAME = "G1-", EXT = ".txt";

  @ Override boolean accept(File dir, String name) {
    return name.startsWith(NAME) && name.endsWith(EXT);
  }
};

Coordinate[] coords;

void setup() {
  if ( (coords = getDataFiles(dataPath(""))) == null )  exit();

  println(coords);
  println();

  java.util.Arrays.sort(coords);
  println(coords);

  exit();
}

Coordinate getDataFiles(String dirPath)[] {
  File f = new File(dirPath);
  if (!f.isDirectory()) {
    println("\nPath \"" + dirPath + "\" isn't a folder!\n");
    return null;
  }

  String[] names = f.list(FILTER);
  if (names.length == 0) {
    println("\nNo data files found inside \"" + dirPath + "\" folder!\n");
    return null;
  }

  List<Coordinate> points = new ArrayList();
  for (String name: names)  for (String row: loadStrings(name))
    parseRowCoords(points, splitTokens(row));

  return points.toArray( new Coordinate[points.size()] );
}

protected void parseRowCoords(List<Coordinate> vecs, String... tokens) {
  for (String t: tokens) {
    int len = t.length();
    if (len < 3)  continue;

    int x = int(t.substring(0, len - 2));
    int y = t.charAt(len - 2) - '0';
    int z = t.charAt(len - 1) - '0';

    vecs.add( new Coordinate(x, y, z) );
  }
}

---

final class Coordinate implements Comparable<Coordinate> {
  byte x, y, z;

  Coordinate() {
  }

  Coordinate(int xx, int yy, int zz) {
    x = (byte) abs(xx);
    y = (byte) abs(yy);
    z = (byte) abs(zz);
  }

  @ Override int compareTo(Coordinate c) {
    return
        x != c.x ? x - c.x
      : y != c.y ? y - c.y
      : z  - c.z;
  }

  @ Override int hashCode() {
    return x<<020 | y<<010 | z;
  }

  @ Override boolean equals(Object o) {
    return o.hashCode() == hashCode();
    //return o instanceof Coordinate && o.hashCode() == hashCode();
    //return o == this && o instanceof Coordinate && o.hashCode() == hashCode();
  }

  @ Override String toString() {
    return "[ " + x + ", " + y + ", " + z + " ]";
  }
}

---