Reading data from txt file

cagdasakalin

Hi all,

We are trying to read the data in the .txt file which include

1   0   0   0   1549,01050
1   0   0   0   1549,08360
1   0   0   0   1549,07956
1   0   0   0   1549,07956
1   0   0   0   1549,07552
1   0   0   0   1549,03956
1   0   0   0   1549,07956
1   0   0   0   1549,08158
1   0   0   0   1549,08360
1   0   0   0   1549,07956
1   0   0   0   1549,04148
1   0   0   0   1549,07350
1   0   0   0   1549,02442

such values in it.

After doing that, what we would like to do is to compare the values just in the 5th column with 1549.05 one by one. If any decimal number in the 5th column is bigger than 1549.05, then in the processing, we may create a array or string or something(Since I am new in Processing and Arduino, I may not know true term ) like that which just includes 1s or 0s.

for example

arry=[]

for i=1

1549.05>1549,01050 => make that; arry=[1]

for i=2

1549.05<1549,08360 => make that; arry=[1 0] . . .

The following code is what I found in the web and try to modify it for my purpose.

After obtaining such an array which I called above "arry", the Processing code will send that data to Arduino and it will turn on or off the pin13 depending on which number(0 or 1) is being sent from Processing to Arduino.

The Processing code that we try to modify

import processing.serial.*; 
import java.io.*; 
int mySwitch=0; 
int counter=0;
int i=0;
String [] subtext;
Serial myPort; 

void setup() { 
  mySwitch=1; 
  myPort = new Serial(this, "COM12", 9600); 
  myPort.bufferUntil('\n');
} 

void draw() { 
  if (mySwitch>0) { 
    readData("C:/Python32/BridgeStrain.txt"); 
    mySwitch=0;
  } 
  if (counter<subtext.length) { 
    myPort.write(subtext[counter]); 
    delay(500); 
    myPort.write('0'); 
    delay(100); 
    counter++;
  } else { 
    delay(5000); 
    mySwitch=1;
  }
} 

void readData(String myFileName) { 

  File file=new File(myFileName); 
  BufferedReader br=null; 

  try { 
    br=new BufferedReader(new FileReader(file)); 
    String text=null;
        while ((text=br.readLine())!=null) { 
      subtext = splitTokens(text, ",");
    }
  }
  catch(FileNotFoundException e) { 
    e.printStackTrace();
  }
  catch(IOException e) { 
    e.printStackTrace();
  }
  finally { 
    try { 
      if (br != null) { 
        br.close();
      }
    } 
    catch (IOException e) { 
      e.printStackTrace();
    }
  }
} 

Arduino Code

int outPin = 13;          //define outpin 
float signal; 
void setup() { 
  Serial.begin(9600); 
  pinMode(outPin,OUTPUT); //set output pin 
} 

void loop() { 
    while (Serial.available()>0) { 
      signal = Serial.parseFloat(); 

      if(signal>1){ 
        digitalWrite(outPin, LOW); 
      } 
      else{ 
        digitalWrite(outPin,HIGH); 
      } 
    } 
 } 

Your valuable comments for solving our problem are very appreciated.

Bests, Çağdaş

Ater

Please, format your code, here's how.

cagdasakalin

Thanks for your warning. I just corrected it.

Chrisir

when you just want to send ONE value at a time (per line in the text file), use String

valueString = "1";

when you analyze MANY lines in one go, use array

What you have to do

so you could read the entire txt in setup() in one go, write the array

String txtFile = loadStrings("name.txt"); 

String[] arry = new String[txtFile.length];

for (i=0; i < txtFile.length; i++) {

    String[] thisLine = split(txtFile[i] , " " );  // space is separating lines, here we split

    float columnFive =  float (trim(thisLine[4] ) );

    if (1549.05>columnFive)  {
       arry[i] = "0";
    }
    else {
       arry[i] = "1";
    } // else 

} //for

and in draw() send the array item by item to the arduino

    for (i=0; i < arry.length; i++) {
          // send to arduino: arry[i]  
    }

cagdasakalin

Let me sent the codes again with the explanation below.

1. Below code pairs(Arduino and Processing) work well for a data such data 1,2,6,2,4,8,2,1,4,8,9,6 in the Text file called BridgeStrain.txt .

Processing Code

import processing.serial.*;
import java.io.*;
int mySwitch=0;
int counter=0;
String [] subtext;
Serial myPort;

void setup(){
 //Create a switch that will control the frequency of text file reads.
 //When mySwitch=1, the program is setup to read the text file.
 //This is turned off when mySwitch = 0
 mySwitch=1;

 //Open the serial port for communication with the Arduino
 //Make sure the COM port is correct
 myPort = new Serial(this, "COM12", 9600);
 myPort.bufferUntil('\n');
}

void draw() {
 if (mySwitch>0){
 // The readData function can be found later in the code.
 // This is the call to read a CSV file on the computer hard-drive.
 readData("C:/Python32/BridgeStrain.txt");

 // The following switch prevents continuous reading of the text file, until
 // we are ready to read the file again.
 mySwitch=0;
 }
 // Only send new data. This IF statement will allow new data to be sent to the arduino.
 if(counter<subtext.length){
 // Write the next number to the Serial port and send it to the Arduino 
 // There will be a delay of half a second before the command is
 // sent to turn the LED off : myPort.write('0');
 myPort.write(subtext[counter]);
 delay(500);
 myPort.write('0');
 delay(100);
 //Increment the counter so that the next number is sent to the arduino.
 counter++;
 } else{
 //If the text file has run out of numbers, then read the text file again in 5 seconds.
 delay(5000);
 mySwitch=1;
 }
} 

// The following function will read from a CSV or TXT file
void readData(String myFileName){

 File file=new File(myFileName);
 BufferedReader br=null;

 try{
 br=new BufferedReader(new FileReader(file));
 String text=null;

 // keep reading each line until you get to the end of the file
 while((text=br.readLine())!=null){
 // Spilt each line up into bits and pieces using a comma as a separator
 subtext = splitTokens(text,",");
 }
 }catch(FileNotFoundException e){
 e.printStackTrace();
 }catch(IOException e){
 e.printStackTrace();
 }finally{
 try {
 if (br != null){
 br.close();
 }
 } catch (IOException e) {
 e.printStackTrace();
 }
 }
}

Arduino Code

int outPin = 13;          //define outpin
byte binary_signal;
void setup() {
  Serial.begin(9600);
  pinMode(outPin,OUTPUT); //set output pin
}

void loop() {
    // check if data has been sent from the computer:
    while (Serial.available()>0) {
    // read the most recent byte
    //Serial.println("0");
    binary_signal = Serial.read();
    //You have to subtract '0' from the read Byte to convert from text to a number.
    binary_signal = binary_signal-'0';

    //Send LOW signal to the output pin if the byte Read = 0
    if(binary_signal>5){
      //Turn off all LEDS
      digitalWrite(outPin, LOW);
      //Serial.println("0");
      //Serial.println(byteRead);
      delay(50);
      }
      //Send HIGH signal to the output pin if the byte Read = 1
    else{
      digitalWrite(outPin,HIGH); //set the output to high
      delay(100);
      //digitalWrite(outPin,LOW); //then set the output to low
      //delay(5);
      }
  }
}

However rest of the story is the same. I need to modify that code to be able to read a .txt file including that data such that

1   0   0   0   1549,01050
1   0   0   0   1549,08360
1   0   0   0   1549,07956
1   0   0   0   1549,07956
1   0   0   0   1549,07552
1   0   0   0   1549,03956
1   0   0   0   1549,07956
1   0   0   0   1549,08158
1   0   0   0   1549,08360
1   0   0   0   1549,07956
1   0   0   0   1549,04148
1   0   0   0   1549,07350
1   0   0   0   1549,02442

After that, you may not need to consider my first message. Sorry for the confusion that I may have made in your mind.

Bests, Çağdaş

cagdasakalin

Dear Chrisir,

First of all, I would like to thank you so much for your help and interest.

When I run the Processing code, it gives a warning, saying "cannot convert from String[] to String".

Also, when the cursor is over the following lines, it shows the warnings that I put one line below of each code line.

String txtFile = loadStrings("C:/Python32/BridgeStrain.txt");

type mismatch, "java.lang.String[]" does not match with "java.lang.String"

String[] arry = new String[txtFile.length];

The global variable "lenght" does not exist.

String[] thisLine = split(txtFile[i] , " " );  // space is separating lines, here we split

The type of the expression must be an array type but it resolved to String.

Thanks again, Çağdaş

Chrisir

please press ctrl-t in the IDE to auto-format your code (indents)

then post your entire code please

Chrisir

this

String txtFile = loadStrings("C:/Python32/BridgeStrain.txt");

must be

String[] txtFile = loadStrings("C:/Python32/BridgeStrain.txt");

cagdasakalin

Dear Chrisir,

Everything looks fine before running. However, when I run the code, it gives the screens in the attachment.

Chrisir

post your entire code please

as text not image

Chrisir

use some println to see what's going on

e.g. println (txtFile[i]);

before the error line etc.

cagdasakalin

I did what you have said. It again gives the same warning which says ""ArrayIndexOutOfBoundsException:". Only difference is that it shows just the first line(which is 1 0 0 0 1549.01050) in the .txt file on message screen.

Bests, Çağdaş

Chrisir

ok

after the line with split

when you type

println(thisLine.length);

what do you get?

Are there space " " between the columns or tabs? how many space?

then you need to fix your split line please

Chrisir

almost there now....

cagdasakalin

In red warning line, it again says "ArrayIndexOutOfBoundsException:4". After terminating the little window, it gives:

Bests, Çağdaş

Chrisir

println(thisLine.length);

should give you 4 or 5, not 1

so you need to fix your split line please

String[] thisLine = split(txtFile[i] , "\t" );

\t is for tab iirc

cagdasakalin

Dear Chrisir,

Thank you so much for your help. I just tried that one

int i;

String[] txtFile = loadStrings("C:/Python32/BridgeStrain.txt"); 

String[] arry = new String[txtFile.length];

for (i=0; i < txtFile.length; i++) {

  String[] thisLine = split(txtFile[i] , "\t" );  // space is separating lines, here we split

  float columnFive =  float (trim(thisLine[4] ) );

  if (1549.05>columnFive) {
    arry[i] = "0";
  } else {
    arry[i] = "1";
  }
  println (arry[i]);
}

Resulting that

according to the comparison that you've made in the code. It works well. Again Thank you so much. I also have to sent this array, including [0 1 1 0 1 1 1 0 0 1 0] to the Arduino, I think, through what they call Serial Port.

Let me guess what changes I should make in Arduino Code here.

int outPin = 13;          //define outpin
int signal;
void setup() {
  Serial.begin(9600);
  pinMode(outPin, OUTPUT); //set output pin
}

void loop() {
  while (Serial.available() > 0) {
    signal = Serial.parseFloat();

     if (signal = 1) {
      digitalWrite(outPin, LOW);
      delay(1000);
    }
    else {
      digitalWrite(outPin, HIGH); //set the output to high
      delay(2000);
   }
  }
}

I may make the following changes on it, but I am not sure about the 2nd one.

float signal                            =>      int signal;
signal = Serial.parseFloat();   =>      ???

Please let me know how we can fix that one.

Chrisir

the code looks ok, without the changes

the error is of course

instead of

if (signal = 1) {

you need

if (signal == 1) {

comparison is always ==

see reference

Chrisir

the delays are different but I guess you want that

cagdasakalin

I do not understand why it does not work. As you said, everything looks good. But, do not know why?

cagdasakalin

The coming data to Arduino is not what they called "float". They are actually 1 or 0 s. We may also need to change the line

signal = Serial.parseFloat();

with something else.

Also, I did what you have said in your last message related with comparison in Arduino.

Bests, Çağdaş

Chrisir

you get a String in arduino

so why parse to float?

can't signal be of type String you read a String - I fon't know the command but like

signal = Serial.getString();

then the if-clause would be:

if (signal.equals("1)) {

GoToLoop

Maybe these forum threads might help ya out: ~O)

• http://forum.Processing.org/two/discussion/12386/arduino-to-processing-importing-variables
• http://forum.Processing.org/two/discussion/12757/transmission-of-a-float-through-a-serial-link

cagdasakalin

Dear Chriris,

For the line, you offered

if (signal.equals("1)) {

gives the warning

cagdasakalin

I realized that the line was missing ". So this is why it gives the warning above. The correct one should be

 if (signal.equals("1")) {

Cheers

Chrisir

And does it work now completely?

cagdasakalin

Dear Chrisir,

Unfortunately, it does not. But as I see on the web, what they used to read the strings is "Serial.read" .

I changed the Arduino code with the following one.

int outPin = 13;          //define outpin
int signal;
void setup() {
  Serial.begin(9600);
  pinMode(outPin, OUTPUT); //set output pin
}

void loop() {
  while (Serial.available() > 0) {
    signal = Serial.read()-'0';

     if (signal == 1) {
      digitalWrite(outPin, HIGH);
    }
    else {
      digitalWrite(outPin, LOW); //set the output to high
   }
  }
}

Then, open "Serial Monitor" under "Tools" on Arduino window and entering 1 turn the pin13 on, and entering other than 1 turn pin13 off.

Also, as I am searching online what I realized is that we have to modify the code for the serial communication by adding the following bunch of lines(or something like that) to the Processing code Chrisir offered before.

void setup() {
  mySwitch=1;
  myPort = new Serial(this, "COM12", 9600);
  myPort.bufferUntil('\n');
}

After the addition of above lines and also the lines related with serial communication in the header, the complete Processing code looks like that:

import processing.serial.*;
import java.io.*;
int i;
int mySwitch=0;
Serial myPort;

void setup() {
  mySwitch=1;
  myPort = new Serial(this, "COM12", 9600);
  myPort.bufferUntil('\n');
}

String[] txtFile = loadStrings("C:/Python32/BridgeStrain.txt"); 
String[] arry = new String[txtFile.length];

for (i=0; i < txtFile.length; i++) {

  String[] thisLine =     split(txtFile[i], "\t" );
  float columnFive  =  float (trim(thisLine[4] ) );

  if (1549.05>columnFive) {
    arry[i] = "0";
  } else {
    arry[i] = "1";
  }
  println (arry[i]);
}

However, it gives the following error and I could not find why it gives that?

When it is ready to running in the red stripe:

After the attempt of running:

Sincerely, Çağdaş

benja

Hey,

you are closing the brackets of your setup-function in line 11. The rest of your code should also be inside of setup(), so try to move that closing bracket from line 11 to the bottom of your sketch.

(I did not try to run your code, because i dont have your file and an arduino right now)

Chrisir

void draw(){

missing in line 12

cagdasakalin

Firstly, thank you very much for your warnings and interest, Shifting curly bracket to the end of the code terminates the error it gives.

I also attached the "draw" function and last view of the code is the following.

import processing.serial.*;
import java.io.*;
int i;
int counter =0;
int mySwitch=0;
String[] arry;
String [] subtext;
Serial myPort;

void setup() {
  mySwitch=1;
  myPort = new Serial(this, "COM12", 9600);
  myPort.bufferUntil('\n');

  String[] txtFile = loadStrings("C:/Python32/BridgeStrain.txt"); 
  String[] arry = new String[txtFile.length];

  for (i=0; i < txtFile.length; i++) {

    String[] thisLine =     split(txtFile[i], "\t" );
    float columnFive  =  float (trim(thisLine[4] ) );

    if (1549.05>columnFive) {
      arry[i] = "0";
    } else {
      arry[i] = "1";
    }
    println (arry[i]);
  }
}

void draw() {
  if (counter<arry.length) {
    myPort.write(arry[counter]);
    delay(500);
    myPort.write('0');
    delay(100);
    counter++;
  } else {
    delay(5000);
    mySwitch=1;
  }
}

// The following function will read from a CSV or TXT file
void readData(String myFileName) {

  File file=new File(myFileName);
  BufferedReader br=null;

  try {
    br=new BufferedReader(new FileReader(file));
    String text=null;

    // keep reading each line until you get to the end of the file
    while ((text=br.readLine())!=null) {
      // Spilt each line up into bits and pieces using a comma as a separator
      subtext = splitTokens(text, ",");
    }
  }
  catch(FileNotFoundException e) {
    e.printStackTrace();
  }
  catch(IOException e) {
    e.printStackTrace();
  }
  finally {
    try {
      if (br != null) {
        br.close();
      }
    } 
    catch (IOException e) {
      e.printStackTrace();
    }
  }
}

As usual, I got the warning, saying that "NullPointerException " as also seen below.

Actually the part after the draw function, as I guess, will read from the string that we called "arry" earlier.

Please let me know did I proceed in a correct way or not?

Chrisir

problem is arry

you declare it before setup and then again in setup which is wrong

remove the String[] in setup()

Chrisir

with String [] in front you make a new array that is just visible inside setup()

bad.

cagdasakalin

Dear Chrisir,

Removing the line

String[] arry = new String[txtFile.length];

causes the same error "NullPointerException" and also indicating the line

arry[i] = "0";

Chrisir

??

no, no........

don't remove the line

just remove the beginning of it: String[]

thus avoiding that you declare arry anew

what you want is referring to the one you declared before setup()

Chrisir

sorry, it's taking you so long

let's hunt the beast down....

cagdasakalin

Dear Chrisir,

Sorry for my misunderstanding. I have changed the code as what you said above. Then, it works perfectly. Thank you so much again for your help.

For the purpose of showing it as a source for others, I am sending both Processing and Arduino codes here.

Working Processing code :

import processing.serial.*;
import java.io.*;
int i;
int counter =0;
int mySwitch=0;
String[] arry;
String [] subtext;
Serial myPort;

void setup() {
  mySwitch=1;
  myPort = new Serial(this, "COM12", 9600);
  myPort.bufferUntil('\n');

  String[] txtFile = loadStrings("C:/Python32/BridgeStrain.txt"); 
  arry = new String[txtFile.length];


  for (i=0; i < txtFile.length; i++) {

    String[] thisLine =     split(txtFile[i], "\t" );
    float columnFive  =  float (trim(thisLine[4] ) );

    if (1549.05>columnFive) {
      arry[i] = "0";
    } else {
      arry[i] = "1";
    }
    println (arry[i]);
  }
}

void draw() {
  if (counter<arry.length) {
    myPort.write(arry[counter]);
    delay(500);
    myPort.write('0');
    delay(100);
    counter++;
  } else {
    delay(5000);
    mySwitch=1;
  }
}

// The following function will read from a CSV or TXT file
void readData(String myFileName) {

  File file=new File(myFileName);
  BufferedReader br=null;

  try {
    br=new BufferedReader(new FileReader(file));
    String text=null;

    // keep reading each line until you get to the end of the file
    while ((text=br.readLine())!=null) {
      // Spilt each line up into bits and pieces using a comma as a separator
      subtext = splitTokens(text, ",");
    }
  }
  catch(FileNotFoundException e) {
    e.printStackTrace();
  }
  catch(IOException e) {
    e.printStackTrace();
  }
  finally {
    try {
      if (br != null) {
        br.close();
      }
    } 
    catch (IOException e) {
      e.printStackTrace();
    }
  }
}

Working Arduino Code:

int outPin = 13;
int signal;
void setup() {
  Serial.begin(9600);
  pinMode(outPin, OUTPUT);
}

void loop() {
  while (Serial.available() > 0) {
    signal = Serial.read() - '0';

    if (signal == 1) {
      digitalWrite(outPin, HIGH);
    }
    else {
      digitalWrite(outPin, LOW);
    }
  }
}

Cheers, Çağdaş

Chrisir

great!

cagdasakalin

It was really working. I tried it many times. But, it suddenly started giving the error(ArrayIndexOutOfBoundsException:4) below.

By the way, the data in the txt file the Processing code is the following.

1       0       0       0       1549.01050
1       0       0       0       1549.08360
1       0       0       0       1549.02956
1       0       0       0       1549.06956
1       0       0       0       1549.01050
1       0       0       0       1549.08360
1       0       0       0       1549.02956
1       0       0       0       1549.06956

Other than this, I also need to ask that how I would have to change the code to read the data with commas(,) instead of dotes(.) such that:

1       0       0       0       1549,01050
1       0       0       0       1549,08360
1       0       0       0       1549,02956
1       0       0       0       1549,06956
1       0       0       0       1549,01050
1       0       0       0       1549,08360
1       0       0       0       1549,02956
1       0       0       0       1549,06956

When it was working perfectly, I tried changing the line

    if (1549.05>columnFive) {

with

    if (1549,05>columnFive) {

But it has not helped at all.

Chrisir

1st question

when the line txtFile[i] is empty or has not enough tabs in it, the split() result won't be as expected

we expect it to be an array of length 5 (one item for each column within the line)

you can check this by asking

if (thisLine.length>=5) {
    // it's save to proceed
    float columnFive=...........
    .... 
}
else  {
     println ("Line has not enough columns ");
}

what have you changed so that the error occured?

one empty line at the end of the text file would be enough to let that error occur.....

2nd question

processing only operates with . not comma

you need to replace the comma with dot

you can replace

  float columnFive  =  float (trim(thisLine[4] ) );

with

float columnFive  =  float (trim(thisLine[4].replace(",", ".") ) );

afaik

maybe you have to look it up (javadoc under String)

cagdasakalin

Dear Chrisir,

You are a great programmer.

Your indication

"one empty line at the end of the text file would be enough to let that error occur....."

was true. That was exactly what it was missing.

Also,

using the line

 float columnFive  =  float (trim(thisLine[4].replace(",", ".") ) );

fixed it.

Thank you so much again again. By the way, where are you from?

Chrisir

I am from Germany, we have the comma in numbers instead of the decimal point as well....