Parsing csv data set

craymond

Hello- New to processing and did a tutorial plotting data onto a map and everything worked fine. Very fun. :)

Excited to move on to trying it with my own data set and I am running into a problem. With my data set, when the cvs data loads it does not seem to recognize line endings. I am dealing with a smaller data set than in my tutorial, only 99 lines and four columns, but when they are processed all of the data is one large blob/paragraph in the console when I println(cvs); it. I have tried many times to 'groom' the data starting with new files and using different tools.

In the tutorial, the cvs parsed out with the each line ending neatly. I can't get mine to do that.

My project only has city, longitude, latitude and a quantity in whole numbers.

Thanks for helping a rookie.

craymond

And should I have posted this elsewhere?

GoToLoop

You should post your attempt along a sample of your ".csv" file.
Read here how to: https://forum.processing.org/two/discussion/8045/how-to-format-code-and-text

craymond

here is the code: http://pastebin.com/Ab7nDyV1 and files: http://we.tl/dMLn0pfra0

koogs

String csv[];
...
println(csv);

it looks like a clump because you're printing an array as if it was a simple variable. try

for (String value : csv) {
  println(value);
}

(untested)

you can access individual values using csv[0] etc

Chrisir

String[] testCSV = loadStrings("kfdjf.csv"); 


for (int i...........................
     println(testCSV[i]);

craymond

I plowed ahead and was able to complete the map even though I wasn't getting those lines to break. As I continued I found two cities that had "city, state" - with the double quotes and extra coma that was giving me a different error. I don't know if I were to go back now and println(csv); at that same point if it would parse correctly. But I have more data sets to try...

craymond

thanks for you help!

GoToLoop

You should try out parseJSONArray() loadTable() for edgy cases w/ ": B-) https://Processing.org/reference/loadTable_.html

blindfish

Looks to me like you'd be better off using loadTable; which appears to be designed specifically to parse CSV. As you've discovered using split(",") on a CSV file is only safe on the simplest of content that you have full control over. Commas within CSV data are perfectly valid as are double quotes - see CSV rules - which means parsing CSV can become somewhat more complicated than a simple split ;)

I think you can also run into issues with missed line endings depending on the end of line characters used in your file: that can depend on your OS, encoding and application used to produce the CSV file...