Help splitting a comma-separated string but ignoring commas in quotes.

Programming Questions

dauglyon

Help splitting a comma-separated string but ignoring commas in quotes.

in Programming Questions • 3 months ago

I have a CSV formated like this:

20120807,1200,"I can't sleep, too much caffeine.",476,100,375,187,411,347,175
20120807,1300,I like bagels,476,101,375,187,431,347,175
20120807,1300,The man said he had "powers",476,101,375,187,431,347,175
20120807,1300,"And the he said, "I hate you."",476,101,375,187,431,347,175

And I am try to import it using this code:

String[] lines = loadStrings("data.csv");
String[] peices = split(lines[0], ",");

However, I have yet to figure out how to get processing to ignore the commas which are contained within quotes.

I've dug around on the web but didn't find anything. If anyone knows how to do this I would really appreciate the help!

Replies(8)

datguy

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

instead of loadStrings, use loadTable

GoToLoop

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

Wow! I didn't think that new Table class was that powerful!

I was thinking that dauglyon would have to code his own parser!

But surprisingly, loadTable() was able to interpret those irregular quotes as a unit!!!

It's the 1st time I tried to experiment w/ any of those brand new composites.

However, it was reasonably easy to figure this 1 out!

Thx a lot, datguy !

Check it out:

final Table lines = loadTable("data.csv");

for ( int i = 0, ii = lines.getRowCount(); i != ii; ) {
  final TableRow row = lines.getRow(i++);
  println();
  for ( int j = 0, jj = row.getColumnCount(); j != jj; 
  print(row.getString(j++) + "   ") );
}

exit();

P.S.: A pity those exotic new composites can't be used w/ enhanced loop:

for (String s: row); // Can only iterate over an Array or an instance of java.lang.Iterable

GoToLoop

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

Seems like there's a built-in Iterator through method rows() which returns a TableRow object!

Unfortunately, no columns() counterpart.

I guess they'd still need to invent class TableColumn for it!

Anyways, behold an even shorter version:

final Table lines = loadTable("data.csv");

for ( TableRow row: lines.rows() ) {
  println();
  for ( int col = 0, num = row.getColumnCount(); col != num; 
  print(row.getString(col++) + "   ") );
}

exit();

datguy

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

Thanks for the detective work, GoToLoop. I knew loadTable was a better fit, but hadn't explored all of its capabilities. The examples you provided are great!

dauglyon

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

I probably should have mentioned that I was using Processing.js, which, as it turns out, does not yet support the table class. Does anyone know a way to do this without it?

GoToLoop

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

As I have mentioned above: You'll have to make your own split parser!

Since the char ',' is both a delimiter and a regular character!

You extract a char 1 by 1 and check whether it's a comma.

Then, also check if it isn't inside a quoted sequence.

If both checks are true, get a sub-string up to delimiter.

Otherwise, keep accumulating the sub-string.

Of course, you flag a boolean variable when you find a quote.

And turn it off when finds next quote.

PhiLho

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

Simple solution: take the source code of the Table class and use it in your sketch. It will be converted to JavaScript. (Ie.: no need to reinvent the wheel, even if it is fun!)

GoToLoop

Re: Help splitting a comma-separated string but ignoring commas in quotes.

3 months ago

Perhaps an even simpler solution would be to modify (or even @Override) the method split() itself!

Just add the quote character on/off algorithm to control

whether a split happens when the comma delimiter is found.

Top Reply