Converting String to float from 2D array

rafe.cope..

Converting String to float from 2D array

in Contributed Library Questions • 1 year ago

I have a csv file with rows of data, some of it numbers, some of it text. I'm trying to store this data in a String[][], access it later and then convert it into a useful format.

There is another question with a similar problem/title, except their problem was solved by converting the entire String array into a float array. However, my String array does not contain only floats. It has text in it as well. I can access an individual value from my String[][] and print it fine, but converting it to a float produces this error:

Exception in thread "Animation Thread" java.lang.NullPointerException
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:989)
at java.lang.Float.valueOf(Float.java:388)
at java.lang.Float.<init>(Float.java:489)
at processing.core.PApplet.parseFloat(PApplet.java:6566)
at processing.core.PApplet.parseFloat(PApplet.java:6561)
at GESUS$Planet.visualise(GESUS.java:526)
at GESUS$Planet.plot(GESUS.java:364)
at GESUS.draw(GESUS.java:105)
at processing.core.PApplet.handleDraw(PApplet.java:1631)
at processing.core.PApplet.run(PApplet.java:1530)
at java.lang.Thread.run(Thread.java:680)

The code is below.

// Welcome to GESUS
// Geospatial Earth Simulation Using Sound (SuperCollider)
import processing.opengl.*;
import javax.media.opengl.*; //use this elsewhere, but removed that code.
import tomc.gpx.*; //maybe there are conflicts with the libraries I am using?
import java.util.Calendar;
import java.util.TimeZone;
import java.lang.Math;
import java.io.File;
GL gl;
PGraphicsOpenGL pgl;
//Equator equator; //class used for solving equations, used elswhere but code removed
Planet earth;
public GPX planetGeography;
XMLElement godDirectory;
void setup() {
size(1600,1000,OPENGL);
background(0);
frameRate(300);
earth = new Planet();
getDataPaths();
}
void getDataPaths() {
godDirectory = new XMLElement(this,"godDirectory.xml");
int godCount = godDirectory.getChildCount();
for(int i=0;i<godCount;i++) {
XMLElement godLayer = godDirectory.getChild(i);
GOD GODLayer = new GOD(godLayer);
}
}

//This is GOD, the class for a Geospatially Overlaid Dataset.
//It takes a data input, parses it and is then called by the Planet to which it is assigned.
class GOD {
String name;
String source;
String path;
String filetype;
String seperator;
int fieldCount;
HashMap metaValues;
XMLElement metaData;
String[][] godLayerData;
ArrayList godLayer;
GOD(XMLElement GODLayer) {
metaData = GODLayer;
getMetadata();
loadData();
passData();
}
void getMetadata() {
name = metaData.getChild("name").getContent();
source = metaData.getChild("source").getContent();
path = metaData.getChild("path").getContent();
filetype = metaData.getChild("filetype").getContent();
seperator = metaData.getChild("seperator").getContent();
fieldCount = metaData.getChild("values").getChildCount();
godLayer = new ArrayList();
metaValues = new HashMap(fieldCount+1,1);
for (int i=0;i<fieldCount;i++) {
metaValues.put(metaData.getChild("values").getChild(i).getContent(),i);
}
}
void loadData() {
if (filetype==null) {
println("filetype for "+name+" is either not set or caught");
}
else if (filetype!=null) {
if (filetype.equals("txt")) {
String[] rawLines = loadStrings(path);
String[][] godLayerData = new String[rawLines.length][fieldCount];
for (int i=1;i<rawLines.length;i++) {
String[] splitLine = split(rawLines[i], seperator);
for (int j=0;j<metaValues.size();j++) {
godLayerData[i][j] = splitLine[j];
}
}
metaValues.put("datasetSize",rawLines.length-1);
godLayer.add(metaValues);
godLayer.add(godLayerData);
}
else if (filetype.equals("csv")) {
String[] rawLines = loadStrings(path);
String[][] godLayerData = new String[rawLines.length][fieldCount];
for(int i=1;i<rawLines.length;i++) {
String[] splitLine = split(rawLines[i], seperator);
for(int j=0;j<fieldCount;j++) {
godLayerData[i][j] = splitLine[j];
}
}
metaValues.put("datasetSize",rawLines.length-1);
godLayer.add(metaValues);
godLayer.add(godLayerData);
}
else {
println("filetype '"+filetype+"' is not recognised");
}
}
}
void passData() {
earth.put(godLayer);
}
}

earth is an instance of Planet, below (there might be some irrelevant variables, I use them in code not included):

class Planet {
float pRadius;
int pAspect;
int pLongitudeLineIncrement;
int pLatitudeLineIncrement;
int pDepth;
int pDarksideVisibility;
String pGeographyFile;
float pRotationVelocity;
PVector pCentrePoint;
float pSolarDeclination;
ArrayList godInputs;
Planet() {
pRadius = 2.0*int(min(height, width)*0.4);
pAspect = 0;
pLongitudeLineIncrement = 15;
pLatitudeLineIncrement = 15;
pDepth = 1000;
pCentrePoint = new PVector(width/2, height/2, -pDepth);
pDarksideVisibility = 150; //set the opacity of the darkside of the Earth (0-255)
pGeographyFile = "continents.gpx"; //GPX file containing the earth's geography
godInputs = new ArrayList();
}
void put(ArrayList input) {
if(input==null) {
println("data input is empty");
}
else {
godInputs.add(input);
}
}
void visualise() {
for(int i=0;i<godInputs.size();i++){
ArrayList godLayer = (ArrayList) godInputs.get(i);
HashMap godMetaData = (HashMap) godLayer.get(0);
String[][] godLayerData = (String[][]) godLayer.get(1);
int datasetSize = int(godMetaData.get("datasetSize").toString());
for(int j=0;j<datasetSize;j++) {
String latitude = godLayerData[j][int(godMetaData.get("latitude").toString())];
println(latitude); //this works absolutely fine!
//float latitudeFloat = float(latitude);
//why can i not convert this string into a float???
}
}
}
}

The weirdest part is that if you run this code as is, it prints latitudes out no problem. However, as soon as you uncomment the "float latitudeFloat = float(latitude);" line, it prints NULL followed by the error!!! Huh? How can handling a variable LATER affect it earlier in the program? Am I making a dumb mistake or is there something sinister going on?

Thanks in advance for the help, and if there are any other tips on how to make this work better I'd love to hear them!

Replies(7)

Chrisir

Re: Converting String to float from 2D array

1 year ago

Hi,

you can format your code with Ctrl-t
thus better indent

it would be cool if you could post the first 10 or so lines of your csv (with mixed float and String) otherwise we can't test

can't you check with if whether the String in the line is covertable into float or not?

Why don't you have draw()?

Greetings, Chrisir

rafe.cope..

Re: Converting String to float from 2D array

1 year ago

Hi Chrisir,

I use an external editor and I guess the format of the tabs wasn't appreciated by the forum, it automatically stripped them sorry about that.

Whoops, I missed out draw() by accident, here is the main pde again:

// Welcome to GESUS
// Geospatial Earth Simulation Using Sound (SuperCollider)
import processing.opengl.*;
import javax.media.opengl.*; //use this elsewhere, but removed that code.
import tomc.gpx.*; //maybe there are conflicts with the libraries I am using?
import java.util.Calendar;
import java.util.TimeZone;
import java.lang.Math;
import java.io.File;
GL gl;
PGraphicsOpenGL pgl;
//Equator equator; //class used for solving equations, used elswhere but code removed
Planet earth;
public GPX planetGeography;
XMLElement godDirectory;
void setup() {
size(1600,1000,OPENGL);
background(0);
frameRate(300);
earth = new Planet();
getDataPaths();
}
void draw() {
earth.plot();
}
void getDataPaths() {
godDirectory = new XMLElement(this,"godDirectory.xml");
int godCount = godDirectory.getChildCount();
for(int i=0;i<godCount;i++) {
XMLElement godLayer = godDirectory.getChild(i);
GOD GODLayer = new GOD(godLayer);
}
}

and below is the first ten lines of my datafile. In the meantime I am using this shortened version of the datafile (I am using a shortened version of the data (ie fewer columns) while I build the basic functionality. it is twitter data and for now I have just left in userid, lat, lon and time). below that is my xml index.

tweettime,uid,lat,lon
2012-02-14 00:54:33,449042910,45.49520374,-122.50906839
2012-02-14 00:55:14,331399206,30.245469,-93.0130302
2012-02-14 00:55:17,380545415,51.55634859,0.10255228
2012-02-14 00:55:36,79779803,25.703768,32.638898
2012-02-14 00:57:17,74612285,-20.44317909,-54.59731936
2012-02-14 00:59:00,83438452,-1.3507028,-48.3619505
2012-02-14 00:59:18,153108928,-22.33553669,-49.07197248
2012-02-14 00:59:41,384004795,53.19835633,-2.3666848
2012-02-14 00:59:36,87531148,49.225837,-122.890148
2012-02-14 00:59:51,93694873,19.046874,-98.218742
2012-02-14 01:00:26,23483019,43.475616,-80.53162167
2012-02-14 01:01:00,265656769,-7.14852879,-34.84109376
2012-02-14 01:01:08,111280541,53.4488477,-2.23826825
2012-02-14 01:02:00,417399305,41.53269525,-87.8140258
2012-02-14 01:02:46,26333024,7.1184667,-73.44095703
2012-02-14 01:03:09,241115784,45.559087,-73.59918156
2012-02-14 01:03:53,141805894,-33.58861995,-71.6133469
2012-02-14 01:04:00,152419666,-33.12768236,-71.5769555
2012-02-14 01:04:29,461233461,-34.467716,-72.024501

<?xml version="1.0" encoding="UTF-8"?>
<god>
<godLayer>
<name>World Twitter Data 2012-02-14</name>
<source>########</source>
<path>godData/tweets_one-day_min.csv</path>
<filetype>csv</filetype>
<seperator>,</seperator>
<values>
<value>time</value>
<value>userID</value>
<value>latitude</value>
<value>longitude</value>
</values>
</godLayer>
</god>

I could check if it was convertable or not, but it kinda has to be. The data is consistent and I need to plot every point.

Thanks,

Rafe

rafe.cope..

Re: Re: Converting String to float from 2D array

1 year ago

in the csv ignore the first empty line, forum added it

Chrisir

Re: Converting String to float from 2D array

1 year ago

wow, that's way over my head...

when you split at the comma , seperator (isn't it separator?) then you get

date/time
one value
long and
lat

the 1st is not float

amnon.owed

Re: Converting String to float from 2D array

1 year ago

We can't run your code. Try to recreate a small runnable example of your problem.

Converting a string from a 2D array into a number (float, double depending on required precision) is possible.

See below.

Code Example

String[][] twodee = new String[1][1];
twodee[0][0] = "-33.58861995345348";
println("String: " + twodee[0][0]);
float f = float(twodee[0][0]);
println("Float: " + f);
Double d = Double.parseDouble(twodee[0][0]);
println("Double: " + d);
exit();

To prevent a NullPointerException just check for null before you do something.

rafe.cope..

Re: Converting String to float from 2D array

1 year ago

Ok yeah sorry let me try and simplify the code. Is making a zip easier/more useful?

But ammon you gave me an idea—what about using the java Float class and calling parseFloat()? Haven't tested it but I feel like that would work. Is there any disadvantage to using Float rather than just a float? Is it slower (there are 2 million lines of data here and there will be many more datasets to go with it)?

Also any tips for speeding up the code would be much appreciated.

And yup, even looked it up, seperate is not a word. I wonder how many years I've been spelling that wrong...

rafe.cope..

Re: Converting String to float from 2D array

1 year ago

Have to apologise for asking such a complicated question with such a simple answer. I just made an error with the code, I read from line one (not zero) because of the header line, and used that same loop number to access the 2d array i was making. so there was never a position zero. So problem solved.

Really sorry to waste everyone's time!

Top Reply