Load data from csv into a HashSet

Programming Questions

dimiro

Load data from csv into a HashSet

in Programming Questions • 1 year ago

hi
i have a dataset with 3 cols and ~20.000 lines that contains long/lat coonrdinates like this:

Country,longitude,latitude
Italy,523452,64236
Italy,5435,645654
Sweden,4364,673657
Sweden,325,265
etc

i.e there are 20 countries so each ,for example, Italy is repeated 1000 times which means that i have 1000 point coordinates and so on.

so i `ve written this

private String []data;
   private String [][] csv;
   private String filename="/home/user/coordinates.csv";
   private final int cols=3;
   private String delimiter=",";
PrintWriter output;
Map < String,ArrayList<PVector>> coords;
//List<Map<String, PVector>> coords = new ArrayList<Map<String, PVector>>();
List<Integer> t=new ArrayList<Integer>();
public void loadData(){
output=pa.createWriter("/home/user/points.txt");
data=pa.loadStrings(filename);
ArrayList<PVector> tempPoints=new ArrayList<PVector>(data.length);
// Set<String> tempNames=new HashSet <String> ();
coords=new HashMap<String,ArrayList<PVector>>(data.length);
csv=new String[data.length][cols];
for (int i=0; i<data.length; i++){
for (int j=0; j<cols; j++){
String [] tempGeo=pa.split(data[i], delimiter);
csv[i][j]=tempGeo[j];
}//FOR J
tempPoints.add(new PVector(Float.parseFloat(csv[i][1]),Float.parseFloat(csv[i][2])));
// tempNames.add( csv[i][0]);
coords.put(new String (csv[i][0]),tempPoints);
}//FOR I
for (String s: coords.keySet()){
output.println(s);
for (PVector pv:coords.get(s))
output.println(pv);
}

i get the names correctly by removing duplicates due to Set`s functionality, but (obviously) it assigns *all* the points to each country and i end up with a file of 20.000(lines)x20(countries)..

how can i assign the coordinates correctly to each country? i.e to take an ArrayList of PVectors for each country that it`s size is equal to the number of times this country appears in the first column?

i tried a couple of ifs by matching Strings but had no luck and i cannot figure out something else to try

thanks

Replies(6)

Ale_

Re: Load data from csv into a HashSet

1 year ago

Is it possible to have the csv file to check the code?

Ale · 60rpm.tv/i

Ale_

Re: Load data from csv into a HashSet

1 year ago

Mmm... I don´t think it´s necessary...
You have to do it quite differently. Don ´t pass everything to the hashMap. First you have to retrieve the key. If key doesn´t exist you create the ArrayList, you put inside the first PVector/coordinates and you add to the map the pair key/value. If key exists you retrieve the ArrayList attached to that key an you add then the new PVector with the coordinates.
I did this sketch that uses a hashMap:
http://www.openprocessing.org/visuals/?visualID=34290
Look for the update() method in class H, maybe it´s clearer than my explanation. ;-)

Ale · 60rpm.tv/i

dimiro

Re: Load data from csv into a HashSet

1 year ago

thanks for the reply.

i understand that i pass everything too early to the HashMap. I checked your code and the comments you have made there but i don `t fully understand the logic behind this :

First you have to retrieve the key. If key doesn´t exist you create the ArrayList, you put inside the first PVector/coordinates and you add to the map the pair key/value.

what do you mean by "if key doesn`t exit". Is this somehow a fixed way of filling hashmaps? or was this necessary in your sketch?

Ale_

Re: Load data from csv into a HashSet

1 year ago

Mmm... my English level is not as good as I want to... ;-)

I mean, suppose we´re at the beginning of the for loop. The first step in the iteration, say. You have your map empty so there isn´t any key (nor value) inside (in our case the key is the name of the country, and the value is the ArrayList full of PVectors composed of coordinates).

//We check current key and the coordinates to put inside the value/arrayList:

String key=csv[i][0];   //key
PVector newCoords=new PVector( csv[i][1], csv[i][2]);   //coordinates > value

/*Now we ask the map if that key is already taken. We know that´s not our case because we´re at the first step and the map is empty thus next statement will be true and will happen:*/

if (coords.get(key)==null) {
       ArrayList coordBunch= new ArrayList<PVector>(); //we create the value for that new key
       coordBunch.add(newCoords);                 //we add this last its first element
       coords.put(key,coordBunch);                                 //and we add the new pair key/value to the map
   }else{

//So now we have our first key with a little ArrayList with one PVector as value.
//Suppose at the second step of the iteration we find the same key (another place in the same country, say)
//So this gonna happen:

     ArrayList coordBunch= coords.get(key)   //we retrieve the current value of that key
     coordBunch.add(newCoords);                 //and we add a new PVector to the value
}

//So now you have in the map only a key (Italy, say) and an ArrayList with two PVectors
//etc. etc.

Hope it´s clearer now! :-)

Ale · 60rpm.tv/i

dimiro

Re: Load data from csv into a HashSet

1 year ago

:) Excellent answer. it worked and most important, i understood it!!

thank you!

Ale_

Re: Load data from csv into a HashSet

1 year ago

;-)

Ale · 60rpm.tv/i

Top Reply