How to write multiple LinkLists line by line in a file and read them back as LinkedList and collectively store them in an ArrayList

Programming Questions

blackrazor

How to write multiple LinkLists line by line in a file and read them back as LinkedList and collectively store them in an ArrayList

in Programming Questions • 6 months ago

I am doing a project in Processing. My project is on Visualizing the retweets for tweets on a give topics (# tags). I am using twitter4j to interact with Twitter API. I always end up exceeding the Twitter API Rate limit (15 queries an hour) which limits my visualization data set.

Now I am trying to write a script which runs overnight with some periodic interval to collect the data from twitter. In this way I think I will not cross the rate limit of Twitter API.

Something like this.

My project goes on twitter to search for tweets according to query parameters and for every tweets it finds it collects its retweets and then extract the user location of the tweet and retweet, pass it to Google Maps Geocoding API and gets the latitude and longitude of the tweet's user location and location of all the retweets for this tweet. Basically, the first node of the linked list stores the latitude and longitude of the tweet and following nodes stores the same of retweets.

Tweet -> Retweet 1 -> Retweet2

[22.4655, 34.5678]->[23.2344, 23.4545]->......

and I have n number of linked list of this kind which I store in an ArrayList. In short I have an ArrayList of LinkedList and the LinkedList's node is a PVector containing latitude and longitude of a location.

Now to handle the rate limit I want to write a script which does the same thing but writes the data to file which my visualization program can read and visualize on WorldMap. For this I want to prepare a LinkedList for a tweet with all its retweet's location and then write that linked list to a file then the same for the second tweet in the second line of the file.

In my visualization program I want to read the data line by line in linkedlists and store them collectively in an arraylist. I want keep this kind of structure because my visualization program is based on this.

I think this is more than sufficient information on what I am trying to do but with my limited exposure to Processing and Java I am confused about how to do it.

To summarize I want to write multiple linkedlist on a file so that one line contains one linked list and then read them line by line in n number of linkedlist and store them collectively in an arraylist.

I will be glad if someone can help me with this :)

Thanks.

Replies(2)

PhiLho

Re: How to write multiple LinkLists line by line in a file and read them back as LinkedList and collectively store them in an ArrayList

6 months ago

Note: the usage of LinkedList is quite limited in Java, we tend to use ArrayLists instead, generally more efficient, except perhaps if you need to do lot of insert / delete operations at the start of the list...
Now, using them isn't blocking, so you can keep them if you prefer.

What you ask is rather simple, but explaining it in the void (not based on some code) is harder...
BTW, why "r ead them line by line in n number of linkedlist"?
This approach was sensible some 10 to 20 years ago, but today we have plenty of memory, so unless your file is gigabytes long, you can probably just load it at once with loadStrings(), parse it with split() (classical CSV handling) and put the content of the arrays you got in your linked list.
Then you can access all the data you need in real time.

blackrazor

Re: How to write multiple LinkLists line by line in a file and read them back as LinkedList and collectively store them in an ArrayList

6 months ago

Thanks a lot for your reply.

I was able to find a solution to this and make it working. Now I can collect data from Twitter day and night :)

Top Reply