I am doing a project in Processing. My project is on Visualizing the retweets for tweets on a give topics (# tags). I am using twitter4j to interact with Twitter API. I always end up exceeding the Twitter API Rate limit (15 queries an hour) which limits my visualization data set.
Now I am trying to write a script which runs overnight with some periodic interval to collect the data from twitter. In this way I think I will not cross the rate limit of Twitter API.
Something like this.
My project goes on twitter to search for tweets according to query parameters and for every tweets it finds it collects its retweets and then extract the user location of the tweet and retweet, pass it to Google Maps Geocoding API and gets the latitude and longitude of the tweet's user location and location of all the retweets for this tweet. Basically, the first node of the linked list stores the latitude and longitude of the tweet and following nodes stores the same of retweets.
Tweet -> Retweet 1 -> Retweet2
[22.4655, 34.5678]->[23.2344, 23.4545]->......
and I have n number of linked list of this kind which I store in an ArrayList. In short I have an ArrayList of LinkedList and the LinkedList's node is a PVector containing latitude and longitude of a location.
Now to handle the rate limit I want to write a script which does the same thing but writes the data to file which my visualization program can read and visualize on WorldMap. For this I want to prepare a LinkedList for a tweet with all its retweet's location and then write that linked list to a file then the same for the second tweet in the second line of the file.
In my visualization program I want to read the data line by line in linkedlists and store them collectively in an arraylist. I want keep this kind of structure because my visualization program is based on this.
I think this is more than sufficient information on what I am trying to do but with my limited exposure to Processing and Java I am confused about how to do it.
To summarize I want to write multiple linkedlist on a file so that one line contains one linked list and then read them line by line in n number of linkedlist and store them collectively in an arraylist.
I will be glad if someone can help me with this :)
Thanks.