Saving JSON data from twitter4j - SOLVED

v.k.

Saving JSON data from twitter4j - SOLVED

in Contributed Library Questions • 1 year ago

I'm getting a lot of java examples from twitter4j github and trying them on Processing to attempt to understand how things works. I succeeded in various, but this one is throwing a null pointer exception that i could not get, highlighting the ' } ' just before the last ' } ' in code . As it uses o lot of stuff i don't know like java.io, i'm stuck.

It should get an status (a tweet) and save it to a file in raw JSON. The example in java is here and my code is like this

(i got my credentials at the right place, but deleted them here):

import processing.opengl.*;
import java.io.*;
import java.util.List;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.json.DataObjectFactory;
Twitter twitter;
void setup() {
size(600, 400);
//Credentials
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey("");
cb.setOAuthConsumerSecret("");
cb.setOAuthAccessToken("");
cb.setOAuthAccessTokenSecret("");
cb.setDebugEnabled(true);
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
System.out.println("Saving public timeline.");
try {
new File("statuses").mkdir();
List<Status> statuses = twitter.getHomeTimeline();
for (Status status : statuses) {
String rawJSON = DataObjectFactory.getRawJSON(status);
String fileName = "statuses/" + status.getId() + ".json";
storeJSON(rawJSON, fileName);
System.out.println(fileName + " - " + status.getText());
}
System.out.print("\ndone.");
System.exit(0);
} catch (IOException ioe) {
ioe.printStackTrace();
System.out.println("Failed to store tweets: " + ioe.getMessage());
} catch (TwitterException te) {
te.printStackTrace();
System.out.println("Failed to get timeline: " + te.getMessage());
System.exit(-1);
}
}//eof setup
private static void storeJSON(String rawJSON, String fileName) throws IOException {
FileOutputStream fos = null;
OutputStreamWriter osw = null;
BufferedWriter bw = null;
try {
fos = new FileOutputStream(fileName);
osw = new OutputStreamWriter(fos, "UTF-8");
bw = new BufferedWriter(osw);
bw.write(rawJSON);
bw.flush();
} finally {
if (bw != null) {
try {
bw.close();
} catch (IOException ignore) {
}
}
if (osw != null) {
try {
osw.close();
} catch (IOException ignore) {
}
}
if (fos != null) {
try {
fos.close();
} catch (IOException ignore) {
}
}
}
}

Replies(9)

PhiLho

Re: Saving JSON data from twitter4j

1 year ago

" this one is throwing a null pointer exception that i could not get, highlighting the ' } ' just before the last ' } ' in code"
Perhaps you can show us the stacktrace... Personally, I cannot run this code.

v.k.

Re: Saving JSON data from twitter4j

1 year ago

Ops, sure:

Exception in thread "Animation Thread" java.lang.NullPointerException

at java.io.Writer.write(Writer.java:140)

at TweetTestsJson.storeJSON(TweetTestsJson.java:128)

at TweetTestsJson.setup(TweetTestsJson.java:102)

at processing.core.PApplet.handleDraw(PApplet.java:1608)

at processing.core.PApplet.run(PApplet.java:1530)

at java.lang.Thread.run(Thread.java:680)

PhiLho

Re: Saving JSON data from twitter4j

1 year ago

Looks like rawJSON is null...

v.k.

Re: Saving JSON data from twitter4j

1 year ago

Yep, you are right. I added this two lines below line 32

println("@" + status.getUser().getScreenName() + " - " + status.getText());
println("rawJASON= "+ rawJSON);

and the result is:

@brainpicker - Polaroid inventor Edwin Land was Steve Jobs long before Jobs and his story is full of timeless entrepreneurship lessons http://t.co/DTLINumO

rawJASON= null

Exception in thread "Animation Thread" java.lang.NullPointerException

at java.io.Writer.write(Writer.java:140)

at TweetTestsJson.storeJSON(TweetTestsJson.java:130)

at TweetTestsJson.setup(TweetTestsJson.java:104)

at processing.core.PApplet.handleDraw(PApplet.java:1608)

at processing.core.PApplet.run(PApplet.java:1530)

at java.lang.Thread.run(Thread.java:680)

The first line is one tweet from my timeline, so the status exist... And is being get from twitter API... any hint?

edit: i found this at twitter4j javadoc, may be linked to this problem i think. I don't really understands how those thread are going, i mean, the twitter listener the setup(). I tried to add a draw() but sme thing.

Note that raw JSON forms can be retrieved only from the same thread invoked the last method call and will become inaccessible once another method call

v.k.

Re: Saving JSON data from twitter4j

1 year ago

Have you looked at the original java code? I tried to post but reached the limit of forum (?)

v.k.

Re: Saving JSON data from twitter4j

1 year ago

To keep it updated:

The thing is that DataObjectFactory is off by default in twitter4j. It needs to be enabled in twitter4j.properties. I have done this using configurationBuilder, but can be done in other ways also (command line or editing the file it self), the line i added was :

cb.setJSONStoreEnabled(true); // cb is the instance of configurationBuilder

So i got the code for storing JSON in a file working with the code below. I added a saveString() because the java way copied from the example did not worked, creating no file. So i deleted it. The Processing way ;)

import processing.opengl.*;
import java.io.*;
import java.util.List;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.json.DataObjectFactory;
ArrayList <String> toWrite = new ArrayList();
Twitter twitter;
void setup() {
size(600, 400);
//Credentials
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey(""); // INPUT CREDENTIALS HERE!!
cb.setOAuthConsumerSecret("");
cb.setOAuthAccessToken("");
cb.setOAuthAccessTokenSecret("");
cb.setDebugEnabled(true);
cb.setJSONStoreEnabled(true);
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
System.out.println("Saving public timeline.");
try {
new File("statuses").mkdir();
List<Status> statuses = twitter.getHomeTimeline();
for (Status status : statuses) {
String rawJSON = DataObjectFactory.getRawJSON(status);
println("@" + status.getUser().getScreenName() + " - " + status.getText());
println("rawJASON= "+ rawJSON);
String fileName = "statuses/" + status.getId() + ".json";
toWrite.add(rawJSON + "\n"+" "+"\n");
System.out.println(fileName + " - " + status.getText());
}
saveStrings("saved", toWrite.toArray(new String[0])); /// Processing way...
System.out.print("\ndone.");
System.exit(0);
} catch (TwitterException te) {
te.printStackTrace();
System.out.println("Failed to get timeline: " + te.getMessage());
System.exit(-1);
}
}//eof setup

v.k.

Re: Saving JSON data from twitter4j - SOLVED

1 year ago

Fyi:

this is reading the saved tweet JSON:

import twitter4j.Status;
import twitter4j.TwitterException;
import twitter4j.json.DataObjectFactory;
String[] rawData;
void setup()
{
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true);
cb.setJSONStoreEnabled(true);
rawData = loadStrings("saved.txt");
try{
Status status = DataObjectFactory.createStatus(rawData[0]);
println(status);
} catch (TwitterException te) {
te.printStackTrace();
System.out.println("Failed to get timeline: " + te.getMessage());
System.exit(-1);
}
}

fari

Re: Saving JSON data from twitter4j - SOLVED

4 months ago

I tried to use createUser and print the name on screen. But it didn't return anything. Any help?

PhiLho

Re: Saving JSON data from twitter4j - SOLVED

4 months ago

faridoon, this question seems totally unrelated to the current one. You should make a new thread, instead.

Top Reply