Preparing a third party resource as a library (Word Predictor). Problem finding Environment database.
in
Contributed Library Questions
•
2 years ago
First I'm not 100% familiar with Java, I did it years ago. This is might not be a processing problem, but you seem like smart people. So I'm sure you can help me out
See this
http://www.projectpossibility.org/viewProject.php?id=5
Ok if you download "New version (with Wikipedia trained prediction!) Windows - 127mb" it is actually not windows specific at all, it works fine on Linux (java 1.6). In keyboard folder ignore the bat you can run the jar from the terminal:
You can get an idea how it works.
To create the library here is the directory structure:
In the resources folder you have dictionary databases which arre a collection of maps for the frequency analysis. So each of those folders contain lot of jdb files, and it basically a trained system, from normal input and also external resources like wikis. It predicts what the next word might be based on previous ones as well as does word completions (which you probably already know by now).
The keyboard is just an example front end, I want to use my own processing interface and word prediction will be part of that. Actually I'm also working on my own physical interface with an arduino prototype, which is a "keyboard" with no keys. Confused? good you should be..haha. This is for my friend and hopefully will spawn a family of assessable interfaces that will help lots of people. This is very early days though. In the future there will be a display+small processor like an android device for previewing, portability and a bunch of cool stuff like training games, but it will also have a "normal" USB keyboard client for plugging in wherever.
Anyhow..I don't need to show all the code as that is not relevant to this question. If you restart processing IDE check that it has found the new library, create a new project. You don't need to import everything, this will do:
If you run you get this error:
Apparently is is having trouble finding the dictionary. But I'm not sure the issue is loading it with File I think is is something do with loading the file with Environment. I tried adding to class path, changing current directory, etc but I'm not sure this is really the problem.
If you look at the source of TreeMapWordPredictor:
http://ss12.info/svn/wordpredict/wordpredictor/trunk/src/org/ss12/wordprediction/TreeMapWordPredictor.java
Line 100 hints that is not really to do with the file path becuase that happens earlier.
I commented that whole constructor out complied the file in its cloned directory structure relative to library:
and then update from librabry directory:
I'm pretty confident this library will work ok, if it can access its data. Any ideas?
See this
http://www.projectpossibility.org/viewProject.php?id=5
Ok if you download "New version (with Wikipedia trained prediction!) Windows - 127mb" it is actually not windows specific at all, it works fine on Linux (java 1.6). In keyboard folder ignore the bat you can run the jar from the terminal:
- $ java -mx512m -jar WordPrediction.jar
You can get an idea how it works.
To create the library here is the directory structure:
- sketchfolder/
- libraries/
- WordPrediction/
- library/
- WordPrediction.jar
- lib/
- resources/
- dictionaries/
- bdb/
- bi/
- dict/
- tri/
- uni/
In the resources folder you have dictionary databases which arre a collection of maps for the frequency analysis. So each of those folders contain lot of jdb files, and it basically a trained system, from normal input and also external resources like wikis. It predicts what the next word might be based on previous ones as well as does word completions (which you probably already know by now).
The keyboard is just an example front end, I want to use my own processing interface and word prediction will be part of that. Actually I'm also working on my own physical interface with an arduino prototype, which is a "keyboard" with no keys. Confused? good you should be..haha. This is for my friend and hopefully will spawn a family of assessable interfaces that will help lots of people. This is very early days though. In the future there will be a display+small processor like an android device for previewing, portability and a bunch of cool stuff like training games, but it will also have a "normal" USB keyboard client for plugging in wherever.
Anyhow..I don't need to show all the code as that is not relevant to this question. If you restart processing IDE check that it has found the new library, create a new project. You don't need to import everything, this will do:
- import org.ss12.wordprediction.WordReader;
- import org.ss12.wordprediction.TreeMapWordPredictor;
- import org.ss12.wordprediction.model.WordPredictor;
- WordPredictor predictor = new TreeMapWordPredictor();
- //WordReader wordReader = new WordReader(predictor);
- //String input = "the cat walked down the r";
- //String[] results = predictor.getSuggestionsGramBased(predictor.processString(input),1);
- // update unigram, bigram and trigram according to user input string
- // for(String r:results) println(r);
If you run you get this error:
- com.sleepycat.je.log.LogException: (JE 3.2.76) Environment home ./resources/dictionaries/bdb/dict doesn't exist
- at com.sleepycat.je.log.FileManager.<init>(FileManager.java:228)
- at com.sleepycat.je.dbi.EnvironmentImpl.<init>(EnvironmentImpl.java:302)
- at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:102)
- at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:54)
- at com.sleepycat.je.Environment.<init>(Environment.java:103)
- at org.ss12.wordprediction.TreeMapWordPredictor.<init>(TreeMapWordPredictor.java:100)
- at encoder_interface_pde.setup(encoder_interface_pde.java:62)
- at processing.core.PApplet.handleDraw(Unknown Source)
- at processing.core.PApplet.run(Unknown Source)
- at java.lang.Thread.run(Thread.java:662)
Apparently is is having trouble finding the dictionary. But I'm not sure the issue is loading it with File I think is is something do with loading the file with Environment. I tried adding to class path, changing current directory, etc but I'm not sure this is really the problem.
If you look at the source of TreeMapWordPredictor:
http://ss12.info/svn/wordpredict/wordpredictor/trunk/src/org/ss12/wordprediction/TreeMapWordPredictor.java
Line 100 hints that is not really to do with the file path becuase that happens earlier.
I commented that whole constructor out complied the file in its cloned directory structure relative to library:
- $ javac -classpath '.:/home/zero/sketchbook/libraries/WordPrediction/library/WordPrediction.jar' TreeMapWordPredictor.java
and then update from librabry directory:
- $ jar uf WordPrediction.jar org/ss12/wordprediction/TreeMapWordPredictor.class
I'm pretty confident this library will work ok, if it can access its data. Any ideas?
1