[SOLVED] Sikuli script and Processing

sidran

[SOLVED] Sikuli script and Processing

in Contributed Library Questions • 1 year ago

How to use Sikuli script within processing? http://sikuli.org/

Sikuli has decent OCR capabilities (AFAIK there is no decent, user friendly OCR library for Processing, why is beyond me)

I tried copying portable version of Sikuli to Code folder of my sketch.. it immediately gives linking errors.

Is it possible to simply link to Sikuli library as is described in Sikuli docs ( http://sikuli.org/docx/faq/030-java-dev.html)

I am missing something.. as it doesnt work.. probably because Sikuli is jar library but has multiple dependent libraries.

Any help?

Thanks

Replies(3)

PhiLho

Re: Sikuli script and Processing

1 year ago

If Sikuli has multiple dependent libraries, you have to put all the dependent jars (and native libraries, if any) in the code folder as well.
But the Java-dev page you point to shows only one jar in the Eclipse project.

" it immediately gives linking errors"
If you don't report the messages, we can't know what happens.
Although it seems interesting, I might try it tonight...

" there is no decent, user friendly OCR library for Processing, why is beyond me"
Because OCR is hard to do correctly? Last time I checked (some years ago...), there was only one open source OCR library, by Google (Tesseract, IIRC), and it was native and not that good. Perhaps some good libraries have emerged since then?

sidran

Re: Sikuli script and Processing

1 year ago

@phi.lho

Thank you for your effort to help me.

My program starts with

import org.sikuli.script.*;

as recommended by Sikuli docs.

First I tried putting sikuli-script.jar in Code folder of my sketch.

Adding following lines to Draw() function (just a test) :

Screen sikuli = new Screen();

try {

sikuli.click("start.png");
// Image of Windows 7 Start button

}

catch(FindFailed e){

e.printStackTrace();

}

Upon start it gives:

[info] Windows utilities loaded.

Exception in thread "Animation Thread" java.lang.UnsatisfiedLinkError: C:\Users\Marko\AppData\Local\Temp\tmplib\VisionProxy.dll: Can't find dependent libraries

Obviously there are some native libraries used.

I then take everything there is in Sikuli folder (very ugly action) and put it in Code folder of my Sketch.

I get exactly the same error message. It cannot find VisionProxy.dll

Then, I try putting all Sikuli files in Code Folder, without any of their folder structure.. all files, no folders

and it still cannot find (same message) VisionProxy.dll which IS in Code folder along with all other dlls and jars.

Do you have any idea what to do? This is very ugly even if it worked.

I also failed to find explanation why we are forced to put everything in Code folder. Very frustrating.

I have no doubts it would work in JAVA as described by sikuli docs.. only if we were allowed to access things the same way.

About OCR. I dont think scrapping onscreen (computer rendered) text is such a scifi operation. IMO it should be part of Processing itself. I am pretty sure Sikuli uses Tesseract.. I dont want to know how :D really! It works in Sikuli IDE. I just need a few numbers scrapped from my games screen.

It is suspiciously strange tho that Processing, which has all kinds of vision/graphics related projects still does not have a simple text scrapping command. It can detect how many fingers I am wagging in front of camera.. but it cannot read time from my bottom right corner of the screen.

Thank you again :)

sidran

Re: Sikuli script and Processing

1 year ago

I am not restarting my machine for weeks so "path" set by Sikuli could not be found :)

I restarted and it actually works!

Sikuli brings so much more power to Processing: OCR, cursor control, image recognition...

Top Reply