Executing python 3 code from within processing

daddydean

Hello!
I'd like to integrate this code which is written in Python into my current code but im not sure how to go about it since the script is in Python 3:
https://github.com/alex-parker/CenturyOfTheSun/pull/1/commits/ed50abc1dce0f88e4e517109910bc4d52dbd6079?diff=unified
Is there a way I could execute this script using processing?
I know Jython only supports up to 2.7, so im trying to find a workaround.
:)

GoToLoop

https://www.Reddit.com/r/processing/comments/6l8itb/python_mode_in_processing/

https://Processing.org/reference/launch_.html

daddydean

Haha, yeah thats me.
I'll try the launch code.
-Im just getting a grey box.. am I supposed to specifiy a destination for the segments?

jeremydouglass

"I know Jython only supports up to 2.7, so im trying to find a workaround."

That is it in a nutshell: "How can I execute Python 3 code in Python 2?"

You don't. You either back-port your 3 code to 2 or you run it externally under a separate process. That separate process is separate -- so you'll need to have it communicate back to your 2 code somehow e.g. via writing data to disk, passing OSC or serial messages, etc. etc.

daddydean

@jeremydouglass
Thanks for the clarification.
How would I backport this code from 3 to 2?
Where do I begin?
I've never used Python before :\
Would be great to have it all in one place, but I suppose if it becomes too difficult I'll just do it separately.

jeremydouglass

If you have never used Python before I do not recommend trying to rewrite it from Python 3 to 2.

If your sketch is just segmenting images once then using Processing launch() or exec() should be fine.

When you say "integrate this code ... with my current code": what does each part do, and what should they do together when they are done?

GoToLoop

https://GitHub.com/GoToLoop/command/blob/patch-1/src/deadpixel/command/Command.java

daddydean

@GoToLoop
Sorry, what does this do?

daddydean

@jeremydouglass
Ok.
I tried the launch code, but it didnt seem to do anything so maybe I am doing something wrong on my end.
The code I am working with is here (thanks to @kfrajer):
https://mega.nz/#F!akA3XBYK!8--3DOBeqXyl6tNlIx89NA
The idea was to segment an image and then apply those segments to make up the input image.
So if I have: http://imgur.com/a/i7eVS
All these squigglies would be cut out, saved to a folder and then applied to make the input image (squiggly 1, squiggly 2, etc...)
So if the input image was a picture of a hand, I'd come out with a squiggly hand.

GoToLoop

• @daddydean, that's merely a more elaborate launch() as a pure Java library. :-\"

• In order to add it to your sketch, hit CTRL+SHIFT+N and name the new tab exactly as Command.java.

• Then you can import it to your sketch like this: import deadpixel.command.Command;.

• Instantiate it this way: Command cmd = new deadpixel.command.Command("notepad");.

• And execute it as: cmd.run();.

• Given in the case above it's been instantiated w/ "notepad", if you're in Windows, it should run its text editor "Notepad.exe".

• Let's say you've got a file called "abc.py" inside a sketch's subfolder "data/" w/ the following content:

#!/usr/bin/env python

print('abc defg hijk lmnop')
print('qrstuvw xyz')

If you've got Python properly installed in your OS, the ".pde" sketch below (together w/ "Command.java" + "data/abc.py") should work for ya: O:-)

/** 
 * Python Command (v1.0.1)
 * GoToLoop (2017/Jul/05)
 *
 * https://Forum.Processing.org/two/discussion/23324/
 * executing-python-3-code-from-within-processing/p1#Item_9
 *
 * https://GitHub.com/GoToLoop/command/blob/patch-1/src/
 * deadpixel/command/Command.java
 */

import deadpixel.command.Command;
Command cmd;

static final String APP  = "python ";
static final String FILE = "abc.py";

void setup() {
  final String path = dataPath(FILE);
  cmd = new Command(APP + path);
  println(cmd.command);

  cmd.run();
  println("Success:", cmd.success);
  printArray(cmd.getOutput());

  exit();
}

daddydean

Hey thanks.
Im pretty slow at all this. Sorry haha.
So I've put the segementing.py in a folder called data inside where my sketch above and command.java are. I have python 3.4 installed.
Where am I putting your deadpixel code?
https://github.com/GoToLoop/command/blob/patch-1/src/deadpixel/command/Command.java
Im getting "no library found for deadpixel.command"

GoToLoop

Where am I putting your deadpixel code?

Already explained that in my previous reply:

In order to add it to your sketch, hit CTRL+SHIFT+N and name the new tab exactly as Command.java.

• Well, at least that is how we create a new empty tab in the PDE.
• Still need to copy & paste the original "Command.java"'s source code into the newly empty tab.
• Of course, if you have the "Command.java" file somewhere else, you can simply copy it into the same folder as your ".pde" file. ;;)

GoToLoop

I'm getting "no library found for deadpixel.command"

• That's just a warning & PDE is dumb! ;))
• Just make sure you're using latest PDE 3.3.5: https://Processing.org/download/

I have Python 3.4 installed.

That's an old version. I've got 64-bit Python 3.6.1 in my OS currently: >-)
https://www.Python.org/downloads/

daddydean

Alright, I downloaded the latest Python. Yep, have 3.3.5 installed.
Im getting the error still though, probably have just mixed something up :|
http://imgur.com/a/x1Hd7
aaahh mann

GoToLoop

Well, I've said exactly as Command.java. L-)
However, according to your screenshot, it's named as command.java instead! #-o

daddydean

Oh, the captal C....derp. Makin' a fool outta myself here.
Ok, I am now getting a
COMMAND ERROR: Cannot run program "pythonC:\Users\myname\Desktop\sketch_170705a\data\segmenting.py": CreateProcess error=2, The system cannot find the file specified
Do I need to have python in the folder as well?

GoToLoop

My example works here. Function dataPath() is responsible to grab the full path to "abc.py" outta sketch's "data/" subfolder. 8-|

daddydean

I've put the full path of segmenting.py into the datapath and im getting: expecting "class", found 'string'
So sorry man, I mean im sure I'll get it eventually haha.
First time in processing and python.

GoToLoop

• Before attempting more complex Python programs, you should begin w/ easy 1s.
• Have you already run my example using cmd = new Command("notepad");and see whether it works?
• Then w/ cmd = new Command(APP + path); for static final String FILE = "abc.py";?

daddydean

Yeah its true, bit in the deep end now. But I need to do something fairly specific for this art project I'm working on, the segmenting code looked pretty great for what I needed, hard to give up this far in.
I didnt quite understand that part, am I running the example from within processing, python?
I've tried both, but neither work
For my datapath function I have:
final String path = dataPath(C:/Users/myname/Desktop/sketch_170705a/data/segmenting.py);

GoToLoop

I've tried both, but neither work ...

Before advancing to your actual objective, 1st you need to run my example sketch successfully w/ the "abc.py" Python program.

... am I running the example from within processing, python?

The external Python program "abc.py"'s execution is oversëered by PDE's Java Mode:

#!/usr/bin/env python

print('abc defg hijk lmnop')
print('qrstuvw xyz')

Once it's finished, it should output like this in the PDE's own bottom console:

Success: true
[0] "abc defg hijk lmnop"
[1] "qrstuvw xyz"

daddydean

So, I have abc.py whose content is
" #!/usr/bin/env python

print('abc defg hijk lmnop') print('qrstuvw xyz') "

Im just not understanding the cmd thing, where am I putting that, in my sketch? Might need to call it a night soon, my brain is clearly dying
Mind showing me a screenshot of what yours looks like?

GoToLoop

"PythonCommand.pde":

"PythonCommand/"

"PythonCommand/data/"

daddydean

Ok, so yeah I have the exact same thing as you, I was just missing ENTER. However I am still getting an error:
Cannot run program "python": CreateProcess Error 2 the system cannot find the file specified
-I put the python.exe in the folder and that made no difference either.

GoToLoop

Are you on Windows? When Python is properly installed, we can run it in a command line terminal by simply typing in: python. And hit ENTER.

If that doesn't work for ya, Processing won't be able to find it either!
Unless we pass its actual full path. Or manually add Python to OS' PATH.

daddydean

Yeah im on windows. Like the regular ol' command prompt?
I tried in the command prompt and yeah its not working. I see.. By path, do you mean where it is installed?

GoToLoop

Everything I've posted so far was assuming you had a properly Python installed in the OS.

And by properly I mean Python can be run from the command line terminal by just typing python in it.

I'm puzzled why it's not working for ya. Python's installation should take care all of that automatically for you, like it did for me here! ^#(^

daddydean

@GoToLoop
Ok, so if I shift-click the Python folder and "open a command window here" and type python it works. But not from just the command window. I currently have it installed on my desktop. So how would I fix the path then..?

GoToLoop

https://Docs.Python.org/3/faq/windows.html

https://Docs.Python.org/3/using/windows.html#setting-envvars

daddydean

Great, ill check this out in the morning and get back to you. Thanks for all the help

GoToLoop

Another thing to be aware of is which installer you've picked among the 1s below: /:)
https://www.Python.org/downloads/release/python-361/

I've chosen this 1 for installation here in my 64-bit Windows 8.1: :>
https://www.Python.org/ftp/python/3.6.1/python-3.6.1-amd64.exe

B/c the normal behavior is for the installer to already take care of including python & pip in Windows' PATH environment variable. :-@

daddydean

I managed to get python working from the command line!
Im using windows 7, 64 bit I think..
Now...haha, im getting a different error:
http://imgur.com/a/wsYav

GoToLoop

The exception comes directly from Python and states there's some SyntaxError at line #1 in "abc.py".
However, the 1st line is merely a comment and shouldn't raise any exceptions aFaIK: :|

#!/usr/bin/env python

print('abc defg hijk lmnop')
print('qrstuvw xyz')

GoToLoop

Since we're in Python Mode's forum category, I've converted my Java Mode sketch to Python Mode. \m/

PythonCommand.pyde:

"""
 Python Command (v1.1.1)
 mod GoToLoop (2017/Jul/06)

 https://Forum.Processing.org/two/discussion/23324/
 executing-python-3-code-from-within-processing#Item_32

 https://GitHub.com/GoToLoop/command/blob/patch-1/src/
 deadpixel/command/Command.java
"""

from Command import Command

APP, FILENAME = 'python ', 'abc.py'

def setup():
    global cmd
    path = this.dataPath(FILENAME)
    cmd = Command(APP + path)
    print cmd.command

    cmd.run()
    print 'Success: ' + `cmd.success`
    print cmd.getOutputAsTuple()

    exit()

Command.py:

from java.io import BufferedReader, InputStreamReader, IOException
from java.lang import Runtime, StringBuilder, InterruptedException
from sys import stderr

class Command:
    _ERR = 'COMMAND ERROR(s):\n'
    _LF, _SPC = '\n', ' '

    def __init__(c, command):
        c.command = command
        c.success = False

        c.outputBuffer = []
        c.runtime = Runtime.getRuntime()


    def __str__(c): return c.command + Command._SPC + `c.success`

    def getOutput(c): return c.outputBuffer[:]

    def getOutputAsTuple(c): return tuple(c.outputBuffer)


    def run(c):
        c.success = False
        del c.outputBuffer[:]

        try:
            process = c.runtime.exec(c.command)
            sb = StringBuilder(Command._ERR)

            out = BufferedReader(InputStreamReader(process.inputStream))
            err = BufferedReader(InputStreamReader(process.errorStream))

            while True:
                read = out.readLine()
                if not read: break
                c.outputBuffer.append(read)

            while True:
                read = err.readLine()
                if not read: break
                sb.append(read).append(Command._LF)
            if sb.length() != len(Command._ERR): print >> stderr, sb

            c.success = not process.waitFor()

        except IOException, e:
            print >> stderr, 'COMMAND ERROR: ' + e.message

        except InterruptedException, e:
            print >> stderr, 'COMMAND INTERRUPTED: ' + e.message

        return c.success

daddydean

Well thank the lord, I got it. It was because I left >>> in the python code in abc.py, so it comes out as true now.
Also just tried your code above in python mode and it comes out true as well!

daddydean

@GoToLoop
Im still stuck in figuring out how to run the segmenting.py code :\
I found: http://www.lfd.uci.edu/~gohlke/pythonlibs/
update: I installed numpy, scipy, pillow, matplotlib...

GoToLoop

I've tried your "segmenting.py" Python script and made it run under my "Python Command.pde" sketch.

However, just like @akb74 from your discussion on Reddit:
https://www.Reddit.com/r/processing/comments/6l8itb/python_mode_in_processing/djt3f1i/

It's loading my png file, then complaining it isn't two dimensional.

I've also got even more errors & warnings like these 2 below:

---

File "C:\Program Files\Python36\lib\site-packages\skimage\filters\thresholding.py", 
line 196, in threshold_local "``block_size`` {0} is even.".format(block_size))
ValueError: The kwarg ``block_size`` must be odd! Given ``block_size`` 40 is even.

---

C:\Program Files\Python36\lib\site-packages\skimage\filters\thresholding.py:222: 
skimage_deprecation: 
Function ``threshold_adaptive`` is deprecated and will be removed in version 0.15.
Use ``threshold_local`` instead.

---

Perhaps this Python script is so old that the libraries which it relies on have changed a lot by now. :-SS

Also "segmenting.py" creates a subfolder in Current Working Directory (CWD) via os::mkdir():
os.mkdir(os.path.join(os.getcwd(), 'segments'))

However, if it already exists, it crashes the script! :-&

I've fixed it by replacing that statement w/ these lines: :ar!

folder = os.path.join(os.getcwd(), 'segments')
os.path.isfile(folder) and os.remove(folder)
os.path.isdir(folder) or os.mkdir(folder)

But that's as far as my Python knowledge reaches. Can't help ya much further! X_X

GoToLoop

update: I installed numpy, scipy, pillow, matplotlib...

I had to install numpy, scipy & scikit_image too.
Others were dependencies of those 3 cited above and got installed along.

I've used Python's builtin utility pip for it.
However, simply typing in "pip install numpy" won't install the version appropriate for Windows!

I had to manually download those 3 packages as ".whl" files from:
http://www.lfd.uci.edu/~gohlke/pythonlibs/

Given I've got 64-bit Python v3.6.1 for Windows installed, I've downloaded these 3 ".whl" files:

1. numpy‑1.13.1+mkl‑cp36‑cp36m‑win_amd64.whl
2. scipy‑0.19.1‑cp36‑cp36m‑win_amd64.whl
3. scikit_image‑0.13.0‑cp36‑cp36m‑win_amd64.whl

Other packages such as pygame, pyglet, matplotlib, etc. would install just fine w/ just: pip install pygame

P.S.: You're probably gonna need to run the command prompt as administrator in order for pip to install packages! ^#(^

GoToLoop

Anyways, I've modified my original sketch to work w/ "segmenting.py".

Problem is "segmenting.py" was creating a subfolder "segments/" inside PDE's installed folder instead of the current running sketch.

So I had to come up w/ a workaround to change CWD before executing "python".
Just found out that "cmd /c cd" + dataPath("") + "&& python " would take care of it. \m/

Besides the path to "segmenting.py", I've also had to concatenate the paths of each of the images as arguments using dataPath() too. #:-S

This time, I've created 2 subfolders inside sketch's subfolder "data/":

1. "scripts/" -> "abc.py" & "segmenting.py"
2. "images/" -> "1.jpg", "2.png" & "3.gif".

Of course, put your own images in "data/images/". Here's newest "Python Command v2.0": :bz

/** 
 * Python Command (v2.0.2)
 * GoToLoop (2017/Jul/05)
 *
 * https://Forum.Processing.org/two/discussion/23324/
 * executing-python-3-code-from-within-processing/p1#Item_38
 *
 * https://GitHub.com/GoToLoop/command/blob/patch-1/src/
 * deadpixel/command/Command.java
 */

import deadpixel.command.Command;

static final String BASH =
  platform == WINDOWS? "cmd /C " :
  platform == MACOSX? "open" : "xdg-open";

static final String CD = "cd ", PY_APP = "python ";
static final String AMP = " && ", SPC = " ";

static final String PY_DIR = "scripts/";
//static final String PY_FILE = PY_DIR + "abc.py";
static final String PY_FILE = PY_DIR + "segmenting.py";

static final String PICS_DIR = "images/";
static final String PICS_EXTS = "extensions=,png,jpg,jpeg";

void setup() {
  final String dp = dataPath(""), py = dataPath(PY_FILE);
  final String prompt = BASH + CD + dp + AMP + PY_APP + py;

  final String pd = dataPath(PICS_DIR);
  final String pics = join(listPaths(pd, PICS_EXTS), SPC);

  final Command cmd = new Command(prompt + SPC + pics);
  println(cmd.command, ENTER);

  println("Success:", cmd.run(), ENTER);
  printArray(cmd.getOutput());

  exit();
}

GoToLoop

Python Mode version too, of course: :P

"""
 Python Command (v2.1.2)
 mod GoToLoop (2017/Jul/06)

 https://Forum.Processing.org/two/discussion/23324/
 executing-python-3-code-from-within-processing/p1#Item_39

 https://GitHub.com/GoToLoop/command/blob/patch-1/src/
 deadpixel/command/Command.java
"""

# from deadpixel.command import Command
from Command import Command

BASH = 'cmd /C ' if this.platform == WINDOWS else\
       'open' if this.platform == MACOSX else 'xdg-open'

CD, PY_APP = 'cd ', 'python '
AMP, SPC = ' && ', ' '

PY_DIR = 'scripts/'
# PY_FILE = PY_DIR + 'abc.py'
PY_FILE = PY_DIR + 'segmenting.py'

PICS_DIR = 'images/'
PICS_EXTS = 'extensions=,png,jpg,jpeg'

def setup():
    dp, py = this.dataPath(''), this.dataPath(PY_FILE)
    prompt = BASH + CD + dp + AMP + PY_APP + py

    pd = this.dataPath(PICS_DIR)
    pics = SPC.join(this.listPaths(pd, PICS_EXTS))

    cmd = Command(prompt + SPC + pics)
    print cmd.command, ENTER

    print 'Success: ' + `cmd.run()` + ENTER
    print joinList1d(cmd.getOutput())

    exit()


def joinList1d(l1d, FORMAT='[%d] %s\n', s=''):
    for kv in enumerate(l1d): s += FORMAT % kv
    return s

daddydean

Wow, you are on your game!
Ok so im at where you are, but where am I putting
"cmd /c cd" + dataPath("") + "&& python " ?
Im getting the errors you described above
http://imgur.com/a/cfk7U
Argh, I can taste it!

GoToLoop

Ok so im at where you are, but where am I putting "cmd /c cd" + dataPath("") + "&& python " ?

Whole main sketch is tasked on getting the String full paths via dataPath(), and then concatenate them together w/ the constant variables to form 1 String and pass it as the argument for class Command.

The resulting String becomes Command's command field, and is then println() to the console. :>

Im getting the errors you described above

As I've stated, Python script "segmenting.py" is buggy.
It's using the imported libraries (numpy, scikit_image, etc.) w/ invalid arguments! :-SS

Perhaps in the past when that was written it may have worked.
You're gonna have to ask its author in GitHub to fix it.
Or you can study those libraries' API and figure out how to handle them. :-<

daddydean

Im not sure if I understand the first part...working on it.
Yeah I tried contacting but to no avail, maybe its missing another library.
I got a response elsewhere today from a guy who said the segmenting.py is written in iPython notebook (Jupyter + Anaconda). Not sure if this helps.
Guess its going through error by error now.
Just curious if you are getting the exact same errors, less or more than me. Which ones are you getting?
-One thing I changed was all the "adaptive" to "local"

GoToLoop

Python has a complex variety around it. I'm Python noob and can't help ya w/ that, sorry. :(

daddydean

@GoToLoop Thats ok, you got me this far!
Python noob here too.
Really appreciate all the help.
I installed "py -3.6 -m pip install ipython" and removed line 67 and 68 and im getting a success: true. No errors, but no images...
I think line 67 and 68 is the problem.

GoToLoop

https://www.Reddit.com/r/processing/comments/6l8itb/python_mode_in_processing/

https://www.Reddit.com/r/processing/comments/6lz48i/python_code_in_processing_pt2/

https://www.Reddit.com/r/Python/comments/6lz2m6/running_a_python_code_in_processing/

GoToLoop

https://GitHub.com/GoToLoop/command/blob/patch-1/src/deadpixel/command/Command.java

https://GitHub.com/ntadej/CenturyOfTheSun/blob/master/segmenting.py

daddydean

@GoToLoop
Yo im getting a success true if I load a monochrome bitmap in, but I get no images..
I feel like it has something to do with grayscale images.

GoToLoop

Can you post that B/W image?

daddydean

I literally just tried a black circle: http://imgur.com/a/Mrbrq
Here at the bottom says the image needs to be grayscale:
https://stackoverflow.com/questions/35325193/the-parameter-image-must-be-a-2-dimensional-array
Nevermind, it gives me a success true even with no images in the folder :|

GoToLoop

I get this, after replacing threshold_adaptive w/ threshold_local, and using your B/W ".png" image: :-&

line 20, in adaptive_threshold return np.invert(binary_adaptive) * 1.

TypeError: ufunc 'invert' not supported for the input types, 
and the inputs could not be safely coerced to any supported types 
according to the casting rule ''safe''

And no idea where to use this excerpt. I dunno Python; only a little Python Mode (Jython): ~:>

im = cv2.imread(train_image)
im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
ngcm = greycomatrix(im, [1], [0], 256, symmetric=False, normed=True)

You should post your own custom "segmenting.py" so others are on the same ground rather than guessing in vain! 3:-O

Recall that "segmenting.py" is run by the external python interpreter.
Processing got nothing to do on what happens there! :-@