Combining two pieces of code

Hey there,
Im trying to combine an image segmentation code with another piece of code that takes the segments from the image segmentation and applies them to a different image. I need some help to figure out how to automate the process.

The image segmentation code is:
The two codes im running in processing are:
which looks like:
The image I am using currently is:
Since the image segmentation code is written in Python, I needed to download the Python libraries Scipy, Numpy, Matplotlib and Scikit from

This is the code that I need to take the segments created and assign them to another image of my choice:
The idea I had was for all the segments that are created by sketch 1 to go into the data folder in sketch 2 and they would be assigned to my image. Right now sketch 2 looks like: with the data folder being empty.
The image I am currently using is:

I need to figure out some way to automate the image container process in sketch 2, so that it takes a certain amount of segments from sketch 1.
I would like to move sketch 2 into sketch 1, so they just become one sketch file.

Right now If I manually take just the first three of the segments from sketch 1 and put them in the sketch 2 data folder I come up with this final product:
Which doesnt quite look right because I haven't figured out how to scale back and overlap the segments either, and it appears to assign the segments to the white area of the image as well.

Any ideas?



  • That is lots of code to merge, or at least, code with different functionalities to merge. Do you want to call the python code right from Processing?

    Here I have looked at your latest sketch where you placed the mini images into the hand image. I have made two changes. The first one is loading n number of images. Now, you can change n to reflect how many images you have in the folder and then it will load them for you if all those images are saved with the same file name formatting that you set initially.

    The second change was introducing the variables STEPX and STEPY. When you read the mini-images, they are resize to these values. Then, when you are stepping in the main image, you load those mini images as pieces in a puzzle.

    You need to keep in mind that you are using imageMode(CORNER) which is the default image mode in Processing. This means that your mini-image's top left corner is placed right at the x and y position. Notice that when you select the mini-image, you are judging the whole mini-selection based on a single pixel. You might reconsider this concept at a later time in your project.

    In a nutshell, hre I have addressed:

    Which doesnt quite look right because I haven't figured out how to scale back and overlap the segments either

    and it appears to assign the segments to the white area of the image as well.

    It assigns an image to the white segment as the white segment also has color. If you do not want a mini-image in the white background, then you will to adapt your algorithm. For example, you could ignore any pixel color that has a white tone. First question though, what happens if you have another image with white background, but the interior of the image contain some feature painted in white? To be able to generate the right algorithm, it is important to understand the nature of the source images. Do you have a set of images that you want to apply this to? I am referring to images similar to your hand.jpg and images similar to the template image where all the mini-images are extracted.


    final int len=25;
    final float thresh=170;
    final int STEPX=18;
    final int STEPY=18;
    boolean newDesign=false;
    PImage pic;
    ArrayList<PImage> imgContainer;
    int n=3;
    void setup() {  
      size(800, 800, P2D);
      colorMode(RGB, 255);
      pic.resize(width, height);
      color c1 = color(200, 25, 25);
      color c2 = color(25, 255, 200);  
      imgContainer=new ArrayList<PImage>();
      for (int i=0; i<n; i++) {
        PImage pimg=loadImage("test2.jpg_"+i+".png");
      noLoop();  //Driven by redraw
    void draw() {
      background(250, 250, 250);
      if (newDesign==false) {
      for (int y = 0; y < height; y+=STEPY) {
        for (int x = 0; x < width; x+=STEPX) {
          // Get the color stored in the pixel
          int index=y*width+x;
          color pixelValue = pic.pixels[index];
          // Determine the brightness of the pixel
          float pixelBrightness = brightness(pixelValue);
          float imgPicked=map(pixelBrightness,0,255, 0, n);
          image(imgContainer.get((int)imgPicked), x, y);
    void mouseReleased() {
  • edited July 2017

    So in sketch 1 the python code is already being called from within processing. I was hoping to somehow just run sketch 1, where sketch 1 puts the segments in its regular segments folder, and then sketch 2 would run automatically after that fact, to then extract those segments out of the segments folder to create the final image. So the user would only need to load in the image that they wanted segmented and the image they want the segments applied to.
    Right now im just dragging and dropping the segments created into sketch 2's data folder, and then running sketch 2.
    Oh I see about white being a colour, that makes sense.
    Hmm, ok. The original idea was from
    where I guess here the text assigned to the image is measured in shades?
    I'd like the output image to be black and white regardless, so if I loaded a black and white hand.jpg in, how would I go about having the segments just applied to where the greyscale area is, so white wouldnt even count in the scenario?
    I've put in your code (thanks so much, this is perfect) and im getting the error:
    IndexOutofBoundsException: Index: 3, Size:3
    As for a set of images I wish to apply the segments to, it doesnt really matter...I'm curious to see it done on any image.

  • im getting the error: IndexOutofBoundsException: Index: 3, Size:3

    I have to say my code was not tested as you see. Can you tell me what line does the error point to? I think you need to provide me the actual line in your code as line numbers will not be of much help, unless you share your current code, for example using pastebin.

    I didn't study the python code bc I am not into python and I cannot set it up at the moment. However, here is an idea. You don't need to move the files generated in sketch1 to a folder in sketch2. What you could do is from your sketch2, to load the files directly from the data folder in sketch one. To do this, you need to provide the full path to access those files in your sketch2. For instance, usually you do this:

    PImage img=loadImage("photo.png");

    Instead do this:

        final String pathStr="C:\\Users\\You\\Documents\\Processing\\sketch1_folder\\";  //Set it in global scope
        PImage img=loadImage(pathStr+"photo.png");

    A better version is to have the files generated by your python script stored in the sketch2's data folder directly. A third option is to save the files in a "middle-man" folder by sketch1 and then sketch2 will read the files from this folder. A middle-man folder means a folder that it is not inside any of the sketch folders but they reside in a temporal folder (TMP dir).

    Related to the white background and assigning images only to shaded areas. It is possible. Regard the following line in my previous post:

    float imgPicked=map(pixelBrightness,0,255, 0, n);

    What this line says is that you project the pixel brightness into a scale that goes from 0 to n-1, inclusive. Notice the brightness can take values between 0 and 255. You need to change this. Instead you can run this example and see by yourself:

    final int ZEROBRIGHTNESS=0;
    final int HALFBRIGHTNESS=128;
    final int LOW=0;
    final int HIGH=8;
    float d=255;  //Full brightness

    The value of d turns out to be 15.93. However, d is larger than my expected value of HIGH. This is how you can tell the color is white, by doing the following checking:

       ...If this is true, then d is a bright color

    In your case, you only want to add mini-images for shaded regions, so you need to change the condition:

         ... a shaded pixel... place a mini-image here
      else {
         ... do nothing: don't place a mini-image this time


  • Sorry, for the index error it is highlighting this line on my end:
    image(imgContainer.get((int)imgPicked), x, y);
    Before I continue further with what you have kindly provided for me, ill see if I can fix this error first. :)

  • edited July 2017

    So I've managed to edit the Python code to specify the output of the segments now, so the segments are now able to go right into sketch 2's data folder, but its going in the data folder as its own folder:
    Would there be a way of specifying this folder location using PImage?
    Still not figuring out the index error.

  • edited July 2017

    In the original "" Python script, it created a subfolder called "segments/" inside what is called the Current Working Directory (CWD):

    In order to force that subfolder to be created in my Python Command sketch's "data/", I had to prepend command cd before invoking python:

    So, the full directory path where the segment images would go into when "" was run is:

    Within Python Command sketch, this statement below determines which CWD is for "":
    final String dp = dataPath("");

    If you wish for some other folder path to be the CWD, simply assign that to variable dp as a String. :)>-

  • edited July 2017

    Hey thanks @GoToLoop, I managed to assign sketch 2's data folder as the final destination for the segments. Here is the updated
    But it goes in the data folder as its own seperate folder like so:
    So I need to change sketch 2 to pick this folder instead.
    To make this probably easier for you and @kfrajer, i've done a screen-record of what everything looks like on my end at the moment.

  • I can't go to b/c I don't have an account there!

    Please post it as a regular YouTube video! :-\"

  • edited July 2017

    Should be good now, put it to public:

  • edited July 2017

    L#76 -> folder = r'C:\Users\yourname\Desktop\sketch2\data'
    L#79 -> filenm = os.path.basename(f)
    L#83 -> segments_folder = os.path.join(folder, filenm[:-4]+"_segments")

    Those 3 statements above are what form the full directory path:

    • filenm[:-4] grabs all of the characters from an image's basename, skipping its extension.
    • And then prepends it to "_segments".
    • So, if your image's filename is called "test.jpg" or "test.png", resultant subfolder name's gonna be "test_segments". :-B
  • Yup! So what im wondering now is, how can I edit sketch 2 now to pick test_segments from within the data folder? The thing is, each time I load a new image in sketch 1, it'll create a new test_segments folder for the segments created.
    So im not entirely sure how to pick out the test_segments folder...being that the next test_segments folder will be called "test_segments2" then "test_segments3" and so on.
    The idea I had was that if sketch 1 and sketch 2 were just combined into one sketch file, then you load the image to be segmented in on one end and then PImage would pick out the most current test_segments folder to work with, and then just assign the segments to the image you want.
    Haha, so complicated.

  • edited July 2017

    ... being that the next test_segments folder will be called "test_segments2" then "test_segments3" and so on.

    The way your Python script is programmed, all subfolders gonna end w/ "_segments".
    If you want diff. names, just use images w/ diff. names! /:)

  • edited July 2017

    Haha yeah...true. Right but if there is a new folder created each time a new image is loaded in sketch 1, im not sure how I would distinguish them in sketch 2's code. Even if the segments weren't in a folder, im still not sure how I would distinguish them. It would be nice to not have to delete whatever segments are put into the data folder of sketch 2 before loading in a new image in sketch 1. It would make most sense for the segments to be deleted after sketch 2 assigns them to the input image, so we could start fresh each time within the data folder, if that makes any sense

  • For the error, I need to confirm you are using these lines in your code:

    float imgPicked=map(pixelBrightness,0,255, 0, n);
    image(imgContainer.get((int)imgPicked), x, y);

    For debugging, introduce this line between the above lines:

    println("DEBUGGING check n="+n+" map returns "+imgPicked);

    And also, can you tell how you are defining the value of n? Are you doing this manually? The value of n should be based on the number of images available. Also, make sure you are reading the right folder inside your data folder:

    PImage pimg=loadImage("test_segments\\"+"test2.jpg_"+i+".png");


  • edited July 2017

    ... how I would distinguish them.

    In the original "Python Command" sketch:

    I've relied on listPaths() in order to grab all image paths (.png & .jpg) from subfolder "data/images":
    final String pics = join(listPaths(pd, PICS_EXTS), SPC);

    Besides files, listPaths() can also grab folder paths if we pass a "directories" to it after the 1st root folder path argument.

    Then we can remove() all subfolder paths which doesn't endsWith() "_segments".
    After that, we can grab all image paths from each "_segments" subfolder, again w/ listPaths()! *-:)
    And finally we can transfer it all to a global String[][] 2D array. B-)

    I've made a set of util functions to do all that hard work below. #:-S
    Just invoke getFoldersOfImagePaths() and assign it to a 2D array variable.
    Use joinStrArr2d() in order to println() the whole 2D array too: :-bd

     * Grab Image Seg Paths (v1.0.1)
     * GoToLoop (2017/Jul/18)
     * combining-two-pieces-of-code/p1#Item_15
     * executing-python-3-code-from-within-processing/p1#Item_38
    String[][] dirs;
    void setup() {
      dirs = getFoldersOfImagePaths();
      println("Segment folders found:", dirs.length);
    static final String joinStrArr2d(final String[][] arr2d) {
      final StringBuilder sb = new StringBuilder();
      int outer = 0;
      for (final String[] arr1d : arr2d) {
        int inner = 0;
        for (final String path : arr1d)  sb
          .append("] ").append(path).append(ENTER);
      return sb.toString();
    String[][] getFoldersOfImagePaths() {
      final String[] dirs = getSegFolders();
      final String[][] all = new String[dirs.length][];
      for (int i = 0; i < all.length; all[i] = getPathsFromFolder(dirs[i++]));
      return all;
    String[] getSegFolders() {
      final File dataFolder = dataFile("");
      if (!dataFolder.isDirectory())  return new String[0];
      final String[] folders = listPaths(dataFolder.getPath(), "directories");
      final StringList sl = new StringList(folders);
      final java.util.Iterator<String> it = sl.iterator();
      while (it.hasNext())  if (!"_segments"))  it.remove();
      return sl.array();
    String[] getPathsFromFolder(final String folder) {
      return listPaths(folder, "extensions=,png,jpg,jpeg");
  • edited July 2017

    Alternatively, "Grab Image Seg Paths" in Python Mode too: :)>-

     Grab Image Seg Paths (v1.0.1)
     GoToLoop (2017/Jul/18)


    def setup():
        global dirs
        dirs = getFoldersOfImagePaths()
        print joinList2d(dirs)
        print 'Segment folders found:', len(dirs)
    def joinList2d(l2d, FORMAT='[%d][%d] %s\n', s=''):
        for outer, l1d in enumerate(l2d):
            s += ENTER
            for inner, path in enumerate(l1d):
                s += FORMAT % (outer, inner, path)
        return s
    def joinList1d(l1d, FORMAT='[%d] %s\n', s=''):
        for kv in enumerate(l1d): s += FORMAT % kv
        return s
    def getFoldersOfImagePaths():
        return tuple(getPathsFromFolder(dir) for dir in getSegFolders())
    def getSegFolders(NAME='_segments', OPT='directories', ROOT='', NONE=()):
        dataFolder = this.dataFile(ROOT)
        if not dataFolder.isDirectory(): return NONE
        dirs = this.listPaths(dataFolder.path, OPT)
        return tuple(dir for dir in dirs if dir.endswith(NAME))
    def getPathsFromFolder(folder, EXTS='extensions=,png,jpg,jpeg'):
        return this.listPaths(folder, EXTS);
  • edited July 2017

    Woah.. that should work yeah!
    Before I proceed with adding this can you clarify something for me:
    Right now the segments created from sketch 1 are going into sketch 2's data folder as its own _segments folder. Changing the python command file here is fixing it before the segments are in sketch 2's data folder or after?
    What would I need to change in sketch 2 then? Confused on this part ..
    -Actually @kfrajer has answered this above.

  • @kfrajer
    My n value is int n=3. Which probably doesnt make any sense. I added the print line and im getting: DEBUGGING check n=3 map returns 3.0

  • edited July 2017

    @daddydean, as explained, those util functions collect the paths of each image inside sketch's "data/" subfolders ending w/ "_segments" as a 2D String[][] array.

    That is, after invoking getFoldersOfImagePaths(), the String[][] returned from that is filled w/ image paths, which are those created by the Python script.

    So you won't need to guess the exact name for each "_segments" subfolder, for they're found automatically if they're inside sketch's "data/". \m/

    You just need to add those util functions to your posted sketch here, declare a global String[][] variable, and call getFoldersOfImagePaths(), assigning it to that variable in setup(): :-B

    String[][] dirs;
    void setup() {
      dirs = getFoldersOfImagePaths();
  • Most probably you're gonna need to declare a 2D PImage[][] array too, in order to store each image loaded w/ loadImage() w/ the paths of the 2D String[][] array. :>

  • edited July 2017

    I see. I think what im getting confused is, what I would put in sketch 2's code then? When you refer to sketch's data folder, you mean sketch 2 obviously right?
    If I am putting your grab-image-seg-paths code in, im putting this directly in the python command code right? Is it from line 28 in the python command onwards that I would delete?
    Sorry im having such a hard time following.

  • edited July 2017
    • Oops, sorry! There was a subtle bug both in Java & Python Mode versions! :-O
    • In getPathsFromFolder(), the "extensions= png,jpg,jpeg" fails for ".png" files! :-\"
    • A comma , is also needed after the assign = character: "extensions=,png,jpg,jpeg" 8-}
    • Just fixed it in both versions 1.0.1 now! 8-|
  • edited July 2017

    When you refer to sketch's data folder, you mean sketch 2 obviously right?

    Yup! Although my set of util functions would work for any sketch which contains image subfolders w/ their names ending w/ "_segments" inside its subfolder "data/". :P

  • edited July 2017

    Oh good haha, I thought for some reason you meant sketch 1's. #-o
    So sketch 2 will need to be changed then as well to accomodate this, I was asking kfrajer about it...but it'd have to be something different now right?
    Is that what you mean by the 2d pimage array?
    So am I adding v1.0.1 after line 28 in the python command code?
    Or am I way off here

  • edited July 2017

    You need to grab the 2D String array w/ getPathsFromFolder() before using loadImage() on them all.

  • edited July 2017

    You can also paste all of my util functions in a separate tab so it won't distract you in the main tab! *-:)

  • edited July 2017

    Ah, ok. So have python command, command java, and a seperate tab with v1.0.1?
    I have to delete after line 28 in python command though right?

  • edited July 2017
    • Only the util functions you may choose to place them in a separate tab.
    • The code within setup() is mere a sample, test-drive; demoing how to use those util functions.
  • edited July 2017

    Ahh my head. Maybe a visual screenshot of what youre looking at might help me at this point, Im a bit lost. I need to reread everything here.

  • edited July 2017
    • Consider those 4 util functions joinStrArr2d(), getFoldersOfImagePaths(), getSegFolders() and getPathsFromFolder() form together 1 library.
    • They're together responsible to scan the running sketch's "data/" subfolder for other subfolders whose names end w/ "_segments".
    • Then they collect all ".png" & ".jpg" paths found in all of those "_segments" subfolders.
    • Their entry point is function getFoldersOfImagePaths(), which in turn calls the other util functions & returns a 2D String[][] array filled w/ all the segment image paths.
    • We can also display those contents, w/ their respective 2D indices too, of that returned 2D String[][] array by invoking joinStrArr2d() and println() its returned String.
    • Now that you've got the 2D String[][] array, you can use loadImage() to load those paths and assign them to a 2D PImage[][] array.
    • It's up to you what to do w/ the resultant 2D PImage[][] for the rest of your own sketch. >-)
  • edited July 2017 So all of this is in what you've written in v1.0.1 right? Im just unclear on where I need to put it within sketch 1. So it should output a string of all the paths, which I will use in sketch 2's PImage?

  • You place those set of functions in the sketch which needs to collect the segment image paths.

  • edited July 2017

    So it would make most sense to be in sketch 2 then I would have to assume, being that the segments are put in the sketch 2 data folder. Im just not sure where in the code I would put it, or am I supposed to run it separately in a tab?
    If I run it on its own it comes up with 'segment folders found: 0'

  • edited July 2017

    If I run it on its own it comes up with 'segment folders found: 0'.

    It means there aren't any subfolder names ending w/ "_segments" inside subfolder "data/" of the running sketch.

  • edited July 2017

    OH haha. So if I put it in sketch 2 just by itself it works and recognizes the folder and all 98 pngs in it. So at this point, im not sure how I combine v1.0.1 with sketch 2, being that sketch 2 is using the PImage that kfrajer suggested. Or is that what you were saying, that I need to change my Pimage and loadimage to something else to accomodate v1.0.1?
    How do I work with loadimage to load in the paths?

  • edited July 2017

    Right now I have in sketch 2's load image:

  • edited July 2017
    • That's called hardcoded literal. It works as long as the source image's name is "test.extension" & the sketch's full path remains the same.
    • However, if another name is used for the image, for example: "example.extension", your sketch won't find those segmented images anymore.
    • Or if you change the location of your ".pde" sketch within your computer's filesystem, or run it on another computer, it will also fail.
    • That's why I've made those util functions, so the segmented images are always findable within the running sketch's subfolder "data/", regardless of the source image's name or the location of the sketch within a computer. :)>-
  • edited July 2017

    Right, so I can run the util functions on their own to find the segments paths within data/, but I dont know how I would change the loadimage in sketch 2 to accept those paths. ? Right now I have your v101 in a separate tab in sketch 2.
    It would be cool to have this all packaged up to work after for people that are interested in image segmentation :)

  • Can you post your current attempt? Recall that getFoldersOfImagePaths() returns a 2D array. L-)

  • Tip: GitHub is an excellent place to post source code w/ multiple folders and data: *-:)

  • edited July 2017

    I dont even really have an attempt, I can locate the segment paths created... (99 of them) Im not sure how I'd make a 2D Pimage though
    Right now Im assuming all the paths created would go into the loadimage
    I found
    But im not sure how this will help.

  • ... all the paths created would go into the loadImage().

    But you're gonna have to store all of the created loadImage()'s PImage objects somewhere, right?
    In the least, you're gonna need a 1D PImage[] array to store them all. 8-X

  • edited July 2017

    Ohhhh, I see. Are all the images being deleted after, and thats why they need to be in an array?
    If your util functions outputs the segment paths, where is it outputting them to exactly, or how do I call it in my Pimage? Right now it looks like this obviously;
    There is
    Would something like this work?

  • edited July 2017

    Are all the images being deleted after, ...

    For all garbage-collect-based languages, like Java, JS, Python, etc., if no variable holds the reference of some object, that object is marked for deletion! :-h

    So after using loadImage(), its resultant PImage needs to be immediately assigned somewhere, or else that "newborn" goes straight to the "trash bin", as if it had never existed. =((

    Arrays got indexed [] slots which act much like variables:

    But the array container itself is assigned to 1 variable.
    It's as if its whole indexed elements shared the same variable name. \m/

  • edited July 2017

    Gotcha about the garbage. Im not sure if I follow what you mean by "resultant PImage". Does it matter where its assigned? Im not sure where to begin here at all to be honest.
    Would it be easier to have a function where the PImage picks the most recently created folder within data/? If there is a function for that..

  • edited July 2017

    ... what you mean by "resultant PImage".

    PImage is the datatype of the resultant object from calling Processing's createImage() and loadImage() functions: :-B


    There are others ways to create PImage objects too, such as:

    Directly using operator new over class PImage: PImage img = new PImage();:

    Or invoking PImage's get() method w/ no arguments: PImage clone = img.get();

  • Right, but what am I putting after Pimage get or PImage new? How do I get the paths created from the util functions?
    Alternatively im wondering if it might be easier to just pick the most recent folder created within sketch 2's data/ folder, no matter what the folder is named, if you know what i mean?

  • edited July 2017

    I'm wondering, how much do you know about Java, Python or any programming in general? :-/

    It seems even when I try to explain some subject in deep, you just think about where to copy and paste it rather than trying to understand the concept. :-@

    Gonna try to come up w/ some other example here...

  • edited July 2017

    I only know a tiny bit, brand new this month. I get the basic functions of things but its difficult for me to put them all together. I get what the util code you have provided does, but theres no way I'd ever be able to write something like that myself at the moment. Im unsure how to route the paths of the image segments into PImage correctly, therefore I was thinking itd be likely easier to just pick the most recent folder created in data/ and keep whatever folders build up in there and remove them myself. I really really appreciate all of your help, and it all works on my end...was just hoping to automate it a bit to make it less time consuming when loading and unloading images. Its not essential, but it could help.

  • edited July 2017

    @GoToLoop, I think it would be easier to actually combine sketch 1 and sketch 2 into one sketch file. That way you could just have one data folder where you load the images, and where the folders are created. I tried combining them myself but im getting a "duplicate method setup error" which Im assuming means that I can't have a "void setup" in sketch 1 as well as a "void setup" in sketch 2.
    I figured out how to scale back the segments in sketch 2, so I am getting a proper image now: but the segments are still being assigned to the white portion of my hand picture. This is the hand image mentioned at the very top.
    Do you mind pointing me in the right direction on how to combine sketch files?
    I found
    Checking it out.

Sign In or Register to comment.