We are about to switch to a new forum software. Until then we have removed the registration on this forum.
Hey there,
Im trying to combine an image segmentation code with another piece of code that takes the segments from the image segmentation and applies them to a different image. I need some help to figure out how to automate the process.
SKETCH 1
The image segmentation code is: https://pastebin.com/0sKxKJDF
The two codes im running in processing are:
https://pastebin.com/wd3Fk9Xn
and https://github.com/GoToLoop/command/blob/patch-1/src/deadpixel/command/Command.java
which looks like: http://imgur.com/a/ggPNb
The image I am using currently is: http://imgur.com/a/87ybb
Since the image segmentation code is written in Python, I needed to download the Python libraries Scipy, Numpy, Matplotlib and Scikit from http://www.lfd.uci.edu/~gohlke/pythonlibs/
SKETCH 2
This is the code that I need to take the segments created and assign them to another image of my choice:
https://pastebin.com/Cs4FfhTV
The idea I had was for all the segments that are created by sketch 1 to go into the data folder in sketch 2 and they would be assigned to my image. Right now sketch 2 looks like: http://imgur.com/a/UUSx3 with the data folder being empty.
The image I am currently using is: http://imgur.com/a/3XYzJ
I need to figure out some way to automate the image container process in sketch 2, so that it takes a certain amount of segments from sketch 1.
I would like to move sketch 2 into sketch 1, so they just become one sketch file.
Right now If I manually take just the first three of the segments from sketch 1 and put them in the sketch 2 data folder I come up with this final product: http://imgur.com/a/sIZuA
Which doesnt quite look right because I haven't figured out how to scale back and overlap the segments either, and it appears to assign the segments to the white area of the image as well.
Any ideas?
:3
Answers
That is lots of code to merge, or at least, code with different functionalities to merge. Do you want to call the python code right from Processing?
Here I have looked at your latest sketch where you placed the mini images into the hand image. I have made two changes. The first one is loading n number of images. Now, you can change n to reflect how many images you have in the folder and then it will load them for you if all those images are saved with the same file name formatting that you set initially.
The second change was introducing the variables STEPX and STEPY. When you read the mini-images, they are resize to these values. Then, when you are stepping in the main image, you load those mini images as pieces in a puzzle.
You need to keep in mind that you are using imageMode(CORNER) which is the default image mode in Processing. This means that your mini-image's top left corner is placed right at the x and y position. Notice that when you select the mini-image, you are judging the whole mini-selection based on a single pixel. You might reconsider this concept at a later time in your project.
In a nutshell, hre I have addressed:
It assigns an image to the white segment as the white segment also has color. If you do not want a mini-image in the white background, then you will to adapt your algorithm. For example, you could ignore any pixel color that has a white tone. First question though, what happens if you have another image with white background, but the interior of the image contain some feature painted in white? To be able to generate the right algorithm, it is important to understand the nature of the source images. Do you have a set of images that you want to apply this to? I am referring to images similar to your hand.jpg and images similar to the template image where all the mini-images are extracted.
Kf
Hey!
So in sketch 1 the python code is already being called from within processing. I was hoping to somehow just run sketch 1, where sketch 1 puts the segments in its regular segments folder, and then sketch 2 would run automatically after that fact, to then extract those segments out of the segments folder to create the final image. So the user would only need to load in the image that they wanted segmented and the image they want the segments applied to.
Right now im just dragging and dropping the segments created into sketch 2's data folder, and then running sketch 2.
Oh I see about white being a colour, that makes sense.
Hmm, ok. The original idea was from http://www.generative-gestaltung.de/P_4_3_2_01
where I guess here the text assigned to the image is measured in shades?
I'd like the output image to be black and white regardless, so if I loaded a black and white hand.jpg in, how would I go about having the segments just applied to where the greyscale area is, so white wouldnt even count in the scenario?
I've put in your code (thanks so much, this is perfect) and im getting the error:
IndexOutofBoundsException: Index: 3, Size:3
As for a set of images I wish to apply the segments to, it doesnt really matter...I'm curious to see it done on any image.
I have to say my code was not tested as you see. Can you tell me what line does the error point to? I think you need to provide me the actual line in your code as line numbers will not be of much help, unless you share your current code, for example using pastebin.
I didn't study the python code bc I am not into python and I cannot set it up at the moment. However, here is an idea. You don't need to move the files generated in sketch1 to a folder in sketch2. What you could do is from your sketch2, to load the files directly from the data folder in sketch one. To do this, you need to provide the full path to access those files in your sketch2. For instance, usually you do this:
PImage img=loadImage("photo.png");
Instead do this:
A better version is to have the files generated by your python script stored in the sketch2's data folder directly. A third option is to save the files in a "middle-man" folder by sketch1 and then sketch2 will read the files from this folder. A middle-man folder means a folder that it is not inside any of the sketch folders but they reside in a temporal folder (TMP dir).
Related to the white background and assigning images only to shaded areas. It is possible. Regard the following line in my previous post:
float imgPicked=map(pixelBrightness,0,255, 0, n);
What this line says is that you project the pixel brightness into a scale that goes from 0 to n-1, inclusive. Notice the brightness can take values between 0 and 255. You need to change this. Instead you can run this example and see by yourself:
The value of d turns out to be 15.93. However, d is larger than my expected value of HIGH. This is how you can tell the color is white, by doing the following checking:
In your case, you only want to add mini-images for shaded regions, so you need to change the condition:
Kf
Sorry, for the index error it is highlighting this line on my end:
image(imgContainer.get((int)imgPicked), x, y);
Before I continue further with what you have kindly provided for me, ill see if I can fix this error first. :)
So I've managed to edit the Python code to specify the output of the segments now, so the segments are now able to go right into sketch 2's data folder, but its going in the data folder as its own folder: http://imgur.com/a/2FHdP
Would there be a way of specifying this folder location using PImage?
Still not figuring out the index error.
In the original "segmenting.py" Python script, it created a subfolder called "segments/" inside what is called the Current Working Directory (CWD):
https://en.Wikipedia.org/wiki/Working_directory
In order to force that subfolder to be created in my Python Command sketch's "data/", I had to prepend command
cd
before invokingpython
:https://en.Wikipedia.org/wiki/Cd_(command)
So, the full directory path where the segment images would go into when "segmenting.py" was run is:
"sketchPath/data/segments/"
Within Python Command sketch, this statement below determines which CWD is for "segmenting.py":
final String dp = dataPath("");
If you wish for some other folder path to be the CWD, simply assign that to variable dp as a String. :)>-
Hey thanks @GoToLoop, I managed to assign sketch 2's data folder as the final destination for the segments. Here is the updated segmenting.py: https://pastebin.com/60i5RMLp
But it goes in the data folder as its own seperate folder like so: http://imgur.com/a/2FHdP
So I need to change sketch 2 to pick this folder instead.
To make this probably easier for you and @kfrajer, i've done a screen-record of what everything looks like on my end at the moment.
I can't go to https://YouTube.com/edit?video_id=Gy2ghdOv-ec b/c I don't have an account there!
Please post it as a regular YouTube video! :-\"
Should be good now, put it to public:
""
L#76 ->
folder = r'C:\Users\yourname\Desktop\sketch2\data'
L#79 ->
filenm = os.path.basename(f)
L#83 ->
segments_folder = os.path.join(folder, filenm[:-4]+"_segments")
Those 3 statements above are what form the full directory path:
"C:\Users\yourname\Desktop\sketch2\data\test_segments\"
filenm[:-4]
grabs all of the characters from an image's basename, skipping its extension.Yup! So what im wondering now is, how can I edit sketch 2 now to pick test_segments from within the data folder? The thing is, each time I load a new image in sketch 1, it'll create a new test_segments folder for the segments created.
So im not entirely sure how to pick out the test_segments folder...being that the next test_segments folder will be called "test_segments2" then "test_segments3" and so on.
The idea I had was that if sketch 1 and sketch 2 were just combined into one sketch file, then you load the image to be segmented in on one end and then PImage would pick out the most current test_segments folder to work with, and then just assign the segments to the image you want.
Haha, so complicated.
The way your Python script is programmed, all subfolders gonna end w/ "_segments".
If you want diff. names, just use images w/ diff. names! /:)
Haha yeah...true. Right but if there is a new folder created each time a new image is loaded in sketch 1, im not sure how I would distinguish them in sketch 2's code. Even if the segments weren't in a folder, im still not sure how I would distinguish them. It would be nice to not have to delete whatever segments are put into the data folder of sketch 2 before loading in a new image in sketch 1. It would make most sense for the segments to be deleted after sketch 2 assigns them to the input image, so we could start fresh each time within the data folder, if that makes any sense
For the error, I need to confirm you are using these lines in your code:
For debugging, introduce this line between the above lines:
println("DEBUGGING check n="+n+" map returns "+imgPicked);
And also, can you tell how you are defining the value of n? Are you doing this manually? The value of n should be based on the number of images available. Also, make sure you are reading the right folder inside your data folder:
PImage pimg=loadImage("test_segments\\"+"test2.jpg_"+i+".png");
Kf
In the original "Python Command" sketch:
https://Forum.Processing.org/two/discussion/23324/executing-python-3-code-from-within-processing/p1#Item_38
I've relied on listPaths() in order to grab all image paths (.png & .jpg) from subfolder "data/images":
final String pics = join(listPaths(pd, PICS_EXTS), SPC);
Besides files, listPaths() can also grab folder paths if we pass a "directories" to it after the 1st root folder path argument.
Then we can remove() all subfolder paths which doesn't endsWith() "_segments".
After that, we can grab all image paths from each "_segments" subfolder, again w/ listPaths()! *-:)
And finally we can transfer it all to a global String[][] 2D array. B-)
I've made a set of util functions to do all that hard work below. #:-S
Just invoke getFoldersOfImagePaths() and assign it to a 2D array variable.
Use joinStrArr2d() in order to println() the whole 2D array too: :-bd
Alternatively, "Grab Image Seg Paths" in Python Mode too: :)>-
@GoToLoop
Woah.. that should work yeah!
Before I proceed with adding this can you clarify something for me:
Right now the segments created from sketch 1 are going into sketch 2's data folder as its own _segments folder. Changing the python command file here is fixing it before the segments are in sketch 2's data folder or after?
What would I need to change in sketch 2 then? Confused on this part ..
-Actually @kfrajer has answered this above.
@kfrajer
My n value is int n=3. Which probably doesnt make any sense. I added the print line and im getting: DEBUGGING check n=3 map returns 3.0
@daddydean, as explained, those util functions collect the paths of each image inside sketch's "data/" subfolders ending w/ "_segments" as a 2D String[][] array.
That is, after invoking getFoldersOfImagePaths(), the String[][] returned from that is filled w/ image paths, which are those created by the Python script.
So you won't need to guess the exact name for each "_segments" subfolder, for they're found automatically if they're inside sketch's "data/". \m/
You just need to add those util functions to your posted sketch here, declare a global String[][] variable, and call getFoldersOfImagePaths(), assigning it to that variable in setup(): :-B
Most probably you're gonna need to declare a 2D PImage[][] array too, in order to store each image loaded w/ loadImage() w/ the paths of the 2D String[][] array. :>
@GoToLoop
I see. I think what im getting confused is, what I would put in sketch 2's code then? When you refer to sketch's data folder, you mean sketch 2 obviously right?
If I am putting your grab-image-seg-paths code in, im putting this directly in the python command code right? Is it from line 28 in the python command onwards that I would delete?
Sorry im having such a hard time following.
,
is also needed after the assign=
character: "extensions=,png,jpg,jpeg" 8-}Yup! Although my set of util functions would work for any sketch which contains image subfolders w/ their names ending w/ "_segments" inside its subfolder "data/". :P
Oh good haha, I thought for some reason you meant sketch 1's. #-o
So sketch 2 will need to be changed then as well to accomodate this, I was asking kfrajer about it...but it'd have to be something different now right?
Is that what you mean by the 2d pimage array?
So am I adding v1.0.1 after line 28 in the python command code?
Or am I way off here
You need to grab the 2D String array w/ getPathsFromFolder() before using loadImage() on them all.
You can also paste all of my util functions in a separate tab so it won't distract you in the main tab! *-:)
Ah, ok. So have python command, command java, and a seperate tab with v1.0.1?
I have to delete after line 28 in python command though right?
Ahh my head. Maybe a visual screenshot of what youre looking at might help me at this point, Im a bit lost. I need to reread everything here.
Right...cool. So all of this is in what you've written in v1.0.1 right? Im just unclear on where I need to put it within sketch 1. So it should output a string of all the paths, which I will use in sketch 2's PImage?
You place those set of functions in the sketch which needs to collect the segment image paths.
So it would make most sense to be in sketch 2 then I would have to assume, being that the segments are put in the sketch 2 data folder. Im just not sure where in the code I would put it, or am I supposed to run it separately in a tab?
If I run it on its own it comes up with 'segment folders found: 0'
It means there aren't any subfolder names ending w/ "_segments" inside subfolder "data/" of the running sketch.
OH haha. So if I put it in sketch 2 just by itself it works and recognizes the folder and all 98 pngs in it. So at this point, im not sure how I combine v1.0.1 with sketch 2, being that sketch 2 is using the PImage that kfrajer suggested. Or is that what you were saying, that I need to change my Pimage and loadimage to something else to accomodate v1.0.1?
How do I work with loadimage to load in the paths?
Right now I have in sketch 2's load image:
C:\Users\username\Desktop\sketch2\data\segments\test_segments\"+"test_"+i+".png");
Right, so I can run the util functions on their own to find the segments paths within data/, but I dont know how I would change the loadimage in sketch 2 to accept those paths. ? Right now I have your v101 in a separate tab in sketch 2.
It would be cool to have this all packaged up to work after for people that are interested in image segmentation :)
Can you post your current attempt? Recall that getFoldersOfImagePaths() returns a 2D array. L-)
Tip: GitHub is an excellent place to post source code w/ multiple folders and data: *-:)
https://GitHub.com
I dont even really have an attempt, I can locate the segment paths created... (99 of them) Im not sure how I'd make a 2D Pimage though
Right now Im assuming all the paths created would go into the loadimage
I found http://processingjs.org/learning/basic/array2d/
But im not sure how this will help.
But you're gonna have to store all of the created loadImage()'s PImage objects somewhere, right?
In the least, you're gonna need a 1D PImage[] array to store them all. 8-X
Ohhhh, I see. Are all the images being deleted after, and thats why they need to be in an array?
If your util functions outputs the segment paths, where is it outputting them to exactly, or how do I call it in my Pimage? Right now it looks like this obviously; http://imgur.com/a/dnoMP
There is https://www.openprocessing.org/sketch/32892
Would something like this work?
For all garbage-collect-based languages, like Java, JS, Python, etc., if no variable holds the reference of some object, that object is marked for deletion! :-h
So after using loadImage(), its resultant PImage needs to be immediately assigned somewhere, or else that "newborn" goes straight to the "trash bin", as if it had never existed. =((
Arrays got indexed
[]
slots which act much like variables:https://Processing.org/reference/arrayaccess.html
But the array container itself is assigned to 1 variable.
It's as if its whole indexed elements shared the same variable name. \m/
Gotcha about the garbage. Im not sure if I follow what you mean by "resultant PImage". Does it matter where its assigned? Im not sure where to begin here at all to be honest.
Would it be easier to have a function where the PImage picks the most recently created folder within data/? If there is a function for that..
PImage is the datatype of the resultant object from calling Processing's createImage() and loadImage() functions: :-B
There are others ways to create PImage objects too, such as:
Directly using operator
new
over class PImage:PImage img = new PImage();
:https://Processing.org/reference/new.html
Or invoking PImage's get() method w/ no arguments:
PImage clone = img.get();
https://Processing.org/reference/PImage_get_.html
Right, but what am I putting after Pimage get or PImage new? How do I get the paths created from the util functions?
Alternatively im wondering if it might be easier to just pick the most recent folder created within sketch 2's data/ folder, no matter what the folder is named, if you know what i mean?
I'm wondering, how much do you know about Java, Python or any programming in general? :-/
It seems even when I try to explain some subject in deep, you just think about where to copy and paste it rather than trying to understand the concept. :-@
Gonna try to come up w/ some other example here...
I only know a tiny bit, brand new this month. I get the basic functions of things but its difficult for me to put them all together. I get what the util code you have provided does, but theres no way I'd ever be able to write something like that myself at the moment. Im unsure how to route the paths of the image segments into PImage correctly, therefore I was thinking itd be likely easier to just pick the most recent folder created in data/ and keep whatever folders build up in there and remove them myself. I really really appreciate all of your help, and it all works on my end...was just hoping to automate it a bit to make it less time consuming when loading and unloading images. Its not essential, but it could help.
@GoToLoop, I think it would be easier to actually combine sketch 1 and sketch 2 into one sketch file. That way you could just have one data folder where you load the images, and where the folders are created. I tried combining them myself but im getting a "duplicate method setup error" which Im assuming means that I can't have a "void setup" in sketch 1 as well as a "void setup" in sketch 2.
I figured out how to scale back the segments in sketch 2, so I am getting a proper image now: http://imgur.com/a/JCBmQ but the segments are still being assigned to the white portion of my hand picture. This is the hand image mentioned at the very top.
Do you mind pointing me in the right direction on how to combine sketch files?
I found https://www.gicentre.net/utils/multiwindow
Checking it out.