Issue with SkeletonFilter and PImage's resize

Programming Questions

orgicus

Issue with SkeletonFilter and PImage's resize

in Programming Questions • 1 year ago

Hi,

I'm trying to do binary image skeletonization and so far have used Jerry Huxtable's class (with a couple of PImage ammends). I'm not getting great framerates when running the filter multiple times in the draw loop, so thought about resizing the image to be processed, filter than, then scale the result back. The problem is skeletonization doesn't work after resizing. I've tried with other images and using PGraphics and have the same behaviour: skeletonization works before resizing, but not after. I'm not sure if the problem is the resize() function, the skeletonization code (probably), or both.

Here's my test code so far:

import java.awt.Rectangle;
import java.awt.image.BufferedImage;
PImage source,result;
SkeletonFilter f;
Rectangle r;
int s = 2;
float invs = 1.0/s;//inverse scale
void setup(){
size(640,480);
source = loadImage("pose9.png");
f = new SkeletonFilter();
}
void draw(){
r = getBounds(source,color(255));
if(r != null) {
int b = -10;
result = source.get(r.x-b,r.y-b,r.width+b*2,r.height+b*2);
// result.resize((int)(result.width * invs),(int)(result.height * invs));
image(result,0,0);
result = skeletonize(result);
image(result,r.x,r.y);
}
}
PImage skeletonize(PImage source){
PImage result = source.get();
int previousCount = source.pixels.length+1;
int diff = source.pixels.length;
while(diff > 0){
result = f.filter(result,1);
int currentCount = pixelCount(result,0xFFFFFFFF);
diff = abs(previousCount-currentCount);
previousCount = currentCount;
}
return result;
}
Rectangle getBounds(PImage source,int chroma){
int minX = width,minY = height,maxX = 0,maxY = 0;
int numPixels = source.pixels.length,w = source.width,h = source.height;
for(int i = 0 ; i < numPixels; i++){
int x = i%w;
int y = i/w;
int c = source.pixels[i];
if(c != chroma){
if(x < minX) minX = x;
if(x > maxX) maxX = x;
if(y < minY) minY = y;
if(y > maxY) maxY = y;
}
}
return new Rectangle(minX,minY,maxX-minX,maxY-minY);
}
int pixelCount(PImage src,int clr){
int result = 0;
int numPixels = src.pixels.length;
for(int i = 0 ; i < numPixels; i++) if(src.pixels[i] == clr) result++;
return result;
}
import java.awt.*;
import java.awt.image.*;
// Jerry Huxtable's SkeletonFilter http://www.jhlabs.com/ip/filters/ modified for PImage (rather than casting to BufferedImage back and forth)
// Based on an algorithm by Zhang and Suen (CACM, March 1984, 236-239).
public class SkeletonFilter {
int newColor = 0xffffffff;
int colormap;
private final byte[] skeletonTable = {
0, 0, 0, 1, 0, 0, 1, 3, 0, 0, 3, 1, 1, 0, 1, 3,
0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 2, 0, 3, 0, 3, 3,
0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 3, 0, 2, 2,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 3, 0, 2, 0,
0, 1, 3, 1, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1,
3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 3, 1, 3, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 3, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0,
3, 3, 0, 1, 0, 0, 0, 0, 2, 2, 0, 0, 2, 0, 0, 0
};
public SkeletonFilter() {
newColor = 0xffffffff;
}
PImage filter(PImage source,int iterations) {
PImage out = createImage(source.width,source.height,RGB);
try {
int width = source.width;
int height = source.height;
int[] inPixels = source.pixels;
int[] outPixels = new int[width * height];
int count = 0;
int black = 0xff000000;
int white = 0xffffffff;
for (int i = 0; i < iterations; i++) {
count = 0;
for (int pass = 0; pass < 2; pass++) {
for (int y = 1; y < height-1; y++) {
int offset = y*width+1;
for (int x = 1; x < width-1; x++) {
int pixel = inPixels[offset];
if (pixel == black) {
int tableIndex = 0;
if (inPixels[offset-width-1] == black)
tableIndex |= 1;
if (inPixels[offset-width] == black)
tableIndex |= 2;
if (inPixels[offset-width+1] == black)
tableIndex |= 4;
if (inPixels[offset+1] == black)
tableIndex |= 8;
if (inPixels[offset+width+1] == black)
tableIndex |= 16;
if (inPixels[offset+width] == black)
tableIndex |= 32;
if (inPixels[offset+width-1] == black)
tableIndex |= 64;
if (inPixels[offset-1] == black)
tableIndex |= 128;
int code = skeletonTable[tableIndex];
if (pass == 1) {
if (code == 2 || code == 3) {
pixel = newColor;
count++;
}
}
else {
if (code == 1 || code == 3) {
pixel = newColor;
count++;
}
}
}
outPixels[offset++] = pixel;
}
}
if (pass == 0) {
inPixels = outPixels;
outPixels = new int[width * height];
}
}
if (count == 0)
break;
}
out.pixels = outPixels;
}
catch (Exception e) {
e.printStackTrace();
}
return out;
}
}

and here's the image I used

Any hints/tips would help !

Thanks,

George

Replies(3)

amnon.owed

Re: Issue with SkeletonFilter and PImage's resize

1 year ago

I'm not exactly sure what's going on, but somehow Processing's resize function seems to break one of your functions that comes after that. I suspect this might be somehow related to the pixel's color values.

I borrowed some code that I wrote to help somebody else, which is a sort of custom resize function. Using this manual method of resizing, your functions do not break. Press the mouseButton to see the actual image sizes.

Perhaps this clue will help solve the mystery...

Adapted Code

import java.awt.Rectangle;
import java.awt.image.BufferedImage;
PImage source, result;
SkeletonFilter f;
Rectangle r;
int s = 2;
float invs = 1.0/s;//inverse scale
void setup() {
size(640, 480);
source = loadImage("pose9.png");
f = new SkeletonFilter();
}
void draw() {
background(200);
r = getBounds(source, color(255));
if (r != null) {
int b = -10;
int w = r.width+b*2;
int h = r.height+b*2;
result = source.get(r.x-b, r.y-b, w, h);
int factor = max(1, 1 + mouseX / 40);
result = resizeImage(result, factor);
if (!mousePressed) { image(result, 30, 10, w, h); } else { image(result, 30, 10); }
result = skeletonize(result);
if (!mousePressed) { image(result, 118, 10, w, h); } else { image(result, 118, 10); }
}
}
PImage resizeImage(PImage in, int factor) {
PImage out = createImage(in.width/factor, in.height/factor, RGB);
for (int y=0; y<out.height; y++) {
for (int x=0; x<out.width; x++) {
PImage segment = in.get(x*factor, y*factor, factor, factor);
color c = averageColor(segment);
int index = x + y * out.width;
out.pixels[index] = c;
}
}
return out;
}
color averageColor(PImage in) {
int r = 0, g = 0, b = 0;
for (int i=0; i<in.pixels.length; i++) {
color c = in.pixels[i];
int rPixel = (c >> 16)&0xff;
int gPixel = (c >> 8)&0xff;
int bPixel = (c)&0xff;
r += rPixel;
g += gPixel;
b += bPixel;
}
r /= in.pixels.length;
g /= in.pixels.length;
b /= in.pixels.length;
return color(r, g, b);
}
PImage skeletonize(PImage source) {
PImage result = source.get();
int previousCount = source.pixels.length+1;
int diff = source.pixels.length;
while (diff > 0) {
result = f.filter(result, 1);
int currentCount = pixelCount(result, 0xFFFFFFFF);
diff = abs(previousCount-currentCount);
previousCount = currentCount;
}
return result;
}
Rectangle getBounds(PImage source, int chroma) {
int minX = width, minY = height, maxX = 0, maxY = 0;
int numPixels = source.pixels.length, w = source.width, h = source.height;
for (int i = 0 ; i < numPixels; i++) {
int x = i%w;
int y = i/w;
int c = source.pixels[i];
if (c != chroma) {
if (x < minX) minX = x;
if (x > maxX) maxX = x;
if (y < minY) minY = y;
if (y > maxY) maxY = y;
}
}
return new Rectangle(minX, minY, maxX-minX, maxY-minY);
}
int pixelCount(PImage src, int clr) {
int result = 0;
int numPixels = src.pixels.length;
for (int i = 0 ; i < numPixels; i++) if (src.pixels[i] == clr) result++;
return result;
}
import java.awt.*;
import java.awt.image.*;
// Jerry Huxtable's SkeletonFilter http://www.jhlabs.com/ip/filters/ modified for PImage (rather than casting to BufferedImage back and forth)
// Based on an algorithm by Zhang and Suen (CACM, March 1984, 236-239).
public class SkeletonFilter {
int newColor = 0xffffffff;
int colormap;
private final byte[] skeletonTable = {
0, 0, 0, 1, 0, 0, 1, 3, 0, 0, 3, 1, 1, 0, 1, 3,
0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 2, 0, 3, 0, 3, 3,
0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 3, 0, 2, 2,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0,
3, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 3, 0, 2, 0,
0, 1, 3, 1, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1,
3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 3, 1, 3, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 3, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0,
3, 3, 0, 1, 0, 0, 0, 0, 2, 2, 0, 0, 2, 0, 0, 0
};
public SkeletonFilter() {
newColor = 0xffffffff;
}
PImage filter(PImage source, int iterations) {
PImage out = createImage(source.width, source.height, RGB);
try {
int width = source.width;
int height = source.height;
int[] inPixels = source.pixels;
int[] outPixels = new int[width * height];
int count = 0;
int black = 0xff000000;
int white = 0xffffffff;
for (int i = 0; i < iterations; i++) {
count = 0;
for (int pass = 0; pass < 2; pass++) {
for (int y = 1; y < height-1; y++) {
int offset = y*width+1;
for (int x = 1; x < width-1; x++) {
int pixel = inPixels[offset];
if (pixel == black) {
int tableIndex = 0;
if (inPixels[offset-width-1] == black)
tableIndex |= 1;
if (inPixels[offset-width] == black)
tableIndex |= 2;
if (inPixels[offset-width+1] == black)
tableIndex |= 4;
if (inPixels[offset+1] == black)
tableIndex |= 8;
if (inPixels[offset+width+1] == black)
tableIndex |= 16;
if (inPixels[offset+width] == black)
tableIndex |= 32;
if (inPixels[offset+width-1] == black)
tableIndex |= 64;
if (inPixels[offset-1] == black)
tableIndex |= 128;
int code = skeletonTable[tableIndex];
if (pass == 1) {
if (code == 2 || code == 3) {
pixel = newColor;
count++;
}
}
else {
if (code == 1 || code == 3) {
pixel = newColor;
count++;
}
}
}
outPixels[offset++] = pixel;
}
}
if (pass == 0) {
inPixels = outPixels;
outPixels = new int[width * height];
}
}
if (count == 0)
break;
}
out.pixels = outPixels;
}
catch (Exception e) {
e.printStackTrace();
}
return out;
}
}

orgicus

Re: Issue with SkeletonFilter and PImage's resize

1 year ago

Thank you very much! That is very helpful.

I think you're right about pixel values...I've done this basic test:

color c1 = color(255);//prints: -1
color c2 = 0xFFFFFF;//prints: 16777215
color c3 = 0xFFFFFFFF;//prints: -1
println(c1);
println(c2);
println(c3);
c1 = color(0);//prints: -16777216
c2 = 0x000000;//prints: 0
c3 = 0x00000000;//prints: 0
println(c1);
println(c2);
println(c3);
println(0xFFFFFFFF + " / " + color(0xFFFFFFFF));//prints: -1 / -1
println(0x00000000 + " / " + color(0x00000000));//prints: 0 / -16777216
println(0xFFFF0000 + " / " + color(0xFFFF0000));//prints: -65536 / -65536

Seems there's an issue with black/white values...not sure why...the color type puzzles me.

amnon.owed

Re: Issue with SkeletonFilter and PImage's resize

1 year ago

The color type in Processing is actually a 32bit ARGB int. Read this.

I guess then the above test results will become understandable. The 'unexpected' values are often due to the alpha. For example on line 8, color(0) is actually color(0,0,0,255) which equals 0xFF000000. So that's fully opaque black, while line 9 and 10 are fully transparent black. Similarly on the second value of line 16 you set the color to the int value 0, which is basically line 8. So again it's fully opaque black that you are comparing to fully transparent black. Hope that clears it up.

Top Reply