How to make an Android Processing sketch fragment semi-transparent

kelleyvanevert

Hi all!

I'm happily using Processing from the Android IDE, allowing me to add certain critical functionality through Android Java directly. For the project I'm now working on, I want to overlay all kinds of nice Processing sketches, in a "virtual reality" way, over a camera view. Because displaying the camera view and doing the required detection is a bit heavy, I'll be doing this with OpenCV instead of Processing. My idea was to then simply overlay the Processing sketch fragment over the OpenCV fragment, thus allowing the cool 3D stuff to be easily programmed in Processing, while the more intensive camera stuff is done with OpenCV.

However, although I've managed to make simple Android fragments semi-transparent, I can't seem to be able to make the Processing sketch fragment semi-transparent. Maybe this is because it's neccessary to go all the way down to the SurfaceView involved and make it semi-transparent or something?

I'd be very much obliged if someone could help me out :)

The code is generally as follows:

The main layout file:

<RelativeLayout android:layout_width="fill_parent" android:layout_height="fill_parent" ... > <FrameLayout android:id="@+id/bg_frame" android:layout_width="fill_parent" android:layout_height="fill_parent" android:background="#cc0000" ... > <FrameLayout android:id="@+id/fg_frame" android:layout_width="fill_parent" android:layout_height="fill_parent" ... > </RelativeLayout>

The Processing sketch:

public class MySketch extends PApplet { ... public void draw() { background(color(255, 50)); // semi-transparent ... } ... }

The activity:

public class MainActivity extends Activity { ... protected void onCreate(Bundle savedInstanceState) { ... getFragmentManager().beginTransaction() .add(R.id.fg_frame, new MySketch()) .commit(); ... } ... }

I have tried (as you can see above) using Processing's background color background(color(255, 50)).

Also, I've tried setting android:background="@android:color/transparent" as well as android:alpha="0.5f" in the fb_frame frame layout.

kfrajer

How do you draw your OpenCV data? I will think the way to do it is to pass the info to the PApplet for it to handle it. In a way, you can draw your openCV data first and then draw the data from Processing on top of it using a PGraphics object and after adjusting the alpha value. If you want both to have semi-transparent values, then this could be handle in processing as well only if Processing can get a handle on your data. Maybe maybe there is a way this could be done without Processing. For example, you paint your canvas using Android native methods. The Processing it is used only to output image data that your application can use and manipulate. However, you are losing lots of the power Processing provides you if you do it this way.

For the overlay effect that you are looking for, can you say a bit more about it?

Can you also tell more about your detection business? I agree doing image processing on Android mode in Processing lags sometimes....

You can search PGraphics in the forum and this will give you ideas of what you could do. You could get also relevant results if you search for transparent.

Kf

kelleyvanevert

Hey kfajer;

Thanks for your quick answer!

As of now I haven't actually started doing anything serious with OpenCV yet, first I wanted to see whether the basics fit together.

As I had it planned, there would only be very minor communication between the OpenCV part and the Processing part: the OpenCV part would only send the results of it's detection, i.e. some coordinate data, to the Processing part. So:

• OpenCV part renders the camera output (possibly with a nice filter to make it more beautiful) and calculates the object position of the camera.

--> sends the output of calculation to the Processing part

• Processing part renders whatever I choose to project onto the image "as if actually there in the real world".

You suggest having OpenCV send the image data itself to Processing and have Processing figure the overlay out. The reason I don't want to do this is because, when prototyping this whole business in Processing only, I got in to too much efficiency problems.

Below are two screenshots of the detection I had been prototyping in Processing, yesterday. Basically, I just want to detect the square of paper. For the prototyping however, I quickly settled for the easier solution of just detecting four green dots that I would draw in the corners. To get Processing to handle this somewhat effectively, I had to cut the read- and calculation- framerate to 15 fps, and the camera input size to 340 x 240, but still Processing had a hard time dealing with the code that I was adding. On the other hand, some OpenCV examples I ran were very very fast and detailed, doing all kinds of filters. In addition, OpenCV seems to have a function exactly for the calculation I want to do: solvePnP.

kfrajer

You suggest having OpenCV send the image data itself to Processing and have Processing figure the overlay out

I thought that was the reason you wanted semi-transparent graphics.

In either case, you should try running examples of the OpenCV library in Processing in Android mode. I think they work and there are some algorithms available for blob detection.

Also you should try using the P2D renderer. Some other posts have described improve in performance but I am not sure if this is valid in Android mode... I would say yes but don't put my name on it.

There are also some fiducial libraries. It is worth to try them to see if it would work in Android mode: https://forum.processing.org/two/search?Search=fiducial

Kf

kelleyvanevert

My plan was to have the entire fragment (or possibly the underlying surfaceview or whatever) be semi-transparent, so to have the Android UI then handle it, whereas your suggestion is to do it inside Processing, using a second PGraphics.

I'm new to Processing, and Processing on Android, so I'm not entirely sure how or when things slow down -- but had the feeling that just the business of piping any large image data through Processing was slowing down the app considerably, compared to having OpenCV draw it to another surfaceview on its own accord (the exact manner of which I haven't looked into). Hence the two fragments plan, instead of having everything piped through Processing.

But you're right, maybe I should look into that too. Using a second overlayed PGraphics object would then indeed be the way to go.

And thanks for the pointer to the fiducial libraries. I'd not seen those before (very new to this), but they seem to be spot on.

Also, I hadn't thought of using P2D, but if it is indeed faster on Android, that'd be a possible solution, having the image processing in the sketch with P2D, and the overlayed PGraphics with the "VR" being rendered with P3D. That is what you mean, right?