Camera / video image slow

mcfetrmatt

Hi there, I am looking into accessing the camera built into my laptop and have that feed back into Processing. I have downloaded the Video library and copied the code from the Capture Class and when I run it I get 90 cameras listed and the processing window (the camera video feed) runs really slow, almost one frame a second. The image also has a green tint and I can only access the front camera. I have a i7 model of the new Surface Pro 4.

Any feedback appreciated,

mcfetrmatt

Am I going about this the wrong way? I thought it would be a simple as selecting the front or rear facing camera, not all the sizes and framerates of each.

mcfetrmatt

Could it have something to do with the camera being a RealSense Camera? It's dealing with depth data as well as RGB which may explain the slow speed. Is there a way to specify the RGB camera? Or even the rear facing camera, it looks as though it's only detecting the front facing camera as all the names are the same.

Ater

A little strange. Does it work using any element from list, when you create actual Capture object?

mcfetrmatt

Hi Aler, sorry what do you mean? Do you mean selecting camera 0 or 1 or 2 etc. from the list? If so I've tried most numbers from 0 to 90 (cause they show different resolutions and framerates in the comments part) and none seem to make any discernible difference.

mcfetrmatt

As an interesting side note I hooked up my Kinect 2 sensor and had no issues what so ever (colour correct, ran at 60 FPS fine etc), it's just the inbuilt cameras which are giving me grief..

Might there be some troubleshooting I could try?

mcfetrmatt

I've just tried running one of the example files and I am still getting a really laggy video feed - Examples > Libraries > Video (capture) > GettingStartedCapture

Is there a way to troubleshoot this? Or has something changed in Processing 3? Any ideas?

Cheers,

koogs

What resolution is the camera?

What resolution is the image?

I'm wondering whether the resize is the problem... Take out the image call, perhaps, and add something that'll print the frame rate instead to get the raw camera speed.

mcfetrmatt

When I take out the image call the window runs at 60fps so maybe it is in the resize? This is the code I used:

`import processing.video.*;

Capture cam;

void setup() { size(1280, 720); String[] cameras = Capture.list();

if (cameras.length == 0) { exit(); } else { cam = new Capture(this, cameras[0]); cam.start(); } }

void draw() { background(255); if (cam.available() == true) { cam.read(); }

//image(cam, 0, 0, width, height);

fill(0); textSize(18); text("FRAMERATE: " + int(frameRate), 50, 50); }`

koogs

What resolution is the camera?

mcfetrmatt

It ranges from 640x360 to 1920x1080, all have the same effect on the speed (see image at top of post for 'available cameras'). Interestingly the actual frame rate Processing puts out differs from the refresh rate of the image.. So in the image below you see the frame rate at 60 but the image still only updates once per second (even if I control it with frameRate()). I wonder if there is a hardware / driver issue going on then? Thoughts please :) - as you can see there is no re-sizing going on either:

Daan

Dear mcfetrmatt, Im having the exact same issue with my Surface Pro 4. Did you manage to get this issue resolved in the meantime? Microsoft could not help me with this issue.. Kind regards, Daan

mcfetrmatt

Hi Daan, no I haven't.. I 'see' so many camera names but they all work the same, nothing changes. It's so weird that the fps viewer will show 60fps but the refresh rate is clearly slower.

When I plug an external camera in I have no issues, I wonder if we're missing a driver. Did Microsoft give you any clues?

Daan

Hi macgertmatt, I see the same list all with the same green thingie. With Microsoft i disabled the front IR cam and the Windows Hello driver but with no result. Disabling the front cam resulted in the same green filter on the back cam.. They said i had to reinstall processing via the Appstore to have a supoerted version, but processing does not exist in the store.. Im affraid the integrated cam just does not work for Surface owners?!

mcfetrmatt

I hope that's not the case. I couldn't get the rear camera to come through so you got further than me! I couldn't find the id name of the surface camera when I was looking to see if I could get any information on it. I wondered if the depth sensor was throwing Processing out too..

Daan

If you want to try; i disabled the fron cam, ir and hello cams in the device manager under system devices. Hope someone will come with an workaround.

bgoosman

mcfetrmatt, Daan, did you ever find a solution? No matter which camera I try on my Surface Book, I get a choppy feed.

CharlesDesign

^ same issue for me on my Surface Book

kfrajer

Do you see improvements using a lower res camera when you init the capture object? Just curious...

Kf

gersart

I am having this trouble as well. I tried some suggestions on another post on a microsoft forum ( https://answers.microsoft.com/en-us/surface/forum/surfbook-surfmusic/surface-book-cameras-exhibiting-green-tint-and/a26bab7a-9917-4f2a-8c1f-d2a6b374161d ) but I cannot find out how to disable hardware acceleration. I 'did' find some preferences one could set by right-clicking on the processing icon, but none of them seemed to do anything. These included scaling, etc. They mentioned turning off hardware acceleration for the display but I was unable to find that in the current windows version.

It would be nice if this laggie camera problem was remedied. I wanted to test openCV but it is useless to do so without carrying around a web cam.

A side note: I drew the framerate in the getting-started-with-video example, and the framerate did not drop below 57fps even with the stuttering, so I think it might be a driver issue in getting images from the camera. Strangely this is not a problem when using openframeworks, so it might be a java thing. I am unsure if it is a microsoft, processing or video library problem.

another side note: in the link I posted there is a suggestion for changing register values which I did NOT do as I don't want to screw up a system that works. If someone more knowledgeable wants to experiment, be my guest. I checked for the values specified but I was unable to find them, so I did not dig any deeper into that rabbit hole. It would be nicer to have some official response to this problem and an official solution or work around...

micuat

I found this article using DirectShow on Processing: http://www.magicandlove.com/blog/2012/04/10/directshow-for-processing/

As in the article you need to copy dsj.dll and dsj.jar in sketchfolder/code (if you're using 64bit processing, use dsj.dll in 64bit folder). I needed to change the line in draw function to fit the texture in the window:

image(img, 0, 0, width, height);

also in DCapture you can change dsi[0][0] to dsi[0][1] for switching to front camera. I would have made a plugin if dsj was open source (it is free only for non commercial purpose).