Combining elements from video and webcam

edited April 2018 in Library Questions

Would it be possible to add elements from prerecorded video on certain fields over live web footage for example overlaying all moving objects with prerecorded footage? and what would be the best way to aproach this problem?

Answers

  • Do you have the live streaming part already? I ask for my own curiosity.

    Check this: https://github.com/processing/p5.js/wiki/Positioning-your-canvas

    Kf

  • edited April 2018 Answer ✓

    I have been experimenting with video effects on live and prerecorded video.

    The sketch below does not completely answer your question, but it uses a live video feed from my laptop's webcam (Mac Book Pro), processes each frame and then renders it on the screen.

    The biggest downside is that the frame capture rate from the live video feed is not very high.

    BTW: Left-click the mouse to switch between the video effects. ;)

    // Video Camera Filter with Modes by Niels Johnson-Laird
    // March 20th, 2018
    //
    // Mode 0 = Pixel Brightness to circle/dot (B&W TV Effect)
    // Mode 1 = Pixel Brightness to color circle/dot (B&W TV Effect)
    // Mode 2 = Semi-Mosiac effect
    // Mode 3 = Color TV with Scanlines Effect
    //
    // Creative Commons Attribution ShareAlike [https://creativecommons.org/licenses/by-sa/3.0]
    //
    // See: https://processing.org/reference/libraries/video/index.html
    
    import processing.video.*;
    
    final int rasterScalingFactor = 2;
    final int screenScalingFactor = 2;
    final int rasterPatternSize = (4 * rasterScalingFactor) - 1;
    final int ellipseDiameter = 2;
    final int rectSize = 12;
    final int mosaicSize = 4;
    
    int modeSelect = 0;
    Capture cam;
    PImage img;
    
    void settings() {
      size(640 * screenScalingFactor, 360 * screenScalingFactor);
    }
    
    void setup() {
      String[] cameras = Capture.list();
    
      if (cameras.length == 0) {
        println("There are no cameras available for capture.");
        exit();
      } else {
        println("Available cameras:");
        for (int i = 0; i < cameras.length; i++) {
          println("#" + i + " " + cameras[i]);
        }
    
        // The camera can be initialized directly using an element from the array returned by list():
        // Available cameras on MacBook Pro (Retina, 13-inch, Early 2015):
        //   #0 name=FaceTime HD Camera,size=1280x720,fps=30
        //   #1 name=FaceTime HD Camera,size=1280x720,fps=15
        //   #2 name=FaceTime HD Camera,size=1280x720,fps=1
        //   #3 name=FaceTime HD Camera,size=640x360,fps=30
        //   #4 name=FaceTime HD Camera,size=640x360,fps=15
        //   #5 name=FaceTime HD Camera,size=640x360,fps=1
        //   #6 name=FaceTime HD Camera,size=320x180,fps=30
        //   #7 name=FaceTime HD Camera,size=320x180,fps=15
        //   #8 name=FaceTime HD Camera,size=320x180,fps=1
        //   #9 name=FaceTime HD Camera,size=160x90,fps=30
        //   #10 name=FaceTime HD Camera,size=160x90,fps=15
        //   #11 name=FaceTime HD Camera,size=160x90,fps=1
        //   #12 name=FaceTime HD Camera,size=80x45,fps=30
        //   #13 name=FaceTime HD Camera,size=80x45,fps=15
        //   #14 name=FaceTime HD Camera,size=80x45,fps=1
        cam = new Capture(this, cameras[4]);
        cam.start();
      }
    
      textSize(36);
    } 
    
    void draw() {
      int c = 0;
      int xyOffset = 0;
    
      // Process a video frame when it is available?
      if (cam.available() == true) { 
        cam.read();
      } else { 
        return; // Nothing to process! :-(
      }
    
      PImage p = createImage(cam.width, cam.height, RGB);
      p.copy(cam, 0, 0, cam.width, cam.height, 0, 0, cam.width, cam.height);
      img = p;
    
      img.loadPixels();
    
      switch (modeSelect)
      {
      case 0: // Old Black and White TV
        background(180);
        xyOffset = ellipseDiameter / 2;
    
        for (int y = 0; y < img.height; y += ellipseDiameter) {
          for (int x = 0; x < img.width; x += ellipseDiameter) {
            c = int( brightness( img.pixels[(y * img.width + x)] ) );
            fill(c);
            ellipse(x * screenScalingFactor + xyOffset, y * screenScalingFactor + xyOffset, 
              ellipseDiameter * screenScalingFactor, ellipseDiameter * screenScalingFactor);
          } // end for - x
        } // end for - y
    
        fill(0, 0, 0);
        text("Old Black and White TV Effect", 10, 10, width, height);
        break;
    
      case 1: // Old Color TV
        background(180);
        xyOffset = ellipseDiameter / 2;
    
        for (int y = 0; y < img.height; y += ellipseDiameter) {
          for (int x = 0; x < img.width; x += ellipseDiameter) {
            c = int( img.pixels[(y * img.width + x)] );
            fill(c);
            ellipse(x * screenScalingFactor + xyOffset, y * screenScalingFactor + xyOffset, 
              ellipseDiameter * screenScalingFactor, ellipseDiameter * screenScalingFactor);
          } // end for - x
        } // end for - y
    
        fill(0, 0, 0);
        text("Old Color TV Effect", 10, 10, width, height);
        break;
    
      case 2: // Semi-Mosiac effect
        int r;
    
        background(180);
    
        for (int y = 0; y < img.height; y += mosaicSize) {
          for (int x = 0; x < img.width; x += mosaicSize) {
            c = int( img.pixels[(y * img.width + x)] );
            fill(c, 127); // fill(rgb, alpha) - alpha = opacity [0 to 255]
            r = int(red(c)) / 6;
    
            if (r < 5) { 
              r = r + int(random(mosaicSize, mosaicSize * 20));
            }
            noStroke();
            //ellipse(x * screenScalingFactor + xyOffset, y * screenScalingFactor + xyOffset, r, r);
            rect(x * screenScalingFactor + xyOffset, y * screenScalingFactor + xyOffset, r, r);
          }
        }
    
        fill(0, 0, 0);
        text("Semi-Mosiac Effect", 10, 10, width, height);
        break;
    
      case 3: // Color TV with Scanlines Effect
        background(31);
        for (int y = 0; y < img.height; y += 1) {
          if ( (y % 2) == 0) continue; // Skip frame lines
    
          for (int x = 0; x < img.width; x += 2) {
            c = int( img.pixels[(y * img.width + x)] );
            fill(c, 127); // fill(rgb, alpha) - alpha = opacity [0 to 255]
            noStroke();
            rect(x * screenScalingFactor + xyOffset, y * screenScalingFactor + xyOffset, 4, 2);
          }
        }
    
        fill(0, 0, 0);
        text("Color TV with Scanlines Effect", 10, 10, width, height);
        break;
    
      default:
        break;
      }
    } // end draw()
    
    void mouseClicked() {
      modeSelect = (modeSelect + 1) % 4;
    }
    
    // End of code.
    
  • Answer ✓

    @Niels

      PImage p = createImage(cam.width, cam.height, RGB);
      p.copy(cam, 0, 0, cam.width, cam.height, 0, 0, cam.width, cam.height);
      img = p;
    

    Try this instead:

      img=cam.get();
    

    Kf

  • @kfrajer Thanks! That should improve performance as it doesn't have to copy the entire frame every time through draw(). =D>

Sign In or Register to comment.