We are about to switch to a new forum software. Until then we have removed the registration on this forum.
I've used this code for multiple projects at this point without many issues but am now running into an unexpected problem which I am unsure how to solve. I have little to no experience with Processing but was getting some help from people a few months back ago when I started to use this code.
The project is a large LED panel installation on 40 panels, the code will run through processing on a raspberry pi so I am using this pi compatible version someone helped me find. I have one video which almost successfully runs on these 40 panels with a bit of lag (which isn't the worst for the project) but it is not the video I intend to use! In comparing it to the video which I do want to use I am not sure what the difference is. I made the first video which works a few months ago so I can't remember if I did anything different but both videos are made in final cut (mov) and then converted to mp4 online. In final cut I made the dimensions of both videos 640 x 360 and both were H264. The first video moves a bit slower and looks more lo-res but I believe those to be qualities from the original source before I imported and edited in FinalCut.
I believe the problem to be related to the amount/size of data being processed. I should mention that both videos will work according to processing and be recognized by the panels but the issue is that the video I want to use does not play consistently. One teensy may send part of the video to the LEDs mostly correct while another teensy's LEDs start to pick up after 10 seconds and then flickers a spectrum of random colors. None of the teensys seem to be synced up with this second video (though all teensys are mounted on the OctoBoards and connected to each other to sync.)
I'm wondering if anyone has any advice for what to do to the video to make it play correctly.
I have tried using HandBrake to also change some qualities of the video but have no clue what I'm doing really and all of my test have yielded no results.
I also noticed that the flickering happens more when theres a bigger change in color supposed to be happening
I suppose the problem could relate to how many rows I have, not sure, but the other video does seem to be working mostly properly. I am inserting some photos that show the project, some of the panels still need some work so thats why one is pink (wrong RGB ordered lights) and the all white ones just have no data input yet.
Anyone have any ideas?
I hope I entered this code correctly...
`/* OctoWS2811 movie2serial.pde - Transmit video data to 1 or more
Teensy 3.0 boards running OctoWS2811 VideoDisplay.ino
http://www.pjrc.com/teensy/td_libs_OctoWS2811.html
Copyright (c) 2013 Paul Stoffregen, PJRC.COM, LLC
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
// To configure this program, edit the following sections:
//
// 1: change myMovie to open a video file of your choice ;-)
//
// 2: edit the serialConfigure() lines in setup() for your
// serial device names (Mac, Linux) or COM ports (Windows)
//
// 3: if your LED strips have unusual color configuration,
// edit colorWiring(). Nearly all strips have GRB wiring,
// so normally you can leave this as-is.
//
// 4: if playing 50 or 60 Hz progressive video (or faster),
// edit framerate in movieEvent().
//import processing.video.*;
import gohai.glvideo.*;
import processing.serial.*;
import java.awt.Rectangle;
GLMovie myMovie;
float gamma = 1.7;
int numPorts=0; // the number of serial ports in use
int maxPorts=24; // maximum number of serial ports
Serial[] ledSerial = new Serial[maxPorts]; // each port's actual Serial port
Rectangle[] ledArea = new Rectangle[maxPorts]; // the area of the movie each port gets, in % (0-100)
boolean[] ledLayout = new boolean[maxPorts]; // layout of rows, true = even is left->right
PImage[] ledImage = new PImage[maxPorts]; // image sent to each port
int[] gammatable = new int[256];
int errorCount=0;
float framerate=0;
void setup() {
String[] list = Serial.list();
delay(20);
println("Serial Ports List:");
println(list);
//serialConfigure("/dev/ttyACM1"); // change these to your port names
//serialConfigure("/dev/cu.usbmodem3550481");
serialConfigure("/dev/tty.usbmodem3999691");
//serialConfigure("/dev/cu.usbmodem3550481");
//serialConfigure("/dev/ttyACM0");
serialConfigure("/dev/tty.usbmodem3645941");
serialConfigure("/dev/tty.usbmodem3694501");
serialConfigure("/dev/tty.usbmodem3766451");
if (errorCount > 0) exit();
for (int i=0; i < 256; i++) {
gammatable[i] = (int)(pow((float)i / 255.0, gamma) * 255.0 + 0.5);
}
size(480, 400, P2D); // create the window
String mpath = sketchPath() + "/../../../media/x.mp4";
println(mpath);
myMovie = new GLMovie(this, mpath);
myMovie.loop(); // start the movie :-)
}
// movieEvent runs for each new frame of movie data
void movieEvent(GLMovie m) {
// read the movie's next frame
m.read();
//if (framerate == 0) framerate = m.getSourceFrameRate();
framerate = 30; // TODO, how to read the frame rate???
for (int i=0; i < numPorts; i++) {
// copy a portion of the movie's image to the LED image
int xoffset = percentage(m.width, ledArea[i].x);
int yoffset = percentage(m.height, ledArea[i].y);
int xwidth = percentage(m.width, ledArea[i].width);
int yheight = percentage(m.height, ledArea[i].height);
ledImage[i].copy(m, xoffset, yoffset, xwidth, yheight,
0, 0, ledImage[i].width, ledImage[i].height);
// convert the LED image to raw data
byte[] ledData = new byte[(ledImage[i].width * ledImage[i].height * 3) + 3];
image2data(ledImage[i], ledData, ledLayout[i]);
if (i == 0) {
ledData[0] = '*'; // first Teensy is the frame sync master
int usec = (int)((1000000.0 / framerate) * 0.75);
ledData[1] = (byte)(usec); // request the frame sync pulse
ledData[2] = (byte)(usec >> 8); // at 75% of the frame time
} else {
ledData[0] = '%'; // others sync to the master board
ledData[1] = 0;
ledData[2] = 0;
}
// send the raw data to the LEDs :-)
ledSerial[i].write(ledData);
}
}
// image2data converts an image to OctoWS2811's raw data format.
// The number of vertical pixels in the image must be a multiple
// of 8. The data array must be the proper size for the image.
void image2data(PImage image, byte[] data, boolean layout) {
int offset = 3;
int x, y, xbegin, xend, xinc, mask;
int linesPerPin = image.height / 8;
int pixel[] = new int[8];
for (y = 0; y < linesPerPin; y++) {
if ((y & 1) == (layout ? 0 : 1)) {
// even numbered rows are left to right
xbegin = 0;
xend = image.width;
xinc = 1;
} else {
// odd numbered rows are right to left
xbegin = image.width - 1;
xend = -1;
xinc = -1;
}
for (x = xbegin; x != xend; x += xinc) {
for (int i=0; i < 8; i++) {
// fetch 8 pixels from the image, 1 for each pin
pixel[i] = image.pixels[x + (y + linesPerPin * i) * image.width];
pixel[i] = colorWiring(pixel[i]);
}
// convert 8 pixels to 24 bytes
for (mask = 0x800000; mask != 0; mask >>= 1) {
byte b = 0;
for (int i=0; i < 8; i++) {
if ((pixel[i] & mask) != 0) b |= (1 << i);
}
data[offset++] = b;
}
}
}
}
// translate the 24 bit color from RGB to the actual
// order used by the LED wiring. GRB is the most common.
int colorWiring(int c) {
int red = (c & 0xFF0000) >> 16;
int green = (c & 0x00FF00) >> 8;
int blue = (c & 0x0000FF);
red = gammatable[red];
green = gammatable[green];
blue = gammatable[blue];
return (green << 16) | (red << 8) | (blue); // GRB - most common wiring
}
// ask a Teensy board for its LED configuration, and set up the info for it.
void serialConfigure(String portName) {
if (numPorts >= maxPorts) {
println("too many serial ports, please increase maxPorts");
errorCount++;
return;
}
try {
ledSerial[numPorts] = new Serial(this, portName);
if (ledSerial[numPorts] == null) throw new NullPointerException();
ledSerial[numPorts].write('?');
} catch (Throwable e) {
println("Serial port " + portName + " does not exist or is non-functional");
errorCount++;
return;
}
delay(250);
String line = ledSerial[numPorts].readStringUntil(10);
if (line == null) {
println("Serial port " + portName + " is not responding.");
println("Is it really a Teensy 3.0 running VideoDisplay?");
errorCount++;
return;
}
String param[] = line.split(",");
if (param.length != 12) {
println("Error: port " + portName + " did not respond to LED config query");
errorCount++;
return;
}
// only store the info and increase numPorts if Teensy responds properly
ledImage[numPorts] = new PImage(Integer.parseInt(param[0]), Integer.parseInt(param[1]), RGB);
ledArea[numPorts] = new Rectangle(Integer.parseInt(param[5]), Integer.parseInt(param[6]),
Integer.parseInt(param[7]), Integer.parseInt(param[8]));
ledLayout[numPorts] = (Integer.parseInt(param[5]) == 0);
numPorts++;
}
// draw runs every time the screen is redrawn - show the movie...
void draw() {
if (myMovie.available()) {
movieEvent(myMovie);
}
// show the original video
image(myMovie, 0, 80);
// then try to show what was most recently sent to the LEDs
// by displaying all the images for each port.
for (int i=0; i < numPorts; i++) {
// compute the intended size of the entire LED array
int xsize = percentageInverse(ledImage[i].width, ledArea[i].width);
int ysize = percentageInverse(ledImage[i].height, ledArea[i].height);
// computer this image's position within it
int xloc = percentage(xsize, ledArea[i].x);
int yloc = percentage(ysize, ledArea[i].y);
// show what should appear on the LEDs
image(ledImage[i], 240 - xsize / 2 + xloc, 10 + yloc);
}
}
// respond to mouse clicks as pause/play
boolean isPlaying = true;
void mousePressed() {
if (isPlaying) {
myMovie.pause();
isPlaying = false;
} else {
myMovie.play();
isPlaying = true;
}
}
// scale a number by a percentage, from 0 to 100
int percentage(int num, int percent) {
double mult = percentageFloat(percent);
double output = num * mult;
return (int)output;
}
// scale a number by the inverse of a percentage, from 0 to 100
int percentageInverse(int num, int percent) {
double div = percentageFloat(percent);
double output = num / div;
return (int)output;
}
// convert an integer from 0 to 100 to a float percentage
// from 0.0 to 1.0. Special cases for 1/3, 1/6, 1/7, etc
// are handled automatically to fix integer rounding.
double percentageFloat(int percent) {
if (percent == 33) return 1.0 / 3.0;
if (percent == 17) return 1.0 / 6.0;
if (percent == 14) return 1.0 / 7.0;
if (percent == 13) return 1.0 / 8.0;
if (percent == 11) return 1.0 / 9.0;
if (percent == 9) return 1.0 / 11.0;
if (percent == 8) return 1.0 / 12.0;
return (double)percent / 100.0;
}`
Answers
so like I said, a few of these panels (marked with Xs--ignore them) just aren't wired right yet... and the rest of the panels are somewhat out of order right now. The problem is the video is not being displayed consistently in the panels which are being fed data. The first photo is the problem video version. The second image shows the video x.mp4 playing correctly.
:(
I have one obvious idea
Other than that, the less compressed a video is the faster, generally, it decompresses (and the larger the file, obviously). So try older formats like mpg4 or mpg2. mjpeg, even.
Doesn't the pi have hardware video decoding for certain formats? Try those. You might need a license.
You might even try replacing the percentage and inverse percentage methods with a lookup table the way you do for the gamma.
And that problem video looks like an incomplete file, like when I try and watch something before handbrake has finished.
Thanks koogs! I tried your recommendations with the mjpeg/ mpg4 /mpg2 and its still about the same... all seem to still have a data date over 100 Mbit/s. I'll try some more throughout the day with handbrake trying to reduce the bitrate/ quality... doesn't seem to change that much so far though.
I don't think they're incomplete files and they are showing clearly in the processing window but maybe there is something else wrong with them? I really wish I knew more about this stuff.
Not sure what the percentage/ inverse methods refer to but ill try to look that up as well.
Anyone else have any other ideas on how to reduce the amount of data / data speed?
I could also run this from a Mac mini instead of a pi... I just don't want to haha. I've tried it running from both the pi and my laptop at this point and its practically the same problems. maybe the pi even works better, but maybe theres a solution that would work on a Mac that wouldn't work on a pi?
Really great advice though! Wish it fixed my weird problem.
Lines 244, 251, 260
actually perhaps part of my problem is the Arduino part?
I've changed the way the teensys feed to panels from the default format... im breaking up the video vertically into 5 strips, placing my x offset 20 percent over each time (I think) ... I assumed this was a percentage, maybe its not?
I've used this on the teensys:
this being the edited part that I am using, editing only the offset by 20 for each board