We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpPrograms › Merging Images with Sound
Page Index Toggle Pages: 1
Merging Images with Sound (Read 467 times)
Merging Images with Sound
Dec 1st, 2008, 5:24pm
 
Hi, I'm doing a project for school and I need some help with it. For my project I'm using a sensor and what I'm trying to do is make a sound file play along with a certain image. Ex. Classic.jpg with Grease.wav. So far I'm unable to have the image with the sound file to work together. Here's what I have so far. Please help me out. Thanks

p.s. I'm working with Processing 0135

import processing.serial.*;
// The serial port:

Serial myPort;

import krister.Ess.*;
AudioChannel myChannel;

void setup() {

   println(Serial.list());
 myPort = new Serial(this, Serial.list()[0], 57600);
 
 size(500,395);
PImage a;  // Declare variable "a" of type PImage
a = loadImage("Classic.jpg"); // Load the images into the program
image(a, 0, 0); // Displays the image from point (0,0)

PImage b;  // Declare variable "a" of type PImage
b = loadImage("Classic1.jpg"); // Load the images into the program
image(b, 0, 0); // Displays the image from point (0,0)

PImage c;  // Declare variable "a" of type PImage
c = loadImage("Classic2.jpg"); // Load the images into the program
image(c, 0, 0); // Displays the image from point (0,0)

PImage d;  // Declare variable "a" of type PImage
d = loadImage("Classic3.jpg"); // Load the images into the program
image(d, 0, 0); // Displays the image from point (0,0)

PImage e;  // Declare variable "a" of type PImage
e = loadImage("Classic4.png"); // Load the images into the program
image(e, 0, 0); // Displays the image from point (0,0)

PImage f;  // Declare variable "a" of type PImage
f = loadImage("Classic5.jpg"); // Load the images into the program
image(f, 0, 0); // Displays the image from point (0,0)
}

{
 // start up Ess
 Ess.start(this);
 myChannel=new AudioChannel("Grease.wav");
 
 myChannel=new AudioChannel("Grease1.wav");
 
 myChannel=new AudioChannel("Grease2.wav");
 
 myChannel=new AudioChannel("Grease3.wav");
 
 myChannel=new AudioChannel("Grease4.wav");
 
 myChannel=new AudioChannel("Grease5.wav");
 
}

void draw() {
  while (myPort.available() > 0) {
   int inByte = myPort.read();
   println(inByte);
}

// we are done, clean up Ess
//void mousePressed() {
 if (myChannel.state==Ess.PLAYING) {
   myChannel.stop();
 }
 else {
   myChannel.play(Ess.FOREVER);
 }
}
public void stop() {
 Ess.stop();
 super.stop();
}

//int getsongnumber(void){
 //int answer =0;
 
//values range from about 70 to about 500


 //return answer;

//}


/*

use it like this

songnumbertoplay = getsongnumber();



*/
Re: Merging Images with Sound
Reply #1 - Dec 1st, 2008, 5:52pm
 
You correctly define a variable per image (although the comments are out of synch...), but you load all the sound samples in the same variable, actually keeping only the last.

In both cases, you might want to use arrays for more flexibility.
Re: Merging Images with Sound
Reply #2 - Dec 5th, 2008, 5:12am
 
Thanks for replying to my question. I've seemed to clean things up but now the problem that I’m having is that only one image and one sound file are both playing repetitively.  Is there a way to linking up certain images to play with the certain sound file chosen (Ex. Classic.jpg with Grease.wav) and also a way to set the sensor to play these at a certain times. Thanks

Also here’s what I have now

import processing.serial.*;
// The serial port:

Serial myPort;

PImage img1, img2, img3, img4, img5, img6 ; //set place holders in memory for our image
int currentPixel;//track the current pixel the mouse position
import krister.Ess.*;
AudioChannel myChannel;
int sampleLength; //keep track of the length of audio file
//set a buffer variable to contain incoming serial data
String buff = "";
int threshold = 50;

void setup() {

   println(Serial.list());
 myPort = new Serial(this, Serial.list()[0], 57600);
 
 size(500,395);
 img1 = loadImage("Classic.jpg");
 img2 = loadImage("Classic1.jpg");
 img3 = loadImage("Classic2.jpg");
 img4 = loadImage("Classic3.jpg");
 img5 = loadImage("Classic4.png");
 img6 = loadImage("Classic5.jpg");
 noStroke();
}
{
 // start up Ess
 Ess.start(this);
 myChannel=new AudioChannel();
 myChannel.loadSound("Grease.wav");
 myChannel.loadSound("Grease1.wav");
 myChannel.loadSound("Grease2.wav");
 myChannel.loadSound("Grease3.wav");
 myChannel.loadSound("Grease4.wav");
 myChannel.loadSound("Grease5.wav");
 myChannel.softClip = true;
 
 sampleLength = myChannel.size;
 println(sampleLength);
 
 myChannel.play(Ess.FOREVER);
}

void draw()  {
  if (myPort.available() > 0) {
   int inByte = myPort.read();
   println(inByte);
}

//use image()to display the images
//currentPixel equals the mouseX position+the number of pixels
//precedding the current mouse Y position in the pixels()Array
currentPixel = mouseX + (mouseY * width);
image(img1,0,0);//display the images at point (0,0)
loadPixels(); //loadPixels will runby self; loads entire canvas
img1.loadPixels(); //loading pixels[] for a single image
img2.loadPixels();
img3.loadPixels();
img4.loadPixels();
img5.loadPixels();
img6.loadPixels();
}

 /*if (myChannel.state==Ess.PLAYING) {
   myChannel.stop();
 }
 else {
   myChannel.play(Ess.FOREVER);
 }*/
 
//clean up Ess
//this must be done every time used in sketch
public void stop() {
 Ess.stop();
 super.stop();
}
Re: Merging Images with Sound
Reply #3 - Dec 5th, 2008, 11:04am
 
Same answer...

Unless I am not understanding how ESS works (I never used it), it is probable that the successive loadSound's just overwrite the previous sound in the channel's buffer, so you keep only the last one.

Likewise, the successive loadPixel calls just overwrite the previous pixels[] array, plus you don't need that to display the images, look at image() instead.

And, again, look at Array, it will ease the process of associating images with sounds.
Re: Merging Images with Sound
Reply #4 - Dec 9th, 2008, 7:31pm
 
Thanks for your help, I've finally got it working. Again Thank You.
Page Index Toggle Pages: 1