text follow voice

aletropdj

Hi all,

Im trying to write a code where the text follow the voice. Something like this but less complicated:

tips are welcomed.

Chrisir

well...

you need a MP3 file that you play trought using library sound.

https://processing.org/reference/libraries/

play it in setup()

https://processing.org/reference/libraries/sound/SoundFile.html

set startTime = millis(); at the end of setup()

then you monitor the time that has passed since the start with millis()

see reference

currentTrackTime = millis() - startTime; 

so currentTrackTime is your reference point

if you want to keep it simple say

void draw() {
    background(0);
    if (currentTrackTime<1200) {
        text("What",29,20);
    }
    else if (currentTrackTime<2200) {
        text("did ",29,20);
   }
   else if (currentTrackTime<2500) {
        text("did ",29,20);
        text("he",29,120);
    }
   else if (currentTrackTime<2800) {
        text("did ",29,20);
        text("he",29,120);
        text("say",29,220);
    }
}

so you need to measure your MP3 where which text is...

for transitions (a word like "what" is moving or the last 3 words above turning up one after another, like above) write extra functions.

Those would get the time from (1200) and to (2800) and a text (did he say) and handle it. But this function displayThreeWords would also have to be called in a else if clause in draw().

But this is hard coded for any song or MP3, it's far from a tool to visualize any given MP3/ voice. Except for the transitions maybe.

Best, Chrisir

;-)

aletropdj

So, the easiest its the manual way. it might be easier using the command say on terminal. will sound kind robotic, but enough for what im looking for. thanks

Chrisir

i don't understand... why did you write "will sound kind robotic"?

i didn't discuss the sound - just the image

what do you mean manual way?

aletropdj

it could be done by a automatic speech recognition (am I watching to many movies) or something like that. I can send messages from processing to terminal, no?

Chrisir

that I don't know...