Highlighting effect to text and/or image similar to be synchronized with audio

Question

I am looking how to approach following problem:

We have application that displays text with audio recorded material. We use Browser Control (Internet Explorer) in Delphi App to do this. We respond to events in Delphi code setting innerHTML for elements if we have to update the style ...

Now, request is to add option to dynamically move the cursor or dynamically highlight the words spoken from the paragraph. It doesn't need to match absolutely the exact word spoken so we will have to dynamically update the content of position of highlighted word based on some timer or something (because it is not text to speach).

What should be the most practical and easy approach to this kind of problem, all answers are greatly appreciated.

Thanks.

Answer 1

let's assume you have a text file with the text to be shown and the annotated time of when to hightlight it (kind of a subtitles file, for example the standard proposal w3c timed text (http://www.w3.org/AudioVideo/TT/) or the SUB - Movie subtitle file format in use by several media players.

Your program must first read and parse the text file, and decode the annotated time. Insert it in a stringlist called Subtitles which items would also keep objects similar to this one

type tSubtitle = class
  num : integer;
  prevTime, fromTime : tdatetime;
  toTime, nextTime: tdatetime;
  text: string;
end;

You might want to extend the object to hold some highlighting attributes as well.

Then you just need to display those objects synchronized with a timer.

procedure TForm1.Timer1Timer(Sender: TObject);
var rt : TDateTime;
    done:boolean; 
    si,st,sb:integer; 
    s:string;
begin
  rt:=now-startTime;
  st:=0;
  sb:=subtitles.Count;  // binary search the subtitle for the current time
  repeat
     si:=(st+sb) div 2;
     s:=TSubtitle(subtitles.Objects[si-1]);
     done:= ((t>=s.prevTime) and (t<=s.nextTime));
     if not done then 
     begin
        if t>s.prevTime then st:=si 
        else if t<s.nextTime then sb:=si;
        if st=sb then done:=true;
     end;
  until done;
  // do what you want with s
end;