1.2 Timed Text Track Api

In notebook:
edX Advanced HTML5
Created at:
2016-01-16
Updated:
2016-01-17
Tags:
To add audio tracks include a ​<track>​ element inside the ​<video>​ element:
  <video id="myVideo" preload="metadata" controls crossOrigin="anonymous">
    <source src="http://...../elephants-dream-medium.mp4" type="video/mp4">
    <source src="http://...../elephants-dream-medium.webm" type="video/webm">
    <track label="English subtitles" kind="subtitles" srclang="en"
           src="http://...../elephants-dream-subtitles-en.vtt">
    <track label="Deutsch subtitles" kind="subtitles" srclang="de"
           src="http://...../elephants-dream-subtitles-de.vtt" default>
    <track label="English chapters" kind="chapters" srclang="en"
           src="http://...../elephants-dream-chapters-en.vtt">
 </video>
  • IE11 and Safari let's you choose which subtitle/caption to display. If has a ​default​ attribute it's loaded
  • Chrome and Opera: no menu choose your track. ​default​ is loaded or none if not ​default​ is not set
  • Firefox - no track menu at all, only if there a ​default​ set
Instead we need to use the Timed Text Track API of the HTML5 spec
To get the included subtitle tracks, use ​querySelector​:
 ​htmlTracks = document.querySelectorAll("track")​ 
(use it inside a ​window.onload​ listener)

They have properties like ​kind​ ("subtitles") and ​srclang​ ("en")
the returned elements (tracks) have a ​readyState​ property:
  • 0 = NONE ; the text track's cues have not been obtained
  • 1 = LOADING ; the text track is loading with no errors yet. Further cues can still be added to the track by the parser
  • 2 = LOADED ; the text track has been loaded with no errors
  • 3 = ERROR ; the text track was enabled, but when the user agent attempted to obtain it, something failed. Some or all of the cues are likely missing and will not be obtained

TextTrack

The above are HTML tracks (html elements).

The ​TextTrack​ JavaScript object is the corresponding object of the HTML track.

The object that contains the cues (subtitles or captions or chapter description from the WebVTT file) is not the HTML track itself; it is another object that is associated with it: a TextTrack object!

It's a the ​.track​ property of an html track e.g.
textTrack = document.querySelectorAll("track")[0].track;​
it has several properties e.g. : ​.kind​ ​.label​ ​.language​

The ​mode​ property

  • "showing": the track is either already loaded, or is being loaded by the browser. As soon as it is completely loaded, subtitles or captions will be displayed in the video. Other kinds of track will be loaded but will not necessarily show anything visible in the document. All tracks that have mode="showing" will fire events while the video is being played.
  • "hidden": the track is either already loaded, or is being loaded by the browser.  All tracks that have mode="hidden" will fire events while the video is being played. Nothing will be visible in the standard video player GUI.
  • "disabled": this is the mode where tracks are not being loaded. If a loaded track has its mode set to "disabled", it will stop firing events, and if it was in mode="showing" the subtitles or captions will stop being displayed in the video player
TEXTTRACK CONTENT CAN ONLY BE ACCESSED IF A TRACK HAS BEEN LOADED! USE THE MODE PROPERTY TO FORCE LOAD A TRACK!

It is possible to force a track to be loaded by setting the mode property of the TextTrack object to "showing" or "hidden". 

Tracks that are not loaded have their mode property equal to "disabled". 

You can force load a subtitle by setting it's ​mode​ property​ to ​hidden​.

To summarise the workflow for loading tracks

  1. use ​onload​ listener
  2. check ​readyState === 2​
  3. if not, set ​.mode = "hidden"​ 
  4. then listen again to ​"load"​ event

  function readContent(track) {
   console.log("reading content of loaded track...");
   displayTrackStatuses(htmlTracks); // update document with new track statuses
}

function forceLoadTrack(n) {
    // first parameter = track number,
    // second = a callback function called when the track is loaded,
    // that takes the loaded TextTrack as parameter
    getTrack(htmlTracks[n], readContent);
}
 
function getTrack(htmlTrack, callback) {
   // TextTrack associated to the htmlTrack
   var textTrack = htmlTrack.track;
   if(htmlTrack.readyState === 2) {
      console.log("text track already loaded");
      // call the callback function, the track is available
      callback(textTrack);
   } else {
      console.log("Forcing the text track to be loaded");
 
      // this will force the track to be loaded
      textTrack.mode = "hidden";
      // loading a track is asynchronous, we must use an event listener
      htmlTrack.addEventListener('load', function(e) {
         // the track is arrived, call the callback function
         callback(textTrack);
      });
   }
}

Contents of a ​​TEXTTRACK object

  • ​kind​  -> is either "subtitles", "caption", "descriptions", "chapters", or "metadata".
  • ​label​ -> same as the HMTL track element ​label​ attribute
  • ​language​ -> same as the HTML track element ​srclang​ attribute
  • ​mode​ -> see above ("disabled", "hidden", "showing")
  • ​cues​ -> list of cues as a TextTrackCueList object. The complete contents of the WebVTT file
  • ​activeCues​ -> used in event listener, corresponds to the cues of the current time segment. returns TextTrackCueList object, the returns all active tracks (if they overlap at the same time)
  • ​addCue(cue)​ -> add a cue to the list of cues
  • ​removeCue(cue)​-> remove a cue from the list of cues
  • ​getCueById(id)​ -> returns the cue with a given id (needs polyfill for some browsers)

​​TextTrackCueList

See above list, ​cues​ property:
A collection of cues. Properties and methods, id, sartTime, endTime, text, getCueAsHTML(), align, line, position, size, snaptToLines​

Events during playback

​enter​ and ​exit​ for cues (not supported yet by all browsers in end of 2015)
​cuechange​ event fired forth TextTrack object (good browser support)
  // track is a loaded TextTrack
track.addEventListener("cuechange", function(e) {
   var cue = this.activeCues[0];
   console.log("cue change");
   // do something with the current cue
 });
​​take note of ​activeCues[0]​ 

then use properties like ​.text​ to display text