Tuesday, December 17, 2013

HTML5 Audio Triumphant (Game Programming)

I've been a little dismayed when I found out that some of the HTML5 Audio properties (duration, currentTime, some control CSS properties) don't work as they do in other browsers. Part of that distress was that I made an assumption. Not a good idea. I had been hearing for a long time about the need for audio sprites because iPhone (and others) couldn't play two sounds at once. Remy Sharp's article made a big impression on a lot of people (http://remysharp.com/2010/12/23/audio-sprites/) and many people decided that audio for HTML5 wasn't going to work. And I notice that a lot of the games I'm playing don't have any audio.

But you need currentTime to work with audio sprites because you need to load one file and then run your audio from whatever position you want to start the playing from. Sounds like that doesn't work. But you don't need it! Then I realized I hadn't tested it, so I wrote a simple test and guess what?

Two audio files can dance with each other!


So now your jump sounds can take place without cutting off your music and your collision sounds can ignore the capture of coins. Dance!

My test program sets up two audio players and lets you listen to both songs, and start and stop each whenever you want. The songs repeat, and you can play with this all day, enjoying the combination of two different songs. I remember doing some singing like this, and I think it was a song called Frère Jacques, where some people would start singing and others would start singing a few measures later. I wonder if this song has been sung in other languages?

Anyway, here's what the program looks like in Firefox OS on my ZTE Open.



The top set of controls are playing oggsong.ogg and the bottom set of controls are playing a different song, oggsong_2.ogg. Start and stop each one by clicking on the | | symbol in each control. So these controls are useful for testing, even though you can't do much more than position them so they don't overlap.

And here's the code, very similar to the code in my first post on Audio Programming (http://firefoxosgaming.blogspot.com/2013/12/html5-audio-game-programming_10.html).

<!DOCTYPE HTML>
<html>
  <head>
    <meta charset="utf-8">
    <title>
      Audio x 2
    </title>
  
      <script>

      // Global variables
      var myAudio;
      var myAudio2;

      // Load the page.
      window.addEventListener("load",
        runFirst, false);

      // Runs when the page is loaded.
      function runFirst() {

        console.log("page loaded");

        // Create two audio objects.
        // We like OGG.
        myAudio = new Audio("oggsong.ogg");
        myAudio2 = new Audio("oggsong_2.ogg");
       
        // Need this to see the controls.
        document.getElementById("myBody").
            appendChild(myAudio);          
        document.getElementById("myBody").
            appendChild(myAudio2);

        // Loop both songs.
        myAudio.loop = true;
        myAudio2.loop = true;

        // Make both controls visible.
        myAudio.controls = true;
        myAudio2.controls = true;
       
        // Position 1st audio control.
        myAudio.id = "player1";
        player1.style.position = "absolute";
        player1.style.top = "70px";
        player1.style.left = "10px";
      
        // Position 2nd audio control.
        myAudio2.id = "player2";
        player2.style.position = "absolute";
        player2.style.top = "200px";
        player2.style.left = "10px";
       
        // Listen for fully loaded audios.
        myAudio.addEventListener("canplaythrough",
          processMyAudio, false);         
        myAudio2.addEventListener("canplaythrough",
          processMyAudio2, false);       
      }

      // The first audio is ready to play.
      function processMyAudio() {
     
        console.log("audio one loaded");
       
        // Event no longer needed.
        myAudio.removeEventListener("canplaythrough",
            processMyAudio, false);
           
         // Play the first audio.
        myAudio.play();     
      }

      // The second audio is ready to play.
      function processMyAudio2() {
     
        console.log("audio two loaded");
       
        // Event no longer needed.
        myAudio2.removeEventListener("canplaythrough",
            processMyAudio2, false);
           
        // Play the second audio.
        myAudio2.play();     
      }
     
       </script>
    </head>

    <body id="myBody">
      <p>Audio Controls</p>
    </body>

</html>


The trick here is very, very simple. For every audio track you want to create, create a new Audio object, give them different names, and process them separately.

For example:

        myAudio = new Audio("oggsong.ogg");
        myAudio2 = new Audio("oggsong_2.ogg");


This makes two Audio objects, each with a different name and each with a different song file (but both of them are OGG, yay OGG).

Add them both to the page, make them both loop and have controls. I then positioned each control so they don't fit on the screen.

        // Position 1st audio control.
        myAudio.id = "player1";
        player1.style.position = "absolute";
        player1.style.top = "70px";
        player1.style.left = "10px";
      
        // Position 2nd audio control.
        myAudio2.id = "player2";
        player2.style.position = "absolute";
        player2.style.top = "200px";
        player2.style.left = "10px";


I'm still working out what happens. I think the controls are centered on the phone page and maybe that's a bug someone should file, or maybe they want it that way? At least the top, left styles (absolute) let me have them not overlap.

Next I set up event listeners to make sure that each audio loads. 

        // Listen for fully loaded audios.
        myAudio.addEventListener("canplaythrough",
          processMyAudio, false);         
        myAudio2.addEventListener("canplaythrough",
          processMyAudio2, false);       


Then, finally, I have a separate function that is called when the audio is ready to play (canplaythrough). Here's the function for the first audio and the second is similar, except for the change in object names.

      // The first audio is ready to play.
      function processMyAudio() {
     
        console.log("audio one loaded");
       
        // Event no longer needed.
        myAudio.removeEventListener("canplaythrough",
            processMyAudio, false);
           
         // Play the first audio.
        myAudio.play();     
      }


And that's it! Start and stop the two tracks and see that they can both play at the same time. So who needs Remy Sharp?

Mangling Mozart
In case you want to listen to the two actual songs, you can download them for free (and you don't need to register there either, just click the Download button). Here are the links.

https://soundcloud.com/thulfram/oggsong
https://soundcloud.com/thulfram/oggsong_2

SoundCloud is a cool service for people who want to share their music and it is very easy to use. Put the songs in the same root folder as your index.html file and they will magically be loaded into the simulator and your phone.

The two songs are both about 9 seconds long. The two songs are mangled versions of a Mozart piece called Palindrome (which means running back and forth). I mangled them by putting the tune into FL Studio (http://www.image-line.com/documents/flstudio.html), loading in a synthesizer (H.G. Fortune's Swamp XT at http://www.hgf-synthesizer.com/), and then using FL Studio's Riff Machine to mangle the original music in two different ways (two different sounds, tempos, everything different).

What I find interesting is that even if you combine the two songs starting at different times, the human mind makes them sound like one song. Weird. But you can stop and start each song independently and see that they really are playing simultaneously.

But the important thing is that you can have two songs playing at the same time, so there's no excuse for not adding music and sound effects to your games. I have one or two more audio posts to do and then we can get back to making games. As I've learned, HTML5 Audio is a little tricky, but worth it.