HTML5 Audio Triumphant (Game Programming)

I've been a little dismayed when I found out that some of the HTML5 Audio properties (duration, currentTime, some control CSS properties) don't work as they do in other browsers. Part of that distress was that I made an assumption. Not a good idea. I had been hearing for a long time about the need for audio sprites because iPhone (and others) couldn't play two sounds at once. Remy Sharp's article made a big impression on a lot of people (http://remysharp.com/2010/12/23/audio-sprites/) and many people decided that audio for HTML5 wasn't going to work. And I notice that a lot of the games I'm playing don't have any audio.

But you need currentTime to work with audio sprites because you need to load one file and then run your audio from whatever position you want to start the playing from. Sounds like that doesn't work. But you don't need it! Then I realized I hadn't tested it, so I wrote a simple test and guess what?

Two audio files can dance with each other!

So now your jump sounds can take place without cutting off your music and your collision sounds can ignore the capture of coins. Dance!

My test program sets up two audio players and lets you listen to both songs, and start and stop each whenever you want. The songs repeat, and you can play with this all day, enjoying the combination of two different songs. I remember doing some singing like this, and I think it was a song called Frère Jacques, where some people would start singing and others would start singing a few measures later. I wonder if this song has been sung in other languages?

Anyway, here's what the program looks like in Firefox OS on my ZTE Open.

The top set of controls are playing oggsong.ogg and the bottom set of controls are playing a different song, oggsong_2.ogg. Start and stop each one by clicking on the | | symbol in each control. So these controls are useful for testing, even though you can't do much more than position them so they don't overlap.

And here's the code, very similar to the code in my first post on Audio Programming (http://firefoxosgaming.blogspot.com/2013/12/html5-audio-game-programming_10.html).

<!DOCTYPE HTML>
<html>
  <head>
    <meta charset="utf-8">
    <title>
      Audio x 2
    </title>
  
      <script>

      // Global variables
      var myAudio;
      var myAudio2;

      // Load the page.
      window.addEventListener("load",
        runFirst, false);

      // Runs when the page is loaded.
      function runFirst() {

        console.log("page loaded");

        // Create two audio objects.
        // We like OGG.
        myAudio = new Audio("oggsong.ogg");
        myAudio2 = new Audio("oggsong_2.ogg");
       
        // Need this to see the controls.
        document.getElementById("myBody").
            appendChild(myAudio);          
        document.getElementById("myBody").
            appendChild(myAudio2);

        // Loop both songs.
        myAudio.loop = true;
        myAudio2.loop = true;

        // Make both controls visible.
        myAudio.controls = true;
        myAudio2.controls = true;
       
        // Position 1st audio control.
        myAudio.id = "player1";
        player1.style.position = "absolute";
        player1.style.top = "70px";
        player1.style.left = "10px";
      
        // Position 2nd audio control.
        myAudio2.id = "player2";
        player2.style.position = "absolute";
        player2.style.top = "200px";
        player2.style.left = "10px";
       
        // Listen for fully loaded audios.
        myAudio.addEventListener("canplaythrough",
          processMyAudio, false);         
        myAudio2.addEventListener("canplaythrough",
          processMyAudio2, false);       
      }

      // The first audio is ready to play.
      function processMyAudio() {
     
        console.log("audio one loaded");
       
        // Event no longer needed.
        myAudio.removeEventListener("canplaythrough",
            processMyAudio, false);
           
         // Play the first audio.
        myAudio.play();     
      }

      // The second audio is ready to play.
      function processMyAudio2() {
     
        console.log("audio two loaded");
       
        // Event no longer needed.
        myAudio2.removeEventListener("canplaythrough",
            processMyAudio2, false);
           
        // Play the second audio.
        myAudio2.play();     
      }
     
       </script>
    </head>

    <body id="myBody">
      <p>Audio Controls</p>
    </body>

</html>

The trick here is very, very simple. For every audio track you want to create, create a new Audio object, give them different names, and process them separately.

For example:

        myAudio = new Audio("oggsong.ogg");
        myAudio2 = new Audio("oggsong_2.ogg");

This makes two Audio objects, each with a different name and each with a different song file (but both of them are OGG, yay OGG).

Add them both to the page, make them both loop and have controls. I then positioned each control so they don't fit on the screen.

        // Position 1st audio control.
        myAudio.id = "player1";
        player1.style.position = "absolute";
        player1.style.top = "70px";
        player1.style.left = "10px";
      
        // Position 2nd audio control.
        myAudio2.id = "player2";
        player2.style.position = "absolute";
        player2.style.top = "200px";
        player2.style.left = "10px";

I'm still working out what happens. I think the controls are centered on the phone page and maybe that's a bug someone should file, or maybe they want it that way? At least the top, left styles (absolute) let me have them not overlap.

Next I set up event listeners to make sure that each audio loads. 

        // Listen for fully loaded audios.
        myAudio.addEventListener("canplaythrough",
          processMyAudio, false);         
        myAudio2.addEventListener("canplaythrough",
          processMyAudio2, false);       

Then, finally, I have a separate function that is called when the audio is ready to play (canplaythrough). Here's the function for the first audio and the second is similar, except for the change in object names.

      // The first audio is ready to play.
      function processMyAudio() {
     
        console.log("audio one loaded");
       
        // Event no longer needed.
        myAudio.removeEventListener("canplaythrough",
            processMyAudio, false);
           
         // Play the first audio.
        myAudio.play();     
      }

And that's it! Start and stop the two tracks and see that they can both play at the same time. So who needs Remy Sharp?

Mangling Mozart
In case you want to listen to the two actual songs, you can download them for free (and you don't need to register there either, just click the Download button). Here are the links.

https://soundcloud.com/thulfram/oggsong
https://soundcloud.com/thulfram/oggsong_2

SoundCloud is a cool service for people who want to share their music and it is very easy to use. Put the songs in the same root folder as your index.html file and they will magically be loaded into the simulator and your phone.

The two songs are both about 9 seconds long. The two songs are mangled versions of a Mozart piece called Palindrome (which means running back and forth). I mangled them by putting the tune into FL Studio (http://www.image-line.com/documents/flstudio.html), loading in a synthesizer (H.G. Fortune's Swamp XT at http://www.hgf-synthesizer.com/), and then using FL Studio's Riff Machine to mangle the original music in two different ways (two different sounds, tempos, everything different).

What I find interesting is that even if you combine the two songs starting at different times, the human mind makes them sound like one song. Weird. But you can stop and start each song independently and see that they really are playing simultaneously.

But the important thing is that you can have two songs playing at the same time, so there's no excuse for not adding music and sound effects to your games. I have one or two more audio posts to do and then we can get back to making games. As I've learned, HTML5 Audio is a little tricky, but worth it.

HTML5 Audio (Game Programming)

HTML5 Audio has been around forever, or at least as long as HTML5. There's something new called WebAudio that has a lot more features, but HTML5 Audio lets you play a tune for your game. I'm exploring HTML5 Audio and I'll share what I find as I confirm what works in Firefox OS.

There are a lot of twists and turns in the audio field. You have to create your music somehow and then make sure it's in the right format. Some of this is changing in Firefox, but here's what seems to work best. Encode your audio in OGG.

What the heck is OGG? Well, you probably have heard of MP3 music and OGG is like MP3 but is open source and not proprietary. Both MP3 and OGG compress your music to make it smaller without losing any of the quality of the sound. OGG lives at http://www.vorbis.com/. You don't need to understand OGG, you just need to save an audio file to OGG if you are creating it, and convert music files to OGG if you have it in some other format like MP3, WAV, or WMA. A great open source tool for converting audio is Audacity, which lives at http://audacity.sourceforge.net/ Audacity has been around forever and is really a great piece of open source free software.

I'll be working with audio as part of my blog, so I'll just say the tools I like. I've used a lot of audio tools over the years, but right now here is my tool chain.

1. Create audio in EnergyXT. This is an inexpensive audio workstation that I like. You can create music pretty easily with it and there's lots of help and tutorials. Find it at http://www.energy-xt.com/. It is on sale for €39. Available for Windows, Mac, Linux, and iOS.
2. Right now I'm having fun with Chip Tune Music and my favorite software instrument is ChipSounds at http://www.plogue.com/products/chipsounds/. Not cheap at $95, but there are lots of open source and/or free chip tune instruments. The basic idea is that you add instruments or pre-made loops together to make a song using a workstation like EnergyXT. There are plenty of other workstatons and tons of instruments and loops. For more info on creating music through software, check out http://www.kvraudio.com/. And while you're at it, read Computer Music magazine at http://www.musicradar.com/computermusic/. Music software makes it easy to create music because you can create music bits and combine them together.
3. Sometimes I like to combine bits of music I've created in a program that let's me work with several tracks. I like Adobe Audition because I've used it a long time and I use an old version, so you might want to look around. The new Audition is expensive. But I use the old one. I've been meaning to look at Ardor, which is a very cool open source alternative. http://ardour.org/
4. And as a final step, I run all the combined tracks through Audacity to tweak and convert. My old copy of Audition doesn't know about OGG. But if you want to work with Firefox OS, you'll want to make OGG your new BFF.

So enough about making music. Make it OGG and be proud of open source!

I wrote some simple code that just plays an ogg file. Here it is:

<!DOCTYPE HTML>
<html>
  <head>
    <meta charset="utf-8">
    <title>
      Simple Audio
    </title>
      <script>

      // Global variables
      var myAudio;

      // Load the page.
      window.addEventListener("load",
        runFirst, false);

      // Runs when the page is loaded.
      function runFirst() {

        console.log("page loaded");

        // Create audio object.
        // We like OGG.
        myAudio = new Audio("oggsong.ogg");

        // Listen for fully loaded audio.
        myAudio.addEventListener("canplaythrough",
          processMyAudio, false); 
      }

      function processMyAudio() {

        console.log("audio loaded");

        // Play the audio.
        myAudio.play();
      }

       </script>
    </head>

    <body>
    </body>

</html>

This code is pretty simple. Use these steps:

1. Make a global variable for your audio object.
2. Wait until the page loads.
3. Create an audio object and give it an audio file to load.
4. Wait until the audio file loads.
5. When it loads, play the audio!

You'll notice a pattern that is often useful. Load something, set an event handler to wait until it loads, and then use it. Because we're working in the mysterious world of browsers, you can't always know when exactly something will load or in what order. Especially if you're loading something from a server or something is slow, like a big file.

Creating the Audio Object

Here's the bit that creates the audio object.

        myAudio = new Audio("oggsong.ogg");

This is going deeper into the DOM and is called a constructor. It makes an object and initializes it with the name of an audio file. This is assuming that your song is in the root directory as defined by the app manifest. Specify the folder in the constructor; you don't need to tell the manifest where the audio file is, it will just pull it in the same as an art file. Always make sure that you use the .ogg file extension (and that your file is OGG based.

Waiting for Audio

Here's the bit that waits for the audio to load.

         myAudio.addEventListener("canplaythrough",
          processMyAudio, false); 

It sets up an event listener that is waiting for the canplaythrough event to take place on the audio object.  The event is fired when all of the audio is loaded. Read more about it on the new-and-improved truly beautiful Mozilla Developer Network page at https://developer.mozilla.org/en-US/docs/Web/Reference/Events/canplaythrough. No, it isn't an special rule for golfers.

Play the Audio

After all this, the command is simple.

        myAudio.play();

The command sits in a function that is called only when the audio is good and ready. For many uses, you could just skip the waiting, but I like to be careful. You don't want to mess up your audio. I've noticed that not all the game I review have working audio and this is sad. Music is such a powerful addition to games and most games that succeed not only have a great game programmer, they have a great artist and a great musician. (And a great web site designer, business manager, and tea boy.)

The song will play once and that's all for now. There's more to the world of HTML Audio, but that's all for now. Stay tuned, but not iTuned!