Thursday, December 12, 2013

HTML5 Audio Followup (Game Programming)

I'm still deep into HTML5 Audio programming as it relates to Firefox OS. I ran into some interesting problems and filed bug 949129 to make sure I'm not crazy. I'll be working on some work-arounds for the problems I'm seeing so you can still use audio in your games.













This post will cover a few things more I've found, in random order.

Audio Codecs

André Jaenisch pointed out that I didn't explain that MP3 and OGG are not lossless. This means that when the audio file is created, some of the bits are removed so that the file size is smaller. A popular format for lossless audio is FLAC. But this blog is all about Firefox OS gaming and right now, FLAC is not supported. As a matter of fact, MP3 is not supported either.

The original non-lossy formats were WAV (Windows) and AIFF (Apple). There might be an original Unix/Linux format (AU?)  but I've not used it. WMA (Windows), AAC (Apple), and MP3 (Fraunhofer) all use compression to make file sizes smaller, but still sound pretty good. Somewhere in the middle is the open-source FLAC, which uses compression but doesn't lose any quality and is considered lossless.

MP3 is an interesting issue for Firefox. For a long time, Firefox didn't support MP3 directly unless your operating system had already set up support. Because of this, when programming audio for the Firefox browser, you had to make two files, one MP3 and the other OGG. And for other browsers, you might have to add in an Apple codec (but we don't speak Apple here). You can read about Firefox support at https://developer.mozilla.org/en-US/docs/HTML/Supported_media_formats, but keep in mind that this is a moving target.

Very recently, Mozilla announced that they will support MP3 and pay the fee. MP3 costs money right now, OGG is free! The good news is that the MP3 patents will expire in a year or two.

Why does all this matter? If you have your music in MP3 and you convert it to OGG, the lossy audio will become more lossy. Right now, the music I download onto my ZTE Open will play even if it is MP3, but if I want to write a program in HTML5, my best bet is OGG. Firefox will let you program a WAV file, but the WAV file size is larger than OGG or MP3, and for a phone, you always want the smallest of everything.

So the point is, if you are making music for the Firefox OS, create your original file in WAV format (or one of the Apple formats), but at some point, convert it to OGG. But save your original file (lossless) so you can convert it later if you want to program with MP3.

My own personal guess is that OGG will be supported in Firefox OS for a long time, so its a safe bet. WAV is supported, but large file sizes will make it a poor option for phones and tablets. MP3 support will be soon, and I wouldn't be surprised to see in the future support for Apple's AAC, which is catching on fast (because it has smaller file sizes and better fidelity than MP3).

And don't forget, OGG is open source and very cool!

So if you think HTML5 audio is confusing, wait until you get to HTML5 video! But the answer is to start working with Web Audio (not the same as HTML5 Audio) which is supported. I'll be poking into that soon, but there's more pressing matters.

Oh, and I misspelled the cool audio workstation Ardour. Sorry, guys. Get it at http://ardour.org/

Globals

According to many wise JavaScript gurus, you should avoid globals. However, I find that I need them to make things simple for small game demos. So in the recent HTML5 Audio post at http://firefoxosgaming.blogspot.com/2013/12/html5-audio-game-programming_10.html,  I had this global definition:

      // Global variables
      var myAudio;


I don't assign it anything yet, because I need to wait until the page loads. This variable is used for the audio object and gets created in the audio constructor here, but only after the page loads:

      myAudio = new Audio("oggsong.ogg");

When working with objects and loading things, you always want to make sure that you know something is loaded before you use it. But also, in this case, if you don't make myAudio global, the function processMyAudio won't know where to find it.

And as part of the making sure, processMyAudio doesn't get called until canplaythrough tells us that the audio is now loaded and ready to go!

If you do these things without some of the safeguards, things might work or they might not. But you want to be sure. But if you don't make myAudio global, the music won't play. Using JavaScript functions is cool, but you want to make sure that everything inside the function can be used.

You can rewrite this program to avoid globals and pass things through function calls, but for now, I want to keep everything as simple as possible. If you want to know more about why globals are evil, check out JavaScript, the Good Parts. The book is difficult but worth reading several times, and I don't always agree with the author, but he clearly knows what he's talking about.

Here's a great picture of the two best books on JavaScript:


The thin book on the left is Crockford's JavaScript: The Good Parts. The thick book on the right is called JavaScript: The Definitive Guide by David Flanagan. Read both and you'll know most of what you need to know. Make sure you get the latest edition of Flanagan, because it gets updated every few years. Note the size difference of the two books, but there's lots of good parts in the bigger book also.

Firefox OS Simulator

The Firefox OS simulator (available to your Firefox browser as a plug-in) is really valuable for debugging audio. Something that runs just fine in the desktop browser may not work on the phone. Why do I bring this up?

If you use MP3 in the Firefox desktop browser, all works well. But not on the phone. If you use OGG, your audio works in the simulator. Before audio, I barely looked at the simulator unless there was a sizing issue, but for audio, it's crucial that the sound works at the simulator level.

What Might Not Be Working

I was surprised to see that two of the audio object's properties, duration and currentTime didn't work in Firefox OS when they work just fine in desktop Firefox. I haven't read all the browser source code yet, but I filed a bug 949129. You can check it out at https://bugzilla.mozilla.org/show_bug.cgi?id=949129. You can read my test code there.

By the way, if you find any kind of bug, make sure you can reproduce it, and file it at Bugzilla. Remember, YOU are Mozilla. Make it better and bang on it today. Especially with Firefox OS, which is still young and growing.

I'll keep you posted.

Next

I'm working on a work-around for the lack of currentTime. I'll also do some short code samples for loop and controls. And there's a sample game I want to get on to next, and maybe some game engines, and oh, yes, more game reviews. And vibration and tilt and oh, the list goes on forever!