HTML5 Audio Followup (Game Programming)

I'm still deep into HTML5 Audio programming as it relates to Firefox OS. I ran into some interesting problems and filed bug 949129 to make sure I'm not crazy. I'll be working on some work-arounds for the problems I'm seeing so you can still use audio in your games.













This post will cover a few things more I've found, in random order.

Audio Codecs

André Jaenisch pointed out that I didn't explain that MP3 and OGG are not lossless. This means that when the audio file is created, some of the bits are removed so that the file size is smaller. A popular format for lossless audio is FLAC. But this blog is all about Firefox OS gaming and right now, FLAC is not supported. As a matter of fact, MP3 is not supported either.

The original non-lossy formats were WAV (Windows) and AIFF (Apple). There might be an original Unix/Linux format (AU?)  but I've not used it. WMA (Windows), AAC (Apple), and MP3 (Fraunhofer) all use compression to make file sizes smaller, but still sound pretty good. Somewhere in the middle is the open-source FLAC, which uses compression but doesn't lose any quality and is considered lossless.

MP3 is an interesting issue for Firefox. For a long time, Firefox didn't support MP3 directly unless your operating system had already set up support. Because of this, when programming audio for the Firefox browser, you had to make two files, one MP3 and the other OGG. And for other browsers, you might have to add in an Apple codec (but we don't speak Apple here). You can read about Firefox support at https://developer.mozilla.org/en-US/docs/HTML/Supported_media_formats, but keep in mind that this is a moving target.

Very recently, Mozilla announced that they will support MP3 and pay the fee. MP3 costs money right now, OGG is free! The good news is that the MP3 patents will expire in a year or two.

Why does all this matter? If you have your music in MP3 and you convert it to OGG, the lossy audio will become more lossy. Right now, the music I download onto my ZTE Open will play even if it is MP3, but if I want to write a program in HTML5, my best bet is OGG. Firefox will let you program a WAV file, but the WAV file size is larger than OGG or MP3, and for a phone, you always want the smallest of everything.

So the point is, if you are making music for the Firefox OS, create your original file in WAV format (or one of the Apple formats), but at some point, convert it to OGG. But save your original file (lossless) so you can convert it later if you want to program with MP3.

My own personal guess is that OGG will be supported in Firefox OS for a long time, so its a safe bet. WAV is supported, but large file sizes will make it a poor option for phones and tablets. MP3 support will be soon, and I wouldn't be surprised to see in the future support for Apple's AAC, which is catching on fast (because it has smaller file sizes and better fidelity than MP3).

And don't forget, OGG is open source and very cool!

So if you think HTML5 audio is confusing, wait until you get to HTML5 video! But the answer is to start working with Web Audio (not the same as HTML5 Audio) which is supported. I'll be poking into that soon, but there's more pressing matters.

Oh, and I misspelled the cool audio workstation Ardour. Sorry, guys. Get it at http://ardour.org/. 

Globals

According to many wise JavaScript gurus, you should avoid globals. However, I find that I need them to make things simple for small game demos. So in the recent HTML5 Audio post at http://firefoxosgaming.blogspot.com/2013/12/html5-audio-game-programming_10.html,  I had this global definition:

      // Global variables
      var myAudio;

I don't assign it anything yet, because I need to wait until the page loads. This variable is used for the audio object and gets created in the audio constructor here, but only after the page loads:

      myAudio = new Audio("oggsong.ogg");

When working with objects and loading things, you always want to make sure that you know something is loaded before you use it. But also, in this case, if you don't make myAudio global, the function processMyAudio won't know where to find it.

And as part of the making sure, processMyAudio doesn't get called until canplaythrough tells us that the audio is now loaded and ready to go!

If you do these things without some of the safeguards, things might work or they might not. But you want to be sure. But if you don't make myAudio global, the music won't play. Using JavaScript functions is cool, but you want to make sure that everything inside the function can be used.

You can rewrite this program to avoid globals and pass things through function calls, but for now, I want to keep everything as simple as possible. If you want to know more about why globals are evil, check out JavaScript, the Good Parts. The book is difficult but worth reading several times, and I don't always agree with the author, but he clearly knows what he's talking about.

Here's a great picture of the two best books on JavaScript:

The thin book on the left is Crockford's JavaScript: The Good Parts. The thick book on the right is called JavaScript: The Definitive Guide by David Flanagan. Read both and you'll know most of what you need to know. Make sure you get the latest edition of Flanagan, because it gets updated every few years. Note the size difference of the two books, but there's lots of good parts in the bigger book also.

Firefox OS Simulator

The Firefox OS simulator (available to your Firefox browser as a plug-in) is really valuable for debugging audio. Something that runs just fine in the desktop browser may not work on the phone. Why do I bring this up?

If you use MP3 in the Firefox desktop browser, all works well. But not on the phone. If you use OGG, your audio works in the simulator. Before audio, I barely looked at the simulator unless there was a sizing issue, but for audio, it's crucial that the sound works at the simulator level.

What Might Not Be Working

I was surprised to see that two of the audio object's properties, duration and currentTime didn't work in Firefox OS when they work just fine in desktop Firefox. I haven't read all the browser source code yet, but I filed a bug 949129. You can check it out at https://bugzilla.mozilla.org/show_bug.cgi?id=949129. You can read my test code there.

By the way, if you find any kind of bug, make sure you can reproduce it, and file it at Bugzilla. Remember, YOU are Mozilla. Make it better and bang on it today. Especially with Firefox OS, which is still young and growing.

I'll keep you posted.

Next

I'm working on a work-around for the lack of currentTime. I'll also do some short code samples for loop and controls. And there's a sample game I want to get on to next, and maybe some game engines, and oh, yes, more game reviews. And vibration and tilt and oh, the list goes on forever!

Putting Music on your ZTE Open Phone (Hardware Review)

I like to listen to music on my phone and the ZTE Open is no exception. But just dropping files onto the folder created when you connect your phone to a computer doesn't always keep the music information, especially not the cover art.

So I poked around, tried some tag editors, and gave up. But I found a solution that works for me and may work for you if you have the following functionality:

1. Windows - preferably Windows 7, not 8, not XP, but just plain Windows 7.
2. MP3 files (I tried some OGG files and I couldn't get tag editor to work)
3. Windows Media Player (it comes with Windows)

As you may guess, Windows Media Player 12 doesn't support OGG files. You can get a plugin to play OGG files, but it doesn't do the tagging and you can't sync it with your phone. But since most of my files are MP3, I'm good to go. If someone has a non-WMP player for Windows that works with ZTE Open, please let me know. Solutions on Mac and Linux are cool too, and I'll post them or you can add them as a comment.

So here's the steps.

First hook up your ZTE Open phone to a PC with a standard USB cable (little plug in the phone, big plug in a PC USB port). If this works, you'll see the ZTE OPEN as a drive in File Manager. The ZTE Open may ask you if you want to connect for remote debugging.

Get your MP3 files together. Make sure you have cover art. If you don't have it or don't want it, Firefox OS will make different colored abstract shapes which are cool too.

Load them into Windows Media Player and import an album. You want to do this from the Library pane of the Player. You get to the library by clicking on the boxes at the top right of the main player pane. Here's an album I imported from a video game soundtrack called Legend of Dungeon. Notice that it has the cover art, names of the album, artist, and a track list.

Note that the ZTEOPEN is on the left as a place where Windows Media Player (WMP) can find files.

Next, drag your album over to the right where the Sync tab is open. It should say ZTEOPEN Linux and show a bar indicating how full the ZTEOPEN is at the moment. After you drag it over to the right, you should see something like this:




Next you want to click on the "Start Sync" button at the upper right. It will chug along and you should see something like this:

The right side shows the status of the synchronization. I took this screen shot at 24% completed.

After the sync, your player should look like this:

It should say "Sync Completed". If you want to check, you can actually click on the ZTEOPEN menu at the left and open it to see the Music on the phone. At the top it says "ZTEOPEN > Music > All music" and I scrolled up so you can see that Legend of Dungeon is now on the phone and has the proper cover art, track list, etc.  (By the way, I made an error and didn't rename the tracks. They say "1. Zombie" for the track name and I should have erased all the numbering on the track names because WMP keeps track of the track number (you can see it under #). Oh, well.)

Now, unplug the phone and open the music player on it. You should see something like this (if you have the same music -- unlikely!):

The Firefox OS player has four views which are on the menu at the bottom (from left to right):

1. Tiled view of all the music by album cover
2. Playlist view where you can select types of album (most played, recent, etc.).
3. Artist view - alphabetical with album covers
4. Album view - alphabetical with album covers (pictured above)

Once in a while I had a problem with the Windows Media Player recognizing the ZTE Open. Rebooting the player makes that work.

I'll look around to see if I can find another player that works with OGG. Or other formats. According to Mozilla, Firefox OS supports these file formats:

• .mp3
• .mp4, m4a (encoded with H.264 or AAC)
• .wav
• .ogg (encoded with Vorbis or Opus)
• .webm (endoded with Vorbis)
• .3gp

Every operating system and browser has a different list of files it supports, but almost everyone supports .mp3 and almost no one (except Windows) supports .wma. I'm not sure who supports FLAC. I've also heard that Firefox has problems with .wav files.

Stay tunes and let me know if you can get other music synchronizing to work with other players and operating systems. Keeping Firefox OS happy with cool music is certainly my goal!