Wednesday, January 8, 2014

Asynchronous Storage (Game Programming)

Recently, I wrote this post (http://firefoxosgaming.blogspot.com/2014/01/local-storage-game-programming.html) on Local Storage, a simple way to store data on your phone between sessions. Simple, easy, works on all browsers. Or so I thought! But storage turns out to be more complicated.



I got some comments from Gabriele Svelto and Dietrich Ayala. They told me that Local Storage is not recommended for Firefox OS. Since they are both on the FxOS team, that's good enough for me. Turns out that for Firefox OS, Local Storage is a performance and I/O problem.

This isn't terribly surprising, giving how critical computing is on a phone with limited resources. One thing I've learned over the years is that synchronous programming can be a problem. What is synchronous programming? It's where you make a call and wait until the call is returned. That's an oversimplified answer, but when things are tight, you don't want to wait.

So if you use Local Storage, your call will take place and nothing else can happen while you are doing it. That can be fine if your calls are small and there's nothing else going on and you have plenty of resources. But you can't afford to have your phone freeze up.

The opposite of synchronous programming is ... wait for it, asynchronous programming. What a difference that first letter of the alphabet makes (if you're using the English alphabet - for the tinkopar, it would be t). Asynchronous programming says that you can make a call and then go do something else. You register a callback (a separate function) and when your call goes through, the function will respond and take action. How cool is that?

So Local Storage is out. Don't use it on Firefox OS. What's the alternative? If you are doing any kind of serious game, you want to let the player save their game for another day. And memorizing codes went out with old game consoles. Well, there's another way to save data, and its name is IndexedDB. This is a more recent way to save data, and is asynchronous. If you want to know more about it, see this page on MDN https://developer.mozilla.org/en-US/docs/IndexedDB. This has been around in Firefox for a long time, but other browsers were slow to catch up and Safari still isn't there at all and some of the other mobile browsers are AWOL as well. But it works in Firefox OS and works well.

However, the API for IndexedDB is a bit more complicated. It uses real database concepts and you need to understand how it works. It uses transactions and the DOM, but not SQL (I hate SQL, don't you?) Read more about it here on MDN: https://developer.mozilla.org/en-US/docs/IndexedDB/Basic_Concepts_Behind_IndexedDB

So you can't use Local Storage and you may not want to use IndexedDB if you're starting out. What is a game programmer to do?

Riding to the rescue of overwhelmed developers everywhere, Mozilla has made a compromise API that is almost as simple as Local Storage, but uses IndexedDB under the hood. This technology comes in the form of a small JavaScript library called async_storage.js. You can get it here: https://github.com/mozilla-b2g/gaia/blob/master/shared/js/async_storage.js. Copy it all and save it as a file named async_storage.js. But before you do, make sure you understand that this is open source under the Apache license and that you follow all the rules. If you don't know those rules, read them here: https://github.com/mozilla-b2g/gaia/blob/master/LICENSE.

Once you've got your async_storage.js file prepared, save it to your project. To use this cool piece of software, insert this line into your web page:

    <script src="async_storage.js"></script>

If the async_storage.js file isn't in your root, change the line above to tell the packager where it is.

What does this new asynchronous storage API do? Pretty much exactly what Local Storage did. It uses the same concepts and names, but with one small difference. Every method must include an extra function call as a callback. And there are no properties now, only methods.

Maybe an example would be nice. I'll use the same example I used in my Local Storage post: http://firefoxosgaming.blogspot.com/2014/01/local-storage-game-programming.html. Here's what the UI looks like.



Enter a key and value in the top two boxes and touch Save to ... save. Retrieve your data with a key you enter in the third box and touch Retrieve. Clear the storage with Clear and get the number of key/value pairs by touching the Number of Items button. Each action will produce a message at the top of the screen (where it says "Watch this space!"

And here's the new code, run and tested on my brand new shiny Geeksphone Peak.

<!DOCTYPE HTML>
<html>
 
  <head>
    <meta charset="UTF-8">
    <title>
      Async Storage
    </title>
   
    <script src="async_storage.js"></script>
 
    <script type="text/javascript">

        // Load event listener
      window.addEventListener("load", getLoaded, false);
       
      // Function called on page load.
      function getLoaded() { 
     
        // Save button listener
        myButtonSave.addEventListener(
          "mousedown", myClickSave,false);

        // Retrieve button listener
        myButtonLoad.addEventListener(
          "mousedown", myClickLoad,false);
         
        // Clear button listener
        myButtonClear.addEventListener(
          "mousedown", myStorageClear,false);
         
        // Length button listener
        myButtonLength.addEventListener(
          "mousedown", myStorageLength,false);       
         
        // Background color
        document.body.style.backgroundColor = "GreenYellow";

            // Output to console.
        console.log("The page is loaded.");
      }
     
      // Save button event handler
      function myClickSave() {
     
        // Save data with key.
        asyncStorage.setItem(myKey.value, myData.value,
          function() {
            console.log("New value stored.");
            info.innerHTML = "Input - Key: " + myKey.value +
              " Value: " + myData.value;      
        });     
      }
     
      // Retrieve button event handler
      function myClickLoad() {
     
        // Get data from key.
        asyncStorage.getItem(getKey.value,
          function(myValue) {
            console.log("The value is ", myValue);
            myValue = "Retrieve - Key: " + getKey.value +
              " Value: " + myValue;          
            info.innerHTML = myValue;
        });           
      }
     
      // Clear button event handler.
      function myStorageClear() {
     
        // Clear the storage.
        asyncStorage.clear(function() {
          console.log("Storage cleared!");
          info.innerHTML = "Storage cleared!";
        });
      }
 
      // Number of Items button event handler.
      function myStorageLength() {

        // Get the number of items in storage.
        asyncStorage.length(function(myLength) {
          console.log("The number of items is ",  length);
          info.innerHTML = "Items: " + myLength;        
        });      
      }
      
    </script>
  </head>
 
  <body>
 
    <p id="info">Watch this space!</p>
 
    <p> Enter key here</p>
    <input id="myKey" autofocus>
    <br>
   
    <p> Enter data here</p>
    <input id="myData">
    <br>
   
    <button id="myButtonSave">Save</button>
   
    <p> Retrieve data with this key</p>
    <input id="getKey"><br>
   
    <button id="myButtonLoad">Retrieve</button>
    <br><br>
   
    <button id="myButtonClear">Clear</button>
    <br><br>
   
    <button id="myButtonLength">Number of Items</button>

  </body>

</html>


Short and simple. Enter some key/value pairs, retrieve them, add more, count them, and clear. If you don't clear the database, and you shut down and come back tomorrow, the data will still be there!

The API details are actually in the JavaScript file, but here's a quick summary:

setItem - saves an item with a key/value pair.

getItem - gets an item value with a specific key.

removeItem - removes an item with a specific key.

clear - clears the database.

length - tells you how many key/value pairs are in the database.

key - gives you the value at a specific location in the database.

Note on the key API: the order stored isn't necessary guaranteed, so use this only to loop through all the items.

Sounds pretty much like the Local Storage API. But there are two differences:

1. You must use the asyncStorage object for all your calls and attach your methods to it.

2. Each method must have an additional parameter that specifies the function that will be called when the asynchronous call is done.

This last one is cool and not that hard. Here's an example:

Instead of typing:

        asyncStorage.clear();

You must type:

       asyncStorage.clear(myClearCallback);

Then create a function called myClearCallback() that will be called when the asynchronous call returns. You don't need to call it myClearCallback, any function name is fine as long as it is unique and has a function to go to.

If you don't want to create a separate function for your callback, you can use an anonymous function. Anonymous functions look like this and you use them inside the parameter. Here is an anonymous function:

       function() {
          // Stuff here you want to have happen
          // when the asynchronous call returns.
       }

It looks like this if you insert it into the asyncStorage.clear function:


        asyncStorage.clear(function() {
          console.log("Storage cleared!");
          info.innerHTML = "Storage cleared!";
        });
 

This displays a console.log message and a UI message when the asynchronous call returns.

The syntax is a bit overcrowded with punctuation, but you can always have a separate callback function if you want. By the way, callbacks are called by event listeners, I just haven't called them callbacks before.

This is a little different from programming you may have seen, in that you can no longer type something like:

       myValue = localStorage(key);

Values are not returned directly. That would be synchronous! So you must insert a callback function in your parameters. If the localStorage didn't have any parameters (like clear and length), you have to add one (and length is now a function, not a property). If localStorage had parameters, add one more!

I used anonymous functions for my four functions. Here is how each bit looks:

setItem

        // Save data with key.
        asyncStorage.setItem(myKey.value, myData.value,
          function() {
            console.log("New value stored.");
            info.innerHTML = "Input - Key: " + myKey.value +
              " Value: " + myData.value;      
        }); 
    

getItem

        // Get data from key.
        asyncStorage.getItem(getKey.value,
          function(myValue) {
            console.log("The value is ", myValue);
            myValue = "Retrieve - Key: " + getKey.value +
              " Value: " + myValue;          
            info.innerHTML = myValue;
        });       
    

clear

        // Clear the storage.
        asyncStorage.clear(function() {
          console.log("Storage cleared!");
          info.innerHTML = "Storage cleared!";
        });


length

        // Get the number of items in storage.
        asyncStorage.length(function(myLength) {
          console.log("The number of items is ",  length);
          info.innerHTML = "Items: " + myLength;        
        });  


myKey  and myData are defined by their input boxes.

So this isn't really that hard to use and lets you be asynchronous with your data. The folks at Mozilla were geniuses to think of a way to be asynchronous with an API that was synchronous. All you need to do is load in the tiny library and add a callback for each function you use.

NOTE: The callbacks are not required for setItem() and clear(). But I recommend using them so you can be sure your items are saved or cleared. Remember the poster on Mulder's wall: "Trust No One." You will need the callbacks for getItem() and length() because otherwise you won't get the data you requested.

NOTE: After more discussion, the callbacks don't tell you much more than something worked. If it didn't work, something very wrong happened. But usually they work. If you need more confidence about success and failure, check out my post on IndexedDB at http://firefoxosgaming.blogspot.com/2014/01/indexeddb-game-programming.html. async_storage.js uses IndexedDB under the hood so if you want more than simple, check out IndexedDB.