Offline App Development

… is a pain.

Mainly because once you add a file to the manifest, and thus to the browser’s app cache, you’re never sure if it’s getting updated. I spent most of last night and much of today just trying to get the browser to fetch the new version of html doc, and then the correct js file. Too bugged. It seems to be an issue primarily with Chrome & Chromium, but they are my primary development platform.

Anyway, if despite all the changes to manifest, restarting servers, clearing browser cache, the browser is still fetching old cached files, here’s a way out:

  1. Open a new tab in chrome/chromium and go to ‘chrome://appcache-internals/
  2. Clear all cached app resources
  3. Refresh the page to confirm there are no more cached app resources
  4. Refresh your test page – it should fetch everything from the server now

There’s another bit I learnt and implemented (though my implementation is pretty simplistic): Dynamic Cache Manifest.

Basically, instead of writing the application manifest, for offline app storage, by hand and changing it every time you change a file, let your server-side script do the work. What I’ve done is :

  1. Hand-wrote a cache manifest file but with a comment line that includes a django tag {{ts}}.
  2. Changed my app.yaml so that instead of serving the file directly, the request for it is sent to my server script.
  3. In the script I use the handwritten cache manifest and replace the ts tag with a random number.

What this ensures is that the browser is forced to fetch all elements in the app manifest every time because of the random number changing cache manifest on every fetch. This ensures that while I’m connected to the server and testing my js scripts, I get the latest versions every time. On the other hand, when I switch off the server and test the offline mode, the script still has resources available offline for rendering.

There is a lot more you can do here with the dynamic cache manifest rather than just plug-in a random number. I learnt this trick from this blog at Cube, and there are a couple more handy tricks being used there, so I suggest reading that resource.

Update: Seems like, as with script-initiated xhr requests, Chrome also makes multiple requests for the cache manifest from the server for every page load. When I use a randomizer function to send a ‘new’ cache manifest every time, this results in the two cache manifests being different to each other and thus Chrome doesn’t save any files. Not what I wanted, now? So, here’s a small change I made

previous randomizer: int(random.random()*10000)
new randomizer: time.strftime(‘%y%m%d%H%M’)

Now, instead of returning a new resource every time, I return a new resource every minute. This means that the two simultaneous requests that Chrome fires, will get the same cache manifest. But a page refresh/load later, will return a new cache manifest. It’s working so far :)
[Ofcourse, I know that there’s still a remote chance that the two requests will hit either side of the minute change. But the probability is small enough for me to take a chance.]

POST using xmlhttprequest

Struggled with this all of yesterday and earlier today. Finally, discovered what seems to be a bug in the implementation (or something that specs missed):

When passing arguments / parameters with a POST request, the javascript: xmlHttpRequest function seems to truncate/modify the first and last arguments.

So, now I’m sandwiching the actual arguments between empty buffer parameters and its working.

Can’t believe how many hours I’ve wasted on this one issue, and that no one addresses it anywhere on the web.

Anyway, the whole app is now working via ajax, and, as a result, is lightening fast. Now, need to include offline storage of files (manifest) and data (localStorage). Before that, wondering if I should do a split version and test passing the whole formatted html div instead of just JSON objects in response to xhr requests.

The Rockstar Weekend

Learnt more, importantly more practical stuff, about ajax, xhr and how to use it within Google App Engine from this post. Then went about restructuring (refactoring?) the code on both sides – client & server – to enable it. Didn’t require much work on the server-side. During the previous round of restructuring, I’d enabled single url calling with a sorter function spreading requests to relevant functions. That helped since I’m now just calling the same function for xhr calls and returning the results as a JSON, instead of a formatted HTML page.

On the client side, had to redo nearly the whole code again. The main HTML is now just a plain page with a header and a body div. Everything is written into it by js functions. This has meant I can’t use the powerful django templates on server-side to generate that html. And that’s a pain. Still thinking if I should just give up and instead of returning the JSON objects, return the formatted html code for relevant div in response. Might try it too and then decide on the outcome based on whichever approach works faster and lighter. Anyway, using the xhr-JSON method, have already got the GET functions to work. The two POST functions are still standing out. Was working on them when Rags woke up and interrupted my work day. Still, glad that I woke up at 4 and immediately got to work. Gave me 6-7 hours clean before she decided to rock my day.

The rest of weekend was almost wasted in work terms. Saw Rockstar, the hindi movie, in the evening. Loved it. Then met JD&R for dinner in Chinatown. Came home and, after briefly checking G+, twitter & techmeme, dropped dead on the bed.

Today’s been uber-lazy. Woke up at 12 after 11 hours of sleep but got out of bed another 90 mins later. That too when Rags brutally pushed me outta bed. Spent the rest of day watching F1, listening to Rockstar OST over and over again, and reading articles over at gigaom and businessinsider. Researched the biggest player in CldNts industry right now. Followed their founder CEO on twitter. Also followed Naval Ravikant, the angellist founder, on twitter. Fella seems interesting. Finally read a businessinsider story on Ron Conway. Scary.

Another code restructuring /

Yesterday I restructured (I think they call it refactoring) all the code to integrate it into a single html template and a single python class. Also moved all calls to POST so nothing is visible, and editable, from the URL.

Today, I realise I may have to restructure (refactor) the application again. Yesterday’s restructuring was to bring in simplicity and order. This time it is required for the ‘offline webapp’ bit. This is what happens when you use the learn as you go (cross the bridge when we get to it) approach.

Looks like I’ve got another few long hours of boring, error-prone, code restructuring ahead of me :/