Never again

mega-pr-3

mega-pr-1

2 mega PRs! They removed a lot of cruft and code debt, and created a huge amount of review debt. (The first one includes changes from its sub-PRs—the second PR above, and another small PR. Its own changed file count was ~140)

I could have split the second PR into three sub-PRs. But the old code was so deeply woven-in that each of those three sub-PRs would also have had 50-70 changed files. The choice was really between 4-5 large PRs with 50-70 files changed, or 2 massive PRs with 100+ changed files. I’m not sure I made the correct choice. I don’t know if there was a correct choice. Perhaps the correct choice was what was—just leave it be.

The relevant code was messy, it is now less messy. It’s still fairly noodley. I just hope this work made it easier to improve for the next person who has a go at it.

My lesson: never ever again do I want to create such massive PRs. Neither do I ever want to be at the other end of such massive PRs—reviewing them.

Continue reading

Online/Offline Daemon

Completed a big task on Friday, though not the way I wanted it – background, auto synchronisation of notes (including changes, deletions, etc) between server and browser app.

HTML5 has introduced the powerful features that provide in-built features for managing actions such as synchronisation when the browser detects an internet connection.

[sourcecode language=”javascript”]
if(navigator.online){
//do sync here
} else {
//store data offline
}

document.body.addEventListener("online", function () {
// do sync here
}, false);

document.body.addEventListener("online", function () {
// store data offline
}, false);

[/sourcecode]

Unfortunately, these features aren’t completely supported, or even uniformly implemented across browsers and, thus, are hardly usable in the current state.

All server requests in my app are implemented through xhr. So, I’m tapping into it and using this workaround:

  1. If an xhr call fails, an alternate function is called which:
  1. Sets a global variable to ‘offline
  2. Calls equivalent local functions which edit/create/delete/list from localStorage
  3. Starts a daemon using setInterval() that checks every sends a ‘Ping‘ xhr request to check if a server connection is available again
  • When the daemon can connect to server, it:
    1. Sets the global variable to ‘online
    2. Starts a function that syncs all local changes to server and then downloads a fresh list from server
    3. Kills the daemon using clearInterval() call

    There are quite a few problems with this approach. The ones I’m worrying about right now are:

    1. Memory, CPU and battery usage by that artificial daemon, specially on mobile devices.
    2. Unnecessary load on the server from handling those Ping requests, even though there’ll be only one successful one for every connect.

    Though, compared to the browser ‘online / offline’ properties, this approach has one advantage too – it can also handle instances when the internet connectivity is up but my server / app is down.

    One change I’ll probably be doing is fetching only the header instead of pinging the app and checking result – It’s faster and might be lighter on the server (need to check this bit). Got the idea for this from this blog post.

    [sourcecode language=”javascript”]
    x.open(
    // requesting the headers is faster, and just enough
    "HEAD",
    // append a random string to the current hostname,
    // to make sure we’re not hitting the cache
    "//" + window.location.hostname + "/?rand=" + Math.random(),
    // make a synchronous request
    false
    );
    [/sourcecode]

    Still, that doesn’t make up for a native browser capability to tell the app when the network connection has gone down. Waiting for it…

    Continue reading

    A night of suffering

    The refactoring continues.

    After moving all xhr calls from JSON to html, backed by django templating on the server, realised that in order to implement offline, localStorage based system, I still need to generate html at the browser. I could have taken the old, working version with html generating via js strings but it will need another load of work when the design of page starts changing. So, spent the night researching browser-based templating systems. Have shortlisted three – mustache, Closure and Pure. All seem to be taking strange approaches. So far Closure seems to be the one I’ll go with, but the final call will happen tomorrow after some more research.

    Meanwhile, the pen drive finally seems to have filled up, so can’t work off it anymore. Have saved all that I wanted to in dropbox and in bookmark syncs. Gonna reformat and recreate the live USB now so I can start working again tomorrow. The new laptop still isn’t featured on dell website though people have started talking about it on twitter. Seems like it’ll be another week or so before I finally get my hands on it and can create a full development environment. The new liveUSB should last till then, I hope.

    Time to sleep now. Ciao.

    Another code restructuring /

    Yesterday I restructured (I think they call it refactoring) all the code to integrate it into a single html template and a single python class. Also moved all calls to POST so nothing is visible, and editable, from the URL.

    Today, I realise I may have to restructure (refactor) the application again. Yesterday’s restructuring was to bring in simplicity and order. This time it is required for the ‘offline webapp’ bit. This is what happens when you use the learn as you go (cross the bridge when we get to it) approach.

    Looks like I’ve got another few long hours of boring, error-prone, code restructuring ahead of me :/