Cloud Computing – Jack Scott's Blog

Xero Accounting

In July this year I embarked on a business venture with a few friends (something I’ll talk about more in a later blog post). All businesses need good accounting software – it’s how you know if you’re actually making money (and how you keep the tax office happy). I’ve used Quicken/QuickBooks and MYOB (both the desktop Accounting and FirstAccounts, as well as the online LiveAccounts) and they all leave a few things to be desired. The desktop versions of MYOB, and the versions of Quicken software that I’ve tried all have a very 1995 feel to them… and no surprise too, I don’t think the software has had a major overhaul since then. LiveAccounts worked ok for me, but it just wasn’t polished enough.

Enter Xero, a New Zealand-based company that specialises in online accounting software. I have to say, I’m impressed. We’ve been using their business product since July, and it’s great. The interface is really easy to use, I had no trouble with it at all and I’m certainly not experienced in business accounting. It hides all the details you usually don’t want, but makes them available when you do (as opposed to MYOB, which just throws every single detail ever at you and expects you to cope). The bank feeds are super reliable, updating with the latest data every night and presenting it ready to reconcile (and if a transaction doesn’t already exist, creating it is trivial). To top it all off, the Android app is incredibly polished, with a lot of thought (an example: logging in with email and password takes forever on a phone keyboard, but you don’t want to be logged in all the time, so it prompts you to create and use a 4-digit PIN).

Recently they announced automatic bank feed support for their personal product, which I had tried previously but given up on because it takes too much effort to manually transfer the data over. I’ve been using it for a couple of weeks, and while I’m impressed with the product generally, there are a few things that I find disappointing:

Bank feeds aren’t as reliable as in the business product. For some reason they don’t use the same functionality. This (instead of talking to the bank through an API) appears to log in to you Internet banking portal using your username and password (points off for that) and downloads the information by manually parsing the HTML. Which only seems to work about 50% of the time, and is usually three days behind what Internet banking shows you when you visit in your browser. The bank feeds work great in the business product, why not leverage on that for the personal product?
It’s not double-entry. This annoys me. For 90% of people single entry is ok, but it doesn’t give you the detail that double-entry does. When I transfer money from one account to another, it doesn’t appear as a transfer; it appears as a withdrawal from one and a deposit in the other. Minor, but annoying.
When entering values of assets and liabilities to calculate net worth, it doesn’t show you history of those items, so it’s difficult to track the value of a single item over time.

Basically, I’d love Xero Personal to be like a browser version of Quicken Personal Plus, but I think that may be a dream too far.

Why I Have A Home Server

It’s pretty much impossible to use a computer these days without also using the Internet. It’s also pretty much impossible to use the Internet without using a cloud service of some kind. Most people I know depend on cloud services entirely, but not me. There are several good reasons I have my own servers, including my own home server.

It’s a learning experience. This is certainly one for the geeks, but hey, I’m a geek. By running my own servers I learn about the building blocks of the Internet. I’m a professional systems administrator, and my own home environment is a good place for me to try out things that I don’t get to try at work, or don’t have time to. Part of IT is constantly learning, and that’s what I try to do.

I can run whatever software I want. I’m not limited by whatever Google decides to put into Gmail. I can run my own Exchange server if I want (I do). It may not be free software, but it gives me huge advantages in syncing between devices. If I want to try something out, I just can.

My own privacy is assured. I don’t have to trust my email provider that they aren’t reading my emails or looking through my online backups. I only have to trust myself with my data, and if you can’t trust yourself, who can you trust? I don’t have anything to hide, but I think we should value privacy far more than most people currently do. After listening to Jacob Appelbaum at linux.conf.au in January 2012, I’m assured of this.

I run backups to my own server, and for geographic protection send self-encrypted files to the cloud. I use GPG to encrypt my data, and so should you. I know DropBox and other like services say they encrypt your data so they can’t read it, but how would you ever know?

I will admit that running a home server can be more expensive than trusting the cloud with all my data, as I have to pay for hardware (I spend about $500 a year just on server hardware, but you could spend much less), for power, for a static IP address, and for software licensing (I spend $450 a year here, but with free software I could spend much less).

All in all, running my own home server gives me great satisfaction, confidence in my own abilities, more freedom and more privacy, at the expense of some time (though now it’s up and running, I probably do 10 minutes of maintenance a month) and a bit of cash. Not a bad deal.

The Ten Commandments of Computer Backups

I recently put a lot of thought into how I perform my computer backups. I’m one of those people that, while I would only be mildly pissed off by the failure of a hard drive, would be quite angry at myself if I lost even the merest hint of data that I wanted to keep. I used to perform my backups manually, using the Windows backup utility to back data up onto an external hard drive. It worked fine most of the time, but it definitely had process defects… the largest of which being, I had to remember to do it. It required my interaction to succeed (because I had to plug the drive in) and this meant there was always a human element involved. And humans are lazy.

So I set about designing myself the ultimate foolproof backup system. There would be multiple storage media, there would be encryption, there would be checks and validations and several custom-written applications. Then I started thinking, “what exactly am I protecting myself against?” It’s a good question. Here’s the list I came up with:

I need my data to be safe from storage media failure. This may mean a single backup DVD being unreadable, or maybe my primary hard disk drives it’s head into the sand.
I need my data to be safe from the failure of every drive of a particular type, simultaneously. It happens more than you would think, and the consequences usually aren’t pretty (whole RAID arrays failing, with all their ‘safe’ data, usually makes people a bit upset).
I need to make sure my data can’t be stolen. If it is stolen (or people I don’t want reading my data try to do so) then it should appear as meaningless gibberish.
I need my data to be safe from being corrupted while in storage, or while being transferred between storage devices.
My data needs to be safe from theft or fire, which could mean every storage device in a particular location is unusable.
My data needs to be safe from natural disasters, which could take out an entire city or state. Unlikely, but it’s the kind of thing most people don’t plan for.
I need to be able to search for data that I’ve accidentally deleted, and I need back.
If my data is anywhere not under my direct control, I need to be able to trust the people who do control them.
I have to assume that if my backup hasn’t been tested (i.e. I haven’t tried to restore from it) then the backup isn’t any good.
Finally, I shouldn’t have to do anything… computers should be smart enough these days to back themselves up.

That was all I could think of, though I’m sure there are additional points (leave a comment or email me, please!). Then I figured out what I had to do in order to prevent these situations from happening.

Points one and two are the easiest to solve, and are really what most people think of when they think of “backup” plans. The solution is simple: keep your data on multiple storage media, and those different storage media should be different types.
Point three is pretty simple to solve: encrypt everything you can possibly encrypt. This also partially side-steps point eight, because if your data is encrypted, you don’t have to trust them to not read it, you only have to trust them to not delete it. And you don’t need to trust them to not delete it if you’ve got the data in multiple locations (i.e. somewhere not under their control).
Point four can be partially solved by taking checksums of the data (which can be done at the same time it is encrypted). If a checksum doesn’t match, something has gone wrong and should be tried again or looked at by a human. There is the issue of what happens if the original data is corrupted. I put this in the too-hard basket for now, though the use of a RAID array can reduce the likelihood of this.
Points five and six are closely related, and also solved together. Every good backup plan should make use of off-site backups, where a copy of data is kept away from the original. Point five might mean keeping a copy in another building (or in my case, at my parent’s house a few kilometres away). Point six means I might consider going further. Ideally I’d like to store a copy of my data on another continent, just in case of nuclear war. If I survive, my data should too.
Point seven means I should be creating archives of data, so that copies of old files are kept so that I can go back in time. I would like to be able to choose copies from every day for a week, then every week for a year. After a year, I’m probably not going to remember that I once had a file.
Points nine and ten are quite possibly the trickiest. To solve them, I have to write automatic scripts to do all these backup tasks, then write automatic scripts to try recovering from the data and make sure it’s in perfect state. I also need to do this manually, just in case my scripts stop working (it is a computer, after all).

So there was my analysis of the backup problem done. Now for the design stage. My current working computer systems consist of a laptop (running Windows 7), a desktop (dual-booting Windows 7 and Debian GNU/Linux), and my home server (which runs Debian GNU/Linux). So I chose to do the following:

I decided that, since it was turned on all the time, my home server would be the primary location for all my treasured data. Every other location for my data would feed off that. My laptop and my desktop will be synchronised to my server using software such as rsync running on a very frequent schedule. Ideally I will code a switch into the script on my laptop that does syncs less often when I’m not at home, to avoid wasting bandwidth. This will give me three or four working copies of my data, depending on how implementation goes.
My server has two hard drives, and I’m going to use this to my advantage. The first hard drive has my primary working copy of data, and the second drive is where the backups go. So I’ll write another script that will take my working copies from my first hard drive, perform archival on them (using tar), encrypt them and checksum them (using encryption that money can’t buy) and copy them to my second hard drive. This gives me the ability to go back in time through my data, if need be. At this stage there are some things I won’t backup, either for legal reasons (I’m fairly sure the MP3 backups of my music collection shouldn’t be stored off-site under Australian law) or for practical reasons (videos are just too large to transfer off-site over the Internet).
I still haven’t solved the problem of off-site backups. To solve this, I’m planning to make use of Amazon S3, which is a cloud backup solution offered by everybody’s ~~favourite~~ ~~friendly~~ forgettable online book store, Amazon. Because my data has now been encrypted, I don’t have to trust them at all. I can just copy it across, mark it as being invisible to the wider world, and forget about it. I will also take up an offer from my friend Jamie to store my data on his NAS, which gives me another off-site backup location. I’m in Tasmania, Jamie is in Queensland, Amazon is in the U.S.A., and my data is safe.
I’m also planning to fit my server with a DVD burner and write a script that backs up my most crucial data (such as financial information and treasured memories) onto a DVD every week or so. Encrypted, of course. The only problem is that I need to remember to go and change the DVD over every week.
Finally, I have to write scripts to occasionally check the consistency of my data, so that nothing suffers from bit rot.

I haven’t completed the process of implementation yet (in fact I’ve hardly started). Already though, I feel safer knowing that I’ve thought about the process of storing my data, and that makes me feel a lot safer. Most people don’t think about backups until it’s too late, and perhaps maybe they should.