John Topley’s Weblog

I’m pleased to report that my AssetsGraphed Ruby on Rails application has been running continuously for over two hundred days now, as the screenshot below taken from my installation of monit shows. Okay, so AssetsGraphed isn’t exactly getting hammered like Facebook, but this level of reliability supports my decision to choose Rails Machine for hosting my Rails applications. Their choice of running Xen-based VPSs on Linux would appear to be a wise one and I know that 37signals themselves are moving in this direction.

The next milestone will be a year’s continuous uptime—let’s hope that posting this isn’t the kiss of death that takes AssetsGraphed offline! Happy New Year everyone.

I went to a great gig in London last Friday with my good buddy John Conners. In a bizarre twist on the traditional format for these things, it was actually a presentation on project management software for software teams, rather than a music gig. If you think that sounds boring, then you’d be right. Fortunately, the person giving the presentation was none other than A-list celebrity software blogger Joel Spolsky of Joel on Software fame. Joel and Fog Creek Software co-founder Michael Pryor were in town to promote the latest version (6.0) of FogBugz, which increasingly embodies Joel’s philosophies on different aspects of the software development process.

The venue was the conference centre at the British Library, which was jam-packed with geeks—mainly male, it has to be said. Joel gave his presentation which included a FogBugz demo, then the floor was opened to questions and finally Joel and Michael stuck around afterwards to answer questions on a more informal basis from the throngs of developers that surrounded them.

The most interesting new feature in FogBugz 6.0 is Evidence Based Scheduling (EBS). Joel himself would probably characterise it as a hand-wavy sounding feature, but from watching his demonstration it actually looks practical and useful. What it boils down to is this: FogBugz makes it as painless as possible for developers to record what they’re working on and then over time it uses statistical modelling techniques to generate fancy charts that tell you a percentage probably of when your team will ship, or how accurate individual developers have been at estimating their work. Joel blogged about the feature in depth a while ago.

FogBugz 6.0 also includes a handy wiki feature that serves as a central project documentation repository. What’s interesting to me about how they’ve implemented this is that it only has a WYSIWYG editing mode i.e. you can’t get at the markup that’s generated under the covers. I suspect that Fog Creek have learned from their experience with CityDesk here, which had all sorts of problems when you switched back and forth between the editor view and the markup view. Joel actually touched on this during the Q & A when someone asked if WYSIWYG is a leaky abstraction: the short answer is that it is.

What I found most useful from a FogBugz point of view was getting a leg-up on how to use the software effectively. I’ve been using the hosted version for a while for AssetsGraphed, but I knew that I wasn’t getting the most out of it. I had trouble coming to terms with the FogBugz model whereby everything is essentially a case within the system, even e-mail. That’s all a lot clearer now and I shall shortly be hooking up the AssetsGraphed contact screen to FogBugz, so that any feedback I receive from users is properly categorised and managed. After that I’ll hook up the crash reporting system, which means that new cases will get created automatically on those rare occasions when the application crashes, instead of the present system which simply e-mails me the details. The beauty of it being that FogBugz is smart enough to recognise when incoming crash details are the same as previous crash details and so won’t create lots of duplicate cases, but rather will append to an existing case.

I’ve also developed an appreciation of what a sophisticated AJAX application FogBugz now is! I think this is something of a well-kept secret because FogBugz doesn’t advertise its use of AJAX in a flashy way, it just makes the experience of using the software faster and more pleasant. One feature that I certainly was aware of from the previous version is the great keyboard support, which is far superior to any other web application I’ve seen. Just as one thing Microsoft got right with Windows from the start was the ability to drive the user interface using only a keyboard, you can get around FogBugz in the same manner.

Joel turned out to be exactly how I expected, which is to say an entertaining and interesting speaker who answered everyone’s questions with patience and charm. Talking of questions, I was a bit surprised that some people used the occasion to glean information that can be easily found on Fog Creek’s website. I guess there were quite a few people there who wanted to find out about FogBugz straight from the source rather than attending because they’re JoS fans. We certainly did find out all about the latest version of FogBugz and very impressive it is too. My only slight disappointment with the event was that there was no FogBugz World Tour 2007 merchandise for sale, such as T-shirts with a list of tour dates on the back. Not very rock ‘n’ roll, Joel!

As part of my process of continuous and sometimes arduous improvement of my AssetsGraphed web application, I’ve recently added slightly better localisation support. Localisation—sometimes called L10n—in the context of software means adding special features for a specific locale. It usually follows internationalisation (i18n), which is the full-blown support of different languages and cultures.

AssetsGraphed isn’t currently internationalised, and unless there is suddenly a massive demand for it to be, I have no plans on changing this. So the user interface text will remain in English and read from left to right, as that’s what I know. I also know that proper internationalisation is a bit more involved than that! What I have changed however is how the display of currencies is handled. As you might imagine this is a fairly significant part of a financial asset tracking application.

AssetsGraphed has supported multiple currencies from day one and users can select which one they want to use on the Settings screen. The example below shows an extract from the Assets screen with the U.S. Dollar as the currency:

That’s all well and good, but it’s not always appropriate to prefix the amount with the currency symbol, and not all cultures group digits using a comma. In France for example, they use a period. ISO 31-0 mandates a space character. To accommodate all this, the digit grouping symbol is now a per-user setting and as the system administrator I can control whether the currency symbol is displayed before or after the amount. Here’s what it looks like with Swiss Francs:

I just got a skeletal Ruby on Rails application running on a Java Virtual Machine using JRuby:

—This is running on Windows XP which is running in a Parallels VM which is running on my Mac. To recap: it’s Ruby on Java on Windows on Mac OS X. Who could have predicted that a few years ago! Next steps: hook up a database and try to deploy a Rails application to a servlet engine. I think this could be very useful in future!

I just noticed something that Flickr got exactly right—you don’t have to give photos a title. My process for using Flickr is usually:

1. Upload a batch of photos to my account, marked as private
2. Use the Organizr (did they drop that name?) to add the photos to a new set, or an existing one
3. Go through the new photos adding any individual tags as appropriate
4. Make the photos that aren’t really bad public and try to come up with a title

It’s that very last step that I find really increases the cognitive load; so much so that I often just give lots of photos the same title. I can’t quite bring myself to have public photos with titles like IMG_0970.JPG. I got to thinking that I never had to give physical photographs a title when I put them in an album, so why should Flickr make me do so? Then I tried it, found out that I don’t have to and smiled. Expect to see lots of anonymous photos in my Flickr photostream in future.

I recently added Shaun Inman’s superb Mint statistics package to my AssetGraphed Rails Machine installation. As the installation wasn’t particularly straightforward, I thought I’d write this little guide for others who may be struggling. I should point out that the real credit for the information here goes to Bradley Taylor from Rails Machine who was extremely helpful and persistent.

First of all, I should point out that these instructions cover installing Mint under your /public directory i.e. yoursite.tld/mint/ rather than the subdomain installation method (mint.yoursite.tld). Also note that → indicates that the line has a break in it in the instructions, but in reality it should all be on one line. With that out of the way, here’s what you need to do:

1. Install PHP on your Rails Machine using these instructions. You should have no problems with them if you follow them exactly.
2. Log in to your Rails Machine as your deploy user then become root by using su - root and entering your root password.
3. Edit /etc/httpd/conf/httpd.conf and search for DirectoryIndex index.html. Add index.php to the end if it’s not present as shown below and save the file:
   
DirectoryIndex index.html index.php
4. Edit /etc/httpd/conf/apps/yourapp.conf and add the following immediately after the RewriteEngine On line:
   
RewriteRule ^/mint/$ /mint/index.php [QSA]
5. Now find the three lines shown below:
   
# Redirect all non-static requests to cluster
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$→
balancer://yourapp_cluster%{REQUEST_URI} [P,QSA,L]—And add the following line immediately before them:

   # Don't redirect requests to /mint
RewriteCond %{REQUEST_URI} !^/mint/(.*)$

6. The full rewrite rules should look like this:
   
RewriteEngine On
RewriteRule ^/mint/?$ /mint/index.php [QSA]# Prevent access to .svn directories
RewriteRule ^(.*/)?\.svn/ - [F,L]
ErrorDocument 403 "Access Forbidden"

   # Check for maintenance file and redirect all requests
RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$ /system/maintenance.html [L]

   # Rewrite index to check for static
RewriteRule ^/$ /index.html [QSA]

   # Rewrite to check for Rails cached page
RewriteRule ^([^.]+)$ $1.html [QSA]

   # Don't redirect requests to /mint
RewriteCond %{REQUEST_URI} !^/mint/(.*)$

   # Redirect all non-static requests to cluster
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$→
balancer://yourapp_cluster%{REQUEST_URI} [P,QSA,L]

7. Save the file. It’s probably read-only. If you edited it using vi then you can save it even though it’s read-only by using :wq!
8. Finally, restart Apache by using /sbin/service httpd reload and log out completely from your Rails Machine.

You can now install Mint itself by adding it to your working copy directory under /public and adding it to your Subversion repository using svn add ./public/mint. Once you’ve deployed to your Rails Machine you’ll be able to complete the Mint installation by visiting http://yoursite.tld/mint/ and following the instructions.

P.S. I have spent the past several hours wrestling with WordPress trying to get the markup in this post to validate, but I have conceded defeat!

You may recall that a while ago I mentioned The Rails Way, which is a site where Rails core team members Jamis Buck and Michael Koziarski review code submissions and illustrate Rails best practices. Well, the big news is that they’ve started reviewing my code! I submitted a subset of the code from the AssetsGraphed application and the first article about it was posted yesterday. Incidentally, in case you’re wondering what I didn’t submit, it’s the code for the Reports feature and for the hidden administration interface that I use to manage users and currencies within the system etc.

The first part of the series is about removing a lot of duplicated code around the incomings and outgoings that each asset has. I always knew that having so much identical code was totally evil, but I wasn’t quite sure how to go about removing it. As is often the way with these things, it does seem kind of obvious now! I have to admit that it’s quite nerve-wracking having my early fumblings with Ruby and Rails subjected to public scrutiny, but it’s also a great way to learn, which is why I’m doing it. I intend to implement all of the improvements featured, although some of them may be more challenging to do than meets the eye because I have the added problem of having to preserve the existing data. I don’t know how many articles are going to be in the series, but there’s certainly lots of areas that I know could be done better.

I’ve also just launched the AssetsGraphed Forums where you can post any feature requests or bug reports or just generally hang out. I’m using Josh Goebel’s and Rick Olson’s Beast forum software. Go take a look!

I’ve just finished moving AssetsGraphed over to Rails Machine. The application was originally hosted by TextDrive, on the same server as this site. I don’t have any complaints about TextDrive but it was clear that I needed something more substantial for AssetsGraphed. I have lifetime hosting with TextDrive which is fine for websites but problematic for Ruby on Rails applications, as you’re limited to 48 MB resident memory and 80 MB virtual memory. If your application exceeds those limits—which are pretty generous for a shared hosting environment—then your Ruby process gets terminated without warning and it’s goodnight to your app. This was proving to be a problem for AssetsGraphed, which is fairly memory intensive due to the image processing required for the graphs.

I did consider TextDrive’s other plans, in particular their Accelerator Hosting plans whereby you get a proportion of a Solaris box and root access. It sounded great but a bit too DIY for me. I don’t mind the odd bit of system administration but I’d rather be writing code with my developer’s hat on. Enter Rails Machine. I’d read good things about them and watched the demo movie on their site. The thing that appealed to me was that they make it really very easy to set everything up and deploy your application, leaving you free to spend your brain power on development. You use their Rails Machine RubyGem and custom Capistrano recipe and it all just works, as if by magic.

I went for a three VPS set up, as shown below. One virtual server is given over entirely to the AssetsGraphed Rails application, a second one will host the forthcoming AssetsGraphed forum and blog and the third is the MySQL database server that all the applications use. It’s a nice, tidy set up:

One of the things that I didn’t appreciate from the Rails Machine site is that the don’t provide any DNS services, so you have to bring your own. I signed up for the cheapest account from DNS Made Easy and all I had to do was go into my GoDaddy domain control panel and enter the names of the nameservers that DNS Made Easy provided me with. Then, in the DNS Made Easy control panel I set up the records for the assetsgraphed.com domain so that they pointed to the IP addresses of my Rails Machine virtual servers.

Because I went for a multiple VPS arrangement, I did have a few configuration problems where my set up deviated from the instructions on the Rails Machine site, which assumes a single VPS per app. For example, something that caught me out was that when I initially set up the Rails Machine for AssetsGraphed, the assetsgraphed.com domain was still pointing to TextDrive, so I set the domain variable in the Capistrano recipe (deploy.rb) to the non-friendly server address assigned to me by Rails Machine. When I did switch the domain over and accessed the application, all I saw was a Welcome to Rails Machine! holding page. That was fixed by setting the domain variable to assetsgraphed.com and then running:

cap setup_web
cap reload_web

—To configure Apache correctly. I should stress that I got excellent help with this and the other issues that I had by dropping in to the Rails Machine Campfire chat and talking directly to Bradley Taylor, who has always been friendly and quick to help, no matter what else he was occupied with at the time!

If you come to Rails Machine from other hosting providers like I did, then you may find it a bit bare-bones at first. As well as no DNS service, there’s no Web-based control panel. It’s all driven through Capistrano and the command line. However, don’t let that put you off because once you wrap your brain around it then it all works extremely well. What you do get for your money is a high performance Rails hosting solution, with the emphasis being on solution. All the hard work of setting up a Subversion repository, importing your code into it and deploying and running your application has been done for you. Highly recommended.

I’ve just put my first Ruby on Rails application online. It’s called AssetsGraphed. It’s a free asset tracking application that also graphs your data. I created it as a vehicle for learning Rails and also because I needed a better handle on where my money goes!

AssetsGraphed lets you create assets and give them a starting balance. An asset can be anything financial that you want to track over time. For example, that dream holiday that you’re saving up for, a bank account, a retirement fund or even your pocket money. You enter incomings or outgoings against each asset and each incoming or outgoing is assigned a category, which might be something like Deposit for an incoming or Electricity Bill for an outgoing. You have complete control over your own categories. Oh, you can select which currency you want to do all this in too. There are eight to choose from, although it’s easy for me to add more. I made it default to the U.S. Dollar because that’s the nearest we have to a global currency. Don’t be offended if it’s not your currency, just select your own on the Settings screen.

The main AssetsGraphed screen gives an overview of all your assets and you can click them to drill down into the detail. Sparkline graphs plot the trends in the incomings, outgoings and overall balance for the asset. You can view or share asset activity by subscribing to its Atom feed. Yesterday I added a Reports page that for now just contains one report. It’s a pretty useful one though as it gives a breakdown of income or expenditure by category, either for all your assets or individual ones. So you can see the movement of your money very easily.

It’s important to understand that because AssetsGraphed is a learning exercise, it’s not a bulletproof web application. That means that it might crash, although I’ve been eating my own dog food on AssetsGraphed for just over a month now and it seems pretty solid. If it does crash on you then I’ll automatically get an e-mail about it and will investigate. You might find it a bit slow too. That’s because it’s actually running on the same shared server as this blog. I’m currently weighing up various options for moving it to a much more powerful hosting environment. I might even add SSL support too.

AssetsGraphed is just the first step on my journey to learn Ruby and Rails. There are lots of bits under the covers that I know could be done in better ways. I shall keep improving it as I learn more. There’s a really great community around Rails. I’d like to thank the Rails Weenie folks for patiently answering my questions and also Khoi Vinh for inspiring me with his Boxes and Arrows colour palette and letting me re-use it on AssetsGraphed.

Now that you’ve learned about AssetsGraphed and you’ve read the disclaimers, why not give it a try and let me know what you think?

I’ve been thinking about Web applications lately and in a U-turn worthy of an unpopular politician have revised my opinion of them. Regular readers may recall that previously I’ve bemoaned the poor user experience offered by Web apps in comparison to their desktop cousins. That’s still true—although not as bad as it used to be—but I just don’t think it matters any more, because Web apps are good enough for what most people want to do most of the time. Of course there are exceptions, for example I don’t think we’ll be seeing Photoshop rewritten using PHP (or even Rails!) any time soon. I don’t see computationally and GUI intensive software leaving the desktop whilst the way the Internet comes down the pipe is architected as it is.

Thanks to developments like AJAX (the widespread adoption of XMLHttpRequest, not the invention of the buzzword), we can now make Web applications that start to approach desktop-bound ones in terms of responsiveness and all-round slickness of experience. In fact, if we’re not careful there are a whole variety of ways now at our disposal that we as developers can break the user’s expectations of how things ought to work. By the way, AJAX should really just be named “A”, because you can use XMLHttpRequest from VBScript if you’re mad enough, and JSON or even text works just as well as XML for the response format. What’s more, the whole shebang can even be invoked synchronously. All of which goes to show that the AJAX moniker is entirely without merit apart from being catchy and widespread, and in the end that’s all that matters because it’s not important what the label is, so long as everyone has a common understanding of what it represents.

It’s been fascinating watching the growth over the past year or so of frameworks geared towards making Web application development easier, particularly in the A(JAX) arena. There’s so much stuff for developers out there now that simply wasn’t available this time last year. Kids today don’t know how lucky they are. And a lot of it is good stuff too, with decent documentation and testing behind it. Step forward the Yahoo! User Interface Library and probably the Google Web Toolkit too.

All this musing on Web apps is all well and good but I’m actually putting my money where my mouth is and paying monthly fees to rent the use of several online applications. Namely Backpack, Flickr, Strongspace and Wufoo in addition to the ones that I use for free: Basecamp, Gmail and Google Calendar. I know that I’m going to be hosed if my broadband goes down or if any of these apps disappears—is anyone else worried that 365 Main have a large data centre in San Francisco?—but I have more confidence in the ability of Google and 37 Signals et al to back up my data than I have in my own ability to do so on a regular basis. Besides, the sheer convenience of being able to access my data that’s held across these services from anywhere there’s a broadband Internet connection has so far outweighed any downsides. I’ve surprised myself by how I’ve gone for this in such a big way. It doesn’t bother me in the least that I don’t have a CD-ROM to show for my money. I’ve still got the installation media for software that I’ve bought over the years and all it does is gather dust and waste space. Also, I don’t consider my data to be interesting enough to get all paranoid about.

I think it’s interesting to take a moment to reflect on some of the ways that Web applications have no equivalent in the desktop world:

• You get to them using URLs rather than by having your OS launch an executable file. URLs can be typed directly, bookmarked or linked to from somewhere else. They can also be mistyped and you might end up somewhere else entirely. Imagine how bizarre it would be if you thought you were starting Excel and you ended up running some program for browsing extra-large clothing!
• URLs can be chopped about. Properly designed Web applications should support URLs that can be hacked. Getting around desktop applications is totally different in that you’re offered a finite number of things that you can click on to get to some place else. You don’t see the internals of how they’re stitched together.
• If you’re not careful, users might bookmark your Web application when it’s in an inconsistent state. Short of doing something unspeakable to their browser, you can’t stop them from hitting Ctrl + D when you least expect it.
• The entry point to commercial Web applications is also usually a Website that’s geared towards selling that application.
• There’s no consistency within the user interfaces of different Web applications. Sometimes you click buttons for actions, sometimes an image made to look like a button and sometimes it’s a hyperlink. Having said that, consistency now seems to be a dirty word amongst desktop applications too. Office 2007 has a brilliant UI that’s totally non-standard and Apple just can’t decide how they want their bundled OS X apps to look.
• There’s no desktop equivalent of a 404 error page!

In spite of all these oddities and potential problems, I think the model of letting software companies take the strain when it comes to not only developing applications, but also to installing, updating and securing them and of backing them up is a good fit for how most people want to use computers. Let’s face it, people like things that just work without too much thought being required on their behalf. We’re all busy enough already.

I didn’t believe Sun Microsystems when I first came across their slogan that The Network is the Computer, but it looks like they might actually turn out to be right to a large extent. Unfortunately for Sun they didn’t figure on most providers of Web applications doing so using commodity hardware and open-source software, but Sun’s loss is our gain as both developers and users.