Ripping the heart out

Building a large application always comes with a lot of problems. One of the most important issues is usually that the implementation doesn't match the way it is used anymore.

This can be solved in two ways: The short term simple way, which is to simply ignore it, or the always dreaded refactor. About 10 days ago (including weekend that is), we were at such a point.

A little background information on the problem might be insightful. At the heart of our application are the so-called events. These events are simple homecare related actions that have to be performed at some point in the future. This can for example be delivering and / or helping a client with his or her medication.

All the events are generated by more general rules that state when certain care has to be provided for, most of the time based on weekly or fortnight intervals. These generated events are what Moves uses to its scheduling.

Well, back to 10 days ago. We were still working with the first version that models all this behavior and it was showing its problems. Events could be generated at multiple points in the system and none of us had a clear view of the inner workings of the system anymore.

Because the major part of the work that was assigned to me involved trying to optimize and improve this model, I dared to ask the always dreaded "Why?" question. At a certain point I started wondering why a certain design choice was made as this choice was making the situation very complex and slow.

I asked Andre and Bart and together we thought about the problem and how it could be solved. On our way back home we were still discussing it in the car and decided to stop developing for the upcoming Moves 1.0 and discuss the entire model the next morning. That night at the new Harry Potter movie, both Andre and I couldn't get the ideas out of our heads (we did enjoy the movie though).

After a good night sleep, we were all ready to do some good thinking. After quite some discussion we came to the conclusion that we only had the two options already mentioned above. Either deal with the current situation and put a huge legacy burden on ourselves, or just refactor the most important and complex part of the system.

Just to make matters a bit more complicated, we had an upcoming deadline for the first completely working version of our product, which made the decision even harder. Because the choice was do it now or never, we had to go for now. Ignoring the situation was an even bigger no no, because of the huge amount of problems that would occur in the future.

So there we were... 10 days to go and ready to delete a couple of thousand lines of code... After branching our new version (so we could go back in case of disaster), we went for it. It took us only 6 days to get the new branch in a reasonable condition, so we merged it back to the trunk. No matter what happened, we were not going back to the old situation. We liked the new model way too much :).

And here we are, just delivered our first version :). That means we made it, and I can say that we are pretty confident that the system is working much better with the new event model in place.

What we've learned from this experience is that you should never wait too long with refactoring a piece of your system if it gets in your way. It will get even more in your way if you let it stay. Rails is pretty forgiving in ripping a part of the system out and ActiveRecord lets you stay pretty flexible. In our case it went much better than we could have hoped for.

So, after 1.0 its time to rip the next part out :). The actual drag & drop interface we use to couple all the events from the clients to the employees has some very nasty properties. It couldn't be done before 1.0, but luckily it didn't have to. It's only the interface and javascript side of the system and it simply has to work the same way, only nicer for the developer.