Not so long time ago I released an application on my mobile broadband blog called Mobile Broadband Logging Monitor. It’s purpose is to help non-techincal users diagnose and discover if they’re affected by the heavy logging problems, that some of the big mobile broadband connections softwares suffer from.

The following story is about the creation of this piece of relatively simple software, and I’d like to share some of the thought I’ve had while making it.

First, a small question. What’s the difference between the applications shown in the pictures below?

Ok, I was cheating for theatrical effect. The pictures are the same. Read more, and you’ll know why I asked.

The story of the software begins with the issues about mobile broadband connection clients logging way too much information. I had made plenty of blog postings on what the issues were and how to solve them, but I wanted to make a diagnostic tool for everyone to use so that they could see if they were affected or not.

So, having thought for a while creating this software for a while and while feeling inspired, I sat down and started coding. I had no formal specification for the software, as I usually have on my job, but of course, I had some goals. The software had to be able to diagnose several different types of mobile broadband software, in various versions, and display their file activity to the user. It furthermore had to display some simple statistical information, the activity pr. minute and forecasted activity pr. hour and work day.

I didn’t take long before have the famous “working protype”. Yes, I could start the app, and it would be able to diagnose HUAWEI Mobile Partner and two versions of Birdstep Techonolgy’s EasyConnect. Golden!

Now, from a feature perspective I was done. The first version of the application had these features:

* Start button – starts the monitoring of the mobile software.
* Stop button – stops the monitoring.
* If the software in question has a fix described in the blog, show a link to that blog post.
* If it is the first time the user start the aplication, show a thank-you-for-downloading web page.
* If user clicks links to blog or feedback page, show the pages accordingly.

I put the software online and people started downloading.

But from other perspectives than the (admittingly vague) requirements, I wasn’t done at all. I had created an application below the norm in many ways if this had this been in a professional context (in my opinion). The application had one window and all the logic of the application was contained in the code behind of that window. This meant lack of testibility (well, there was no testibility at all!). Also, I had ideas for adding more features and even exposing the core application logic (finding and monitoring the software) as an API for others to use.

I decided to take on refactoring the application big time, without adding any new features to the application. The goals for the application were:

* Make most of the application logic covered by non-integration unit tests.
* Following the above, to have the part of the application which interacts with the “runtime” and/or external systems made as “thin” as possible, making it very easy to code these interactions and also ensure that they behave correctly in unit tests.
* Compose the application with components, that have clear purposes and one concern pr. component only.
* To open up for injectibility and IOC, hopefully to make use of Castle Windsor or the like in the future.
* Open up for making the application Open Source and thus expose the inner workings of the application (people might be hesitant to download arbitrary software from blogs, I know I would)

The goals for me personally was:

* To make it “right”. No bullshit.
* To make use of the many inputs I’ve had over the years, once and for all, making the software a sort of mini experiment of how it could be done in a professional situation.
* To be able to add new features in a structured way instead of the trial-and-error way that the software invited to in its current implementation.
* Not necessarily use the last tools or frameworks, but at least use sensible implemenation patterns what would make structured change easy.

Oh yes, big goals in small scale! :o)

The following are the stages of refactoring I went through with the application. If you don’t want the bloody details, I suggest you skip the description of these stages, and go directly to the “You had a question, Mr. Ostengaard?” section.

Stage 0
This was the working prototype that I started out with. I uploaded the software to Microsoft’s Codeplex. You can browse the code here.

Stage 1
In this stage I seperated a lot of the applications logic into components (ie. objects) with well defined purposes. The application only had one really structured part so far, which was about what mobile software the application supported. I could add new types of software pretty easy, even in the first edition. The rest of the application, however, was all code behind spaghetti voodoo. I started to identify the concerns of the code. You know, identifying these things in retrospect is not an easy task. I took a lot of time, and you start to see how entangled things really are when you code like this: Functionality for seaching for the software, starting / ending monitoring, UI state management, reading from configuration, handling events from directory watcher, collecting simple statistics and cross-thread UI updates. There’s was a little of everything, not much, but enough to make the identification of concerns pretty fun ;o)

I ended up creating some components and introduced interfaces for each of these, and every component got their dependencies injected (I chose the constructor strategy). You can browse the code of this stage here.

Stage 2
At this stage, efter going through the refactoring of business logic, I turned my focus on the UI. I had most of the business logic of the application seperated from the UI logic now. Not all, but almost. There was still some things, such as reading from the configuration, opening web pages and such in the UI code behind. But I was really really thirsty for more. I wanted to have the UI logic testible through unit tests in a non-integrated way too. I wanted full control over everything that the application did, at least as far as possible.

Now, when I had my own company I was involved in developing a Windows Forms application. At that time, I used a simple MVC framework to create views and sub-views and their corresponding controllers. The implementation had its forces and its drawbacks. But since then, I hadn’t done anything so structured in Windows Forms, and therefore I had some inherit mistrust for the strategy of that tiem. No need to not learn from the past, right? ;o) So, after having poked around for a while and tested some strategies, and honestly being a bit confused about which strategy to use, I ended up with the MVP pattern. Many thanks to input from Steve Bohlen on that, you have a beer waiting in Copenhagen.

But here’s an interesting thing: When you have the ambition to cover as much logic as possible in non-integrated unit tests, transforming traditional UI logic into testible code is quite an eye opener. You have to ask yourself what a dependency really is? In the case of UI logic, it’s can be the many things that you usually just take for granted in a quick-and-dirty Windows Forms application: Reading configuration files, displaying MessageBox (and child windows) and in this case also opening web pages. These simple things now became obstacles for my unit testing goals and something had to be done about them. I therefore made components to handle these things, disregarding the sense of overkill at the time. This would, hoverever, when creating unit tests, turn out to be a very good idea: Everything was injectible, everything was mockable.

You can browse the code of this stage here.

On a side note, the same is true for UI logic in .Net web applications. Traditional code behind is not very testible, so if you want to have that control over your software, similar strategies has to be taken into use.

Stage 3
At this point there were still some things in the business logic that were not covered by unit tests, especially the parts handling statistics. And to make that testable I had to control the applications sense of time. Just as writing “new SomeComponent();” is avoided in the name of injectibility and testibility, so is writing “DateTime.Now;”. If you use DateTime.Now in your code you have code that is less testable because your code is dependant on a very fluent component, namely the system time of the computer. How would you for example test a time-based statistics component when you cannot control time? It is possible in some ways, but if you want solid results and the test to run in short time, you’ll have a challange. Would you choose to have your unit tests run in real-time? That’s way too counter productive – these non-integration unit test has to be blazingly fast and give quick feedback to the developer - and because of this, time has to be encapsulated as well and made injectible. The IDateTimeContainer was introduced which contains the method GetNow() returning nothing more but DateTime.Now.

In this stage a new project was included in the solution. Until now the business logic had been contained in the same project as the UI, but with 100% seperation. Now I moved the logic into its own project (API), to ensure this seperation fully and also to make the core logic available without any relation to the UI.

You can browse the code of this stage here.

Final product
After going through all these stages I still had an application with no new features. But it was much better structured and had the qualities that I wanted: thoroughly tested through unit tests, and open for structured and controlled change. 

New versions
After this rearrangement of code I have relesed a new edition of the software. Change was adding a new child window to the mani window. The child window and its related presenter, and even the changes in the presenter for the main window, was added with ease. It was really a joy doing that in a controlled invironment.

You had a question, Mr. Ostengaard?
Now, the headline of this blog post asked “What’s the lesson of this story?" I ask because I actually want your input on what part of this whole story you think is the actual success story. I can think of some viewpoints, maybe you can think of others.

* One viewpoint could be that creating a “working prototype”, like I basically did here, is fair enough, as long as you have enough dicipline to throw the original code away or a least invest a proper amount of time restructuring it before adding new features. 

* Another viewpoint could be that things has to be thoroughly analyzed and components identified (whatever that means) before going into development. That would mean getting many of the benefits from unit tests and structured change from the early beginning. But it also means that the customer has to pay more up front and wait longer before gettign return on investment. (I’m not going to go into numbers here.)

* A third viewpoint could be not to focus so much on unit testing and other forms of control and feedback at all, and just fulfill the requirements and have testers assert that the requirements have been fulfilled. When the application get past these tests, the release is considered a success.

I ask the question above and put forward these three examples of viewpoints, because I am surrounded with people who carries (roughly) one of these three viewpoints and discussions often about this often occur. Some will say that the early release is the way for success (early business value), others will say the pre-study and analysis must be done, and others simply don’t have a clue about structured and quality-oriented application production (or simply don’t have it as a business goal).

The first viewpoint is good for early business value, I agree, but the problem with this strategy is that the application is born with a quite large techincal debt. But I have never heard about a customer (or software consultant house for that matter), who would pay for 4-10 times as much time as it took to create the original version to (re-)make it into a structured application and this without gaining basically any new features for that money. Have you? And if that doesn’t happen, all you have done is lured yourself and taken the first step in following the third viewpoint, where technical debt is not considered bad. But to me, the third strategy can never be the way to go. It will end up in a system, that carries so many problems, that change in the end will be seen as evil. Change will be avoided. It’s on a death march...

I hope that you can see, that I am, in most cases, a proponent of the second viewpoint. I can live with the first viewpoint, if actions are taken to rearrange the software. Unfortunately I do believe that many people start out by going with the first strategy, then skipping/postponing the restructuring, only to later find themselves trapped far into to third strategy with almost no possibility of returning.

And if you still wonder about the pictures, the difference between the two applications lies in the quality of the implementation and openness to change in the future. One sucks, one doesn’t.

Tell me something from your part of the world… :-)