My first Code Retreat - Legacy Code Retreat in Frankfurt on Sep 15, 2012

If you are not familiar with the concept of a code retreat, listen to this podcast (or read the transcript).

While I knew about Code Retreats for a while, this was the first I actually managed to attend). It was organized by the German Software Craftsmanship community group, hosted by Namics, and facilitated by Nicole Rauch and Andreas Leidig. And it was great, thanks to everyone involved in putting up the event!

The format has been described by others, so I won't cover that. I have to say though that I really like the format and I wish I started socializing (in software related matters) first at a code retreat instead of conferences or usergroups - the format of the event guarantees one doesn't have to worry about uncomfortable silences to be filled with smalltalk. The day starts with coding, the retrospective is group talk, and with the exception of the lunch, the breaks are only five minutes long, and you are searching for the next programming pair during that time anyway. Great way to get more comfortable interacting with strangers about software! (And if you do want to socialize, just come early for breakfast and stay after the event).

I wonder if being familiar with automated testing is a pre-requisite

My assumption is that one could attend a legacy code retreat even if (s)he has no experience with automated testing, since

• You could learn the basics of testing from the pairs you are working with
• You can see it applied in the real world. The most common objection I hear from people recently introduced to automated testing/TDD is that it might work on greenfield projects, but cannot be applied on their existing project

So if you are (or know of someone who is) a person who attended such a code retreat with no prior testing experience, please let me know - I would love to know whether the above hypothesis matches your experience! Unfortunately all my pairs had prior experience, so it's still just a hypothesis.

Iteration impressions, lessons learned

• Dynamic language IDEs still have a long way to go, so for now I'll probably stick to Vim for python
• While it's interesting to take a guided tour of a language you don't know, the focus of the codebase is not on datastructures (only uses lists/arrays) and thus you only catch a glimpse of the language. I'll have to attend a normal code retreat to see whether this would be different there
• Giving a language demo is interesting, and you learn a lot about the language too. People new to a language tend to ask questions about things you take for granted, yet you may not know the answer to
• Taking baby steps and not assuming anything is a Good Thing ™ - the codebase is devious one, crafted with care to make you trip over. I.e.: it is a proper legacy codebase, despite its small size!
• The "never assume" advice holds especially as you move between iterations. During one iteration we made a mistake that wasn't caught by the regression tests. Since in the previous iteration (with another pair) we had 100% (line) coverage, the fact that in the next iteration we might not have that didn't occur to me...
• Discipline is hard. I was totally carried away refactoring during the last iteration. I had this craving to actually make progress with the refactoring, and I caught myself saying things "were we responsible coders, we would now stop to write some tests, but let's just move on now", as well as tugging multiple pieces of the spaghetti at the same time. While here I might be forgiven (after all, the last iteration was a free to choose what to do (with) this codebase), it's an important reminder that I should watch myself at work - I would have never expected myself to get so off track in a matter of 10-15 minutes. And I used to pride myself that I realize when I'm in a dead end and have no trouble throwing away code to start from the last known good state!
• The code retreat format is great for teaching people the importance of prototypes, I will keep that in mind for the future. During the functional iteration we haven't made much progress, but on the train home I did a quick experiment to start making it functional from the outside in, starting at the main() method, introducing the GameState as a subclass of Game, and each step returning a new GameState (while still modifying the old game state, since the refactoring was incomplete, as it usually is the case). This approach didn't occur to me the first time, and had I not started from a clean slate, I would not have thought of it if I were to continue where I left off the previous attempt.
• While the facilitators keep going around, we didn't always get deep into the issues they commented on (e.g.: I think if the test case and the test name express clearly the domain and the scenario, it is totally fine to use a variable called sut, etc.).
• However, there is a lot of time available to discuss with your pair, not having to worry whether or not the code will be finished, which is great. One caveat is that you do have a time limit on the discussion, since you don't want to bore your pair and want to actually write code, so you are forced to condense your thoughts. Luckily, this limit is not as bad as twitter's
• Theory vs. practice, a.k.a. talking the talk vs. walking the walk. I've been guilty of this myself, describing how my ideal test case would look like in theory, and what guiding principles I follow while writing an actual test case. Then the pair politely points out that the theory is great, but what we have here in the code is not a manifestation of those principles...

The Iteration I wish was there - working towards a change request

Each iteration had a different focus, and I assume that there isn't a static final (pun intended) list of possible restrictions and it evolves. So despite this being my first ever code retreat and being told that these ideas wouldn't fit the format, I'm documenting them here, so that I can refer back to it after my next code retreats to see whether I still feel the same about these, since now I think they would be similar restrictions like during the traditional code retreat when one is not allowed to speak or use if statements in the code.

I really missed having a clear functional goal for the iterations, since one usually refactors legacy code when some new feature/enhancement is needed - and it has a huge impact on how one approaches a refactoring.

One mistake I have (seen) made when working with legacy code is going on a refactoring spree, touching parts of the codebase which we don't need to change. The danger of it is that we can easily code ourselves into a corner for days and slip on the original delivery. If it ain't broken, don't fix it (and this doesn't contradict the boyscout rule). This issue has been exposed during the iterations, many of us refactored one part of the application that wasn't business logic heavy, but was a low hanging fruit. While one iteration wouldn't be enough time to finish testing that part, the conversation around it (what test cases would be needed to provide sufficient code coverage, what's the minimum refactoring we need to do to achieve that, etc.).

I raised it during the final retrospective, and people agree it's an important aspect, but they suggested it's not fit for the format of the code retreat.

The other great benefit of having a clear goal is that they demonstrate how fragile the regression characterization tests can be. A neat little change request to the core business logic would have left us without the safety net again, and would have made us think back to the previous iterations when we felt skipping writing a specific test is safe. While everyone knows it, that doesn't mean we wouldn't fall victim to it..

And if you prefer to see a concrete example, instead of just reading through this abstract text, I have something like the Double Dawg Dare in mind.

Some technical notes for attending a code retreat:

• doublecheck with the organizers what you'll need to attend. They probably plan to send out a reminder/notification email before the event, but I so rarely use my laptop in an online environment that their notice was too late for me to actually prepare my laptop for the event.
• know your settings & IDE. There are a ton of yaks to be shaved, and many minutes have been wasted by setting things up. It doesn't take away from the experience, but it did stress me a bit the first time

• either know how to use git, or just create two copies of the codebase so you can easily revert to a clean codebase after the sessions. We had some problems with this.

  git clean -x -f -n # remove -n to really remove them git reset HEAD . # remove everything from the changelist in case you added it git checkout -- . # revert everything below the

• bring a USB stick, and if you are not using your own laptop during all the sessions, make a copy of the golden master textfile onto it after each of your sessions in a new programming language (my laptop was only used during the first and the last iteration, so for the last we had no sample output textfile we could work against, and it took some time to obtain it.

• bring your own keyboard and know how to change a mac/linux/windows machine's keyboard layout (or install one). I have not been typing in a number of sessions because of this (try typing on a German mac keyboard, when you are used to windows US layout!)

In Summary

It's a great event, you meet great people, and I would be surprised if you came away from a code retreat not having learnt anything new.

Data Migrations As Acceptance Tests

While I have previously said that on migration projects both verification and regression tests are important, does it mean that the two should be separate? Like first, let's migrate the data, and then we'll rewrite the functionality, right? Or let's do it the other way around - we'll talk with the customer, incrementally figure out their requirements, deliver the software (with a proper regression test suite) that satisfies them, and then we migrate. Both approaches have problems:

• customers want to use the software with their current, real data - having only the data and no application to use it with is no value to them. Neither is having only an application with no data in it


• real data has lots of surprising scenarios that the domain expert might have forgotten to specify (see caveats though)


• requirements are not static, and new ones will be introduced during the development process, that inevitably will cause the new application's models to change, which means that the migration has a moving target it needs to migrate to.

Doing them in parallel

If the data source is organized chronologically (order by date in the migration script), and organized in a format that resembles what the system's end users will enter into the system, then we can use the new application's outmost automatable entry point (Selenium, HTTP POST, a module's public API) to enter this data during the migration from the old system to the new.

Why

While a clear disadvantage of this approach is speed of the migration - it will be likely slower than an INSERT INTO new_db.new_table select .... FROM old_db.old_table join ... where .... statement, but in the case of non-trivial migrations it will likely compensate for the slowness, because:

• changes to the new system's code/data structure become a problem localized to the new application code - no headaches to update the migration scripts in addition to the code


• when the client requests the demo deployment to be in sync with the old system, the code is ready (spare for the part to figure out which records have changed)


• the legacy data edge cases provides focus - no need to invent corner cases, for there will be enough in the data


• likely there will be releasable functionality sooner than with either of the above approaches

How

First, create the acceptance tests for the migration:

• Pick the data to be migrated


• find the view in the original system that displays this data to the users and find a way to extract the data from there


• define the equivalent view in the new system (it's about end to end, user visible features!)


• write the verification script that compares the above two (be sure to list the offending records in the failure message!)


• define the data entry point in the new system


• write the migration script - extract from the old system, transform if needed (only to match the entry points expectations of the format - no quality verification as in classic ETL!), then send it into the new system (using the above defined entry point)

At this point both the new view, and the data entry points are empty. From here on, the TDD cycle becomes a nested loop

• run the migration script. See which record it failed for


• analyze the failing acceptance test, and find the missing features for it


• (A)TDD the missing features


• run the migration script to restart the cycle

Caveats

While the existing data makes one focus on the real edge cases instead of the imagined one, beware - not everything has to (or can be) migrated - for instance, in a payment system, the system used to accept many currencies in the past, but now only €. IN this case, possibly the currency exchange handling logic could be dropped in the new system (and just to store the currency in a char field for the old ones); or in some other domains, maybe only the last ten years' data is needed. However, this should be a business decision, not a decision for a developer!

Source Data Quality is often a problem, one that will likely cause issues. If data needs to be fixed (as above, ask the stakeholders!), it should stay out from your application's code, and be in the Transform part of the migration script.

Testing Strategy On Migration Projects

It's common for migration projects (a.k.a.: rewrite) to specify scope initially as "to behave the same way as the old system does". Thus the testing approach to automatically compare the new system's results to the old system's seems to be a perfect choice (of course, in addition to the data migration, calculations should be reconciled too - at least for the records that were calculated using the latest version of the old system).

However, once the application goes live and the old one is decommissioned, we cannot rely on these tests anymore (there is no old system anymore), and we have no regression suite to rely on for the future change requests/enhancements.

Just to avoid any misunderstanding: I don't advocate not having automated reconciliation checks (verifications), on the contrary, I think they are immensely valuable. We can write the best specifications/code, but still miss some details, which, luckily for us, do pop up in the real data. These automated checks give everyone on the project team the peace of mind they so need before go-live.

The point I'm trying to make here is that while these checks are essential, they are not enough for the long term health of the system. These are a good starting point, but just as we do with the specifications, as we work on the new system, when we find and uncovered logic case (e.g.: as part of the calculation reconciliation), we need to add a test case to the new application's test suite to ensure proper regression coverage that we can rely on in the years to come. And adding this test case is easy - we can just copypaste (after making it anonymous of course) the input that caused the problem with the verification into the unit/functional tests, implement the missing functionality, and move on. But saving on these few seconds costs a lot later down the road.

Unit/functional/integration/system tests are supposed to be self contained - we would like to create (a) clean database(s), which we put into a known state before the tests (some frameworks support this out of the box, e.g.: Django, but we can easily implement this ourselves). Migration reconciliations, by their very nature, need to work on the live (snapshot) data. Also, as described earlier, these reconciliation tests are temporary artifacts, while the other tests supposed to be permanent (at least until the client decides to change the requirements). Separation of Concerns also applies to the test suite - running tests in the same suite with different assumptions (live db we shouldn't touch vs. empty test db we can read/write as we wish) is more than risky - keep them physically separated, both at runtime and in source control.

Even if the delivered project could be summed up in that vague sentence (does what the old does), this summary is never true - few start rewrite projects just to get the exact same functionality. Usually these projects are sponsored because the old application became unmaintainable and the client is missing opportunities, because the software is not supporting, but hindering their goals. Without the self-contained test suite, our shiny new migrated application is going to become another one of these.