There Is More To Clean Code Than Clean Code

A post written by Uncle Bob in January (I'm behind my reading list) offended me. I absolutely agree with Uncle Bob's analysis regarding the code itself, and I also prefer the refactored version, but I have a problem with insulting the programmer(s) reluctant to appreciate the change.

We write code in programming languages, and there are different levels of proficiency in a language.

As I'm currently learning a new spoken language, I'm painfully aware of this - initially I probably sounded like a caveman. The first impression you get about me is totally different depending on the language I speak - but I am the same person!

The learning curve of a language is not smooth - the steepness between consecutive levels of proficiency is different. Going from not speaking any German to speaking A1 (tourist) level was easy, getting the basic grammar required for the low intermediate (B1) level wasn't too bad, but to get my German to the level where my English is will take more effort than the sum of all my previous investments¹.

Since it is my third foreign language I'm learning, I have no difficulty accepting that the level I think I speak is higher than the level I actually speak. Because of that, whenever someone rephrases my sentences in proper German ², I start from the assumption that likely their version is better, even if I don't understand first why - and I take the effort to understand their reasoning ³. I do that despite that I was of course convinced that when I spoke, I expressed my thoughts in the best possible way.

However, I don't have much at stake - no ego to hurt, no reputation to loose, and the roles are clear: I'm the beginner, and the people around me are the more experienced ones. In a software team, the roles might not be so clear - I had told colleagues almost twice my age how they should write code after only a few weeks of working there. Bad idea. Since then, I have learned not to start improving the coding style of a team by rewriting the code they have written, and showing off how much better my version is. Rather, I wait until a situation arises when they don't mind having me demonstrate some code improvements. I demo it, and explain why I do it that way. In my experience, the second approach is more effective, though it doesn't have that instant satisfaction and relief the first provides.

As the joke goes, you need only one psychologist to change a light bulb, but the light bulb has to want the change real bad.

Driving Technical Change is hard, because it requires a mental/cultural change, and that change has to come from the inside - but can be catalyzed from the outside of course⁴. But just forcing practices or ways of working on unwilling recipients generates resistance (e.g.: the story of the EU technocrat appointed to recalculate the 2009 Greek budget).

I would like to see more public code reviews and public refactorings (e.g.: Andrew Parker, GeePawHill), but I would like to see less public judgement passing on people at the lower proficiency levels of programming.

---

¹ there is a great Hanselminutes episode on learning a foreign language if interested. Beware, it may contain programming!

² German readers might disagree, since most Germans I meet speak Frankish :)

³ Which of course, is sometimes harder for natives to properly explain than for novices to ask questions pointing out the seeming irregularities of the grammar

⁴ And we won't always be able to foster change in all environments (note: this does not mean the others are at fault for not changing!). The same programmer can be highly productive in one team, and be the one slowing down another team. There is nothing wrong with changing jobs after realizing we are a net loss to a given team.

Data Migrations As Acceptance Tests

While I have previously said that on migration projects both verification and regression tests are important, does it mean that the two should be separate? Like first, let's migrate the data, and then we'll rewrite the functionality, right? Or let's do it the other way around - we'll talk with the customer, incrementally figure out their requirements, deliver the software (with a proper regression test suite) that satisfies them, and then we migrate. Both approaches have problems:

• customers want to use the software with their current, real data - having only the data and no application to use it with is no value to them. Neither is having only an application with no data in it


• real data has lots of surprising scenarios that the domain expert might have forgotten to specify (see caveats though)


• requirements are not static, and new ones will be introduced during the development process, that inevitably will cause the new application's models to change, which means that the migration has a moving target it needs to migrate to.

Doing them in parallel

If the data source is organized chronologically (order by date in the migration script), and organized in a format that resembles what the system's end users will enter into the system, then we can use the new application's outmost automatable entry point (Selenium, HTTP POST, a module's public API) to enter this data during the migration from the old system to the new.

Why

While a clear disadvantage of this approach is speed of the migration - it will be likely slower than an INSERT INTO new_db.new_table select .... FROM old_db.old_table join ... where .... statement, but in the case of non-trivial migrations it will likely compensate for the slowness, because:

• changes to the new system's code/data structure become a problem localized to the new application code - no headaches to update the migration scripts in addition to the code


• when the client requests the demo deployment to be in sync with the old system, the code is ready (spare for the part to figure out which records have changed)


• the legacy data edge cases provides focus - no need to invent corner cases, for there will be enough in the data


• likely there will be releasable functionality sooner than with either of the above approaches

How

First, create the acceptance tests for the migration:

• Pick the data to be migrated


• find the view in the original system that displays this data to the users and find a way to extract the data from there


• define the equivalent view in the new system (it's about end to end, user visible features!)


• write the verification script that compares the above two (be sure to list the offending records in the failure message!)


• define the data entry point in the new system


• write the migration script - extract from the old system, transform if needed (only to match the entry points expectations of the format - no quality verification as in classic ETL!), then send it into the new system (using the above defined entry point)

At this point both the new view, and the data entry points are empty. From here on, the TDD cycle becomes a nested loop

• run the migration script. See which record it failed for


• analyze the failing acceptance test, and find the missing features for it


• (A)TDD the missing features


• run the migration script to restart the cycle

Caveats

While the existing data makes one focus on the real edge cases instead of the imagined one, beware - not everything has to (or can be) migrated - for instance, in a payment system, the system used to accept many currencies in the past, but now only €. IN this case, possibly the currency exchange handling logic could be dropped in the new system (and just to store the currency in a char field for the old ones); or in some other domains, maybe only the last ten years' data is needed. However, this should be a business decision, not a decision for a developer!

Source Data Quality is often a problem, one that will likely cause issues. If data needs to be fixed (as above, ask the stakeholders!), it should stay out from your application's code, and be in the Transform part of the migration script.