PASS Book Review by: Grant Fritchey
Rating:   (5 out of 5)
 

Overview

Refactoring as it relates to Agile (Scrum, Iterative, Team, Extreme, Rapid, etc.) development techniques is well documented with huge amounts of information on how, when and where to apply individual refactorings. This information is collected in theory and in specific language and tool documentation. Most people refer back to Martin Fowler’s Refactoring: Improving the Design of Existing Code as one the earliest complete references on the theories involved in refactoring. With all this accumulated information, theory and practice, there are almost no books on applying Agile methodologies to the database. Enter Scott Ambler and Pramod Sadalage. Mr. Ambler’s first book on Agile database design and development was called Agile Database Techniques. It discussed the theory & some application of agile development and how best to incorporate your database work into that framework. This book takes the theory and turns into a long detailed discussion of how, exactly, to go about implementing refactoring within your database, designing build & testing methodologies around the database and finally rolling your accumulated changes out to production, safely and efficiently.

The book efficiently outlines how to apply refactoring within your agile development framework. The first four chapters define process around establishing development and test environments, choosing and applying refactorings, testing methods, and deployments. Chapter five takes the time to establish business process around getting your changes out to the community that you’re supporting. Then the book changes gears, radically, and spends the next six chapters outlining specific refactorings and transformations, grouped by purpose. These refactorings are each outlined with a motivation, why would you want to refactor, tradeoffs caused by the refactor, and specific methods for updating the schema, migrating the data and modifiying access code. Basically, everything you need to implement your own refactors within an Agile development methodology.

The book is clearly written and very detail oriented. It’s not specific to any language although examples are written in Java. Unless you’re already a refactoring export, you’ll want to read the first five chapters, completely, in order. The last six you can read or not, but keep the book handy as a reference. You’ll need to look up individual refactorings on a regular basis.

Target Audience

Anyone involved in database development that is either trying to incorporate it into an existing Agile process or you’re looking to reap the benefits of Agile development by getting a process started. Developers and technical project leads will learn what’s involved in creating incremental change within a database environment, which just isn’t as easy as making the same changes within code. DBA’s involved in development can pick up the methods they need to run with their development brothers. DBA’s who’s primary responsibility falls into maintaining a production system will get the information they need to know what’s coming, how it’s developed and recommended practices around implementing it.

Strengths

The main strength of the book comes from the clear expression of the authors' experience with the topic. These are not merely ivory tower academics suggesting a better way to develop applications who have never had to deal with deadlines, business users and all the assorted headaches that add to the challenges of the development process. They clearly speak from the stand point of someone who has been in the trenches. The back half of the book, with all the various refactorings and their associated steps in terms of data changes, etc., will be invaluable to anyone implementing these types of processes.

Weaknesses

Since the book is writen for a "database" audience and not specifically to a SQL Server audience, some of the suggested methods & approaches may be a bit off. Nothing was egregious or even stands out in my memory specifically. I simply recall reading and once or twice thinking to myself, well, I understand what they mean, but that's not how I'd go about doing it. Honestly though, that's the worst thing I can think of to say about the book.

Chapter Breakdown

1: Evolutionary Database Development

A few pages explore the theory behind Agile development processes in general and refactoring in particular. These few pages are quickly left behind as we move right into describing the architecture and processes needed to implement iterative change to a database. These include such topics as automated regression testing, configuration management, and developer sandboxes. The quick discussion on the impediments to implementation of these processes is some of the more important in the whole book.

2: Database Refactoring

This chapter goes through the what, how and why of refactoring a database. The authors take the time to outline what will be needed for implementing these methods within a nice simple system, one database, one application, and very complex systems, multiple applications hitting multiple databases. They outline for the first time database smells, or categories, of refactoring.

3: The Process of Database Refactoring

Using a test case refactor, Move Column, the authors walk you through the implementation of a basic refactor. They start from the basics of verifying that you should refactor at all (does the change make sense, is it needed now, is it worth the effort) and move on to choosing the correct refactoring. Deprecation, planned obsolescence, with a specified transition period and methods for maintaining old structures while application changes are implemented is defined. The most important part of implementing an Agile process is testing. As they put it in the book, “Test Before, During and After.” You can only know that the change introduced will work if you have tests that you can run at every stage of the process. As you may expect, since this is the most important part of the process, it’s also the hardest to implement. While the author’s don’t get into details on tool sets (for SQL Server, TSQLUnit available on SourceForge or the new Visual Studio Team Development for Database Professionals available from Microsoft are your two best choices). The book walks through the basics of implementing the change, migrating the source data, refactoring external programs as needed, and running regression tests. Another aspect of implementing these methods is source control. You must be able to recreate versions of your database at a moment in time. This means getting all the structures, not just the stored procedures, into source control management (Visual SourceSafe or Team Foundation Services are the most common for Microsoft SQL Server shops).

4: Deploying to Production

This chapter has a great deal of meat if you’re trying to implement this approach to development. You’ll need to define methods for moving your code between environments (Sandbox to Integration, Integration to Testing, Testing to Staging, Staging to Production, or whatever your own process will be). Bundling refactorings into single releases (unless you want to run hundreds of little scripts by hand) is another process you’re going to create within your own environment. Finally, you’ll have to deploy these changes to production and remove any deprecated schema. This chapter gives a lot of information on some of the better ways to approach each of these topics.

5: Database Refactoring Strategies

I found this to be a very useful chapter as the authors outline issues that they’ve run into as they implemented their own Agile methods. They outline it as a series of “lessons learned.” Some of the topics that especially caught my eye as I’ve been attempting to integrate my own agile processes included “Smaller changes are easier to apply,” “Implement a large change by many small ones,” “Prefer triggers over views or batch synchronization” (a real eye-opener in terms of process as well as implementation), “Be able to easily set up a database environment” (this has been a continual sticking point within my company as we wrestle with bandwidth of DBA’s & developers while ensuring sufficient data security on the production systems), and “Beware of Politics” (to which I’ll only add a hearty Amen).

6: Structural Refactorings

The remaining chapters are lists of refactorings that you can apply to your systems. Each one is broken down with a description of the refactoring, a Motiviation behind the refactoring, Potential Tradeoffs that could result, Schema Update Mechanics which generally outline how to implement the refactoring, Database-Migration Mechanics which show what, if any, and how to move the data around, and finally Access Program Mechanics which outlines refactorings that are probably necessary in your TSQL and/or application code. This particular chapter pretty much does what is says, lists refactorings that change the structure of the database schema. Some examples include, Drop Column, Introduce Surrogate Key, Rename Table, and Split Column.

7: Data Quality Refactorings

“Data quality refactorings are changes that improve the quality of the information contained within a database. Data quality refactorings improve and/or ensure the consistency and usage of the values stored within the database.” In other words, this chapter outlines more structural changes, but ones that modify the schema for purposes of data quality improvement as opposed to outright structural change. The thing to remember about refactoring is, that it’s usually composed of lots of little changes. So making one of these refactorings may also require you to make a Structural Refactoring & vice versa. Some examples are Add Lookup Table, Apply Standard Type, Introduce Default Value.

8: Referential Integrity Refactorings

“Referential integrity refactorings are changes that ensure that a referenced row exists within another table and/or that ensures that a row that is no longer needed is removed appropriately.” These can be quite painful refactorings to introduce since they almost always entail cleaning data. Examples include Add Foreign Key Constraint, Introduce Hard Delete, Introduce Soft Delete, Introduce Trigger for History.

9: Architectural Refactorings

“Architectural refactorings are changes that improve the overall manner in which external programs interact with a database.” This is much more along the lines of adding new procedures, code and processes to the database, but, when you’re in an Agile development process, every change is a refactor of one kind or another. Some of these refactorings are Add CRUD methods, Introduce Index, Migrate Method to Database, Use Official Data Source.

10: Method Refactorings

What they mean by methods are stored procedures, user defined functions and triggers. These then get broken down into two, pretty logical categories, those refactorings that will change the interface, requiring you to refactor an external program, and those that are strictly internal changes to the stored proc alone. Some examples here are Add Parameter, Rename Method, Replace Parameter with Explicity Methods, Extract Method, Replace Nested Condition with Guard Clauses, Split Temporary Variable.

11: Transformations

These are changes that change the database by adding new features. Pretty simple. The refactorings include Insert Data, Introduce New Table, Introduce View.

Perspective of Reviewer

I've been working with development teams within an Agile type of development process for almost two years. During that time, we've managed to impliment quite a few of the steps & methods outlined here. We've still got quite a ways to go until the database is treated more like another peice of code, another service in the Service Oriented Architecture, rather than a special little domain that the DBA's deal with. Our main victories have been achieved as we've adopted the developers language as our own. When we talk about pasing a unit test in a stored proc after we make an internal change for performance, the developers have a strong faith that we haven't broken their code. It makes for much smoother communication overall. Being able to begin to refer to database "smells" and having documented approaches to addressing them will move us further down that road.

PASS Book Review by: Grant Fritchey 
Rating:   (out of 5)
From the Publisher: Addison-Wesley
ISBN: 0-321-29353-3
Published: March 2006
Pages: 350
Publisher's Price: $49.99