Convincing project teams of the need to normalize

Convincing project teams of the need to normalize

In our last design challenge we agreed that we should normalize regardless of database platform, and we discussed a number of benefits the normalized design can offer. Some of these benefits are longer term and have broader scope than the current project to which you are assigned. This can sometimes cause conflict with a project team who has short term objectives to complete the project deliverables within tight deadlines. Project teams that are “under the gun” are less concerned about long term benefits and heavily focused on the deliverables for next week or next month. Have you ever heard phrases like “We don’t have time for that exercise” or “Don’t hold up our project”? Please address the following two tasks as part of this design challenge:

  • Describe at least one technique or tactic that has worked to convince or educate a project team that the benefits of normalization outweigh the near term additional time and money investment? Why do you think it worked?
  • Describe at least one technique or tactic that has NOT worked. What could have been different to make it work?

Note that this is not an easy challenge. We are dealing less with the facts around relational database theory and more around emotions running high when project deadlines are fast approaching.

Response

This response will discuss an underlying theme in all successful normalization debates, followed by 3 techniques that have worked and one technique that has not worked.

“What’s in it for them?”

Although there are several techniques we’ll discuss that have worked, they are all driven by the same underlying theme. This theme is important in convincing anyone to do anything (even outside of modeling). This theme focuses on putting yourself in your audience’s shoes and identifying why they should listen to you. If you’ve ever read a book on selling or marketing, thinking of the needs of the audience is usually mentioned somewhere in the first paragraph. It is so basic and commonsense, that sometimes we forget to do it. Several respondents to this challenge echoed this important theme of “What’s in it for them?”. Moshe Japha says “…if you want buy-in, you must demonstrate why your project leader, programmer or analyst will benefit from your insistence to normalize.  Merely counting on good corporate citizenship – ‘It’s the right thing to do’, doesn’t cut it.”  And Frank Palermi said “You always have to show them the ‘what’s in it for me’ aspect to make them go along with it…” I had a manager one time, who in a very pleasant and friendly way, was able to convince teams of the value of data modeling including the need to normalize by focusing on her audience.

What worked?

Once we’re aware of applying the “what’s in it for them” theme, let’s discuss some specific techniques that have worked and why.  We’ll talk about 3 of them: a bit of compromise, “I’ll prove it to you”, and company objectives.

A bit of compromise

I’ve worked on a data mart project very recently where the project team manager had a very bad perception of normalization. Using the “what’s in it for them” approach I identified that their primary concerns for their reporting application was, as you might expect, to have friendly and fast data structures. So the project team manager didn’t really care if the structures were normalized or not, as long as the resulting data model lead to a friendly and fast design. So I built close to a fully normalized design. I then “hid” this design from the team manager with a series of database views. These views gave the perception of flattened dimension tables, producing a fairly user friendly structure. However, each view accessed all of the underlying normalized tables. If performance was ever an issue we could simply materialize the view. So this technique was successful because we kept the flexible structure and yet met the project manager’s concerns. However, we did lose some flexibility as a change that would not usually impact a normalized design could potentially now require us to update our flattened database view (and this actually happened several times).

“I’ll prove it to you”

A very effective technique is the “I’ll prove it to you” technique. There are two ways to apply this technique. One way is to lose the normalization discussion with the project team and then when they keep coming back to you with change requests, explain that normalization would have produced a more flexible design that would have reduced this extra work. This way of applying this technique usually does little do build team spirit and can isolate the modeler from the rest of the frustrated team.

The other way of applying this technique is to start with a more normalized design (if we have the opportunity to do so) and then impress the team with its flexibility and ease of maintenance. That is, make it work smoothly and the team will tend to lean towards a more normalized design in the future. Moshe Japha discusses an application of this technique: “The code for the initial vendor had been hard-coded specifically to that vendor’s rules, status-codes and responses.  The newly-hired data analyst designed a normalized solution with the ability to keep both approaches in effect in parallel – based on such simple concepts as m:m and 1:m relationships versus the hard coded single-thread approach.  The implementation team, originally having the design shoved down their throats became over-night converts when they saw the implementation go off without a hitch, one day processing sales through one vendor, and the next through the new one.  They realized that the normalized design saved THEM sleepless nights and the stress of crisis mode.”

Company objectives

Sometimes an effective technique is to relate the benefits of normalization to the broader goals of an organization. Eric Swanson works for a government agency and had this to say: “Because we are not a business concerned about turning a profit, we can focus on the long-term goals more easily…Many systems at the State are in place for a long period of time.  This means that the operating staff may be replaced many times before the system itself is upgraded. A good set of documentation goes a long way towards helping a new crew to understand the inner workings of a system.”

What didn’t work?

The major technique that I have seen fail (because I’ve tried this in the past), has been attempting to convince teams to go with a normalized approach based solely on flexibility.

Just flexibility, that’s all

A fairly weak argument is when the only benefit we give for normalizing is that it is flexible. That is, the argument of future flexibility simply doesn’t work without concrete and specific examples. Most project teams have a fixed time and budget and doing something that might alleviate next year’s work can be unimaginable when they have deadlines for next month. Moshe Japha says “What doesn’t seem to work is the promise of future flexibility:  If you make this design super-table-driven, and allow for all of these normalized relationships where you want a single table, when the requirements change in 6 months, the design will support it.  This really doesn’t promise to make the implementers’ lives easier, since they know that whatever you do to anticipate future changes will at best be only partially right.  Rework will be required in most cases anyway.  In fact, what the implementers see is more up-front pain, since coding to a normalized design is admittedly more difficult.”

Combining the flexibility approach with some very specific and concrete examples can aid the normalization debate. When you think about it, the term “flexible” is as nebulous as the term “normalization” for many people. However, if you can communicate that this normalized design will allow your customer to have not just two accounts, but any number of accounts, and prove that having three or four accounts might become possible in the very near future, the entire team (or at least the team manager) might agree that the normalized structure is the way to go.

1 Comment

  1. Terry Mason 3 years ago

    Normalization is pretty much essential for successful database implementations, but it really helps to use an intelligent data modelling tool.

    What works: When I have had the luxury of control over the evolution of the LDM, PDM and database design the project has always been successful. I put this down to the design consistency and rigor using a modelling tool, and sharing each iteration of the model using poster size paper copies with the developers.

    What fails: The unfortunate reality in some projects is that the data modelling skills are very weak, and the participants are too proud to admit it. The result is that when the system goes into testing the number of defects due to lack of data integrity goes through the roof. Typically such projects fail to invest in a proper data modelling tool which could help them flag up design flaws and enforce data model consistency. Instead they employ a drawing tool instead of a modelling tool. They end up paying the price with major cost overruns.

Leave a reply

Your email address will not be published. Required fields are marked *

*