When IT Projects Attack

By now everyone has heard about the catastrophic rollout of the Obamacare website. Everyone has an opinion, no one can figure it out why it wasn't ready.

"They had three years to figure this out"

"They've spent a half a billion dollars, why doesn't this work?"

They used a Canadian firm to create this nightmare. "

Now I'll explain just why none of that mattered. I've spent the last six years of my life as an independent consultant as an IT Project Manager (PM). I understand how this happens "politically", and I'm not referring to governmental politics. I'm talking about project politics.

First, to be fair, this was an absolutely humongous project to undertake. And no, the project team could not have possibly been working on this for three years. Think about it. The law passed, no one knew what was in it and the insurance industry people could not get answers about how things should work. Some of the states weren't participating in the exchange. Many in the industry were waiting on the fed to interpret the rulings. The law is one thing, but then there are likely tens of thousands of pages of instructions and interpretations. 

And then there would have to be time to put together a large IT team. Even if they had the full three years, there wouldn't have been enough time to pull it off. There simply isn't.  It's been reported there were 55 different outside vendors associated with the project. The simple process of just selecting the vendors takes a great deal of time.  It's also been reported that coding didn't even begin until this spring. I would estimate that's probably accurate. Which also means that left no time for the remaining project process. 

Literally every project I've worked on has had a way too aggressive deadline and was way under budget. I'm normally hired after the project has gotten approval and some CIO or other executive has made the selection of the third party contractor, given the timeline and budget to the board and suddenly they realize they need an internal PM. When I'm hired, I just nod and know that at some point that executive will be forced to make a realistic assessment when the deadline is approaching and the software is just not ready.

The first real step involves documenting the requirements of the system. This is an extremely painful, long drawn out process where you get too many people in the room and you break down the entire process step by step. You investigate each route the system might need to take. You develop flow charts to you can visually see the path any one type of decision might take.  

And the team misses something. That's just a fact of life. Everyone pretends (or God forbid actually beliefs) that every process and step will be caught. Never happens. 

In the meantime, you have people deciding the architecture, which is essentially what type of equipment you need and the methods of delivering information between other systems. From what I understand this is one of the big issues with the ACA website.

Think of it this way. It starts with a database table of records. That is virtually a big long list of information. Most likely there are millions and millions of records in one table. Let's just say this table has the name and SSN's of the primary insured. Then another table might include the list of all the dependents of the primary insured, another table will have some code link of eligible health plans. When the software is working together, there will be significant volume of traffic needing to find the connective link between those necessary tables.

If the architects are not aware of how often one table may need to link to another, they will build it inefficiently. (This happens a LOT) They will not properly index the tables so everytime the software has to provide a user information from many tables (sometimes dozens and dozens of tables) the system has to go through every record on each single table before it can display the information to you in a seamless, instantaneous manner. Now remember, unless you have a very knowledgable team who understand how the system should work this architecture work can sometimes be started in a silo before the requirements are even outlined. Of course it shouldn't but more often than not it does. They think they are saving time to meet the project deadline.

And too often, the right people are not in the room when the requirements are being gathered. You will always have someone who acts like they know the process and will insist on putting things in the requirements that are simply wrong. You have others who think things are very simple and cannot see that their simplistic view does not address a real process. The code word they use is "just". "Well, you can just....." Beware of those people and there is always at least one in every project and they have too much authority for their position.  Rarely does logic and evidence change their belief they have full knowledge of the process.  They are too stupid to realize how stupid they are. Dangerous combination. 

Once everything comes together, you should be able to take the requirements and develop very specific tests to be performed in the software. This includes attempting at least to mimic the amount of network traffic so you can test the efficiency of the software.  

One of my clients had developed a great amount of documentation for the requirements. And then due to an imaginary deadline the executives had promised they rolled out a very large, integrated project with NO testing. By the time I left, only just a few minor processes had been delivered. And about a dozen tests out of several thousands that couldn't be performed were conducted with several issues which were not resolved.  

I realized early on that no one on the team had EVER had any project experience. That was unfathamable to me. It was a very large project and every attempt I made to get the management to understand they could not go live was a failure. They ignored the years I had spent involved with the exact type of project they were dealing with and listened to the lies of some of the internal employees who had no experience for the job they were hired to do. I finally gave up and realized I wasn't going to be around to help them clean up the mess. When I left, two weeks before the go live date I was sure someone would make the last minute decision to extend the go live date.  

They didn't. And the system came crashing down around them. They were unable to make payments to their customers and ended up calculating thousands of payments manually and late and incorrect. They are still digging out of the mess and I can tell you they will be finding errors for years to come. It was the most egregious episode of mismanagement of any project I was involved with.  One of my last conversations with the project sponsor included, "This isn't my first rodeo". It was definitely his first and yet he said I was only being negative.  I would love to be unprofessional and send him an email to ask him how that worked out for him, but I learned early on not to make too many enemies in this town. And he's his own worse enemy anyway. 

Back to our issues with the ACA website October 1st deadline, hardcoded if you will, the big-mouth-know-nothing's in Washington.

Every project has a dance with the deadlines. They are set by people who do not understand the systems or the requirements. Some better project sponsors (the executive) will push the date farther out than they actually think will be needed. And it's always not enough time. Poor sponsors try to get their project approved by shrinking that timeline and giving an impossible deadline.

Some PM's believe in participating in the dance while the floor is falling out from under them. Usually these are PM's who have no real world experience and they limit their activities to looking at the project plan and emailing people for updates. They don't get down and dirty with the details because they can't understand them. Then they go before executives and lie about what the team has been telling them, painting a rosy picture of the project fires.

I've never understood that method and hate working with PM's who operate in that manner. They play politics over appropriately informing the project sponsors of the risks.  And this is what happens when they do. 

I've always considered it my job to make sure the sponsor is fully aware of the risks and it's their job to decide if the risk is worth taking. I've seen some gutsy moves that make me nervous, but if you have a good sponsor then normally the risk was worth the reward. I've also witnessed idiots who insist on going live when they have not mitigated any of the risks. It's their call, it's their job. And it's theirs to live with.

I know this probably bored every one here, but listening to all these discussions in the news regarding this rollout of the ACA website I can just imagine exactly what happened behind the scenes.

If Obama and the Democrats understood anything about IT projects, they should jump on the offer to delay this by at least a year.

The project simply wasn't ready. And I don't have to be their hired PM to tell them why.