It is always easy to Monday-morning quarterback the failure of such a huge government project as HealthCare.gov. But in talking through what went wrong, we have found we have learned (or just remembered) some of the things to avoid in projects and products we are working on.
Recently, Build.Automate’s CEO and Founder Vaughn Bullard and Director of Digital Strategy and Marketing Deanna Murray, sat down to talk through the failings of this system. Their conversations determined the failures do not fall to the left or to the right – but to some fundamental breakdowns in the product conception, capturing of product requirements and communication during project build.
Below, they share their insights in a Q&A format in order to further encourage discussion and learning.
What are the biggest learning points that other companies can take away from the problems with the ObamaCare website?
Bullard: No matter how much you plan, something will always go wrong. From my perspective, it looks as though nothing was planned other than the actual marketing of a website that is, at this point, going on 7 weeks into public rollout and not completely functional. It would appear they were more worried about the consequences of the product not rolling out on time.
Unfortunately, the consequences of rolling out with a broken, half-baked product are worse than if they had delayed rollout of the website. This should never have gone public. The contractor telling stakeholders it was ready despite the obvious flaws in the system is baffling. If I had an employee who did something similar, I would have terminated their employment.
It’s pretty simple. Quality and integrity count in all things.
Was choosing the wrong contractor one of the main problems? If so, how can a company make sure they choose the right contractor for a project?
Bullard: ObamaCare was too big a project and too important for it to fail. The government should’ve from day one, skipped the business-as-usual. They should’ve sole sourced it from Silicon Valley corporations who have experience building large-scale systems like this in an agile way.
They would’ve got it right and even if they had implementation issues, would’ve been able to recover in a professional manner.
Murray: A contractor builds what he/she is told to build. If through shoddy communication, convoluted specs and unclear expectations, a contractor isn’t given what is needed to build a successful project then the blame falls on the contracting employer.
In a project of this magnitude, unrealistic deadlines, a total miscalculation of scope and the inability to foresee and anticipate possible fail scenarios seems to be what has caused the majority of issues.
In my opinion, very poor strategy implementation, project management, quality assurance and system integrity testing is the main culprit. Not to mention lack a cohesive communication throughout conception, build, deployment and post launch.
Is it important to have a project manager in charge of the whole project? Do you think this was a problem with the ObamaCare website?
Bullard: It is blatantly obvious either the project managers were either inept, inexperienced or led down the wrong path by government COTRs. As a certified project manager and someone who has built systems for millions of users, I know the scale of this is much larger than the administration had anticipated. Unfortunately, it took the complete failure of the system in order for them to figure it out.
Murray: Without a doubt — a strong project manager, or a project manager team in this case, is most certainly the lynch pin that holds a project together. If this person, or group of people, does not take into account every single detail from beginning to end within a project, effectively communicate it and also build in fall back scenarios if something fails, then the project is more apt to fail. Exemplary project management, while seen by some to simply be a clerical function, is what winning, successful products are made of.
It is my experience companies tend to throw the PM job to someone who may not understand all the aspects of strategy, build, deployment and post project line items. But in actuality, the PM should have enough knowledge about all areas to be able to guide the teams tasked with carrying them out so he can communicate with authority and make decisions when needed.
One of the apparent deadfalls for the heathcare.gov website was simply the lack of stress testing on the system, complete user-test case scenarios and browser testing — all of these are basic testing processes a PM should have built into a plan and should have insisted the site pass before it was launched. Our firm performed these basic tests on the launched site and found many elementary issues that should’ve been caught in prelaunch testing — including browser compatibility, load time issues and simple editorial errors within copy).
How do you choose the best project manager for the job?
Bullard: There is no special magic dust to find the best project manager for a job of this scope. You are talking a project manager that would have to manage many project managers.
Unfortunately, many project managers do not have a technical background or even project management certifications. Simply having a computer science degree or a project manager certification would not have saved this from failure. This required project managers with vast experience on building large scale systems fast. Those project managers are typically in the private sector making top dollar for financial companies on Wall Street and in Silicon Valley
Murray: Finding a certified and trained project manager is always a good start — but we have used several who haven’t been officially certified and are some of the best in the field. Project management is an art form of organization, communication and knowledge. It takes into account a strict timeline, allowing for almost every single fail scenario and a fix for it.
When one of our clients is searching for a project manager, we always recommend the following:
- Has excellent communication skills i.e. able to relay business goals, keep meetings on track, identity and relate problems, etc.
- Has the ability to organize and categorize tasks
- Has an overall understanding of every aspect of project and what it takes to get there
- Has the ability to implement a complete testing strategy and plans for every core function of project AND for launch
- Has the ability to make hard decisions based on facts and business needs
- Has the respect and trust of team
- Has the ability to see the whole picture and help others see it as well
- Has the ability to filter project priorities to make sure dates are met
How do you best manage the vendor relationship?
Bullard: Clear communication. Telling the vendors precisely what you want, how you want it and how you want it built is paramount to the success of any delivered project. Precisely delineating your responsibilities and those of each vendor from the project start is absolutely essential.
Murray: Vendor relationships can be tenuous — especially if the hiring party isn’t quite sure how something is done and is even more unsure of how to communicate what an end product should be. But the bottom line is, the hiring company IS BOSS. But all too often, companies seek out vendors for projects when they are, themselves not sure of what the end goals should be. They give a contractor the beginning and the end goal and don’t even give a clue as to how to what should come in between.
While a good contractor knows how to get there, their perspective may only be ONE of the hundreds of ways to get there and it may be the inefficient way or the absolute wrong way.
For a contracting body to not know themselves, and have well documented end-project scenarios for a contractor to build towards is probably how many hiring companies end up with products that achieve the desired result but are not at all really what the company envisioned.
While it is very elementary in its illustration, think about the old Tootsie Pop commercials with the owl licking the sucker then suddenly taking a bite out of it to tell the child how many licks it takes to get to the center. The owl chose the shortcut — the end result was still what was needed but the getting there was completely off base.
I say all that to say that communication and planning is the best way to manage a vendor relationship — and also for a business to be extremely proactive in understanding every step of what is being done by the vendor and calling them on it when it’s not what you had in mind.
It seems that glossing over the problems and not being honest when early problems erupted – and even now with the site – were some of the main problems with the ObamaCare website. How can other companies avoid this? How important is it to be truthful when there are glitches?
Bullard: Be honest. People can accept honesty, even when it’s bad news. Having a defined strategy for how you will handle the good and bad attention is extremely important here.
In addition, having one voice that is accountable as the brand ambassador for your company or product and is the liaison for all things customer related is an absolute necessity. Being accountable for even the smallest things, good or bad, instills confidence in your company, product or in the case of ObamaCare, your website.
Let’s face it: ObamaCare is a brand. Having the President’s name associated with this is and was a risky brand strategy; and it is not doing well at all. How many businesses and brands in American history have simply disappeared because of an unsuccessful brand strategy? I’m not a statistician, but I would hazard to guess millions.
Murray: One of the most telling moments for me during this entire time, as a marketing expert and seasoned project manager, was hearing leadership say they were angry at the results after the product launched. To have the person who conceptualized and for all intents and purposes is the ‘business owner’ of the product act as all the issues were a shock told me right away project management and communication to stakeholders was a huge issue. While I don’t expect people to roll up every issue to the president of the US or even the CEO of a major company, I do expect full communication.
From a marketing and PR perspective, the damage was great. In corporate America, these kind of mishaps would get someone fired — in fact, probably many people. Stock prices would go down, people would be losing money and even more would be seeing red and asking for blood.
Successfully spinning this was paramount but was not done at all. How can you spin something like this? Well, truth is a great step. Roll back. Throw a BETA sign on the site. Ask people to report bugs. Make it about a product that is out there to do good and wonderful things for people but could not be tested properly in simulated scenarios — so you had to roll it out to get it right. Make it a country-wide beta test. NEVER SAY, AS THE PROJECT OWNER and BRAND AMBASSADOR, that you were basically clueless as to what could happen. Take the spotlight off of the unpreparedness and unprofessionalism of something so central to our lives.
Then, when the spin is done, get down and dirty and figure out what went wrong, fix it and … lastly, conduct a full and thorough post mortem so nothing like it happens again.
There will be occasional failures, despite the best efforts. How can a company manage the failures? How can they prepare for failure?
Bullard: System Architecture 101: You don’t build systems for 100 users if you have 100 users, you build them for a thousand or thousands of users. Systems will always fail at some point in time. It’s up to you, or by codified regulation, to decide the level of risk you’re willing to assume.
Good backup strategies with not only primary and secondary systems, but tertiary systems allow you to mitigate that risk for your company. Our company, Build.Automate is a growing company but we have backup and failover systems in place at the same level as large corporations like IBM or HP.
We don’t take chances with our data OR our customers’ data or the systems that we both rely on. That’s not being risk averse, that’s being risk aware and strategically focused on the longer term objectives.
Murray: One of the best ways to make sure you are on top of any failure is to monitor bug reports and any other analytical data several times a day. For a project of this magnitude, once a day or once a week would never work. It is a full-time job for a person or group of people to make sure all areas are covered and working properly.
If something does break, which it will, throw an alert on the front of the site, take down the section and fix it as quickly and thoroughly as possible BUT also make sure to take into account any and all changes you made and how they affect other areas or other outcomes.