Lessons from www.healthcare.gov: 3 Tips not to blow up your critical systems implementation

MSNBC’s Morning Joe team was lamenting this week on how they all miss the era of desk phones, those Mama Bell stronger than a bulldozer devices, capable of being slammed down hard during an argument. It even gave you a comforting post-slam ‘ring’ from the mechanical bell just to reinforce you genuinely were that angry and had made your point. Try that today with your phone, and chances are the post-slam ring will be replaced with the not-comforting and too soft sound of shattering plastic. Not only were products simpler, they could survive occasional abuse beyond original specs.

Recent example of how not to build and launch a mission-critical system:  built  by a contractor then recently fired by the Ontario Gov’t for underperforming, and with the cost going from the original bid of $55.7M to $292M and counting, healthcare.gov is still seriously troubled and attracting a media firestorm.  The contractor who messed it up (to the point where some tech firms are wondering if it has to be scrapped and rewritten) is deflecting blame to the Business-side (here the ‘politicians’) while being forced to fix it on the fly, which is always the hardest way to fix anything. Anyone who has ever used the analogy ‘replacing the engine mid-flight’ has never been a pilot, let alone fixed a complicated, multi-stage high transaction volume system. The ‘fire the managers’ cry is becoming louder and louder.

But, what we can get from this experience is an open discussion of why critical systems developments initiatives fail so often. Here is our ‘Big 3’ and how to avoid this outcome:

1- Ineffective continuous Business involvement with the project

There’s no such thing as an IT project, short of installing new servers or systems software.  If someone other than a Systems Administrator or Technical Application Owner is going to use the system, it’s a Business project.  Typically, the Business has a vision of functional needs and expected volumes, but then throws development and deployment ‘over the fence’ without transferring necessary contextual information or a fully thought out set of detailed target functionality as seen from the ultimate end user’s point of view, which is more specific than the typical User Stories required by Agile oriented IT. Regarding healthcare.gov, one of the biggest complaints, after someone logs in, is an illogical user interaction flow.  The Business (here, the Gov’t) should have performed Human Factors research and testing at the least, pre-launch, as part of their responsibility.  It was their job to declare ‘done done’ and promote to production.

Follow this rule of thumb – encourage your IT group to refuse taking on any complex project without a well thought through Business Requirements document (which doesn’t have to be longer than War and Peace). Even in IT’s increasingly Agile world, the end-state has to be fully described to ensure everything is architected and built to fulfill the underlying Business need. To put it in modern vernacular, unless you have a Business Requirements document, you can never be truly done-done and the stress and User Acceptance testing scenarios can miss the mark by a wide margin . It is the Business’ responsibility to ensure the new system meets user experience and strategic business requirements, not IT’s.

2- The Business does not have robust systems implementation hands-on experience to manage the project

How many times per year does the average Business-side project manager hands-on implement a complex, sophisticated system?  Chances are the correct question should start with ‘how many times per decade…’ Unlike those desk phones of old, today’s systems are multi-vendor, patched together for speed to market, requiring complex data exchange between interacting systems and feeds, with each component having its own specific ‘calls’, timing, edits, definitions and new release cycles.  Using Agile development methods, IT groups often describe iterations and related sprints around 2 week time blocks, not as a series of increasingly powerful Business relevant feature deliveries of short but varying durations. Unless you can debate the finer points of iterations, it’s hard to manage a project beyond reviewing status reports.

Follow this rule of thumb – control the project from the Business side, including retaining authority to pay for accepted work product by project phase, not transferring funds to IT for them to then spend based on time X FTE’s X internal costs calculations.  Either inside or outside sourced, ensure someone is specifically tasked and empowered with having your back, bridging communications gaps between groups, and asking hard, probing questions that continue to propel the project forward but surface critical (and often not obvious) issues while they can be readily addressed.   Typically, a generalist or part-time Project Manager is not a good fit for this tech-savvy role.

3- Critical technical resources spread too thin on too many projects – and the Business caused it.

We have seen where many firms have enough journeymen developers and team leaders, but too few deeply knowledgeable legacy application SMEs.  These SME’s, in turn, are often over-allocated across multiple projects. In our experience, legacy application specific SME’s are the best people to knit multiple systems together into the correct Enterprise Solution architecture.  Any delay in their participation can lead to missing a QA testing schedule and then having to find a new opening in a future Systems Production Release window, affecting revenue projections and anchor customer commitments. A viable backup plan is to avoid using individual contractors, but instead rely on a systems integration specialist firm with a history of knitting systems together as a team. We know of one instance where a fairly straightforward new Digital product was delayed for over a year. Welcome to recession-reduced IT, and using flex-staffing for deep inter-application integration through low cost  development resources is not cost effective, no matter how tempting in the short run.

Follow this rule of thumb – pre-kickoff and thereafter, review the actual names of all personnel assigned to a project, especially the System SMEs, their experience level, concurrent project responsibilities, and ‘what-if the other project runs late’ contingency planning.  Make sure any flex-staffing is used properly.

We’ve written recently in this blog of the many companies suddenly finding themselves in the Consumer Electronics business as their normally straight forward products are being redefined through customer facing technology. Websites, mobile apps, transaction processing, and customer support systems are all increasingly sophisticated, supporting this more complex set of products.  Going forward, systems will have to support customer facing staff assisting customers in making smart product choices, where poor information can hit repeat business and brand reputation almost immediately.  Customer facing staff will have to explain more complex product features, choices, and use, all during the few minutes this buyer is at the point of purchase before fear or frustration sets in and they walk away empty handed or go back to Google.  Near-future systems may explain product options to customers, then send their decision to a nearby specialized vending machine, like those Best Buy vending machines in airports, or to an online forms application linked to a rules engine, or in the near future, to a nearby 3D printer. The world isn’t simple anymore and system failures are now in the public eye, incurring brand reputational risk.

Desktop phones are great nostalgia, but like simple systems, they are the past and the Business needs to step up and actively take charge of your own projects and thereby your future.

Richard Eichen is the Founder and Managing Principal of Return on Efficiency, LLC,  http://www.growroe.com , focusing on companies, initiatives and products where technology is the primary means of delivery and revenue. He is one of their senior Turnaround, Transformation, Program Rescue and Process Rescue leaders.  As a Change Agent, Trusted Advisor, Program Leader and Interim Executive, Rich has over 25 years hands-on experience reshaping companies, Operations, IT/Systems Integration and strategic initiatives.  He can be reached at richard.eichen@growroe.com, and followed on Twitter, @RDEgrowroe