Spend enough time in IT and you’ll see plenty of failed dev projects. Although there is a huge amount of wisdom to be gained from a failed project, I have never seen that benefit reaped by an organization. That’s really unfortunate, but it could easily be avoided.
I want to begin by talking about how an organization identifies a failed project. Failure may seem like something that would be self-evident, but unfortunately it is not. That’s the first major reason organizations don’t learn from it: They don’t realize it happened. The fact is they often never correctly identify a failed project as a failure. How is that possible?
Reason #1: Redefining Success
As the project begins to fail, the organization lowers the bar and redefines success to be whatever the project deliverables are regardless of how far below the original requirements they fall. It’s like kids’ soccer when everyone goes home with a trophy.
Reason #2: Denial
In the Agile process there is a retrospective after each sprint. Part of that retrospective is discussing what went well and what didn’t. I’ve worked in companies where discussion of the negatives never took place, and other companies where criticisms were grudgingly allowed to be recorded during the retrospective, but discussion of them wasn’t. Criticism or even recognition of failure was taboo, which brings me to…
Reason #3: Refusal to Blame
Failure and blame go together, whether we like it or not. Many organizations are uncomfortable assigning the blame for failure on either their processes or their people. Process change is difficult, and there is often great resistance to it. In the typical organization, blame is not something most people are comfortable with on either the giving or receiving end, and in some workplaces it even violates the corporate culture.
Reason #4: Escapism and Scapegoating
When a project looks like it’s going to fail, those responsible often head for the door while their reputation is still intact. Managers cover themselves by firing those below them, or do a reorg to thwart accountability.
Project failures cost a lot of money. They cost the wasted salaries of those involved. They cause negative ROI and lost revenue when they underperform or fail at release. The most insidious cost—one that I believe is rarely counted—is the opportunity cost. It’s the benefit of positive revenue a success would have generated and the developer hours that could have been productive.
Software has no intrinsic value, so it is often discarded when it doesn’t fulfill its requirements. I have never seen a failed software project refactored successfully. Refactoring can make it usable until it can be replaced, but I don’t believe you can repair your way to success. Successful software requires a clear design vision and sound architecture. Failed projects rarely have these and they can’t be retrofitted.
As a developer it hurts to spend months of time on a project only to see it be rejected by the business owner or fail QA repeatedly. And it especially hurts when you knew it was going to fail before you wrote the first line of code! You might ask yourself why a developer wouldn’t push back at the start of a project that had failure written all over it, and the answer is simple: Corporate culture doesn’t like negativity. Positivity is a great thing and it makes everyone feel better, but that doesn’t mean negativity is a bad thing. Hearing the potential down-side of a proposed project while it’s still in the design phase can have huge benefits, the biggest of which is the ability to limit the damage of bad decisions to the design phase. The longer the project lives, the more time and energy have been invested in it, the harder it is to change course.
Here’s a real-world example
One of my coworkers was given a project to create a variation of an existing page for an A/B test. I went over the design with him and identified multiple problems in the UI design, in usability, in functionality and in implementation and integration with our existing code. The number of flaws was hardly surprising, since the design was provided by a data scientist rather than a UX expert, and he had not received any input from anyone familiar with the code or the feature being modified. He had dashed out the design in a few hours, basing it on a feature he’d copied from someone else’s website. The feature he copied was not the same feature we were implementing. They just looked similar. I knew the project would fail if developed as designed. We both should have pushed back on the design immediately, but we didn’t.
All the problems I identified before any code was written became obvious as soon as development began. The result was that the project had to go through many cycles of redesign, redevelopment and retesting until it actually worked. Instead of the project taking the original estimate of one week to develop, it took four months! And it looks and behaves like the hodge-podge it is. As an A/B test it underperformed the original experience. In other words, it was a total fail.
This failure occurred for one reason: No one called out the flawed design. The developer tasked with the design was new to the organization and he wanted to make a good impression. He didn’t feel confident enough to express his opinion. I had the experience and confidence to push back, but it’s because I had experience that I didn’t. I had previously pushed back on a design proposed by this same business owner, who was a V.P. in the organization. The result was that I was immediately pulled off that project and it was reassigned to someone else. That project also had to be completely redesigned for all the reasons I identified, but that did not vindicate me or buy forgiveness. I was never given another project and was relegated to working bugs and doing routine updates from that point on—the developer equivalent of being sent to the gulag. That single episode of “negativity” virtually ruined my career prospects at the organization. No way I was going to speak out again, nor would it have made a difference. And that’s a real shame.
Every organization I’ve ever worked for has spent tons of money teaching me and other developers how to avoid failure with many painful real-life lessons. Yet organizations refuse to recoup that investment by actually listening to those of us who learned the hard way when we warn them in advance of how a project can or will fail. How can this be changed?
“More hurry, less speed”
Business owners and dev managers are under pressure to produce results, so they rush projects through the design phase. I have never worked on a project where one hour of design didn’t pay me back ten-fold in the dev phase. My advice is to take the time to rigorously criticize a design while it is still only a design. Do your best to tear it down. Search tirelessly for ways to improve it. A great design can stand up to these assaults and can get stronger by receiving input. A fatally flawed design will end up where it belongs—in the garbage can instead of in dev. Thinking hard and striving for excellence are difficult, but great results don’t happen by magic during the dev phase. The design phase is not the place to be lazy! I say take as much time as you need to get it right.
Seniority doesn’t equal expertise
I’ve been consistently overruled by senior management whose only expertise is their personal use of the web. Unfortunately in corporate America, there tends to be the attitude that everyone’s opinion counts, but that an opinion matters more if it comes from someone higher up the chain. What they fail to grasp is that while the customer service director may have an opinion, the developer may actually be speaking from knowledge, expertise and experience. So while some people are debating opinions, others are trying to convey facts. The lesson here is to listen to the people who know what they’re talking about no matter where they are on the corporate ladder, and most importantly: Don’t try to make non-technical stakeholders into dev managers. Really. I could write volumes on that one mistake.