Complex systems need solid engineering best practices rather than individual human brilliance

In this article, I want to share with you a real life experience which made me understand a very important principle:
"Complex systems need solid engineering best practices rather than individual human brilliance"

Some six years back ,while I was working for a reputed supply chain management company. The company bagged a very prestigious project of building a complete supply chain planning solution for a global oil and gas major. It was a very huge and a complex project (>200 million dollars), as supply chain planning was done for the first time for a continuous flow industry (that time SCP was only done for discrete manufacturing companies).

TThe complexity of the system can be judged by the fact that, this Oil and Gas Company had around 80+ refineries, 300-400 oil wells, 5 primary distribution centers (each having at-least 20-30 secondary distribution centers with 100s of ships, vessels, barges, trucks, trains, storage tanks etc.). Apart from the scale, the material that was supposed to be planned can be in different forms (Solid, Liquid, Gas) and can change its characteristics at different points of time. Overall a very complex project.

The project was supposed to be executed by a team of around 200 engineers and managers working across the globe. I was a part of the team based out of India.

As I could understand, the SCM company employed some of the best brains in the industry including software engineers, project managers and domain experts ( but not business analysts) from the oil and gas industry etc. All I can say most of the employees working on that project were brilliant and were graduates, postgraduates and PHD holders from some of the best universities in the world.

What do you think should have been the outcome of this project , given the brilliant team that was working on this project?

After three years of development, with a lot of customer dissatisfaction and iterations of re-planning, the project was shelved and around 100+ million dollars were written off as bad debt.

If you ask me...

What according to me were the reasons behind the failure of the project?

I would simply say "Absolutely no analysis, very minimal documentation" and "Too many project managers". As there was absolutely no analysis and modeling, forget about planning their supply chain , we were just not able to understand the complexity of their business.

As there was absolutely no analysis and modeling... Forget about planning their supply chain , we were just not able the understand the complexity of their business.

This project failure made me think about a very fundamental question.

How can a project fail when there were some many brilliant people working on this project?

This project made me realize one of the biggest limitations of human beings wrt dealing with complex systems.

I realized a very simple fact that individual human brilliance can not be sufficient while dealing with complex systems.

One of the problems that I found out was...

While interacting with a foreman of an oil well, the foreman was able to describe every other aspect of his oil well that he could visualize at that moment. But for the other aspects of the oil well that he was not able to visualize , he was not able to describe the same, although he knew about the same very well. Hence the requirements given by the foreman were just a fraction of the actual requirements .The same thing applied to the person who was gathering the requirements as well . He was able to capture every other thing that he was able to visualize but for the aspects of the oil well that were not visible, it didn't exist for him.

This was the case, nearly with most of the people involved in requirements gathering. Individually each of them understood just a fraction of their respective parts, but no one was able to understand the complete picture and the resultant complexity.

This was the first time, when I realized the importance of visualization and was the first step in finding out a very important human limitation called as "Single Frame of Vision" , which is greatly responsible for the failure of software systems.

The problem was multiplied by the fact that no standardized models were used to understand the complexity of the existing system. There was no standard way of communicating the requirements to various stakeholders. The requirements were actually communicated in the form of UI Snapshots with no other supporting documents and models.

Now I would say that it is just unimaginable that , how did the management think of implementing this huge project in such a maverick style solely dependent on the individual brilliance of the engineers.

This project also helped me understand that although the professionals working on this project were brilliant researchers or innovators, but were bad at engineering.

Complex system needs more engineering best practices to manage the complexity rather than individual brilliance.

Hemant Jha
Founder - VPlanSolutions
Researcher, Trainer