The main design and technical problem (ignoring for the moment Weinberg’s maxim that “No matter what they tell you, it’s always a people problem”) was that the contractor had decided that this was an opportunity to develop a reusable software system, and that they could develop this system on the customer’s dime.
This resulted in classic framework-itis. The contractor was no longer just trying to solve the customer’s problem, they were trying to solve all problems that looked like it. So for that reason alone the project would have required 10x the original time and money.
As I ponder on these points, I think of Paul Graham‘s essay Taste For Makers, where he states, “Good design solves the right problem.” When I think about getting at the heart of the problem, I think of the chapter about Use Cases in The Pragmatic Programmer. If the goal is to design a non-trivial library for thousands of users with many different usage patterns, how do you get at all the use cases? You can’t. The thing to do is to make something that solves one problem well, and adapt it, one by one, to solve other problems. Eventually, it will solve a lot of problems.
I think the Linux kernel is a good example of this. The first Use Case was Linus Torvalds running it on his own computer. In time it was adapted to run on many different computers and to do many different things. It had fewer growing pains than other things that were built Cathedral-style.