So the difference is only knowingly or not? Exploiters, who see the traps but transform them into margins, or "fools", who do not see or understand them?
New products or services are made for more effectiveness or efficiency when used or even enable things that could not be done before. They require modeling how they will be used. And this is not so easy.
However, in the article it is recommended to follow five rules when innovating:
- Recognize that a model is needed making assessments about risk and return
- Acknowledge its limitations
- Expect the unexpected
- Understand use and user
- Check the infrastructure
Models can be formal or intuitive, but in quant finance, computational knowledge to price and allocate products is indispensable. Modeling their usage is in-scenarios, what-if analysis and what have you. So if the models for the single valuations are already intrinsically flawed, too complex, not robust, ... the model of their applications are consequently wrong. But if core models are right, the use models can still be wrong because of new complexity, wrong operation or interpretation.
Take the trap of tightly coupled complex systems - because of the complexity you have unintended consequences but the tight coupling does not give you enough time to react to them. You have done "three things" right but the result is wrong.
Or you want to, say, price an exotic instrument - you know the right model (and solvers), market data and how to calibrate (with clever regularization techniques). And then: the fit is right, why is the price so wrong? If the total complexity is too high, little errors will become horrible in interplay.
Models are not only constrained by their users' understanding. If they are not organized orthogonally, the infrastructural complexity and timing may cause a hidden problem that is also difficult to manage (hidden in plain sight).