Andreas in The Beauty of Metal Skis describes a very general problem in a prototypical case.
You have a model that works perfect under lab conditions but how to transfer it into the conditions of the real working space?
To me a model is "perfect" in the sense of "speculative realism" if it contains all "influencing" parameters that we know from the behavior of the real wold system (a kind of an isomorphic map). If you also have an isomorphic map from a theorem described by mathematical symbols you have a mathematical model. But this does not say anything about their real quantities.
I divide this mathematically modeled systems into 2 classes: those that allow for the extraction of informative data from the "process" and those that can't.
Work with informative or uninformative data?
Like in Andreas example it is often about material properties. In the case of metallurgical materials you may obtain the required parameters in the lab or during the process (like in forming …) Wood, composite materials, .. are very difficult.
We discuss this with our quantum physicists, Michael and Stefan, and they say: material research emphasizes on the micro-, meso- and macro-levels (say, the continuum mechanics). You need to dive into the micro level, if you, for example, want to understand crack propagation, creeping, ... in a, say, forming, cutting, welding ... process.
But also in mechanism where you want to control optimal paths (multi axes machine tools, robots, satellites, antenna systems ...), you need to calibrate the control system to the real system behavior. You may have applied an advanced Discrete Mechanics and Optimal Control approach, but the real system has special properties …
Riding the waves of observed data
Shifting from the lab world into the "real"world is calibration. An inverse problem. You will do it wrong if you don't know the traps and how to overcome them (by regularization techniques, ...) or if you have uninformative data.
In The Blank Swan of Metal Forming I pointed out that some physics problems and quant finance problems are not so different.
You have just structured a great new exotic option type. You replicate it by, say, a Heston model that is calibrated to vanilla options. You used the most advanced solvers (and cross-model validation) and did the calibration right.
The fit is so well, but the price may be so wrong, because you don't have market prices of members of your option class. Your market data are too uninformative.
What to do? You need to make provisions for using adequate real time market data (if you get some) in the real trading process. Recalibrate constantly by riding on the price waves.
BTW, the momentary "battle" between the "econophysicist" and the economists is also about this: are physicists problems of better nature to create computational knowledge, because they have always informative data? I will come back to this here.
Picture from sehfelder
