Is it enough to assess, whether a model or methodology is fit for its purpose?
No. What is the subdomain where it is? Can it be easily extended to become more adequate covering a broader variety of deal types (extrapolation is dangerous)?
Not surprisingly the quality assessment of model risk is not so different from any assessment of a mathematical problem solving process.
- is a model describing the problem applicable in principle?
- is it correct in describing the reality?
- can it be transformed in a form that can be solved in the expected accuracy?
- are there numerical schemes that approximate the model adequate?
- are the inverse problems of calibration solved correctly?
- are the implementations adequate, precise, stable and robust?
- are the data right and curated?
In short, models need to be theoretical sound, mathematical adequate, correctly calibrated and solved and error-free implemented.
It is obvious that model validation is easier when the assessing models, solvers and implementation are organized orthogonal.
Models need to be consistent with market practices and contractual terms and it is vital to know, how they impact variations in model parameters, say, under stress conditions.
And because process-through simulation across many factors is important, they provide the most comprehensive assessment of model risk, it is obvious that speed matters.
We all have observed that models can be calibrated perfectly to price a set of liquid instruments, but mis-price exotics. The fit is so well, the parameters obtained exhibited good time stability and robustness, why is the price from different volatility models so different? This raises the question of the required frequency of re-calibration and the impact of the models.
For such a complex task, model validation teams might want to be equipped with a technology platform including pricing and calibration engines and a domain specific declarative programming environment for defining higher level valuation and simulation tasks. They need to run scenarios and cross-model tests independent from the front office model selections and valuation practice.
They need to come as know-how package - UnRisk-Q.
They need to come as know-how package - UnRisk-Q.
