“All models are
wrong, some models are useful"
-George .E.P. Box
Model is by
definition an approximation of reality - why would someone want to model
reality in full detail and precision. A model is developed to help you with
something - answer a question to some degree of precision that is useful or let
the complexity shine through with a certain degree of insight. A model will never
be perfect, but a model can still be useful if one knows what to use it for. So
even highly chaotic systems can be modeled and should be modeled. Modeling
serves many purposes beyond forecasting –not the least of which is to device
adaptive strategies for highly chaotic environments.
Problems with
Modeling: Most problems with models are that people make simplistic
assumptions, sometimes fail to make these assumptions explicit or assume that
future can be predicted by simply looking at past history. Users of the models
don’t bother to look at assumptions, assume that they live in a linear world
where everything is normally distributed and proceed to use a point prediction
made by a model.
Many “soft factors” can be modeled and should be modeled to guide better decisions: When you choose not to model them, you are essentially saying that they have no impact – so it is still there in your model, but with a multiplier of 0. If soft factors are important to a problem, then a model should incorporate them – yes the results will not be very precise and you will not get a point prediction, but you are likely to have fewer errors than if you completely ignored those factors. There is a large body of work in system dynamics modeling and agent-based modeling where soft factors are routinely modeled and used for many applications. The problem with all the "soft" modeling is that when it comes to reality, and making real-time control decisions with optimization, not marketing or lab type estimates, one tends to want a "good estimate" or a more correct estimate", or a prediction that is "favorable most of the time", note that continuous reference to time, and the "fuzziness" in the colloquial sense. Eventually in the real world, if these predictions are not "safe" enough, stuff blows up. So then you "constrain" or "limit" solutions with a safety net that is in many cases "intuitive" or "based on experience" to the point where the solution may become over-constrained or useless.
Forecast is Useful: You simply can’t model reality properly, too many
permutations, unknowns, interactions, uncertainties, non-linearity, with
"singularities" etc. It is true that you cannot make point predictions,
but that is not the reason for doing forecasting. The point of forecasting (which
implies a range of outcomes) is not to predict what precisely will happen, but
to help you make decisions and take actions today. Point predictions are
wrong, but forecasts can be useful. The principles to do modeling:
- Ignore soft factors in models at your own peril.
- Job of models are not prediction or perfection; it is to guide actions today.
- It is possible to model very complex system behavior via very simple components
- Understanding the limitations of each model you use is, therefore, the key. Using models without understanding their limitations exposes you to big surprises.
0 comments:
Post a Comment