Analytics Myth: Can we Predict the "Unpredictable"?

The prediction of the future is based on the analysis of the past. The best prediction is optimal.

Business analytics is on its progressive and “forward-thinking” journey, moving from descriptive analytics (analyze what happened) to predictive analytics (what will happen) and prescriptive analytics (how to respond to it). Still, it’s an emerging scientific discipline and even a digital management transformation. There are many puzzles need to be solved and numerous questions waiting for the answers. Here is an interesting debate: Can we predict the "unpredictable"?

The prediction of the future is based on the analysis of the past. The best prediction is optimal. Bellman's optimality principle provides a global maximum on the set of all variants. Much depends on the evaluation criterion. The principle that an optimal sequence of decisions in a multistage decision process has the property that whatever the initial state and decisions are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decisions.

Analysts/scientists think the optimization is more applicable to decision making using multivariant forecast. What is done in discussed method is actually converting multivariant forecast into monovariant forecast using optimization with specific (haos-related) criterion. The optimal prediction system should have the best results in different areas according to the specific or customized quality criteria. In this sense, obtaining statistical validation is a matter of time. The accuracy of the prediction will depend on the completeness of the input information. Dynamic programming was created for this task.

The analysts/scientists do not pretend that they can predict something which is random, but something which is chaotic. A chaotic process is deterministic therefore it is theoretically predictable. The data analysts do not have or try to compute a function that represent the evolution of the market time series. The only thing they do is that they observe the level of chaos of the market time series is constant then they generate the future points in order that the new whole time series conserve the same level of chaos. It is an optimization process in which, at each step, they select, among a set of randomly generated potential points, the one that minimize the distance between the new value of chaos level of the new time series when they include this point and the initial value of level of chaos measured on the original time series.

The generation of a new point is an optimization process and there is no guaranty that the optimal value is unique. Note that good universal algorithm should in some cases produce "no forecast" as result - just because random events can not be predicted by definition. The measure of chaos is a function but more often the data analysts do not have a function that describe the time series values; or the prediction algorithm pretends to be universal: sequence of segments with random length, odd segments represent sinusoid slightly randomly distorted, even segments represented by constant value slightly randomly distorted too. Switching points correspond to certain external events.

The characteristics of digitalization are complexity, uncertainty, unpredictability and ambiguity, data analytics is the significant tool to help digital professionals (from leaders/managers to front desk/customer reps) make the right decisions (both strategic and tactical) at the right time, on one side, technology is making significant progress, but there is still a long way to reach its full maturity; the talent data analysts/scientists empowered by smart tools can make forecast more accurately in varying industries, still, there’re many things that are unpredictable yet, you just have to prepare for the different scenarios.

Posted in: Big Data