Motivation and evolution of ensemble forecast .1 Chaos

Lorenz (1963) mathematically described the unstable characteristic of the fluid dynamics. Given a finite systems of deterministic ordinary nonlinear differential equations which is designed to represent the hydrodynamic flow, the solution is sensitive to the initial conditions. A slightly differing initial states will quite frequently end up in considerably different states, which correspond to calm or stormy weather ('Edward Lorenz'). However, the chaotic nature of the fluid dynamics and the limited observation data certainly lead to uncertainty in the estimation of true initial state. As a consequence, it is hard for a single set of initial states to properly predict the final state.

This influential paper established the field of chaos theory and have led to the theoretical development of ensemble forecasting.

Incidentally, Sanders (1963) investigate the uncertain nature of subjective probability forecasts by averaging subjective weather forecasts, illustrating the superiority of the combined procedures. Latter, Bosart (1975) also tested on the performance of the average subjective weather forecasts and confirm Sanders (1963)’s results.

5.1.3.2 Early development

Stochastic method was applied to “assess the value of new or improved data by considering their to assess the value of new or improved data”(Epstein 1969, 739). The Monte Carlo experiment results shows stochastic dynamic predictions have outperformed traditional deterministic procedure significantly in terms of mean square errors.

The ensemble conduct continued, as I mentioned before, in 1974, Craddock who worked in Meteorological Office was invited to the discussion of Newbold and Granger (1974) has commented on forecast combination in the perspective of ensemble weather forecast. In a consensus of combined ideas, professors from economic and business cooperated with meteorologists. Winkler and Murphy (1977) showed the outperformance of the average of subjective precipitation probability forecasts over

individual opinion made by numerical weather forecaster. Clemen and Murphy (1986) average the subjective weather forecasts and model output statistics, finding again the superiority of the combined product over the individual one. Inspired by meteorology, professors from economic and business applied forecast combination techniques in the context of weather prediction. Clemen (1985) based on the Bayesian framework outlined by Morris (1974) to discuss whether human weather forecaster can bring new information to the mechanical guidance forecast and whether on human weather forecaster bring incremental information over the other, in the context of precipitation probability forecasts.

To sum up, meteorologists has tried to model weather system since 1951 but the innate chaotic nature prevent them to easily do so. Ensemble forecast has been put into practice since 1963 but was mature till 1999 which has to thank to the idea from forecast combinations (Newbold and Granger 1974) and the advanced computing power.

Professors in economic and business were also inspired by meteorologists, doing forecast combinations researches in the context of weather forecast, such as applying forecast combinations technique (Winkler and Murphy 1977) or investigating the informational contribution in the context of weather forecast combinations (Clemen 1985). A brief summary is provided in Exhibit 9.

Ensemble forecasting provides more information to weather forecasters. In the process of producing ensemble forecast, researchers gain some benefit from comparisons. If the estimated forecasts vary a lot, researchers can thereby calculating a probability of the uncertainty for the final product. On the other hand, if the estimated forecasts are all very similar, forecasters may therefore have more confident on the accuracy of the final product. ('Ensemble Forecasting'). The application of ensemble forecasts has not been restricted to weather forecasting, it also applies to flood forecasting and bunches of projects (Cloke and Pappenberger 2009).

Exhibit 9 Evolution of ensemble forecasts 5.2 Seismology and Ecology

To enrich the findings in the application of combined ideas, embedded combined ideas in seismology and ecology are illustrated in this thesis as well, which potentially enlarge the modified exhibit in Chapter 4.4.

5.2.1 Seismology

The well-known fact that “[e]arthquake prediction research has been conducted for over 100 years with no obvious successes” (Geller 1997, 425) provide me an impetus to dive into this branch of study. No matter to address uncertainty in the choice of proper prediction models (Cooper and Nelson 1975; Nelson 1972) or to improve forecasting accuracy (Sanders 1963; Bates and Granger 1969), combination techniques were widely applied as a nature corresponding answer. However, it’s until late 1990s, seismologists start to use combination techniques in papers. Most of the review of seismology still ignores the developing combination techniques (Ben-Menahem 1995;

Agnew 2002). Seismologists still mainly used the term ‘combination’ in physical meaning, such as the ‘combination of point forces’.

Without doubt, the practical concern is the most important spirit that motivate seismologists to use combination techniques. After all, the more precise we can avoid the destruction of earthquake, the more safety we enjoy. Combination techniques has applied in earthquake forecasting and also applied in the earthquake building engineering such as seismic response spectra and nonlinear dynamic analysis to reach reliable estimate.

Marzocchi et al. (2012) illustrated the uncertainty problem seismologists

have met with and clearly pointed out the practical value of merging models, saying that:

[I]n practice, we never know which of the candidate models will be the best in a long testing phase. We also note that the best candidate model may capture one important part of the earthquake generation process well, while others might suitably represent secondary, or at least more subtle, features (2577).

The spirit and the response of the cited paragraph are just the same as that in economic forecasting. Both of them used the combination techniques in address of uncertainty problems (Winkler 1989, 608):

In our uncertain and rapidly changing world, I think that adhering strictly to this [the development of ‘true’ model] ideal is counterproductive in most important forecasting situations. I prefer to view forecasts as information and the combining of forecasts as the aggregation of information. The key question is how best to accomplish this aggregation (606).

Regarding the uncertain problem, Neither Marzocchi et al. (2012) agree with the practice that “simply adopt the model that has performed best so far and disregard all others” (2577) nor do several scholars who do economic forecast. They all reason the advantage of forecast combination for some of its component models may outperform over the up-to-now best model in other period.

5.2.1.1 Earthquake forecasting

Combination techniques in earthquake forecasting had been happened relatively late. Fedotov et al. (1977) mentioned it while compared two earthquake statistics method. And just one sentence did he said about the combination techniques in the middle of his paper: “Thus, a combined use of various methods seems to be one of the hopeful ways of increasing efficiency of prediction” (320). No more in introduction and no more in conclusion as if nobody would care about his suggestion.

To my knowledge⁶, the next time that seismologists came up with the idea of combination had to wait until 1990s. Sobolev et al. (1991) followed Fedotov et al. (1977) drawing compiling maps of expected earthquakes and both of them found positive forecasting ability in real time. And more pertinent to this paper, Sobolev et al. (1991)

6 Though Sobolev et al. (1991) mentioned Aki (1981) as a forerunner of precursors combinations as well, I found that Aki (1981)’s work is more pertinent to an universal measurement that would be useful for

discussed further the issue of combination. By using Bayesian formula, they combined the probability of expectation of a large earthquake with prognostic precursors. Up to 1989, three earthquakes occur within the areas of expected earthquakes but outside the center. But there were to areas indicated high possibility of earthquake but no strong earthquakes have been reported yet. Criticism though admitted that this paper pushed forward the concept of using combination techniques to improve forecasting ability, pointed out that the interdependence of the combined elements may be a potential problem (Shebalin et al. 2014).

Other than the Bayesian method, because “[i]t is well known that combining many models ……may yield higher performances than any individual member” (37), seismologists are working on their way to combine forecasts in their method. Shebalin et al. (2014) proposed a rate combinations method. It transforms different model outcome (for example: some measures in level, some measure in number) into one base and then multiplies the parameter derived with earthquake occurrence rate to get a new combined model.

After 2000s, more papers are aiming at investigating combined short-term and long-term models and correspondingly exhibited a ‘suspected’ close relation between earthquake forecasting and forecast combinations. I use the word ‘suspected’ because though no directly citation from Granger or other scholars in economic and business, the similarity in the tool and the spirit indicate a strong tight between the two subjects.

Rhoades and Gerstenberger (2009) combined short-term earthquake probability (STEP) forecasting model with long-range earthquake forecasting model EEPAS in their paper.

Each model typically based on time, density and location and a Poisson function to generate, a prediction, an earthquake occurrence rate. The authors combine the rate-based model by using the relative performance of the model as measurement to choose weights in the weighted average formula. Just as same as forecast combinations.

Moreover, they also evaluate model performance by some conventional statistical tools, such as AIC. Technically, the difference between forecast combinations and their mixture models is just that seismologists tend to write down likelihood function and use simplex method, a popular algorithm for linear programming, to solve the optimization problem. Rhoades (2013) extended the study to a more wide ranging.

5.2.1.2 Seismic analysis

Seismic analysis is an earthquake building engineering and is a subset of

structural analysis. A basic method of structural analysis methods is Seismic Response Spectra, a subset of Response-spectrum analysis (RSA) which measures the contribution from each natural mode of vibration to indicate the likely maximum seismic response of a structure. Therefore, the total response of a structure can be defined as a combination of natural mode of vibration. As a note, the term ‘spectra’ in the method name means that, for each mode, a response is read from the spectrum.

Seismic response spectra is the application of response spectra in earthquake engineering. By finding out the natural frequency of a structure and the peak response of that, the forces that a structure must be designed to resist can be derived. (Gupta 1992)

Regarding the complex response of buildings to earthquakes, more complex models were proposed, such as nonlinear dynamic analysis. The response of detailed structural model to ground motion was recorded and combined to lower the uncertainty of estimation. Similar to the case in weather forecast, the outcome forecast is sensitive to the initial condition. The response of buildings to earthquakes is very sensitive to the characteristics of the ground motion, the input, as well. As a consequence, a combination technique is needed to derive the final reliable estimate (Wilson et al.

1972).

Today, precise and timely earthquake prediction is still difficult if not impossible but scientists have already reflect on the uncertain property of the natural phenomenon.

More and more seismic techniques that used combined skill have already put into use, not only for earthquake forecasting but also for understanding the earth structure deeply, such as the combination skills in seismic tomography (Valentine and Woodhouse 2010).

5.2.2 Ecology

The complexity of ecology has put biologists in a great debate about how to model it. In the beginning of 20^th century, Clements (1916) proposed the formation of a plant community as a complex and integrated organism. Phillips (1931) approved with Clements view. And a politian coined the term ‘holism’ with a plethora explanation on the integrated biotic community (Lefkaditou and Stamou 2006). In disagreement, Tansley (1935) advocated a mechanical interpretation on the ecology system called ecosystem. Besides, in rejection of the downward causation proposed in holism, Gleason (1926) advocated for the idea that each community is interacted with its own fickle environment and therefore to study the temporary and fluctuating

community, a population-centered view of ecology is encouraged.

The debate continued to 1960s. Eugene Odum and Howard Odum adopted the view of holism with that of reductionism, which was noted as systems ecology.

While they are promoting the concept of an integrated community, they also stress on the physical interpretation of the ecosystem (Odum 1969). But Levins and Lewontin (1980) criticized them as just forming a large-scale computer models and “do not fit under the heading of 'holism' at all” (50). On the other hand, MacArthur and Wilson (1967) held a Newtonian worldview promoting to form the knowledge of the complex ecosystem by investigating the basic components. The debate has continued half a century; however, the debate is trivial in a sense. Because holism has never put their ideal in practice, there is actually only one modelling method in use, that is the method of reductionism (Lefkaditou and Stamou 2006).

Richard Levins’ critiques toward system ecology made him seemingly properly to be categorized as a reductionists in the dichotomy; however, philosophically he held a holistic perspective (Levins 1974):

The most difficult general problem of contemporary science is how to deal with complex systems as wholes. Most of the training of scientists, especially in the United States and Great Britain, is in the opposite direction. We are taught to isolate parts of a problem and to answer the question “What is this system?” by telling what it is made of (123).

But there was no proper holistic method. To Levins, modelling a one-to-one reflection of the complex ecology is difficult in practice if not impossible (Levins 1966).

More importantly, he criticizes that system ecology is just a form of large-scale reductionism (Levins and Lewontin 1980). By addressing the problem that there is no proper holistic approach available, he brought breakthrough.

5.2.2.1 Model the complexity

To deal with complexity and “work with manageable models which maximize generality, realism, and precision” (Levins 1966, 422), Levins proposed to build on several models that trade-off generality, realism, and precision. Though he did not explicitly mention that information should be collected through model combinations but in the end of his paper he concluded that “a satisfactory theory is usually a cluster of

models” (431). The underlying spirit is that building up a universally applicable model does not guarantee its validity but “generat[ing] good testable hypotheses relevant to important problems” (430) practically enables researchers to do validation test and only a cluster of verified models validates a theory.

In response to his 1966 work, Levins then proposed ‘loop analysis’ to partially specify the system or, say, combine some of the models (Levins 1974).

Different from forecast combinations, loop analysis is a qualitative method and is known as ‘qualitative model’ in ecology also. Loop analysis is typically drawn as diagrams, called ‘signed digraphs’, with circles and lines that represent the relation between two variables. For example, if there is a direct positive effect of one variable upon another, the line will be ended in a pointed arrow; if a negative interaction, the line will end with a solid circle; otherwise, will be an absence of line. The three signed representation can be converted to three numbers -1, 0, 1 and form community matrix.

The next step is usually using some matrix algebra functions to test the system or predict the behavior of system response to a disturbance. (Puccia and Levins 1991).

This innovation approach is intended to correct the “one-sided analytical quantitative approach” then (Levins 1974, 137). The trial is rewarded. Because its simplicity, loop analysis is now accepted as a standard method in biology.

Levins does not put more emphasis on combined model after Levins (1974), but stresses more on the use of dialectic materialism to merge the gap between the mechanistic materialism and idealism. He encourages the exploration of the complex interrelationship in ecology by qualitative method, a dialectic way, but not numerically synthesizing results of models (Levins and Lewontin 1980; Levins 2006). His thought was not noticed in the 1960s and 1970s, but then a bunch of papers dealing with his strategy of modelling as well as his philosophical positions has appeared.