Our technology forecasting system is an improved version of the Delphi technique, consisting of the five phases below:
Phase 1 Scanning We first scan the scientific literature, media, Internet, interviews, and other sources to accumulate background data.
Phase 2 Analysis Next we organize the scanning data into an analysis consisting of a summary, other forecast data, the event being forecast, and trends opposing and driving each technology. These key elements are defined below. The analyses are valuable because they summarize the best available knowledge on any technology in a convenient format.
Selected Adoption and Forecast Data summarize available information on the present adoption level of each technology, any forecasts that are available, estimates of economic demand, and other facts relevant to the Event Being Forecast.
Event Being Forecast is a precisely defined adoption level, milestone, or target to be forecast for each technology. Technologies are tracked as they pass through the typical life cycle of commercial introduction (intro.), mainstream use (30%), etc. See Tools/Adoption Stages. We usually use the 30% adoption level because technologies enter the economic mainstream at this point, but others are used where appropriate.
Cons consist of trends opposing the adoption of the technology. Cons can take the form of limited technical performance, high cost, political obstacles, lack of social acceptance, limited business development, and other factors.
Pros comprise trends driving the technology. They typically describe technical breakthroughs, business investment, examples of successful adoption, changes in government policy, statements by prominent authorities and the like.
Phase 3 Expert Survey The experts go online to integrate all this information using their best knowledge and judgment . The panel is listed under Experts and includes a rich mix of differing perspectives.TechCast strives to enlist competent authorities with advanced degrees, extensive publications, relevant experience, and breadth of knowledge. They are asked to focus on areas they feel most knowledgeable about, so not all respond to all technologies. Delphi studies are considered reliable if they include a dozen or more experts, and we surpass that criterion considerably. (1)
Phase 4 Results The system aggregates these estimates to forecast: 1) The Most Likely Year each technology will reach the specified adoption level. 2) The potential Market Size of the technology when it matures, 3) Our Experts’ Confidence in this forecast. These data are presented in a Forecast Data Analysis table. Frequency distributions are also provided to enhance transparency. The data are presented in various formats that enhance understanding, as in the Forecast Tables and the bubble charts highlighting strategic analyses for each field.
Phase 5 Iterations Experts’ comments and new background data are incorporated in an updated forecast, and this process is repeated every year or so to track the forecast over time, allowing us to extrapolate the best possible results. Arrival dates are also noted to evaluate the accuracy of forecasts. TechCast analyzes the results to identify an Optimism-Pessimism rating for each expert and to identify which qualities make more accurate experts.
Accuracy
The obvious question raised by Delphi forecasts of this type is, “How accurate are the results?”
TechCast has been using this method for 15 years on a variety of projects, and analyses of these results show that the variation among forecasts averages +/- 3 years, with standard deviations averaging 4.3 years.[1] Some technologies vary more widely because they are controversial, while others show little variance because they are well established. We have also recorded “arrivals” of several technologies, all roughly within this likely error band of +/- 3 years. These results are compelling when it is recalled that the expert panel changed over this time, as did the prospects for various technologies, and other general conditions.
The field of “Knowledge Management” (KM) offers a useful perspective for understanding the rationale underlying this methodology. From a KM view, the TechCast approach can be understood as a “learning system.” conducted by a “community of practice” to “continually improve” results. This process of gathering background information, organizing it into a coherent analysis, surveying experts, and using results to improve the system allows the experts to continually learn and thereby approach a “best possible forecast” based on a “scientific consensus.
Some contend that methods replying on expert judgment are subjective, whereas quantitative methods are more precise. Quantitative methods also involve large amounts of uncertainty because of underlying assumptions that must be made. The TechCast approach subsumes quantitative forecasts into the analyses provided to experts, and then allows their considered judgment to resolve the uncertainty that remains.
This consensus can be in error, of course. But it represents a synthesis of the best available background information and authoritative knowledge to produce the best possible answer to a tough question. Experts may have their own bias, naturally, but it is usually distributed normally, washing out in the aggregate results.
Overall, if the present uncertainty is defined as 100%, we have found through experience that this process reduces uncertainty to about 20-30%. Some think of the outcome as “good enough to get a decision-maker into the right ball park.” As noted above, results can become even more accurate by using expert comments to improve the background analyses and by tracking forecasts over time.
--------------------------------------------------------------------------------
1. Harold Linstone and Murray Turoff, The Delphi Method (Reading, Mass: Addison-Wesley, 1975)
2. William E. Halal et al, “The GW Forecast of Emerging Technologies” Technological Forecasting & Social Change 59 89-110 (1998)