|
Our technology forecasting system is an improved version of the Delphi technique consisting of the five phases below:
Phase 1 Scanning We first do extensive scanning of the scientific literature, media, Internet, interviews, and other sources to accumulate background data.
Phase 2 Analysis Next we organize the scanning data into a detailed analysis consisting of a summary, other forecast data, the event to be forecast, and trends opposing and driving each technology. These key elements are defined below. The analyses are considered especially valuable because they summarize the best available knowledge on any technology in a convenient format. Detailed analyses for all technologies inbcluding the Expert Survey Forecasts are listed under the Forecasts page.
Other Forecast Data summarize available information on the present adoption level of each technology, any forecasts that are available, estimates of economic demand, and other facts relevant to the Event Being Forecast.
Event to be Forecast (or a “milestone”) is a precisely defined adoption level to be forecast for each technology. We usually use the 30% adoption level, but other levels are used where appropriate. The 30% level is of particular interest because technologies usually enter the economic mainstream at this point. This method is so flexible that almost any event or variable can be forecast.
Cons consist of trends opposing the adoption of the technology. Cons can take the form of limited technical performance, high cost, political obstacles, lack of social acceptance, limited business development, and other factors.
Pros comprise trends driving the technology. They typically describe technical breakthroughs, business investment, examples of successful adoption, changes in government policy, statements by prominent authorities and the like.
Phase 3 Expert Survey The experts go online to integrate all this information using their judgment and knowledge. The panel is listed in “About” and includes a rich mix of differing perspectives.TechCast strives to enlist competent authorities with advanced degrees, extensive publications, relevant experience, and breadth of knowledge. They are asked to focus on areas they feel most knowledgeable about, so not all respond to all technologies. Delphi studies are considered reliable if they include a dozen or more experts, and we surpass that criterion considerably. (1)
Phase 4 Results The system aggregates these estimates to forecast: 1) The Most Likely Year each technology will reach the level specified adoption level. 2) The Market Size of the technology when it matures, 3) Our Experts’ Confidence in this forecast. These data are presented in a Forecast Data Analysis table. Frequency distributions are also provided to enhance transparency. The data are presented in various formats that enhance analysis, as in the tables under Forecasts and the bubble charts highlighting strategic analyses for each field of study.
Phase 5 Iterations Experts’ comments and new background information are incorporated in an updated breakthrough analysis, and this process is repeated every year or so to “track” the forecast over time, allowing us to extrapolate the best possible forecast. “Arrival” dates are also noted to evaluate the accuracy of forecasts. TechCast analyzes the results to identify which qualities make more accurate experts.
Accuracy
The obvious question raised by Delphi forecasts of this type is, “How accurate are the results?” TechCast has been using this method for 15 years on a variety of projects, and analyses of these results show that the variation among forecasts averages +/- 3 years, with standard deviations averaging 4.3 years. Some technologies vary more widely because they are controversial, while others show little variance because they are well established. We have also recorded “arrivals” of several technologies, all roughly within this likely error band of +/- 3 years. These results are compelling when it is recalled that the expert panel changed over this time, as did the prospects for various technologies, and other general conditions.
The field of “Knowledge Management” (KM) offers a useful perspective for understanding the rationale underlying this methodology. From a KM view, the TechCast approach can be understood as a “learning system.” conducted by a “community of practice” to “continually improve” results. This process of gathering background information, organizing it into a coherent analysis, surveying experts, and using results to improve the system allows the experts to continually learn and thereby approach a “best possible forecast” based on a “scientific consensus.”
Some contend that methods replying on expert judgment are subjective, whereas quantitative methods are more precise. Quantitative methods also involve large amounts of uncertainty because of underlying assumptions that must be made. The TechCast approach subsumes quantitative forecasts into the analyses provided to experts, and then allows their considered judgment to resolve the uncertainty that remains.This consensus can be in error, of course. But it represents a synthesis of the best available background information and authoritative knowledge to produce the best possible answer to a tough question. Experts may have their own bias, naturally, but it is usually distributed normally, washing out in the aggregate results.
Overall, if the present uncertainty is defined as 100%, we have found through experience that this process reduces uncertainty to about 20-30%. Some think of the outcome as “good enough to get a decision-maker into the right ball park.” As noted above, results can become even more accurate by using expert comments to improve the background analyses and by tracking forecasts over time.
|