An Architecture for Distributed Scenario Building and Evaluation: Collaborative Learning from the Future

A Web portal can support collaborative learning and help individuals and

organizations plan for an uncertain future.

M

(

)

PLANNING TO ADDRESS THE INCREASING UNCERTAINTIES THEY MUST FACE

. U

-TAINTY FORCES ORGANIZATIONS TO RECOGNIZE SEVERAL PLAUSIBLE FUTURES AND

DEVELOP STRATEGIES AND POLICIES THAT APPEAR BEST IN LIGHT OF THE UNCERTAIN

-TIES DEFINED

[10]. F

,

-NOLOGY DROPS OR AN INDUSTRY FACES NUMEROUS DEBT DEFAULTS

? H

STRATEGIES AND POLICIES NEED TO BE CHANGED TO ENABLE BUSINESSES TO MOVE

QUICKLY TO CONFRONT SUCH EVENTS

?

A

N

A

R C H I T E C T U R E F O R

D

I S T R I B U T E D

SCENARIO

BUILDING

A N D

E

VA L U AT I O N

By Rosemary H. Wild, Kenneth A. Griggs, and Eldon Y. Li

During scenario planning, factors that might affect the outcome of such future events must be identified and categorized by their degree of uncer-tainty and likely impact on outcomes. For example, in light of the 9/11 attacks, government planning for homeland security must take into account threat assessment, counter-terror activities, threat reduc-tion, and disaster recovery. These types of events

might be categorized as wild cards, that is, they are the most uncertain but would have an enormous impact if they occurred. Scenario planning cannot predict the future, but it can help organizations become more aware of possible outcomes by encouraging planners to envision, learn from, and prepare for potential futures [4].

Longer-term scenario planning requires

ration by teams of stakeholders, which may include economists, sociologists, financial analysts, and domain experts. Royal Dutch/Shell created one of the first scenario-planning protocols in the 1970s, now known as the Shell method,

which has served as a model for scenario planning for many U.S. firms. The Shell method involves workshops at company locations around the world and “learning journeys” in which executives visit sites across the globe to gain insight into how their scenarios might play out in an attempt “to benchmark [their] visions against assumptions” [5].

Some of the difficulties in suc-cessful scenario planning revolve around the fact that planning teams operate in a remote, asyn-chronous environment in which disparate teams make decisions governed by rules and policies not widely shared across the enterprise. In addition, for longer-term planning in which planning teams travel to geo-graphically dispersed locations over an extended period, much of the momentum accrued dissi-pates as the planning process

drags on. Thus it becomes increasingly difficult to test the robustness of strategies under each scenario as conditions shift over time. Even more important, remote planning teams may not possess the shared weltanschauung, or world view, required to guide a planning process that affects the entire enterprise and its stakeholders.

To address some of these difficulties, we propose a Web portal architecture (see the “Web Portals” side-bar) that enables enterprise planning teams to create scenarios and experiment with a decision support sys-tem (DSS) model of key planning processes, jointly, in a virtual community environment. The potential benefits of such an approach to enterprisewide policy and decision making are promising. First, experimen-tation with a DSS model enables knowledge creation. Likely future scenarios serve as a springboard for assessing the effects of a variety of policy implementa-tions on critical success factors for the firm. Second, a

Web portal enables teams that often operate in a stovepiped fashion to collaborate in an integrated electronic environment and share the results of sce-nario planning experiments. Third, if designed

prop-erly, a Web portal can assist teams collaborating on the planning process to develop and share a world view of the organization and maintain a repository or history of the planning process. Fourth, distributed experimentation with a DSS model of the planning environment exposes risks and permits outcomes to surface in light of uncertainties.

DSSs are currently used for planning in non-dis-tributed environments, such as spatial planning using geographical information systems [8] and local law enforcement planning using a myriad of information technologies [2]. The U.S. military uses DSS for a variety of purposes, including strategic planning [1] and delivering military lessons learned [11]. All of these applications could benefit from an architecture that supports distributed planning through virtual sce-nario building and evaluation.

Li fig 1 (11/05)

Scenario Data

Model Data, Rules, Constraints Research Reports, External Data User Profiles Lessons Learned Collaboration Support DB Web Portal Server Database Tier Browser Tier • What-If Models • Optimization Models • Simulation Models • Goal Seeking Models • Statistical Models Model Manager Collaboration Engine Presentation Manager Data Management Scenario Builder Rule Manager DSS Engine

Figure 1. Web portal architecture for distributed scenario building and evaluation.

WEB PORTALS

Just as people or teams in an organization involved in planning through scenario building form a community dedicated to making decisions in light of likely but uncertain futures, the technology required to support this activity must possess attributes that embody a sense of community. A Web portal architecture reflects a virtual view of an enterprise and can be customized for a specific

purpose, such as scenario development and evaluation. It contains features that permit users to collaborate and communicate in a virtual but focused community in which teams interact remotely and often asynchronously.

A Web portal is a secure, single point of interaction with diverse information, business processes, and people, personalized to a user’s needs, preferences, and responsibilities. The characteristics that make a portal unique are:

•Customization: When a user authenticates (logs in) to the portal, the authentication information

determines what the user will see;

•Personalization: A user can select and store a personal set of appearance and content

characteristics that may differ for every user;

•Adaptiveness: The portal gets to “know” the user through information supplied by the user and

information it is programmed to gather about the user. As a user’s role changes, the portal will detect the change and adapt to it without human intervention; and

•Desktop orientation: The portal becomes the user’s point of entry not just into the organization’s

Web spaces, but also to his or her desktop computer.

University campuses have been a natural focal point for the development of portals to serve as gateways to information, points of access for constituent groups, and community/learning hubs [7]. For example, Blackboard®offers a portal that permits integration of course management systems, student information systems, e-commerce systems, and other campus data systems and their portals. Campus portals are also used to provide access to university services such as student records, advising, and the campus library. The next generation of campus portals will employ intelligent agents to serve their users.

Businesses use a variety of portal types. Employee portals, for example, are internally focused and contain benefits information, report management and distribution information, function-specific kiosks (such as benefits enrollment), problem tracking, and help desks. Consumer portals are externally focused and used for a variety of applications such as customer orders, e-commerce services, personal-ized services, warranty registration, and support services. Business partner and supplier-focused portals provide corporate information that must be shared among business partners. Applications include inventory and channel management, private trading networks, shared application services, order management, and business self-service [3].

Regardless of the intended purpose and users of a specific portal, its platform includes interacting layers of application functionality that reside on top of existing systems and infrastructures. The systems layer provides the fundamental infrastructure containing databases, Web servers, and content repositories. The server layer runs, manages, and maintains Web-based applications. The integration layer enables connectivity between components with rules and logic that govern business processes and application services. The interaction layer manages connections between the user interface and services and content, supplying the rules and logic governing the behavior of user-to-application

connections. Lastly, the presentation layer supports access to applications and content through multiple devices and application forms. Technologies such as J2EE and Microsoft’s .NET architecture supply the adaptability necessary to support a comprehensive portal platform. c

ANARCHITECTURE FORDISTRIBUTED SCENARIOBUILDING

A unique architecture is required to sup-port distributed scenario building and evaluation. First, the architecture needs to allow disparate groups to operate as a vir-tual community with shared goals. Issues related to the development and nurturing of virtual teams are many and have been reported throughout the information sys-tems literature. For example, Sarker and Sahay [9] provide a detailed study of the development of virtual teams, and Jarven-paa and Leidner [6] discuss issues associ-ated with communication and trust in virtual teams. Second, the architecture must provide the functionality for remote teams to develop potential scenarios, experiment jointly with a DSS model that captures key planning processes, and manipulate factors affecting future out-comes. Third, it must provide mecha-nisms for remote teams to view and discuss the results of their experiments.

Fourth, rules and policies enforced by disparate teams must be accessible to everyone involved in the planning process for scrutiny and evaluation. Fifth, to encourage the development of a learning commu-nity, the architecture should contain a lessons learned repository that is available to all stakeholders involved.

Figure 1 depicts our proposed Web portal archi-tecture, which employs a variety of integrated com-ponents interacting over a wide-area distributed network, satisfying the needs outlined above. Peer-to-peer networking can be employed to keep each node current with relevant data and reduce network traffic. In the proposed architecture, the main “commu-nity center” for the portal environment is the collabo-ration engine, which provides a common real-time enterprise view of the scenario planning process, enabling and encouraging engagement by each plan-ning team. The collaboration engine manages user profiles, authentication, forums, chat rooms, and community information. Thus, users may meet in secure, virtual rooms that contain the unique charac-teristics of the community as well as community objects. The community objects include documents, multimedia materials, scenarios, and DSS models. In addition, the virtual rooms provide the connection infrastructure required (for example, protocol conver-sion for high-bandwidth international connectivity). The scenario builder tool creates a script that con-trols the organization of resources and sequencing of

events needed for a given scenario that is run using the appropriate DSS models. For example, data

from geographically distributed sources may need to be integrated with local data to create a composite data source for the executed DSS models.

A rule repository and accompanying rule manager store and manage rules and assumptions driving the DSS model. For example, if a simulation DSS model is employed, typically each planning team will have unique simulation parameters to manipulate. The rule repository and rule manager translate these parameters into a common form that is understandable by all par-ticipants. The rule manager serves as an editor for rules and data held in the repository. Ontological techniques could be used to develop a common DSS parameter language.

A DSS engine consists of a DSS application that allows users to set key model parameters (defining a specific scenario), specify experiment conditions (such as model assumptions and time frame), and run each scenario-driven model. It accesses data from opera-tional databases, invokes appropriate DSS routines, and emulates the dynamics of the planning process, such as the movement of objects in the model’s envi-ronment. The associated model manager handles DSS model integration.

A “lessons learned” repository uses standard data-base techniques implemented in the datadata-base tier level of the architecture. Lessons learned consist of an

COMMUNICATIONS OF THE ACM November 2005/Vol. 48, No. 11 83

Figure 2. Web portal interface.

indexed collection of text, charts, and graphics repre-senting a record of the events and outcomes of previ-ous or ongoing virtual planning scenarios.

Navigation through the data is handled by the pre-sentation manager and implemented as a collection of Web pages with hyperlinks. The server-based presen-tation manager also handles the relatively complex display characteristics of the application and provides data push functionality. Through data push, the por-tal can update user data without requiring a user request; this can be combined with peer-to-peer net-working to guarantee that participants have relevant data at the same time. The architecture is n-tier and distributed, allowing additional tiers to be integrated into the architecture as the need arises.

PORTALINTERFACE

The portal architecture provides an infrastructure that keeps all participants aware of the components of each defined scenario and the results of each DSS model executed. For the portal to imitate an actual community of decision makers, it needs an interface that encourages discussion, chats, and personalized views of the scenario building process. Figure 2 depicts a hypothetical portal interface consisting of a collection of views for which each view is detachable. The division of the portal into views is required given the complex nature of the interactions, the need

to have a multitude of open windows (with accompa-nying screen clutter), and the desire to segment the application into functions that parallel the underlying planning process. The views are summarized as follows:

• Home contains the authentication and general information screen.

• Data View includes sub-views of the rules and lessons learned repositories. It provides access to all current and legacy data used in planning and supports the creation of interactive maps, tables, and charts.

• Model/Scenario View includes the scenario man-agement tool used to plan, create, and execute complex scenarios. It allows users to select the appropriate DSS models, run the models, and store and display model results.

• News gives users access to a variety of internal and external news sources as well as chat traffic. • Composite View integrates the data, model, and

news views.

• Collaboration View incorporates the features of the composite view with additional collaboration functionality. It provides users with real-time, secure, room-based chat functionality, a group view of specific scenarios created (including rules invoked) along with model results associated with

each planning scenario, a common whiteboard, shared applications, interactive maps and charts, and audio and video connectivity.

To illustrate the functionality of the proposed architecture, we describe a prototype scenario planning DSS simulation model that was developed, tested, and validated for the U.S. Navy at standalone sites [12]. The proto-type was developed as a proof of concept of the ability of the DSS to provide the functionality nec-essary for distributed scenario planning. The proposed Web portal architecture evolved as a product of the insights gained in developing the scenario planning model itself and has not yet been implemented.

84 November 2005/Vol. 48, No. 11 COMMUNICATIONS OF THE ACM

Generalized DSS Scenario

Planning Factors U.S. Navy

Key Planning Process Process Critical Success Factor(s)

Planning Teams

Rules, Regulations and Policies to be Manipulated by Planning Teams Input Options: Scenarios

Factors and Driving Forces Classes of Entities to be Modeled in the DSS Entity Attributes and Actions

Process Models Associated with Planning Activities Output Options: Metrics related to critical success factors

Display and Trace Facilities

Manpower, Personnel, and Training Process (MPT)

Fleet Readiness: having the right people with the necessary skills and knowledge in the right place at the right time

• Recruiting/Classification/Selection • Training Management

• Distribution and Assignment • Community Management • Force Structure Management

A repository of rules, regulations, and policies enforced by each of the planning teams identified above

Future commissioning and de-commissioning scenarios and their associated probabilities

Specific rules, regulations, and policies enforced by planning teams (with priority rankings) to achieve fleet readiness for each scenario; quantitative data • People: recruits

• Billets: jobs or work order requisitions

• Processes: associated with planning teams (such as the recruitment process) Descriptive and status information for people, billets, and processes

as well as actions taken at decision junctures

Processes that depict the movement of people and billets throughout the system

• Performance metrics related to fleet readiness (e.g., quantity of billets filled; quality of assignments made; appropriate mix of rank and job rates among geographical regions)

• Graphical displays • Trace files

A PROTOTYPEMANPOWER PLANNINGSCENARIOMODEL

Many of the same challenges facing business organi-zations plague the U.S. Navy. Functional units oper-ate in a stovepiped fashion in which data, information, and knowledge are not distributed and shared across the enterprise. To address this problem, the Navy has created a plan for a “networked Navy.” The prototype we describe is one application among many in a multidimensional research endeavor to discover applications that will foster collaboration in distributed environments.

Our prototype DSS models the Navy’s manpower, personnel, and training (MPT) process and allows manpower planning teams to jointly develop plan-ning scenarios that reflect potential future events, assign their probability of occurrence, experiment with rules and factors that affect fleet readiness, and assess the impact of uncertainties and current policies on future performance. The factors identified for the MPT planning process, as well as planning scenarios in general, are outlined in the accompanying table.

This simulation model mimics the movement of Navy personnel over time, assigning people to jobs (billets), moving personnel from place to place and ship to shore, and processing new recruits, promo-tions, rate changes, attripromo-tions, and so on. As billets become available or personnel become eligible for advancement, an optimization model is invoked to match people to billets. Assignments are driven by the specific scenario defined and the associated factors and rules that planning teams specify as input values for each simulation run. The impact of a scenario on fleet readiness can be observed over any number of years. This impact may be in the form of costs (such as training costs and permanent change of station costs), quality of life issues for personnel (including the number and type of permanent change of station moves experienced), quality of matches of personnel to billets (how well a person’s skill and training fit the billet), the ability to recruit the required number of the right people, on-time arrival of personnel to bil-lets, the ability to have the right mix of ranks and seniority at different locations, and the ability to pro-vide the necessary training when it is needed.

The proposed architecture (see Figure 1) evolved as a model for distributing the power of the MPT simu-lation model. For example, when a planning team selects the simulation and optimization DSS models, the data associated with master personnel records and master billet files stored in the database tier of the architecture will be passed to the DSS engine. Remote planning teams will develop scenarios using the sce-nario building tool through the Web portal and run

the model for a specified length of time, say eight years. A sample scenario might specify future changes in force structure based on probabilities associated with likely commissioned and de-commissioned activities. The planning teams further articulate dri-ving forces associated with each scenario by designat-ing rules to invoke or relax (for example, a minimum of three years of obligated service is required for

trans-COMMUNICATIONS OF THE ACM November 2005/Vol. 48, No. 11 85

Organizations cannot

create or predict the

future. To survive and

thrive, however, they

must be able to plan for

the future. Scenario

planning is one method

that attempts to define

the driving forces and

uncertainties facing an

organization and promote

an understanding of how

plausible futures may

impact the policies

and processes to be

implemented today.

fer to Hawaii), setting priorities for assigning person-nel to billets (for example, maximize the number of “no cost” moves or maximize the Navy Enlisted Clas-sification utilization—a measure of how well a per-son’s skill level matches the skill requirements of a billet), setting training class sizes and starting dates, and estimating attrition and advancement rates.

Through the portal interface, authenticated users develop scenarios, view and evaluate the rules various planning teams employ, share results of a simulation planning run, and discuss the risks of each plan asso-ciated with a particular scenario either through chats, bulletin boards, or live real-time videoconferencing. As insights emerge, they are stored in the lessons learned repository.

Our prototype simulation DSS model demon-strated its ability to emulate the MPT process of the Navy for the rate classification sonar technicians. It was able to produce valuable explicit knowledge such as costs related to training, permanent change of sta-tion moves, and community skills and seniority mixes. It was also able to produce important tacit knowledge about the effect of rules and policies on fleet readiness. It demonstrated the power of the scenario planning modeling environment, even though teams that exper-imented with the prototype at this stage in its devel-opment designed scenarios and shared insights and results using conventional means such as email and telephone. Our proposed Web portal architecture cap-tures the functionality and component integration required to distribute the power of this DSS model to MPT planning teams throughout the Navy.

CONCLUSION

Organizations cannot create or predict the future. To survive and thrive, however, they must be able to plan for the future. Scenario planning is one method that attempts to define the driving forces and uncer-tainties facing an organization and promote an understanding of how plausible futures may impact the policies and processes to be implemented today.

To address shortcomings of traditional scenario planning protocols, we proposed a Web portal archi-tecture that permits geographically dispersed plan-ning teams to experiment with planplan-ning scenarios using DSS models. The architecture embodies a vir-tual community that encourages knowledge sharing. We illustrated its functionality with a sample proto-type manpower planning application developed for the U.S. Navy. However, we see the potential for many other domain areas to benefit from the pro-posed architecture. For example, distributed planning teams in the banking industry may benefit from the proposed architecture by being able to pose future

corporate power and government control scenarios, which would drive a DSS model that captures demo-graphic data and macroeconomic trends. They could use this model to plan for technological changes that might reshape the structure of financial institutions. The energy industry can benefit from the architec-ture by experimenting with DSS energy models vir-tually to observe the effects of scenarios such as the adoption of renewable energy sources or possible new fuel technologies on industry policies. Regardless of the domain, the proposed architecture has the poten-tial of distributing the power of DDSs to geographi-cally dispersed planning teams so they can jointly assess the impact of potential futures on decisions made today.

References

1. Adkins, M., Burgoon, M., and Nunamaker, J.F. Using group support systems for strategic planning with the United States Air Force. Deci-sion Support Syst. 34 (2002), 315–337.

2. Chen, H., Schroeder, J., Hauck, R., Ridgeway, L., Atabakhsh, H., Gupta, H., Boarman, C., Rasmussen, K., and Clements, A.S. COPLINK connect: Information and knowledge management in law enforcement. Decision Support Syst. 34 (2002), 271–285.

3. Duffner, R. Portals unlock the knowledge that drives business value. Designing Portals: Opportunities and Challenges. A. Jafari and M. Shee-han, Eds. Idea Group Publishing, Hershey, PA, 2003, 202–219. 4. Fahey, L. and Randall, R.M. Learning from the Future. John Wiley &

Sons, New York, 1998.

5. Frieswick, K. Follow the signposts. CFO Mag. 18, 2 (Feb. 2002), 65–66.

6. Jarvenpaa, S.L., and Leidner, D.E. Communication and trust in global virtual teams. Organizational Sci. 10, 6 (1999), 791–815.

7. Jafari, A. and Sheehan, M. (Eds.) Designing Portals: Opportunities and Challenges. Idea Group Publishing, Hershey, PA, 2003, 2–3. 8. Roeder, S., and Voss. A. Group decision support systems for spatial

planning and e-government; 195.228.254.144/Eloadasok/Streeam3/ Wednesday_16hr/Stephanie_Roeder/roeder_gsdi.pdf (Accessed 6/10/03).

9. Sarker, S. and Sahay, S. Understanding virtual team development: An interpretive study. J. Assoc. Inf. Syst. 4 (2003), 1–38.

10. van der Heijden, K. Scenarios, Strategies and the Strategy Process. Nijen-rode University Press, Breukelen, The Netherlands, 1997.

11. Weber, R.O., and Aha, D.W. Intelligent delivery of military lessons learned. Decision Support Syst. 34 (2002), 287–304.

12. Wild, R.H., Vance, D.E., II, and Griggs, K.A. SimPersonnel: A prototype policy simulation model for enterprise-wide manpower management in the U.S. Navy. Int. J. Internet Enterprise Manage. 1, 3 (2003), 316–335.

Rosemary H. Wild (rwild@calpoly.edu) is an associate professor of information systems in the Orfalea College of Business at the California Polytechnic State University, San Luis Obispo, CA.

Kenneth A. Griggs(kgriggs@calpoly.edu) is an associate professor of information systems in the Orfalea College of Business at the California Polytechnic State University, San Luis Obispo, CA.

Eldon Y. Li (eli@calpoly.edu) is University Chair Professor of the College of Commerce at the National Chengchi University in Taiwan. He is on leave from the Orfalea College of Business at the California Polytechnic State University, San Luis Obispo, CA.

Permission to make digital or hard copies of all or part of this work for personal or class-room use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

© 2005 ACM 0001-0782/05/1100 $5.00