Getting Started in Simulation Modeling

Jerry Banks & Randall Gibson

Simulation modeling is similar to other engineering disciplines. It requires training and experience to become competent, and isn't really as easy as some people might have you believe. It's an art as well as a science. While it's not necessary to have an advanced degree in simulation, industrial engineering, or some other related discipline, there are several steps or guidelines you should consider if you intend to become an effective modeler.

Many articles and books have been written on various aspects of the art of simulation modeling –guidelines for project success, proper management techniques, why projects fail, selecting software, etc. If you're planning a career in simulation modeling, you might read these articles and keep a copy in your personal library. Too often, however, first time simulation modelers simply select a software package and set out to solve a problem. Instead, they would be better off following asset of guidelines like those outlined below. These guidelines, combined with common sense and sound judgment, should help you make wise decisions as you get started in simulation modeling.

Our guidelines for getting started could be written as 10 steps or 15 steps; there is nothing magical about the number 12. What is important is that an orderly process is provided, and the contents of each step need to be accomplished.

Step 1: Define the problem Perhaps the most important, but often the most overlooked, aspect of simulation modeling is defining exactly the problem that needs to be solved. A common misconception among those unfamiliar with simulation is that once a model is built, it will be able to provide answers to any and all questions relating to the simulated system. Like other computer applications, a simulation model can only do what it was designed to do and it's impractical to design it to do everything. Therefore, careful determination of the model design the scope and level of detail represented in the model is critical. A good way to begin is to construct a list of the questions that the model will answer. Be specific. For example, ask, "What is the maximum length of accumulation conveyor used?" rather than "How big should the system be?" The questions should be open ended. For example, ask, "How many sortation lanes are utilized" rather than "Are 10 sortation lanes needed?"

Once all the questions are listed, rank order them and separate them into two classes:

Key questions are those that the model must be able to answer; Desirable questions are those that we may want the model to answer, but they are not critical

Classifying them helps focus the model construction effort, and tells the model developer what may be left out if needed to meet a schedule or budget constraint.

During this step, suspend judgment until the system is better understood. Avoid the temptation to influence the questions if you believe you already have the answer, or consider that aspect of the system not important or unrelated to the problem solution. It is also helpful at this step to quantify the benefits anticipated from obtaining answers to each question. This process will help pare the list of questions to only the important ones. Too many questions will lead to a large and cumbersome model, which will be difficult to construct, validate, and use.

It's important to solve the right problem. If the completed model can't answer relevant questions, the project may be judged a failure even though the model was "correct." You may need to revisit this step later (especially after Steps 2 and 3) to ensure that the model focus is still on target. Step 2: Understand the system After determining the problem to be solved, the next step is to gain a thorough understanding of the system or facility being simulated. "You can't model what you don't understand" is a simple axiom, but it is often ignored by first time modelers in their rush to construct a "baseline" model. The importance of this point cannot be overstressed.

Start with a tour of the facility. If the system is a "greenfield design" (e.g. the system does not exist), tour a similar facility, then conduct a thorough review of the facility layout or design drawings with someone responsible for the design. List the system components: machines, storage locations, buffer or queue locations and their sizes, tool locations, critical resources, material handling units, machine operators, robots, and maintenance operators. Develop an understanding of the operational characteristics of the facility, such as processing logic, scheduling rules, operating schedules and breaks, interaction between processing, and material handling. Talk with supervisors or operators. Ask questions. Make notes and later summarize them in a Description of Operations document for the system. If possible, ask facility supervisors to review and comment on the document. We say "supervisors" since no one person understands the entire system.

After the facility tour, get CAD drawings of the system. These will help in understanding the system and in framing the questions to be answered. Also, they can be used as a basis for developing an animation for the model.

Finally, apply common sense: Do you have a sufficient understanding of all aspects of the system relevant to answering the key questions? Can you define a "paper" model design that will address each question? for example, if a questions related to order completion time in a distribution center, but the order release and batching logic is not completely understood, then you have more homework to do before proceeding.

Step 3: Determine your goals and objectives Once you have gained an understanding of the system to be simulated, the next step is to plan the simulation project. Start with setting explicit goals and objectives for the project. This is very closely related to determining which questions the model must answer; and you may need to repeat these two steps until they are consistent and accepted by management. Your goal as a modeler should be to limit the size and scope of the model to only what is required to address the project objectives and answer the key questions. As with other computer applications, the effort required to build and validate a model grows exponentially with the requirements!

Begin with deciding what scenario(s) are to be simulated e.g., a peak demand day, or an average day with no exceptional conditions, or a third shift at the end of the month. Obtain the normal operating cycles of the facility so that you can determine the period or length of time to be included for each scenario. For instance, if your objective is to determine capacity requirements for peak demand (a typical simulation requirement), then you need to learn from historical data which is the busiest day and limit the data collection effort (Step 8). Additional scenarios can be specified, but be aware that each new scenario will require a careful review of the model capabilities and scope, and possibly a new input data set.

Next, determine what evaluation criteria will be used when analyzing the model results. What are the key measures of performance? what resolution is required in order to be useful to management? Again, separate these into required and desirable categories–some measures may require significant additional model complexity, and may not be necessary!

Finally, summarize the project objectives in writing and have them reviewed by all involved with the project. Be certain that all involved are in agreement. If not, stop now and resolve the different expectations or you risk the model credibility and–ultimately–later project success.

Step 4: Learn the basics Although it is advisable to complete this step before undertaking a specific simulation project, many people find it easier to complete if they are motivated by a real problem they must solve. This might be called "just-in-time learning." If you are one of these, then be sure to leave time in your project schedule to allow completion of this step.

There are several ways of going about this task. You could obtain and study one (or more) of the many texts available. You could attend the introductory tutorials at the annual Winter Simulation conference usually held during the second full week of December. You could take a one-to three-day public short course offered through IIE or several universities. If there are several people in the firm who need to learn the basics, it may be more economical to arrange an on-site short course. We recommend that you start with attending a short course, or if possible, the tutorials at the Winter Simulation Conference. Follow this up with studying one of the texts and additional readings fro one of the professional journals featuring simulation articles, such as IIE Solutions.

Step 5: Confirm that simulation is the right tool Once you have completed the first four steps, you should pause before proceeding and ask "Is simulation the right tool to solve this problem?" There are other tools available, perhaps less expensive or quicker to apply. These might include a static spreadsheet "model" or "closed form" analytical solutions using some form of mathematical programming. From a practical standpoint, simulation models do not provide optimal solutions–they simply report on system performance for a set of conditions that the user inputs. The user must "guide" the simulation experiments and analysis to obtain an acceptable solution or answer.

Key attributes of a problem that would rule out the use of these other solution techniques include complex system interactions, and system dynamics. Complex interactions between system components are difficult to represent in a constraint-based set of equations necessary for a mathematical programming solution. System elements that interact and change over time–and whose time variant behaviors are important to capture–are difficult to represent in a spreadsheet or other static solution technique (which generally provide only average values to solution measures).

If you are not sure about which solution technique to use, consider engaging a consultant to help you determine the answer.

Step 6: Attain support from management This is another step that seems obvious, but is often overlooked in the rush to get a project moving forward. You must obtain clear support from management at the beginning of the project, or management will be reluctant to accept the results. (If the results are not used, the simulation project will be deemed a failure.)

A key component of this step is education. Management in most companies has little or no experience with simulation modeling. Many managers will have very unrealistic expectations regarding the process, usually relating to the project schedule and model scope or capabilities. The first task in obtaining support from management is to help them understand the process and establish reasonable expectations for the project.

Begin with an overview meeting to review the project goals and objectives, time frame, model scope, questions that will be answered, and measures of performance to be used. Provide background information on the process.. Discuss what models can and cannot do. A helpful technique is to review the highlights of a similar project at another company–what was learned, how long it took, and any management comments about the project that are appropriate.

Hold regular, but short meetings to keep management informed of the progress. For those managers who are not available for a meeting, circulate a memo summarizing the key topics. When the model is completed and verified, include management in the analysis process–make them part of the discovery and decision process. Having sat through some of these sessions, they will gain insight into the process, and understand and share in "ownership: of the results.

Attaining management support is a continuing requirement–it doesn't happen just once (e.g. at the beginning of the project). In theory, simulation models are built to answer difficult questions–questions in which management may have keen interests. The more involved managers are, the more they will appreciate the process and learn from the project.

Step 7: Learn about software tools for simulation Making a choice from the vast amount of software that is available for discrete-event simulation is bewildering for the newcomer to the field. We suggest that you proceed with this step in an orderly fashion by narrowing your search to only a few of the more than 50 products that were mentioned in the 1996 Simulation Buyer's Guide in the May issue of IIE Solutions. Narrow your search by asking such questions as the following: Do we want specialized manufacturing software, or do we want a simulation language? Do we want software for scheduling or do we want software for process control? Do we need animation, and if so, does it need to be 3-D? Are material handling constructs important, or can we assume these are just time delays?

Once you have narrowed the search to about five software products using the questions above, and others appropriate to your situation, then concentrate on considerations pertaining to input capabilities, processing capabilities, output capabilities, the simulation environment, the support, and the cost. Each of these considerations raises numerous issues, too many for us to tackle in such a short space. We do want to offer these suggestions, however:

Accuracy and detail may be extremely important Powerful capabilities can make for simulation modeler productivity increases. Get the greatest speed that you can afford, because the greater expense is the modeler's waiting time. Beware of fancy ads and demos that do not really inform the potential buyer of software capabilities. Beware of checklists; go beyond the 'yes' or 'no' check mark. Implementation and capability is what is important.

The bottom line question is, "Which simulation modeling tool is the right one?" When purchasing the software, it's a good idea to do the following:

Have simulation vendors solve a small version of your problem. Seek references that can attest to the software's capabilities and limitations. Seek the opinion of consultants who use several products. Seek the opinions of companies with similar applications. Attend user group meetings and annual symposia. Visit with the vendors at the Winter Simulation Conference or other similar events Take training courses in using the software.

Step 8: Determine what data is needed and what is available There are several types of data that are used in simulation. These include time-based data such as interarrival times, demand rates, loading times, unloading times, processing times, time to fail, time to repair, and travel times. Other types of data used in simulation include fraction failing inspection, fraction requiring each machine, and fraction of each part type.

Data problems occur when we have small samples, summary data (for example, the mean only), qualitative data, guesstimates (estimates that are only guesses), and data in the wrong format (for example, last year's data instead of this year's data or ordering rather than shipping data).

Data sources include direct observations, data from time studies and history, and automated data. However, automated data may be flawed (Is the machine idle because it is down, or has the operator gone to lunch?) Sometimes, data from similar systems may be used. But, inferences from such a source may be risky.

Another data source is operator estimates. But, these are highly suspect because humans are such poor estimators. They forget the extremes and emphasize the present.

Vendor claims or specifications are another source. However, these can be highly optimistic (i.e., the reliability of the system can be grossly over estimated). It is often the case that no data exists. Consider a system that is in the design phase (the system does not exist). Another possibility is when improper data have been collected.

There are techniques that may be used when no data or insufficient data exist. For example, if the underlying process is truly random, under certain general conditions, an exponential distribution can result.

Step 9: Develop assumptions about the problem A major concern is the successful accomplishment of this step, for numerous reasons. For example, the assumptions include the appropriate system boundary. If the boundary is too big, additional time and expense will be encountered in completing the simulation. If the boundary is too small, the questions that are asked of the simulation may not be answered. Similarly, the complexity of the simulation model should be determined. The complexity should be sufficient to answer the questions that are asked, but not more than that.

From another standpoint, the assumptions are important at the end of the simulation. It is an unenviable position to be hit with criticism at the end of the simulation that the wrong problem has been solved. Such criticism can be halted by pointing to the assumptions that were agreed upon the outset of the project.

The best situation is to have complete agreement of the assumptions and to verify that the assumptions are valid as the simulation progresses. In case of a difference of opinion, it may be necessary to negotiate an extension in time or resources to complete the simulation.

Step 10: Determine the outputs needed to solve the stated problem There are many possible outputs including throughput, makespan, WIP, utilization, process time, number of jobs late, and the time that jobs are late to mention just a few. These can be presented as questions to be answered as follows:

Will the system meet throughput requirements? What happens to response time at peak periods? What is the recovery time when short-term surges cause congestion and queuing? What is the system capacity? If problems occur, what is their cause?

But, the purpose is understanding, not numbers. Furthermore, this understanding involves statistical considerations. Many people are uncomfortable with the statistical aspects of simulation. Perhaps, they never took a course in engineering statistics, hypothesis testing, or whatever name is appropriate. Perhaps, they took the course long ago and have forgotten the principles. Perhaps, they took the course but did not really understand the meaning behind all of the equations. Whatever the case, there is output analysis capability built into many simulation software packages. There are also add-on packages that can be purchased for specific software. Another option is software that analyzes simulation output.

Whichever technique is used requires understanding. And, there are lots of aids to this understanding built into simulation software.

Step 11: Simulation will be conducted internally or externally? Actually, there are a number of combinations for conducting a simulation. The simulation could be conducted internally, externally, or via some combination of the two.

For first-time users, conducting the simulation internally may be unwise. There are too many decisions that have to be made, and they could be wrong. Having a consultant perform that initial simulation might be useful to indicate the appropriate application of the technology. The firm can learn by interaction and osmosis. Another possibility is to have a consultant work with the firm for that first application of the technology. A variation of this is the "jump start." The consultant helps get the project going, then turns over the reins to the internal group.

For infrequent users, consideration must be given to the time for relearning. For example, if simulation is used once per year for three months, the not used for nine months, then the analyst (if it is the same person) must relearn the software (or different software) every year. Additionally, the same software does change as a function of later versions. Consider the case where there is a short deadline for the completion of the simulation. Spending time in relearning subtracts from the time available to the simulation activity. A consultant can certainly prove valuable in this case.

For frequent users of simulation, an internal group can be formed. The internal group can be enhanced with the service of consultants if the demand is great, or when there are deadlines to be met.

Step 12: Kickoff the project Some points to consider for the formal kickoff meeting are the following:

Agenda should have small chunks for topics. The kickoff meeting should be upbeat, fast moving, and maintain the interest of the firm. Show milestones. There should be many milestones. This is useful in keeping the decision makers involved in the problem solution. If they maintain their involvement, there is a much greater probability of implementation at project completion. Plan for much interaction. Not only should interaction be anticipated, it should be encouraged. This insures continued interest by the decision makers.

Respond to concerns and objections. This shows that there is interest in what decision makers and others not on the simulation team have to say. Also, there may be some very good points in the concerns and objections. Great care must be taken not to alienate the decision makers and others not on the simulation team. Be realistic. If there is any intention of completing more than one simulation project, follow this point. Wild promises of what can be accomplished and how quickly and cheaply they can be accomplished can prove to be the undoing of a simulation project. Finish strongly. The end of the kickoff meeting should be a resolve that all persons in attendance are in this together. And, that all persons are going to give what is needed to make for success.

For further reading:

Banks, J. [1996] "Output Analysis Capabilities of Simulation Software," Simulation, Vol. 66, No. 1, 1996

Musselman, K. , "Guidelines for Simulation Project Success, " Proceedings of the 1994 Winter Simulation Conference, ed. J.D. Tew, S. Manivannan, D.A. Sadowski, A.F. Seila, 1994.

Nordgren, W.B. [1995] "Steps for Proper Simulation Project Management," Proceedings of the 1995 Winter Simulation Conference, ed. C. Alexopoulos, K. Kang, W.R. Lilegdon, and D. Goldsman, 1995.

Robinson, S., and V. Bhatia [1995] "Secrets of Successful Simulation Projects," Proceedings of the 1995 Winter Simulation Conference, ed. C. Alexopoulos, K. Kang, W.R. Lilegdon, and D. Goldsman, 1995.