The National Science Foundation's (NSF) Tokyo Office periodically receives and disseminates reports on research developments in Japan that are related to the Foundation's mission. NSF-sponsored researchers currently working in Japan prepare many of these reports. These reports present information for use by NSF program managers and policy makers; they are not statements of NSF policy.
Special Scientific Report #00-08 (October 11, 2000)
Mr. Larry Chao, a graduate student in Mechanical Engineering, Stanford University, prepared the following report. Mr. Chao was a participant in the 2000 Summer Institute sponsored in the United States by NSF/NIH/USDA and the Science and Technology Agency and Japan Science and Technology Corporation in Japan. Dr. Kazuo Mori, of the Mechanical Engineering Laboratory in Tsukuba, hosted Mr Chao. Mr. Chao can be reached via email at: lpchao@stanford.edu.
Motivation
The Mars Climate Orbiter was launched by NASA on December 11, 1998, for a journey towards the red planet. It was to be a key craft in NASA’s exploration of the red planet. However, it vanished on September 23, 2000, after a rocket was fired to put the craft into orbit. An investigation concluded that NASA engineers had failed to convert the rocket thrust from English units of pounds of force to metric units of Newtons. One English pound of force equals 4.45 Newtons, and this small difference caused the spacecraft to approach Mars at too low an altitude and the craft is thought to have smashed into the planet's atmosphere and was destroyed.
The purpose of this research is to prevent errors like this which occur in the design process from escaping to the field. By devising and evaluating error management techniques and tools for the design process, we can prevent incidents like that from occurring. Accomplishing this means work on two fronts:
1. The development of design strategies and tools to predict potential errors which may occur during the design phase of a project
2. The development of error-proofing methods for the design phase which prevent errors like that from occurring
By learning more about the design process and the errors which occur, we can better reach the goal of this analysis, which is to develop tools to use in other stages of the life cycle, in particular, to diagnose the general design process.
Background
This research is a continuation of the research of error throughout the lifecycle of products. The research began by first becoming familiar with the design process and errors which may occur during it. This was first done by tracing the entire lifecycle of a product component, from development through production and usage to retirement. Second, actual root cause analyses of failures which had escaped to the field were studied. By studying this empirical data, it is possible to get a better idea of what types of errors occur during the design process. In addition, research on the design process and nature of error was done.
From this research, a design process FMEA (Failure Modes and Effects Analysis) was devised. Based on an assembly process FMEA which uses a question-based analysis, questions for the design process were created. The design process FMEA however is slightly different because of the different nature of the design process. The design process FMEA requires a structured design process to be analyzed because the design team members work off the structure for their analysis.
Design management
The development process, including planning and design is one of several important steps in the lifecycle of a process, which includes
Planning
Design
Production
Distribution
Operation
Maintenance
Disposal
However, it is very important because at these early stages many of the costs are being committed.
There are many reasons for an organization to implement a structured, systematic design process. Systematic design process has its own advantages. Already, much research has been done in systematic design. Many guidelines and standards (like ISO9000) exist and are used by companies. Organizations have developed diagnostic methods like Capability Maturity Model (CMM) to assess the design process.
Design management already implies certain tasks, including the scheduling of design tasks, deadlines, and reviews, the management of design teams and information flow, and decision making at key points to balance issues like cost, time, and features. Management is the best way to formalize the design process. Once it is consistent and structured, the design process can be viewed as a product itself, and each project can be decomposed into design tasks much like a product or system can be broken down into subassemblies or parts.
Failure Modes and Effects Analysis (FMEA)
Unfortunately, things can go wrong in the design of a product. “Error” is a generic term to encompass all those occasions in which a planned sequence of mental or physical activities fails to achieve its intended outcome, and when these failures cannot be attributed to the intervention of some change agency. Failure Modes and Effects Analysis is a tool used to predict the likely sources of failure in a design. The basic analysis asks what functions must occur, how can they fail, and what can cause each type of failure. These three questions are used to try to identify all possible ways a system can fail. With this list, the analysis continues by rating how likely each failure is and what the failure’s effects on the system will be.
A goal of this research is to develop a process FMEA for the design and development process of an organization. Because the design process takes longer (weeks to years) than manufacturing (hours or days), it is impractical to analyze it after the fact. In addition, there is usually greater variation from one development process to the next. It makes more sense to analyze the general design process rather than tracking the design of a specific product. Also, the design process is often much less tangible because ideas and concepts are being worked with rather than physical components which are fabricated or assembled. Because of that, it is more difficult to foresee all the problems that may occur.
Error-proofing was developed by Shigeo Shingo, an industrial engineer at Toyota. Poka-yoke, which is Japanese for mistake-proofing, involves the use of devices either to prevent the special causes that result in defects, or to inspect inexpensively each item that is produced to determine whether it is acceptable or defective. The goal of error-proofing is to predict and prevent errors before they can occur rather than inspect for errors after they have been built into the system. Error-proofing is typically done in design for the manufacture/assembly stage. Because many human errors that occur in the design process are often simple, it would be good to be able to develop poka-yoke analogs for the design process.
Method
To perform a detailed FMEA on the design process as currently prescribed, a structured design process is necessary. However, there are significant differences between the organization as documented and the organization as it actually operates. To determine the feasibility of this analysis as well as learn more about the nature of error in the design process at organization, several things were done this summer.
Company surveys
Working in Japan with the Mechanical Engineering Laboratory in Tsukuba provided the opportunity to visit several leading Japanese companies and discuss the management of error in the design process. With assistance from my host researcher, Dr. Kazuo Mori of the Mechanical Engineering Laboratory’s Manufacturing Information Division, and my advisor, Professor Kosuke Ishii of Stanford University’s Manufacturing Modeling Laboratory, I was able to visit several companies in Japan. Companies visited this summer include Hitachi, Toshiba, Canon, and Matsushita (National-Panasonic). At several sites, a presentation of the design process error management techniques was made and the subsequent discussions were used to discuss not only proposed error management techniques but also their current error analysis and prevention systems. A simple two-page questionnaire was devised which surveyed the organization’s design process and common errors and techniques for dealing with errors. The survey asked the organization to rate themselves in six different areas relating to the structure and discipline of the design process, the sharing of knowledge, and so on. The survey asked the organization to rate itself with agreement to each statement, with one of six choices ranging from strongly agree to strongly disagree.
A structured design process strategy is in place.
Your organization documents previous designs and processes and makes them accessible to design teams.
Your organization uses guidelines, checklists, or documentation of best practices.
Your design teams explicitly record observations and assumptions throughout the design process.
There are regularly scheduled design reviews.
The design managers use performance metrics to evaluate the design process.
In addition, comments or recommendations on the proposed FMEA techniques were recorded.
The company sites visited and surveyed included Matsushita (National-Panasonic), Hitachi, Canon, and Toshiba. Conversations were held with engineers and managers from design and manufacturing to discuss their current design issues.
To supplement these visits, several other companies in both the U.S. and Japan were contacted and sent the survey and presentation via e-mail.
Results
So far, we have received 13 surveys back from various companies in the United States and Japan. The companies surveyed produced a wide variety of products ranging from consumer electronics to copiers and printers to aircraft engines. Several more returns are expected, and additional companies will be visited and additional surveys will be sent out in the near future.
The responses from the survey were plotted using the listed numerical values for the level of agreement:
|
Agreement |
Quantitative Value |
|
Strongly agree |
2 |
|
Agree |
1 |
|
Neither agree nor disagree |
0 |
|
Disagree |
-1 |
|
Strongly disagree |
-2 |
|
Not applicable |
|
Table #1: Quantitative agreement rating scale
The eleven surveys reviewed showed an overall average of slightly less than “agree” (about 0.8) for the six statements, which indicates a fair agreement. The standard deviation was also about 0.9. The highest scores were on documentation and design reviews, with the lowest score on the use of metrics. The scores ranged from disagree to strongly agree (-1.0 to 2.0).
The most common errors cited are unanticipated or oversight errors. These include unanticipated changes in requirements during the design process requiring redesigns, or simple human errors such as the oversight of portions of a task.
For the most part, the most common error management tool used is the design review. There are very few tools used to anticipate errors before they happen. Many guidelines and standards (like ISO9000) are used by companies. Most companies reported the use of documentation throughout the process (the average score was about 1.4). However, a later question asking of the use of documentation was slightly lower overall (about 0.9); however, many companies had a difference of a full point or more.
The visits and interviews so far have shown that although companies are interested in implementing a design process FMEA tool, they do not have anything like it at this time. A few companies spoke of attempts to do so, but have not committed the time or resources to complete it. There were no examples of error-proofing at all in the design process from any of the companies in either the U.S. or Japan. Many engineers were not even aware of the term “error-proofing” or poka-yoke, even in a manufacturing setting.
An insight by some consumer electronics companies showed that though they wanted to implement a design process FMEA like tool, they felt that given the type of products they produced, there was no rush, when compared to certain other industries which dealt with more life-threatening consequences to failure, such as medical devices or aircraft engines. The attempts at developing a prediction tool have often times been delayed due to other pressing projects.