 |
|
|
|
|
This project ended on 2002-10-01. This web space is maintained only for archival reasons.
An interaction-driven system is a distributed system whose primary function is organized around the interactions of complex, loosely coupled components. Since interaction-driven systems were appearing frequently in the manufacturing context, a project to respond to the growing need for relevant testing expertise was begun in 1998. The mission of the TIMS project was to close the gap between our testing capability and the testability of interaction-driven manufacturing systems. We did this by enhancing our testing capability with improved test methods, by defining design-for-testability criteria for specification writers, and by applying tools and techniques to improve the quality of system-level specifications.
Principal investigators: David Flater, K.C. Morris.
The background study performed in the opening months of the TIMS project set the stage for the work to follow. The results of the study are available as an unabridged technical report or as a shortened workshop paper:
K.C. Morris, David Flater, Don Libes and Albert Jones, "Testing of Interaction-driven Manufacturing Systems," NISTIR 6260, National Institute of Standards and Technology, Gaithersburg, MD, 1998.
K.C. Morris and David Flater, "Standards-Based Software Testing in a Net-Centric World," in Proceedings of the Ninth International Workshop on Software Technology and Engineering Practice (STEP '99), Pittsburgh, Pennsylvania, IEEE Computer Society, September 1999, pp. 115-122.
A follow-on report titled "Specifying Interactions in Integrated Manufacturing Systems" was presented at the 4th International Conference on Design of Information Infrastructure Systems for Manufacturing.
We defined a new testing strategy that focuses on the faults that are introduced by imperfect integration of legacy components. We conducted a trial application of this strategy to some prototype shop floor control software to determine whether it can improve the yield of faults over "plain old" black-box testing. A report titled "Testing for Imperfect Integration of Legacy Software Components" documenting the strategy and the results of the trial application was presented at the 1st Asia-Pacific Conference on Quality Software.
In 1997, NIST researchers conducted a series of informal performance tests to establish confidence that an existing manufacturing process control system could successfully be replaced by a Common Object Request Broker Architecture (CORBA)-based system. In response to continued outside interest, this low-key activity was finally documented in "Testing for Feasible Performance of a Distributed Manufacturing System: a Brief Experience Report," which was presented at the 4th World Multiconference on Systemics, Cybernetics and Informatics.
(Note: the following information is also available in a standalone newsletter article.)
The TIMS project experimented with "harmonized testing" of the Object Management Group's (OMG's) PDM Enablers standard and the PDM Schema that is being developed for the ISO Standard for the Exchange of Product Model Data (STEP). A technical report detailing the following results is available online:
A revision of that report was presented at the 25th Annual Computer Software and Applications Conference.
Another report, "Design of a Flexible, Integrated Testing System for STEP and OMG Standards," concentrates on the harmonized testing approach, while yet another report, "Improving PDM Testability through Standards Harmonization," documents the technical details of how the two standards were integrated.
Additionally, the following materials are available from OMG:
To fully demonstrate the value of harmonized testing, it was originally planned that the test cases defined for PDM Enablers would be used to generate SDAI test cases based on the PDM Enablers scenarios. However, that part of the work was not completed in time to be useful and so was abandoned.
Because they are so dynamic, agent-driven systems are difficult to test and debug, yet little had been done to address testability issues in work on agent infrastructures. Using a job scheduling scenario and a publicly available agent infrastructure, we conducted a two-phase case study in debugging agent interactions. The first phase was enhancing the visualization of the agent community to reveal the causes of failed negotiations. The second phase was formalizing the problem using a Temporal Calculus of Communicating Systems (TCCS) and attempting to find a solution. An experience report, "Debugging Agent Interactions: a Case Study," was presented at the 16th ACM Symposium on Applied Computing. It concludes with a specific recommendation for improving testability in agent infrastructure standards.
A white paper that attempts to generalize our experiences into recommendations for how to enhance testability in standards for interaction-driven systems is hosted by the OMG's Test & Validation Special Interest Group. We worked through the Testability Work Group of the Manufacturing Domain Task Force to put it into practice in the PDM Enablers and CAD Services standards.
In the final months, we analyzed testing requirements for RosettaNet, OAG, and OASIS specifications in service to other projects.
For more information, use this hyperlink to email David Flater or refer to his home page for other contact information.
David
Flater
2002-10-08
[Disclaimer/Privacy]