SUMEX STANFORD UNIVERSITY AEDICAL EXPERIMENTAL COMPUTER RESOURCE RR-00785 ANNUAL REPORT-YEAR 15 Submitted to DMEDICAL RESEARCH TECHNOLOGY PROGRAM NATIONAL INSTITUTES OF HEALTH June 1, 1988 STANFORD UNIVERSITY SCHOOL OF MEDICINE Edward H. Shortliffe, Principal Investigator Edward A. Feigenbaum, Co-Principal Investigator DEPARTMENT OF HEALTH AND HUMAN SERVICES PUBLIC HEALTH SERVICE NATIONAL INSTITUTES OF HEALTH DIVISION OF RESEARCH RESOURCES BIOMEDICAL RESEARCH TECHNOLOGY PROGRAM ANNUAL PROGRESS REPORT PART I., TITLE PAGE 1. PHS GRANT NUMBER: 5 P41 RR00785-15 2. TITLE OF GRANT: SUMEX Stanford University Medical Experimental Computer Resource 3. NAME OF RECIPIENT INSTITUTION: Stanford University 4. HEALTH PROFESSIONAL SCHOOL: School of Medicine 5. REPORTING PERIOD: 5a. FROM: 5b. TO: 08-01-87 07-31-88 6. PRINCIPAL INVESTIGATOR: 6a. NAME: 6b. TITLE: Edward H. Shortliffe, M.D., Ph.D. Associate Professor of Medicine and Computer Science 6c. SIGNATURE: 7. DATE SIGNED: 8. TELEPHONE: 415-725-3387 5P41-RR00785-15 Table of Contents I. Title Page II. Description of Program Activities 1I.A. Scientific Subprojects II-B. Books, Papers, and Abstracts 1I.C. Resource Summary Table III. Narrative Description 1II.A. Summary of Research Progress III.A.1. Resource Overview III.A.l.1. SUMEX-AIM as a Resource III.A.l.2. Significance and Impact in Biomedicine III.A.l.3. Summary of Current Resource Goals lll.A.2. Details of Technical Progress lll.A.2.1, Progress Highlights lll.A.2.2. Core ONCOCIN Research lll.A.2.3. Core Al Research lll.A.2.4. Core System Development lll.A.2.5. Relevant Core Research Publications lll.A.2.6. Resource Equipment lll.A.2.7. Training Activities III.A.2.8. Resource Operations and Usage 1II.B. Highlights lll.B.1. PROTEGE -- Developing Knowledge Acquisition Tools for Clinical Trial Advice Systems 111.8.2. A Speech Interface to ONCOCIN lll.B.3. SIMPLE/CARE -- Emulation of Parallel Computing Architectures lll.B.4. Toward the Distributed SUMEX-AIM Community 1II.C. Administrative Changes 1II.D. Resource Management and Allocation III.D.1. Overall Management Plan lll.D.2. 2060 Cost Center 1Il.E. Dissemination of Resource Information 1II.F. Suggestions and Comments IV. Description of Scientific Subprojects 1V.A. Stanford Projects 1V.A. 1. BBICU Project IV.A.2. GUIDON/NEOMYCIN Project IV.A.3. MOLGEN Project IV.A.4. ONCOCIN Project IV.A.5. PROTEAN Project IV.A.6. RADIX and PENGUIN Projects IV-B. National AIM Projects 1 3 5 5 5 9 11 14 14 19 26 39 83 89 97 100 114 115 116 117 118 119 120 120 121 123 125 127 128 129 133 137 144 159 168 181 E. H. Shortliffe 5P41 -RR00785- 15 1V.B. I, ATTENDING Project IV.B.2. INTERNIST-I/QMR Project IV.B.3. MENTOR Project 1V.C. Pilot Stanford Projects IV.C.l. REFEREE Project IV-D. Pilot AIM Projects IV.D.l. Dynamic Systems Project IV.D.2. Pathfinder Project IV.D.3. Knowledge Engineering for Radiation Therapy 1V.E. AIM Communications Users Appendix A. Knowledge Systems Laboratory Brochure Appendix B. Lisp Performance Studies Appendix C. Lisp Environment Specification Appendix D. AIM Management Committee Membership Appendix E. Scientific Subproject Abstracts References 182 186 191 196 197 203 204 211 217 220 225 237 251 273 277 301 E. H. Shor-tliffe ii 5P41-RR00785-15 List of Figures Figure 1: Core System Development Schedule 49 Figure 2: Recent ARPANET Coverage Map 77 Figure 3: SUMEX-AIM DEC 2060 Configuration 90 Figure 4: SUMEX-AIM SUN-4 Configuration 91 Figure 5: SUMEX-AIM Sun-3 File Server Configuration 92 Figure 6: SUMEX-AIM Xerox File Server Configuration 93 Figure 7: SUMEX-AIM VAX File Server Configuration 94 Figure 8: SUMEX-AIM Develcon X.25/TCP-IP Gateway Configuration 95 Figure 9: SUMEX-AIM Ethernet Configuration 96 Figure 10: Total CPU Hours Consumed by Month 101 Figure 11: Monthly CPU Usage by Community 103 Figure 12: Monthly Terminal Connect Time by Community 104 Figure 13: Cumulative CPU Usage Histogram by Project and Community 106 Figure 14: Table of Resource Use by Project 107 Figure 15: Public Data Network Terminal Connect Time 112 Figure 16: INTERNET Terminal Connect Time 112 Figure 17: 2060 Downtime Summary -- Hours per Month 113 Figure 18: Overall 2060 Reliability Summary 113 Figure 19: 2060 Cost Center Performance 122 Figure 20: Run Times for BBI and SOAR 245 Figure 21: Compilation Times for BBI and SOAR 246 Figure 22: Run Times for BBI under Various Compiler Settings 247 Figure 23: Compilation Times for BBI under Various Compiler Settings 248 Figure 24: Differences Between Normal and Reduced Ouput SOAR Run 249 *.. III E. H. Shortliffe 5P41-RR00785-15 Description of Program Activities II. Description of Program Activities This section corresponds to the predefined forms required by the Division of Research Resources to provide information about our resource activities for their computerized retrieval system. These forms have been submitted separately and are not reproduced here to avoid redundancy with the more extensive narrative information about our resource and progress provided in this report. LA. Scientific Subprojects Our core research and development activities are described starting on page 14, our training activities are summarized starting on page 97, and the progress of our collaborating projects is detailed starting on page 127. 1l.B. Books, Papers, and Abstracts The list of recent publications for our core research and development work starts on page 83 and those for the collaborating projects are in the individual reports starting on page 127. II .C. Resource Summary Table The details of resource usage, including a breakdown by the various subprojects, is given in the tables starting on page 100. 3 E. H. Shortliffe 5P41 -RR00785 15 Narrative Description Ill. Narrative Description 1II.A. Summary of Research Progress 1II.A. 1. Resource Overview This is an annual report for year 15 of the SUMEX-AIM resource (grant RR-00785) the second year of a 5-year renewal period to support further research on applications of artificial intelligence in biomedicine. For the technical and administrative reasons discussed in earlier reports, the SUMEX project now includes the continuation of work on the development and dissemination of medical consultation systems (ONCOCIN) that had been supported before 1986 as resource- related research under grant RR-01 631. Progress on core ONCOCIN research is therefore now reported here as well. These combined efforts represent an ambitious research program to: . Continue our long-range core research efforts on knowledge-based systems, aimed at developing new concepts and methodologies needed for biomedical applications. . Substantially extend ONCOCIN research on developing and disseminating clinical decision support systems. o Develop the core systems technology to move the national SUMEX-AIM community from a dependence on the central SUMEX DEC 2060 to a fully distributed, workstation-based computing environment. . Introduce these systems technologies into the SUMEX-AIM community with appropriate communications and managerial assistance to responsibly phase out the central resource and DEC 2060 mainframe in a manner that will support community efforts to become self-sustaining and to continue scientific interactions through fully distributed means. o Maintain our aggressive efforts at training and .dissemination to help exploit the research potential of this field. 1II.A. 1.1. SUMEX-AIM as a Resource SUMEX and the AIM Community In the fifteen years since the SUMEX-AIM resource was established in late 1973, computing technology and biomedical artificial intelligence research have undergone a remarkable evolution and SUMEX has both influenced and responded to these changing technologies. It is widely recognized that our resource has fostered highly influential work in biomedical Al -- work from which much of the expert systems field emerged -- and that it has simultaneously helped define the technological base of applied Al research. The focus of the SUMEX-AIM resource continues to emphasize research on artificial intelligence techniques that guide the design of computer programs that can help with the acquisition, representation, management, and utilization of the many forms of 5 E. H. Shortliffe Resource Overview 5P41 -RR00785-I 5 medical knowledge in diverse biomedical research and clinical care settings -- ranging from biomolecular structure determination and analysis, to molecular biology, to clinical decision support, to medical education. Nevertheless, we have long recognized that the ultimate impact of this work in biomedicine will be realized through its assimilation with the full range of methodologies of medical informatics, such as data bases, biostatistics, human-computer interfaces, complex instrument control, and modeling. From the start, SUMEX-AIM work has been grounded in real- world applications, like systems for the interpretation of mass spectral information about biomolecular structures, chemical synthesis, interpretation of x-ray diffraction data on crystals, cognitive modeling, infectious disease diagnosis and therapy, DNA sequence analysis, experiment planning and interpretation in molecular biology, and medical instruction. Our current work extends this emphasis in application domains such as oncology protocol management, clinical decision support, protein structure analysis, and data base information retrieval and analysis. All of these research efforts have demanded close collaborations with diverse parts of the biomedical research community and the integration of many computational methods from those domains with knowledge-based approaches. Even though in the beginning the "Al- in-medicine" community was quite small, it is perforce no longer limited and easily- defined, but rather is spreading and is inextricably linked with the many biomedical applications communities we have collaborated with over the years. Driven both by the on-going diffusion of Al and by the development of personal computer workstations that signal the practical decentralization of computing resources, we must develop new resource communication and distributed computing technologies that will continue to facilitate wider intra- and inter-community communication, collaboration, and sharing of biomedical information. The SUMEX Project has demonstrated that it is possible to operate a computing research resource with a national charter and that the services providable over networks were those that facilitate the growth of Al-in-Medicine. SUMEX now has a reputation as a model national resource, pulling together the best available interactive computing technology, software, and computer communications in the service of a national scientific community. Planning groups for national facilities in cognitive science, computer science, and biomathematical modeling have discussed and studied the SUMEX model and new resources, like the recently instituted BIONET resource for molecular biologists, are closely patterned after the SUMEX example. The projects SUMEX supports have generally required substantial computing resources with excellent interaction. Today, with the dramatic explosion of high- performance workstations that are more and more generally available, the need for a central source of raw computing cycles has significantly diminished. In place of being a distributor of CPU cycles, SUMEX has become a communications cross- roads and a source of Al and computer systems software and expertise. SUMEX has demonstrated that a computer resource is a useful "linking mechanism" for bringing together electronically teams of experts from different disciplines who share a common problem focus. Al concepts and software are among the most complex products of computer science. Historically it has not been easy for scientists in other fields to gain access to and mastery of them. Yet the collaborative outreach and dissemination efforts of SUMEX have been able to bridge the gap in numerous cases. About 40 biomedical Al application projects have developed in our national community and have been supported directly by SUMEX computing resources over the years -- many more have benefitted indirectly through access to the software, information, and advice offered by the SUMEX resource. The integration of Al ideas with other parts of medical informatics and their dissemination into biomedicine is happening largely because of the development in the 1970's and early 1980's of methods and tools for the application of Al concepts E. H. Shortliffe 6 5P41 -RR00785 I5 Resource Overview to difficult professional-level problem solving. Their impact was heightened because of the demonstration in various areas of medicine and other life sciences that these methods and tools really work. Here SUMEX has played a key role, so much so that it is regarded as "the home of applied Al." SUMEX has been the home of such well-known Al systems as DENDRAL (chemical structure elucidation), MYCIN (infectious disease diagnosis and therapy), INTERNIST (differential diagnosis), ACT (human memory organization), MOLGEN/BIONET (tools for DNA sequence analysis and molecular biology experiment planning), ONCOCIN (cancer chemotherapy protocol advice), SECS (chemical synthesis), EMYCIN (rule- based expert system tool), and AGE (blackboard-based expert system tool). In the past four years, our community has published a dozen books that give a scholarly perspective on the scientific experiments we have been performing. These volumes, and other work done at SUMEX, have played a seminal role in structuring modern Al paradigms and methodology. The Future of SUMEX-AIM Given this background, what is the future need and course for SUMEX as a resource -- especially in view of the on-going revolution in computer technology and costs and the emergence of powerful single-user workstations and local area networking? The answers remain clear. Basic Research on Al in Biomedicine At the deepest research level, despite our considerable success in working on medical and biological applications, the problems we can attack are still sharply limited. Our current ideas fall short in many ways against today's important health care and biomedical research problems brought on by the explosion in medical knowledge and for which Al should be of assistance. Just as the research work of the 70's and 80's in the SUMEX-AIM community fuels the current practical and commercial applications, our work of the late 80's will be the basis for the next decade's systems. The report of the panel on medical informatics [3], convened late in 1985 by the National Library of Medicine to review and recommend twenty-year goals for the NLM, listed among its highest priority recommendations the need to greatly expand and aggressively pursue an interdisciplinary research program to develop computational methods for acquiring, representing, managing, and using biomedical knowledge of all sorts for health care and biomedical research. These are precisely the problems which the SUMEX-AIM community has been working on so successfully and which will require work well beyond the five year funding period we have requested. It is essential that this line of research in the SUMEX-AIM community, represented by our core Al research, the ONCOCIN research, and our collaborative research groups, be continued. The Changing Role of the Central Resource At the resource level, there are changing, but still growing, needs for computing resources for the active AIM research community to continue its work over the next five years. The workstations to which we directed our attention in 1980 have now demonstrated their practicality as research tools and, increasingly, as potential mechanisms for disseminating Al systems as cost-effective decision aids in clinical settings such as private offices. The era of highly centralized general machines for Al research is rapidly coming to an end and is being replaced by networks of distributed but heterogeneous single-user machines sharing common information E. H. Shortliffe Resource Overview 5P41 -RR00785 15 resources and communication paths among members of the biomedical research community. Most of our community groups have been able to take advantage of local computing facilities, with SUMEX-AIM providing a central cross-roads for communications and the sharing of programs and knowledge. In its core research and development role, SUMEX-AIM has its sights set on the hardware and software systems of the next decade. We expect major changes in the distributed computing environments that are just now emerging in order to make effective use of their power and to adapt them to the development and dissemination of biomedical Al systems for professional user communities. In its training role, SUMEX is a crucial resource for the education of badly needed new researchers and professionals to continue the development of the biomedical Al field. The "critical mass" of the existing physical SUMEX resource, its development staff, and its intellectual ties with the Stanford Knowledge Systems Laboratory (KSL -- see Appendix A for a summary of current KSL research activities), make this an ideal setting to integrate, experiment with, and export these methodologies for the rest of the AIM community. We will continue our experimental approach to distributed systems, learning to build and exploit distributed networks of these machines and to build and manage graceful software for these systems. Since decentralization is central to our future, we must learn its technical characteristics. Resource Sharing An equally important function of the SUMEX-AIM resource is an exploration of the use of computer communications as a means for interactions and sharing between geographically remote research groups engaged in biomedical computer science research and for the dissemination of Al technology. This facet of scientific interaction is becoming increasingly important with the explosion of complex information sources and the regional specialization of groups and facilities that might be shared by remote researchers [2, 11. Another of the key recommendations of the NLM medical informatics planning panel [S] was that high-speed network communication links be established throughout the biomedical research community so that knowledge and information can be shared across diverse research groups and that the required interdisciplinary collaborations can take place. Recent efforts to establish a national NSFNet, largely to support the supercomputer projects funded by NSF but also to replace and upgrade part of the national research community linkage that the now aging ARPANET has supported, have made important progress. Still, these efforts do not encompass the broad range of biomedical research groups that need national network access and to date, the NIH has not played an aggressive role in the interagency Research Internet coordination efforts. We must work to build a stronger institutional support for a National Research Network. SUMEX continues to be an important pathfinder to develop the technology and community interaction tools needed to expand community system and communication resources. Our community building effort is based upon the developing state of distributed computing and communications technology and we have therefore turned our core systems research to actively supporting the development of distributed computing and communications resources to facilitate collaborative project research and continued inter-group communications. E. H. Shortliffe 8 5P41 -RR007851 5 Resource Overview Summary of Long-term Goals . Maintain the synergistic relationship between SUMEX core system development, core Al research, our experimental efforts at disseminating clinical decision-making aids, and new applications efforts. . Continue to serve the national AIM research community, less and less as a source of raw computing cycles and more and more as a transfer point for new technologies important for community research and communication. We will also continue our coordinating role within the community through electronic media and periodic AIM workshops. . Maintain our connections to ARPANET, TELENET, and our local Ethernet and assist other community members to establish similar links by example, by integrating and providing enabling software, and by offering advice and support within our resources. . Focus new computing resource developments on more effective exploitation of distributed workstations through better communication and cooperative computing tools, using transparent digital networking schemes. . Enhance the computing environments of workstations so that only minimal dependency on central, general-purpose computing hosts remains and these mainframe time-sharing systems can be phased out eventually. Remaining central resources will include servers for communications, community information resources, and special computing architectures (e.g., shared- or distributed-memory symbolic multiprocessors) justified by cost-effectiveness and unique functionality. . Incrementally phase in, disseminate, and evaluate those aspects of the local distributed computing resource that are necessary for continuing national AIM community support within this distributed paradigm. This will ultimately point the way towards the distributed computing resource model that we believe will interlink this community well into the next decade. . Gradually and responsibly phase out the existing DEC 2060 machine as effective distributed computing alternatives become widely available. We expect this to be possible sometime during the next year of the continuation resource. . Continue the central staff and management structure, essentially unchanged in size and function during the five-year transition period, except for the merging of the core part of the ONCOCIN research with the SUMEX resource. M.A. 1.2. Significance and Impact in Biomedicine Artificial intelligence is the computer science of representations of symbolic knowledge and its use in symbolic inference and problem-solving processes. Projects in the SUMEX-AIM community are concerned in some way with the application of Al to biomedical research and the resource has given strong impetus and support to knowledge-based system research in biomedicine. For computer applications in medicine and biology, this research path is crucial. Medicine and biology are not presently mathematically-based sciences; unlike physics and 9 E. H. Shortliffe Resource Overview 5P41 -RR00785-15 engineering, they are seldom capable of exploiting the mathematical characteristics of computation. They are essentially inferential, not calculational, sciences. If the computer revolution is to affect biomedical scientists, computers will be used as inferential aids. The growth in medical knowledge has far surpassed the ability of a single practitioner to master it all, and the computer's superior information processing capacity thereby offers a natural appeal. Furthermore, the reasoning processes of medical experts are poorly understood; attempts to model expert decision-making necessarily require a degree of introspection and a structured experimentation that may, in turn, improve the quality of the physician's own clinical decisions, making them more reproducible and defensible. New insights that result may also allow us more adequately to teach medical students and house staff the techniques for reaching good decisions, rather than merely to offer a collection of facts which they must independently learn to utilize coherently. Perhaps the larger impact on medicine and biology will be the exposure and refinement of the hitherto largely private heuristic knowledge of the experts of the various fields studied. The ethic of science that calls for the public exposure and criticism of knowledge has traditionally been flawed for want of a methodology to evoke and give form to the heuristic knowledge of scientists. Al methodology is beginning to fill that need. Heuristic knowledge can be elicited, studied, critiqued by peers, and taught to students. The importance of Al research and its applications is increasing in general, without regard for the specific areas of biomedical interest. Al is one of the principal fronts along which university computer science groups are expanding. The pressure from student career-line choices is great. Federal and industrial support for Al research and applications is vigorous, although support specifically for biomedical applications continues to be limited. All of the major computer manufacturers (e.g , IBM, DEC, TI, UNISYS, HP, and others) are using and marketing Al technoiogy aggressively and many software companies are putting more and more products on the market. Many other parts of industry are also actively pursuing Al applications in their own contexts, including defense and aerospace companies, manufacturing companies, financial companies, and others. Despite the limited research funding available, there is also an explosion of interest in medical Al. The American Association for Artificial Intelligence (AAAI), the principal scientific membership organization for the Al field, has 7000 members, several thousand of whom are members of the medical special interest group known as the AAAI-M. Speakers on medical Al are prominently featured at professional medical meetings, such as the American College of Pathology and American College of Physicians meetings; a decade ago, the words artificial intelligence were never heard at such conferences. And at medical computing meetings, such as the annual Symposium on Computer Applications in Medical Care (SCAMC) and the international MEDINFO conferences, the growing interest in Al and the rapid increase in papers on Al and expert systems are further testimony to the impact that the field is having. Al is beginning to have a similar effect on medical education. Such diverse organizations as the National Library of Medicine, the American College of Physicians, the Association of American Medical Colleges, and the Medical Library Association have all called for sweeping changes in medical education, increased educational use of computing technology, enhanced research in medical computer science, and career development for people working at the interface between medicine and computing. They all cite evolving computing technology and (SUMEX- AIM) Al research as key motivators. At Stanford, we have vigorous special programs for student training and research in Al -- a graduate program in Medical Information E. H. Shortliffe 10 5P41 -RR00785 15 Resource Overview Sciences and the two-year Masters Degree in Al program. All of these have many more applicants than available slots. Demand for their graduates, in both academic and industrial settings, is so high that students typically begin to receive solicitations one or two years before completing their degrees. 1II.A. 1.3. Summary of Current Resource Goals The following outlines the specific objectives of the SUMEX-AIM resource during the current three-year award period begun in August 1986. It provides an overall research plan for the resource and provides the backdrop against which specific progress is reported. Note that these objectives cover only the resource nucleus; objectives for individual collaborating projects are discussed in their respective reports in Section IV. Specific aims are broken into five categories: 1) Technological Research and Development, 2) Collaborative Research, 3) Service and Resource Operations, 4) Training and Education, and 5) Dissemination. I) Technological Research and Development SUMEX funding and computational support for core research is complementary to similar funding from other agencies (including DARPA, NASA, NSF, NLM, private foundations, and industry) and contributes to the long-standing interdisciplinary effort at Stanford in basic Al research and expert system design. We expect this work to provide the underpinnings for increasingly effective consultative programs in medicine and for more practical adaptations of this work within emerging microelectronic technologies. Specific aims include: . Basic research on Al techniques applicable to biomedical problems. Over the next term we will emphasize work on blackboard problem- solving frameworks and architectures, knowledge acquisition or learning, constraint satisfaction, and qualitative simulation. . Investigate methodologies for disseminating application systems such as clinical decision-making advisors into user groups. This will include generalized systems for acquiring, representing and reasoning about complex treatment protocols such as are used in cancer chemotherapy and which might be used for clinical trials. . Support community efforts to organize and generalize Al tools and architectures that have been developed in the context of individual application projects. This will include retrospective evaluations of systems like the AGE blackboard experiment and work on new systems such as BBl, CARE, EONCOCIN, EOPAL, Meta-ONYX, and architectures for concurrent symbolic computing. The objective is to evolve a body of software tools that can be used to more efficaciously build future knowledge-based systems and explore other biomedical Al applications. o Develop more effective workstation systems to serve as the basis for research, biomedical application development, and dissemination. We seek to coordinate basic research, application work, and system development so that the Al software we develop for the next 5-10 years will be appropriate to the hardware and system software environments we expect to be practical by then. Our purchases of new hardware will be limited to experimentation with state-of-the-art workstations as they become available for our system developments. 11 E. H. Shortliffe Resource Overview 5P41 -RR00785- 15 2) Collaborative Research . Encourage the exploration of new applications of Al to biomedical research and improve mechanisms for inter- and intra-group collaborations and communications. While Al is our defining theme, we may consider exceptional applications justified by some other unique feature of SUMEX-AIM essential for important biomedical research. We will continue to exploit community expertise and sharing in software development. . Minimize administrative barriers to the community-oriented goals of SUMEX-AIM and direct our resources toward purely scientific goals. We will retain the current user funding arrangements for projects working on SUMEX facilities. User projects will fund their own manpower and local needs: actively contribute their special expertise to the SUMEX-AIM community; and receive an allocation of system resources under the control of the AIM management committees. We will begin charging "fees for service" to Stanford users as DRR support for the DEC 2060 is phased out. Fees to national users will be delayed as long as financially possible. . Provide effective and geographically accessible communication facilities to the SUMEX-AIM community for remote collaborations, communications among distributed computing nodes, and experimental testing of Al programs. We will retain the current ARPANET and TELENET connections for at least the near term and will actively explore other advantageous connections to new communications networks and to dedicated links. 3) Service and Resource Operations SUMEX-AIM does not have the computing or manpower capacity to provide routine service to the large community of mature projects that has developed over the years. Rather, their computing needs are better met by the appropriate development of their own computing resources when justified. Thus, SUMEX-AIM has the primary focus of assisting new start-up or pilot projects in biomedical Al applications in addition to its core research in the setting of a sizable number of collaborative projects. We do offer continuing support for projects through the lengthy process of obtaining funding to establish their own computing base. 4) Training and Education o Provide documentation and assistance to interface users to resource facilities and systems. o Exploit particular areas of expertise within the community for assisting in the development of pilot efforts in new application areas. o Accept visitors in Stanford research groups within limits of manpower, space, and computing resources. . Support the Medical Information Science and MS/Al student programs at Stanford to increase the number of research personnel available to work on biomedical Al applications. . Support workshop activities including collaboration with other community E. H. Shortliffe 12 5P41 -RR00785 15 Resource Overview groups on the AIM community workshop and with individual projects for more specialized workshops covering specific research, application, or system dissemination topics. 5) Dissemination While collaborating projects are responsible for the development and dissemination of their own Al systems and results, the SUMEX resource will work to provide community-wide support for dissemination efforts in areas such as: o Encourage, contribute to, and support the on-going export of software systems and tools within the AIM community and for commercial development. . Assist in the production of video tapes and films depicting aspects of AIM community research. . Promote the publication of books, review papers, and basic research articles on all aspects of SUMEX-AIM research. 13 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 1II.A. 2. Details of Technical Progress This section gives an overview of progress for the nucleus of the SUMEX-AIM resource. A more detailed discussion of our progress in specific areas and related plans for further work are presented beginning on Page 19. Objectives and progress for individual collaborating projects are discussed in their respective reports on Page 127. These collaborative projects collectively provide much of the scientific basis for SUMEX as a resource and our role in assisting them has been a continuation of that evolved in the past. Collaborating projects are autonomous in their management and provide their own manpower and expertise for the development and dissemination of their Al programs. 1II.A. 2.1. Progress Highlights In this section we srtmmarize highlights of SUMEX-AIM resource activities over the past year (May 1987 - April 1988), focusing on the resource nucleus. . We made excellent progress in core Al research. We have begun to explore the design and use of very large knowledge bases with the hypothesis that both the problems of brittleness and over-specialization in current knowledge-based systems can be addressed by constructing large, multi-use knowledge bases (LMKB). A LMKB would 1) encode domain knowledge in greater depth and breadth than required for any specific task, 2) encode knowledge that cuts across many domains of expertise, and 3) serve as a core repository of knowledge to be accessed by large numbers of specific applications. Research has also progressed on several fundamental issues of Al, including knowledge representation, blackboard frameworks, parallel symbolic computing architectures, and machine learning. Work continues in PROTEAN and BBI on the explicit representation of control knowledge, on the representation of geometric problem-solving knowledge in PROTEAN and PEAKS, on the representation of diagnosis expertise and the integration of a numerical simulator of a model system with an expert system in ABLE, and on the flexible, rich representation of control knowledge to facilitate modeling of problem-solving at the strategic level as well as at the tactical level. We have continued to develop the BBl blackboard architecture for systems that reason about (control, explain, and learn about) their own actions. The BBI system is being applied in a new domain, the real time monitoring of patients in an intensive care unit (BBICU). The parallel architectures work has developed a CAD (Computer Aided Design) system for hierarchical, multiple-level specification of computer architectures (SIMPLE); a parameterized, multiprocessor array emulation defined in SIMPLE's specification languages and running on SIMPLE's simulator (CARE); a set of extensions to Lisp for studying expressed concurrency in functional programming, object-oriented, and shared- variable models of concurrent computation (LAMINA); and two alternative parallel blackboard frameworks for expressing application problems (POLIGON and CAGE). These have been applied to several signal understanding problems with promising problem- solving speed-up. The machine learning work has concentrated on explanation-based generalization and chunking work in the SOAR framework, inductive rule E. H. Shortliffe 14 5P41-RR00785-15 Details of Technical Progress learning, "apprenticeship" learning, and tools for acquiring knowledge and debugging knowledge structures. Our research in machine learning has focused on several distinct problem domains including medicine (NEOMYCIN/HERACLES), physics (ABLE), and biochemistry (PROTEAN) in addition to domain-independent investigations. Work has also continued on reasoning with uncertainty to find ways of combining formal and informal approximate reasoning methods. Specifically, this research seeks to (1) develop techniques for using knowledge about problem-solving tradeoffs to dynamically optimize system performance for the user, (2) construct efficient algorithms for probabilistic reasoning, and (3) investigate pragmatic techniques for the elicitation of knowledge from experts. o We have made continued significant progress in the core ONCOCIN research to generalize the tools for clinical trial management from the initial cancer chemotherapy management application. A major accomplishment this year from our work on the examination of the structures of protocols across medical subspecialties other than cancer chemotherapy (e.g., hypertension and insulin diabetes treatment) was the creation of a generalized knowledge acquisition tool designed to encode descriptions of clinical trials. The system is called PROTEGE and produces as its output a computer program which is an OPAL-like clinical trial protocol definition system specifically tailored for an arbitrary clinical area such as hypertension. This OPAL-like system can then be used to create the knowledge base (e.g., for hypertension) that drives an ONCOCIN-like patient/protocol management system. We continued development of the OPAL system for graphical knowledge acquisition to facilitate protocol definition and knowledge base entry for the ONCOCIN oncology application area. We began to explore alternative platforms for developing OPAL-like systems such as HyperCard on the MAC II. We also tested the usefulness of a "generic protocol viewing tool" based on a relational data base storage mechanism. We performed several experiments aimed at developing a single underlying storage mechanism (termed a "cell") from which various interface systems including the Interviewer flowsheet, a generalized spreadsheet utility, and the OPAL schema entry flowchart interface could be derived. We have begun a project to explore the integration of speech- recognition technology into the interface to ONCOCIN. The project uses a commercially available continuous speech recognition product and a prototype ONCOCIN adaptation now permits users to navigate the graphical interface and enter clinical data using spoken commands. The development of this system exploits the significant experience we have gained in distributed computing since the phonetic device, initial parsing software, and the ONCOCIN system all reside on different pieces of hardware. We released a new version of our ONCOCIN object language which has proven to be the most stable and powerful version to date. We have also continued to examine the issues of disseminating the ONCOCIN system into actual clinical settings. . We have made excellent progress on the core system development work targeted at supporting the distributed AIM community. We elaborated 15 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 our systems development plans further at a site visit held in August 1987 in response to our request to the National Advisory Research Resources Council to restore the final 2 years of our grant award. Following a special study section review and reconsideration by the Council, our plan and the full 5-year grant award were approved. In line with the review group recommendations, we have moved to sharply focus our development resources on a small number of standardized hardware and software configurations. After consideration of many requirements and alternative systems for AIM community computing needs for the coming years to replace and upgrade the services of the 2060, we have chosen Apple Macintosh II workstations as the general computing environment for researchers and staff, TI Explorer Lisp machines (including the microExplorer Macintosh coprocessor) as the near-term high-performance Lisp research environment, and a SUN-4 as the central system network server (wide- and local-area network interfaces, file services, printing services, etc.). The bulk of this hardware was purchased with DARPA research funds and we are now beginning the installation and integration process. We have concentrated our current systems efforts on getting basic capabilities operational, such as for text processing, filing/archiving, printing, graphics, office management, system building tools, information resource access, and distributed system operation and management) etc. Development work for the Mac/Explorer/SUN environments has been limited because of manpower cuts necessitated by NIH cuts in our funding awards. Available resources have focussed on providing remote access between workstations, integrating a solid support of the TCP-IP network protocol, and building a distributed electronic mail system. We have significantly refined the prototype distributed EMail system developed for the Xerox D-machines last year (a number of people are using the system routinely to manage their mail) and are porting this to the Explorer and adapting it for the Mac II. We have started a project based on the MacWorkstation software licensed from APPLE which is designed to allow the Macintosh to start up programs on mainframes or other workstations and receive graphic and text output to the MAC in a seamless fashion. We plan to write a Common Lisp window package (based on Common Windows) that uses MacWorkstation so we can connect (via Ethernet, AppleNet or RS-232CYmodem) to any of our Common Lisp engines and run the same piece of code on any of them. One of the key issues in selecting the systems for our distributed computing environment was the performance of Common Lisp and to help make this evaluation, we undertook an informal survey of the performance of two KSL Al software packages, SOAR and BBl, on a wide variety of machines. Within a factor of two of the best performance, a considerable range of workstations based on stock microprocessor chips as well as specially microprogrammed Lisp chips have comparable performance. Even though performance gaps between microprogrammed Lisp systems and stock workstation implementations are narrowing, there still remains a significant difference in the quality of the development environments. We have attempted to distill the key features of the Lisp machine environments that would be needed in stock machine implementations in order to make them attractive in a development setting. E. H. Shortliffe 16 5P41 -RR007851 5 Details of Technical Progress We have installed an interface between our Ethernet environment and the TELENET network by which many AIM users gain access to SUMEX that allows full access to distributed SUMEX resources. We have also continued to maintain the DEC 2060 environment to stay current with operating system and other environmental upgrades. . We have continued the dissemination of SUMEX-AIM technology through various media. We have reorganized the distribution system for our Al software tools (EMYCIN, AGE, and BBl) to academic, industrial, and federal research laboratories, in order to make it more efficient and require less research staff time. We have also continued to distribute the video tapes of some of our research projects including ONCOCIN, and an overview tape of Knowledge Systems Laboratory work to outside groups. Our group has continued to publish actively on the results of our research, including more than 45 research papers per year in the Al literature and a dozen books in the past 5 years on various aspects of SUMEX-AIM Al research. We assisted and participated actively in the AIM Workshop sponsored by AAAI and held at Stanford (Ramesh Patil from MIT was the Program Chairman) and hosted a number of AIM community visitors at our Stanford research laboratory this past year. o The Medical information Sciences program, begun at Stanford in 1983 under Professor Shortliffe as Director, development over the past year. has continued its strong The specialized curriculum offered by the MIS program focuses on the development of a new generation of researchers able to support the development of improved computer- based solutions to biomedical needs. The feasibility of this program resulted in large part from the prior work and research computing environment provided by the SUMEX-AIM resource. It has recently received enthusiastic endorsement from the Stanford Faculty Senate for an additional five years, has been awarded renewed post-doctoral training support from the National Library of Medicine with high praise for the training and contributions of the SUMEX-AIM environment from the reviewing study section, and has received additional industrial and foundation grants for student support. This past year, MIS students have published many papers, including several that have won conference awards. o We have continued to recruit new user projects and collaborators to explore further biomedical areas for applying Al. A number of these projects are built around the communications network facilities we have assembled, bringing together medical and computer science collaborators from remote institutions and making their research programs available to still other remote users. At the same time we have encouraged older mature projects to build their own computing environments thereby facilitating the transition to a distributed AIM community. A substantial number of projects have moved to their own computing resources, including SOAR, under Dr. Paul Rosenbloom at USC/IS1 in Los Angeles; the Logic Group projects (DART, Intelligent Agents, and MRS) under Professor Michael Genesereth at Stanford; Hierarchical Models of Human Cognition (CLIPR), under Professors Walter Kintsch and Peter Polson at the University of Colorado: Problem Solving Expertise (SOLVER), under Professors Paul Johnson and William Thompson at the University of Minnesota; RXDX, under Professor Robert Lindsay at the University of Michigan: and Computer-Based Exercises in Pathophysiologic Diagnosis, under Professor J. Robert Beck at Dartmouth College. 17 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785 15 SUMEX user projects have made good progress in developing and disseminating effective consultative computer programs for biomedical research. These systems provide expertise in areas like cancer chemotherapy protocol management, clinical diagnosis and decision- making, and molecular biology. We have worked hard to meet their needs and are grateful for their expressed appreciation (see Section IV). E. H. Shortliffe 18 5P41 -RR00785- 15 Details of Technical Progress lll.A.2.2. Core ONCOCIN Research ONCOCIN is a data management and therapy advising program for complex cancer chemotherapy experiments. The development of the system began in 1979, following the successful generalization of MYCIN into the EMYCIN expert system shell. The ONCOCIN project has evolved over the last eight years: the original version of ONCOCIN ran on the time-shared DEC computers using a standard terminal for the time-oriented display of patient data. The current version uses compact workstations running on the Ethernet network with a large bit-mapped displays for presentation of patient data. The project has also expanded in scope. There are three major research components: 1) ONCOCIN, the therapy planning program and its graphical interface; 2) OPAL, a graphical knowledge entry system for ONCOCIN; and 3) ONYX, a strategic planning program designed to give advice in complex therapy situations. Each of these research components has been split into two parts: continued development of the cancer therapy versions of the system, and generalization of each of the components for use in other areas of medicine. This annual report will describe our work on each of the components: implementation of ONCOCIN workstations in the Stanford clinic, knowledge acquisition research, and research to generalize ONCOCIN for application in clinicai trial domains other than medical oncology (E-ONCOCIN). A major highlight of this year was the creation of a generalized knowledge acquisition tool designed to encode descriptions of clinical trials. The system, named PROTEGE, was the Ph.D. thesis work of Mark Musen. The output of PROTEGE is an OPAL-like input system designed for one clinical area such as hypertension. This input system (HTN-OPAL) can then be used to create the hypertension knowledge base for an E-ONCOCIN like system. This experiment was carried out this year for both the hypertension and oncology domains. Details of this project are described later in this report. 1 - Overview of the ONCOCIN Therapy Planning System ONCOCIN is an advanced expert system for clinical oncology. It is designed for use after a diagnosis has been reached, focusing instead on assisting with the management of cancer patients who are receiving chemotherapy. Because anticancer agents tend to be highly toxic, and because their tumor-killing effects are routinely accompanied by damage to normal cells, the rules for monitoring and adjusting treatment in response to a given patient's course over time tend to be complex and difficult to memorize. ONCOCIN integrates a temporal record of a patient's ongoing treatment with an underlying knowledge base of treatment protocols and rules for adjusting dosage, delaying treatment, aborting cycles, ordering special tests, and similar management details. The program uses such knowledge to help physicians with decisions regarding the management of specific patients. A major lesson of past work in clinical computing has been the need to develop methods for integrating a system smoothly into the patient-care environment for which it is intended. In the case of ONCOCIN, the goal has been to provide expert consultative advice as a by-product of the patient data management process, thereby avoiding the need for physicians to go out of their way to obtain advice. It is intended that oncologists use ONCOCIN routinely for recording and reviewing patient data on the computer's screen, regardless of whether they feel they need decision- making assistance. This process replaces the conventional recording of data on a paper flowsheet and thus seeks to avoid being perceived as an additive task. In accordance with its knowledge of the patient's chemotherapy protocol, ONCOCIN then provides assistance by suggesting appropriate therapy at the time that the day's treatment is to be recorded on the flowsheet. Physicians maintain control of the 19 E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 decision, however, and can override the computer's recommendation if they wish. ONCOCIN also indicates the appropriate interval until the patient's next treatment and reminds the physician of radiologic and laboratory studies required by the treatment protocol. 2 - Implementation of the ONCOCIN Workstation in the Stanford Clinic In mid-1986, we placed the workstation version of ONCOCIN into the Oncology Day Care clinic. This version is a completely different program from the version of ONCOCIN that was available in the clinic from 1981-1985-- using protocols entered through the OPAL program, with a new graphical data entry interface, and revised knowledge representation and reasoning component. One person in the clinic (Andy Zelenetz) became primarily responsible for making sure that our design goals for this version of ONCOCIN were met. His suggestions included the addition of key protocols and the ability to have the program be useful for clinicians as a data management tool if the complete treatment protocol had not yet been entered into the system. Additional fellows are being trained on a very stable release of ONCOCIN that became available in early 1988. A version of the system was sent to the University of Pittsburgh for evaluation. In addition, ONCOCIN will become available for presentation at the National Library of Medicine Artificial Intelligence Demonstration Center. For these various efforts, Janice Rohn has created an extensive user manual, sample patient interactions, and reminder cards to shorten the training period for ONCOCIN. The process of entering a large number of treatment protocols in a short period of time led to other research topics including: design of an automated system for producing meaningful test cases for each knowledge, modification of the design of the time-oriented database and the methods for accessing the database, and the development of methods for graphically viewing multiple protocols that are combined into one large knowledge base. These research efforts will continue into the next year. In addition, some of the treatment regimens developed for the origin& mainframe version are still in use and can be transferred to the new version of ONCOCIN. We also received new insights about the design of the internal structures of the knowledge base (e.g., the relationship between the way we refer to chemotherapies, drugs, and treatment visits). We will continue to optimize the question-asking procedure, the method for traversing the plan structure in the knowledge base, and consider alternative arrangements used to represent the structure of chemotherapy plans. Although we have concentrated our review of the ONCOCIN design primarily on the data provided by additional protocols, we know that non-cancer therapy problems raise similar issues. The E-ONCOCIN effort is designed to produce a domain-independent therapy planning system that includes the lessons learned from our oncology research. 3 - E-ONCOCIN: Domain Independent Therapy Planning During the past two years, our E-ONCOCIN research has concentrated on understanding how protocols in medicine vary across subspecialties. We are examining several application areas: the intensive care unit, insulin treatment for diabetes, hypertension protocols, and both standard and complex cancer treatment problems. The diagnosis and therapy selection for patients in the intensive care unit is a natural application area because it is based on changing data and the need to determine the response to therapy interventions. In addition, it is an area where reasonable mathematical models of the respiratory system can be integrated into the expert system. We also felt that the area of insulin treatment for diabetes would be a E. H. Shortliffe 5P41 -RR00785-15 Details of Technical Progress good area to explore. Like cancer chemotherapy, the treatments for diabetes continues over a long period of time and has been the area of intensive protocol development. Unlike cancer chemotherapy, the treatment plan must handle multiple treatments in one day and deemphasizes the use of multiple drugs (although there are a variety of types of insulin). During 1987, using the medical literature and several internists in the medical computer science research group (Mark Frisse, Mark Musen, and Michael Kahn), we performed knowledge acquisition experiments for insulin treatment of diabetes. The proposed structure for the knowledge base was implemented using the object-oriented programming language upon which ONCOCIN has been based. These experiments, like those of adding more protocols to ONCOCIN, demonstrated the need for changes in the way that the knowledge base can access the time-oriented data base that records patient data and previous conclusions. The relationships between the different doses and types of insulin treatments will also require alternative ways of building treatment hierarchies. Thus, our initial experiments have shown that many of the elements of the ONCOCIN design are sufficiently general for other application areas, but that some specific elements (particularly the representation of temporal events) will have to be generalized. A description of our revised temporal representations will appear in a forthcoming thesis by Michael Kahn of University of California at San Francisco. During the coming year, we will continue our knowledge acquisition experiments and design a version of the E-ONCOCIN system that is separate from the ongoing "clinic version." 4 - OPAL: Graphical Knowledge Acquisition Interface OPAL is a graphical environment for use by an oncologist who wishes to enter a new chemotherapy protocol for use by ONCOCIN or to edit an existing protocol. Although the system is designed for use by oncologists who have been trained in its use, it does not require an understanding of the internal representations or reasoning strategies used by ONCOCIN. The system may be used in two interactive modes, depending on the type of knowledge to be entered. The first permits the entry of a graphical description of the overall flow of the therapy process. The oncologist manipulates boxes on the screen that stand for various steps in the protocol. The resulting diagram is then translated by OPAL into computer code for use by ONCOCIN. Thus, by drawing a flow chart that describes the protocol schematically, the physician is effectively programming the computer to carry out the procedure appropriately when ONCOCIN is later used to guide the management of a patient enrolled in that protocol. OPAL's second interactive mode permits the oncologist to describe the details of the individual events specified in the graphical description. For example, the rules for administering a given chemotherapy will vary greatly depending upon the patient's response to earlier doses, intercurrent illnesses and toxicities, hematologic status, etc. For example, one form permits the entry of an attenuation schedule for an agent based upon the patient's white count and platelet count at the time of treatment. Tables such as this are generally found in the written version of chemotherapy protocols. Thus OPAL permits oncologists to enter information using familiar forms displayed on the computer's screen. The contents of such forms are subsequently translated into rules and other knowledge structures for use by ONCOCIN. 21 E. H. Shortliffe Details of Technical Progress 5P41-RR00785-15 4.1 - Status of the OPAL System The OPAL is one of the few graphical knowledge acquisition systems ever designed for expert systems. Even fewer are designed to be used as the main method for entering knowledge as opposed to a proof of concept implementation. We have pursued three directions in the development of the OPAL system, also in response to the large number of protocols entered through this system during the last year. The first direction is the modification of graphical forms needed to allow the entry of facts that did not show up in the protocols used to test the initial version of OPAL. OPAL continues to assume that most of the knowledge to be entered will have very stereotyped forms, e.g., dose attenuations for most treatment toxicities are based on a comparison of only one laboratory measurement at a time, such as using the BUN to adjust for renal toxicity. We sometimes need much more complex ways of stating the scenarios in which dose adjustments may be necessary. This need has led us in a second direction, towards a "lower-level" rule entry approaching the syntax of the reasoning component of ONCOCIN, but using graphical input devices where applicable. A major accomplishment of this last year was to experimentally combine the OPAL and ONCOCIN programs into one working program, and to completely enter knowledge from OPAL using both the high level tools and lower level rule editors, but without needing to make changes at the ONCOCIN side of the system. Work on graphical replacements for low level rule entry is the master's project of Eric Sherman. The OPAL program maps the information provided on the graphical forms into a complex data structure (called the IDS) that represents the required knowledge to snecify the contents of a protocol. This data structure is used for copying information from one protocol to another, and as the basis for the creation of the ONCOCIN knowledge base. Our experiments with OPAL, and our intention to generalize OPAL use outside of oncology protocols, suggests that we reorganize the OPAL program to use a relational database to store its knowledge. We have patterned the database after an existing database query syntax. Because no databases exist for the InterLisp language upon which OPAL is based, we reimplemented the database from its written description. The database structure is completed, and was the basis for the PROTEGE knowledge acquisition experiments. With changes in the future of dedicated lisp processors, we began to explore alternative platforms for developing OPAL-like systems. We have begun experiments using HyperCard on the MAC II. We also tested the usefulness of a "generic protocol viewing tool" based on a relational-database storage mechanism, in preparation for a OPAL design based on relational DBMS technology. We also performed several experiments aimed at developing a single underlying storage mechanism (termed a "cell") from which various interface systems including the Interviewer flowsheet, a generalized spreadsheet utility, and the OPAL schema entry flowchart interface could be derived. 5 - Generalized Knowledge Acquisition through PROTEGE Mark Musen designed and implemented the first version of the PROTEGE knowledge-acquisition-system development tool. PROTEGE is used to collect information which describes the concepts (both entities and their relationships) in an application area for which a skeletal-planning type of expert system would be useful, concentrating on clinical trials. The system acquires the "ontology" of a domain through a series of fill-in-the-blank forms and a "flowchart-entry" tool. These concepts are then mapped onto a set of generic forms, which, in turn , create a knowledge acquisition tool for the application area. PROTEGE makes use of the forms management system built for the original OPAL, E. H. Shortliffe 22 5P41 -RR007851 5 Details of Technical Progress and a newly developed relational database management system written for the Xerox Interlisp-D workstations. The output of PROTEGE is an OPAL-like set of forms tailored to the special structures of the application area. To test these ideas, we first reimplemented portions of the OPAL interface from a high level description of oncology. After the translation process to the ONCOCIN reasoning program, a consultation was run that matched the manually built system. This experiment was then repeated for the area of hypertension protocols for which ONCOCIN had never been specifically designed. With some minor generalizations to the oncocin reasoner and interviewer, we were able to run a hypertension consultation. 6 - Speech Input to Expert Systems 6.1 - Prototype Speech Hardware/Software System In 1987 we began a project to explore the integration of speech-recognition technology into the interface to ONCOCIN running on the XEROX lisp workstations. The project uses a commercially available continuous-speech-recognition product loaned by the vendor, Speech Systems, Inc. (SSI) of Tarzana, California. The speech recognizer consists of a custom processor, called the Phonetic Engine(R) and a suite of software modules called the Phonetic Decoder(TM). The Phonetic Decoder(TM) is running on a SUN 3/75. The development of this project requires significant experience in distributed computing since the phonetic device, initial parsing software and the ONCOCIN system all reside on different pieces of hardware. One of the early steps is to allow the Lisp machine to remotely control the parsing software on the SUN. We built an interpreter for communicating between the speech software library running on the SUN and the Xerox Lisp machine. This interpreter reads lisp-style function calls corresponding to the speech library routines and returns lisp-style results as remote procedure call (RPC) mechanism. We then wrote the library of corresponding Lisp stub routines and a function to connect to the SUN workstation (through a programmatic TELNET) and start the server. In normal operation, we call the `C'- based speech library routines from Lisp as if they were Lisp functions. We did a number of updates to both components of the server program (on the SUN and Lisp machine) as various SSI updates came out, including a major revision when SSI doubled the number of library routines, greatly increasing the server's capabilities. Eventually we plan to replace the programmatic TELNET and custom RPC language with a new version of the system that uses the SUN RPC language. This will include the Xerox Lisp interface to the SUN RPC package developed by SUMEX; we were not able to use this interface initially as it did not exist early enough and required a release of Xerox Lisp that supports Common Lisp which the ONCOCIN system was not yet using. This eventual change will allow more efficient communications between the machines, allowing us to move larger data structures much faster. This will also tie in to the proposed method of communications between Lisp and non- Lisp routines that Xerox plans to use when their system migrates to the SUN workstations. The Lisp routines that correspond to the various SSI library routines will not appear to change to the application programs so they should not require any modification when the underlying RPC mechanism is changed. We created a prototype system that permits users to navigate the graphical interface and enter clinical data using speech. The system uses the location of the cursor on the screen to provide a context for choosing candidate grammars with which to attempt recognition of a user's utterance. The system dynamically re-orders the list of candidate recognition grammars based on the dialog history. Albeit with limitations on the legal grammars, it is now possible to carry on most of the ONCOCIN data 23 E. H. Shortliffe Details of Technical Progress 5P41-RR0078515 acquisition steps using speech alone or speech plus pointing with the mouse. In addition, some elements such as the neural toxicities can be entered as textual descriptions and automatically encoded as one the 1-4 point scale used on flowsheet forms. In order to translate an utterance back into an action that can occur in the ONCOCIN interface, we need the ability to reparse the text string returned by the SSI equipment. The SSI equipment uses (potentially complex) syntax, built up of various classifications, to understand sentences but returns just the ASCII component of the actual sentence: you can not get it in terms of the original classifications in the syntax (which are generally semantically significant). To overcome this problem, we devised a whole new syntax format based on Lisp and our OZONE object language in which one devises a grammar from which the SSI syntax files are generated. Then when the ASCII string is returned, the original syntax object is used to parse the string into a parse tree that relates directly to the grammar definition. We can now process the returned information at a much higher level than was possible with the simple ASCII text. In addition, the semantic elements of the parse can be added to syntactic structures used to encode the sentence. The latest additions to this part of the system include exhaustive and random sentence generation as Lisp data structures (previously only partially available just on the SUN workstation) as well as word list generation from the syntax objects. These features generate both in-core representations as well as file representations which will potentially free us from any dependency on SSl's equipment (versus another manufacture's speech input device). We wrote graphics software that plots the sentence returned by the speech equipment along with the changes in amplitude, pitch, accuracy scores and other information. This helps us to understand how SSl's `black box' operates, particularly in situations where it fails. In addition, we wrote programs to make it possible to return all the best syntactic matches from the speech equipment. The current SSI software can either return the one best answer or all the possible word mappings (unfortunately regardless of syntax). We believe the latter would be useful when the most likely parse is associated with a low reliability score. We could then look at all the highest scoring candidates and evaluate what they do and do not have in common for creating a clarifying query to the speaker. The candidate sentences are composed from the word mapping data which is immense and needs to be compressed, filtered and manipulated to be of any use (the parser/generator is used as a filter to remove the non-syntactic word mappings). Unfortunately, though it runs faster with each improvement, this doesn't look reasonable for real time use; currently a three word utterance takes about two to four minutes to process into a candidate list of half a dozen possible sentences. It gives us an indication of what would be useful data structures to process on the SUN (to improve speed and lower data transmission). 6.2 - Speech Experiments We are performing experiments to (1) enhance the system's grammars with a wider range of phrases clinicians actually use when talking to a computer and (2) gain insights into clinicians' models of spoken interaction with advice systems so that we may ground our interface design in observed practice. In order to assess how physicians would speak to a computer in an ideal situation without constraints or prior assumptions, we are conducting a series of experiments which simulate continuous-speech understanding by computers. The setting of these experiments includes a hidden computer operator simulating the output of ONCOCIN if it had the ability to understand the spoken input, as well as a video camera to E. H. Shortliffe 5P41 -RR00785-15 Details of Technical Progress record both audio and visual clues. Typed responses from the operator are translated back as actions on the computer display as well as audio responses through the use of a speech synthesizer. It appears to the subject as if the computer is understanding and responding to their speech. The physicians use ONCOCIN in the same manner as it is used in the clinic when they see patients, but with the added capability of speech input. These experiments enable us to both build up a basic vocabulary for the speech system as well as examine subtle linguistic issues to guide future directions. One component of these experiments was the use of speech synthesis as an output medium. Using an inexpensive board based on the General Instrument ASCII-to- phoneme and phoneme-to-voice chip set, we built a driver for the Xerox Lisp machines. This driver included an application software transparent misspelling dictionary facility to correct for inaccuracies in the speech board. 7 - Object Language Support for ONCOCIN Project We released a new version of our object language at the start of this past year which has proven to be the most stable and powerful version to date. There have been a number of minor bug fixes and several feature additions over the course of the year but for the most part the system as required much less attention than in previous years. The number of new systems being built on it (like our speech work) continues to increase. Future planning for the system consists of determining whether or not it should be converted to Common Lisp, based on whether object systems available under Common Lisp are sufficient for our needs, and if we do convert it what it would it look like if properly integrated with that language. 8 - Personnel Samson Tu has been primarily responsible for the design of E-ONCOCIN, Michael Kahn has developed the temporal representations used by the system, Clifford Wulfman has been involved with extensions the the data entry interface and the extensions to the interface in order to add speech input. Samson and Cliff were responsible for extensions to their programs to support the PROTEGE effort. David Combs has been involved with the knowledge acquisition interface and provided major programming support for the PROTEGE effort. Janice Rohn has been involved with the entry of protocols, interaction with physicians using the system, documentation of the system, and execution of the speech experiments. Ellen Isaacs, a Ph.D. student in Psycholinguistics has helped to design the speech experiments. Christopher Lane has developed the object-oriented systems software upon which the entire ONCOCIN system is designed and the systems software and parsing programs used in the speech project. 25 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 lll.A.2.3. Core Al Research 1 - Rationale Artificial Intelligence (Al) methods are particularly appropriate for aiding in the management and application of knowledge because they apply to information represented symbolically, as well as numerically, and to reasoning with judgmental rules as. well as logical ones. They have been focused on medical and biological problems for well over a decade with considerable success. This is because, of all the computing methods known, Al methods are the only ones that deal explicitly with symbolic information and problem solving and with knowledge that is heuristic (experiential) as well as factual. Expert systems are one important class of applications of Al to complex problems -- in medicine, science, engineering, and elsewhere. An expert system is one whose performance level rivals that of an human expert because it has extensive domain knowledge (usually derived from an human expert); it can reason about its knowledge to solve difficult problems in the domain; it can explain its line of reasoning much as an human expert can: and it is flexible enough to incorporate new knowledge. without reprogramming. Expert Systems draw on the current stock of ideas in Al, for example, about representing and using knowledge. They are adequate for capturing problem-solving expertise for many bounded problem areas. Numerous high-performance, expert systems have resulted from this work in such diverse fields as analytical chemistry, medical diagnosis, cancer chemotherapy management, VLSI design, machine fault diagnosis, and molecular biology. Some of these programs rival human experts in solving problems in particular domains and some are being adapted for commercial use. Other projects have developed generalized software tools for representing and utilizing knowledge (e.g., EMYCIN, UNITS, AGE, MRS, BBl, and GLisp) as well as comprehensive publications such as the three-volume Handbook of Artificial Intelligence and books summarizing lessons learned in the DENDRAL and MYCIN research projects. There is considerable power in the current stock of techniques, as exemplified by the rate of transfer of ideas from the research laboratory to commercial practice. But we also believe that today's technology needs to be augmented to deal with the complexity of medical information processing. Our core research goals, as outlined in the next section, are to analyze the limitations of current techniques and to investigate the nature of methods for overcoming them. Long-term success of computer-based aids In medicine and biology depend on improving the programming methods available for representing and using domain knowledge. That knowledge is inherently complex: it contains mixtures of symbolic and numeric facts and relations, many of them uncertain; it contains knowledge at different levels of abstraction and in seemingly inconsistent frameworks; and it links examples and exception clauses with rules of thumb as well as with theoretical principles. Current techniques have been successful only insofar as they severely limit this complexity. As the applications become more far- reaching, computer programs will have to deal more effectively with richer expressions and much more voluminous amounts of knowledge. Expert systems are being developed that impact nearly every field of human endeavor: medicine, manufacturing, financial services, diagnosis of machinery, geology, molecular biology and structural design, to name a few. Each new instance is a confirmation of the hypothesis that knowledge is power. in each system, expert level problem-solving performance is obtained by using relatively simple and uniform reasoning methods which access an extensive body of domain knowledge. The ability of these systems lies not in their superior reasoning capabilities but in the E. H. Shortliffe 26 5P41 -RR00785-15 Details of Technical Progress specific concepts, facts, methods, models, etc. that can be brought to bear on the problem. The know/edge is power hypothesis has received so much confirmation that we now assert it as the knowledge principle. A corollary to the Knowledge Principle is that significant improvements in the power of knowledge-based systems will be derived primarily from the ability to access large amounts of knowledge. During the past year we have begun to explore the design and use of very large knowledge bases. In the last twenty years we have learned how to build intelligent programs that perform at a high level of competence on specialized tasks within narrowly defined domains. These programs traditionally access small to modest- sized knowledge bases specialized to the prescribed task. In contrast, we have started on a long-range research effort that will result in a large, multi-use knowledge base (LMKB). We believe construction of a LMKB is an essential step toward resolving two fundamental problems plaguing the current generation of expert systems. The first is brittleness: current systems can exhibit only a very narrow range of expert behavior, and their performance falls off precipitously at the limits of their expertise. The second problem is over-specialization: a knowledge base constructed to support of one type of expert task (e.g., diagnosis) cannot be used to support other types of tasks (e.g., design). Our hypothesis is that both the problems of brittleness and over-specialization can be addressed by constructing large, multi-use knowledge bases. A LMKB would 1) encode domain knowledge in greater depth and breadth than required for any specific task, 2) encode knowledge that cuts across many domains of expertise, and 3) serve as a core repository of knowledge to be accessed by large numbers of specific applications. This report documents progress on the basic or core research activities within the Knowledge Systems Laboratory (KSL), funded in part under the SLlMEX resource as well as by other federal and industrial sources. This work explores a broad range of basic research ideas in many application settings, all of which contribute in the long term to improved knowledge based systems in biomedicine. 2 - Highlights of Progress In the last year, research has progressed on several fundamental issues of Al. As in the past, our research methodology is experimental; we believe it is most fruitful at this stage of Al research to raise questions, examine issues, and test hypotheses in the context of specific problems, such as management of patients with Hodgkin's disease. Thus, within the KSL we build systems that implement our ideas for answering (or shedding some light on) fundamental questions; we experiment with those systems to determine the strengths and limits of the ideas; we redesign and test more; we attempt to generalize the ideas from the domain of implementation to other domains; and we publish details of the experiments. Many of these specific problem domains are medical or biological. In this way we believe the KSL has made substantial contributions to core research problems of interest not just to the AIM community but to Al in general. Progress is reported below under each of the major topics of our work. Citations are to KSL technical reports listed in the publications section. 27 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785 15 2.1 - Know/edge Representation How can the knowledge necessary for complex problem solving be represented for its most effective use in automatic inference processes? Often, the knowledge obtained from experts is heuristic knowledge, gained from many years of experience. How can this knowledge, with its inherent vagueness and uncertainty, be represented and applied? Work continues in PROTEAN and BBl, with its explicit representation of control knowledge (see the summary of Blackboard Architectures below). In particular, we have advanced our methods for representation of geometric problem solving knowledge in PROTEAN and PEAKS (see PROTEAN section of this report.) We have developed an application in a new domain of diagnosis and correction of errors in a linear accelerator beam line, the ABLE project. In this we have explored issues of representation of diagnosis expertise, and have developed a method that incorporates a numerical simulator of a model system with an expert system (see discussion under Knowledge Acquisition and Learning below.) In addition, we continued research on NEOMYCIN which has a component for using a flexible, rich representation of control knowledge to facilitate modeling of problem solving at the strategic level as well as at the tactical level. [See KSL technical reports KSL-87-58 and KSL-87-62.1 2.2 - Blackboard Architectures and Control How can we design flexibie control structures for powerful problem solving programs? How can we use these structures effectively in many problem domains? How can we represent processes and reason about their behavior, and perform intelligent actions under real-time requirements? We have continued to develop the BBl blackboard architecture for systems that reason about -- control, explain, and learn about -- their own actions. In the last year, we have significantly extended the system-building support and run-time capabilities of the BBI system. These extensions include (a) declarative representation of large bodies of factual and heuristic knowledge; (b) integration of multiple reasoning skills in a single system; (c) dynamic control under real-time constraints. We have also implemented the following application-independent components: (a) declarative representation for device structure, function, faults, and repairs; (b) reasoning modules for associative and model-based diagnosis: and (c) an asynchronous communications interface. During the past year, we began work on an advisory system, called BB-ICU (see also the separate collaborative project report on Page 129) to support patient monitoring in a surgical intensive-care unit (SICU). Briefly, intensive care patients are critically ill individuals who require life-support devices, such as respirators or dialysis machines, to perform some of their vital functions. During their stay in the intensive care unit, patients are monitored closely and gradually weaned from life- support devices in coordination with their changing physiological status and other therapeutic interventions. We began by visiting the SICU at the Palo Alto VAMC to observe monitoring procedures and operations. We worked with Dr. Adam Seiver to enumerate and characterize component intensive-care monitoring tasks and to delineate the space of relevant knowledge. We developed an ontology and representation scheme for important categories of knowledge (e.g., anatomy, physiology, pathology, therapy) and assessed our approach by implementing a small amount of knowledge in each category. We then enumerated key architectural requirements for BB-ICU and identified those not met in existing Al architectures. E. H. Shortliffe 28 5P41 -RR00785-15 Details of Technical Progress During the fall of 1987, we elaborated our initial ideas in the context of more focused design and implementation activities. Exploiting and extending our BBl architecture [KSL-84-16, KSL-88-221, we developed: (a) an asynchronous I/O subsystem to provide integrated and asynchronous perception, action, and cognition; (b) an intelligent I/O mediator to translate, interpret, and filter low-level data on behalf of the application system; and (c) an I/O manager to coordinate the mediator's activities with an application system's dynamic attentional focus. Working within our BB* knowledge representation environment [KSL 86-381, we implemented representations of the anatomy, physiology, and pathology of the respiratory system as instances of corresponding elements of a generic flow system. We developed reasoning components for continuous data interpretation and associative diagnosis of observed symptoms. We also developed reasoning components that instantiate generic models, such as the flow system model, to explain the causal relations underlying associative diagnoses or to hypothesize plausible diagnoses in the absence of associative knowledge. We demonstrated the application of the reasoning components and respiratory knowledge to interpret, diagnose, and explain respiratory data of the sort continuously monitored in the intensive-care unit. BB-ICU Demo-l comprises independently implemented versions of each of these system components. BB-ICU Demo-2, which we completed in April, integrates these system components. Some of this work is reported in recent technical reports [KSL 87-31, KSL 87-67, KSL 88-20, KSL 88-22). Other reports are in preparation. In addition, we have given talks describing this work at: .the Carnegie-Mellon University Symposium on Architectures for intelligence, Boeing Computer Services in Seattle, Wa., Advanced Decision Systems in Mountain View, Ca., Austin, TX. and the DARPA Planning Workshop in 2.3 - Advanced Architectures Many applications, such as process planning and control, maintenance, troubleshooting, environmental control, and crisis management require knowledge- based systems that can cope with large amounts of data and that produce responses in real-time. The current hardware and software architectures for knowledge-based systems cannot support such requirements. The most promising approach for achieving orders of magnitude improvement in the quantitative performance of knowledge-based systems is by exploiting concurrency on multiprocessor systems. Based on near-term projections for integrated circuit technologies, it is clear that highly parallel multiprocessor computers consisting of 100's to 1000's of processors and realizing a variety of concurrent architectures can be built. The major issue is whether such computers can be effectively used to enhance the performance of knowledge-based systems. Since 1985, the Knowledge Systems Laboratory at Stanford University has been investigating this issue. The goals and technical approach of this project, largely supported by DARPA under the Strategic Computing Program, have been discussed in previous annual reports. To summarize briefly, we seek to achieve two to three orders of magnitude speedup in the execution of knowledge-based systems, by identifying and exploiting sources of concurrency at all levels of system design: the application level, the problem solving framework level, the programming language level and the hardware systems architecture level. Due to the inherent complexity of the task and the lack of theoretical foundations for parallel computation with ill-structured problems, we have taken an empirical approach. During the first phase of the project, which was concluded in July 1987, we made specific choices at each of the system levels, i.e. taken a "vertical slice" through the design space, and have conducted several experiments to investigate the effects of a wide variety of parameters on performance. 29 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 Our research methodology is: . Select specific knowledge-based system applications, primarily signal understanding applications. . Encode these applications following various proposed concurrent software models. . Evaluate the qualitative and quantitative performance of the applications running on simulated multiprocessor machines with respect to varying hardware parameters, for example, number of processors and communication protocols, and varying software organizations, for example, degree of control centralization. In the following discussion, we present the major components of our project, and for each component we describe its current status. 2.3.1 - SIMPLE/CARE Multiprocessor Simulation System Simulation of systems at an architectural level can offer an effective way to study critical design choices if (1) the performance of the simulator is adequate to examine designs executing significant code bodies -- not just toy problems or small application fragments, (2) the details of the simulation include the critical details of the design, (3) the view of the design presented by the simulator instrumentation leads to useful insights on potential problems with the design, and (4) there is enough flexibility in the simulation system so that the asking of unplanned questions is not suppressed by the weight of the mechanics involved in making changes either in the design or its measurement. SIMPLE/CARE is a simulation system which satisfies these requirements. It forms the foundation for our empirical investigations of software architectures and hardware system architectures for concurrent knowledge-based systems. SIMPLE is a CAD (Computer Aided Design) system for hierarchical, multiple level specification of computer architectures and includes an associated mixed-mode, event-based simulator. CARE is a parameterized, multiprocessor array emulation specified in SIMPLE's specification languages and running on SIMPLE's simulator. Our simulation system is in use by several research groups at Stanford, and it has been ported to several external sites including NASA Ames Research Center. A tutorial was held in January, attended by representatives from the DOD, NASA and Boeing, which described the CARE/SIMPLE system, as well as the LAMINA programming interface (see below). The attendees received instruction in use of the system for making measurements of the performance of various simulated multiprocessor applications. The SIMPLE/CARE system is currently implemented in ZetaLisp and executes on Texas Instrument Explorer and Symbolics 3600-class Lisp workstations. We have recently started a reimplementation of the system in Common Lisp. One of our proposed objectives during the coming year is to complete this reimplementation. A Common Lisp version of SIMPLE/CARE will make it portable to a wide variety of computer systems including Sun and MicroVAX workstations. This development will necessarily be an ongoing task as Common Lisp standards, in particular, window standards, evolve and as the inevitable commercial reinterpretations of standards emerge. The SIMPLE design specification system has design operators for automatically generating array type multiprocessor architectures from a "unit cell" specification. There is considerable interest, both at Stanford and elsewhere, in using the system E. H. Shortliffe 30 5P41 -RR007851 5 Details of Technical Progress to specify and simulate other types of multiprocessor architectures. A second proposed objective is to augment SIMPLE's design operators with recursive operators for the generation of architectures using, for example, hierarchical busses or recursive interconnection nets such as Omega nets. 2.3.2 - LAMINA Programming Interface LAMINA provides extensions to Lisp for studying expressed concurrency in functional programming, object oriented, and shared variable models of concurrent computation. The implementation of the support for all three computational models is based on the common notion of a stream, a data type which can be used to express pipelined operations by representing the promise of a (potentially infinite) sequence of values. LAMINA also provides system support for the management of software pipelines and dynamic structure creation, relocation, and reclamation in a multiprocessor, multi- address-space system. Algorithms and applications written in LAMINA may be run on the SIMPLE/CARE simulation system in order to study their execution on alternative multiprocessor architectures. The development of LAMINA was essentially completed during the past year, and the software is now reasonable stable. In order to make LAMINA available in the community, we intend to port it to Common Lisp. We also expect that the application research will motivate various extensions to the LAMINA programming interface. 2.3.3 - Poligon Problem Solving Framework Poligon [KSL 86-19, KSL 88-041 is a framework for the development of Blackboard- like applications on a (simulated) multiprocessor. It consists of: . A compiler, which compiles a high-level description of the Blackboard's structure and the Knowledge to be applied by the system, to run on a distributed memory multiprocessor. . A run-time system which provides a debugging and testing environment for Poligon programs as well as run-time support. Both the compiler and the run-time system are thoroughly integrated with the program development environment of TI Lisp machines, the machine on which the execution of Poligon programs are simulated. Serial Blackboard Systems are implemented with the Nodes being represented as records on the Blackboard. The Knowledge is encoded in Knowledge Sources. These are typically compiled into procedures which are invoked by the Blackboard System's kernel. There is some form of scheduler for the Knowledge, which invokes one Knowledge Source after another. The Blackboard and the Knowledge Base both share the same address space, though they are functionally distinct. Knowledge Sources are "invoked" (executed) as a result of changes in the Blackboard placing that change event in a queue used by the scheduler. The scheduler repeatedly picks a Knowledge Source which is interested in the type of event at the end of the queue. Experiments with Poligon are by no means complete, but we have learned a number of lessons thus far. Some of these lessons are enumerated below. 31 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 . it is very hard to write any program which implements either a framework, such as Poligon or an application such as those which have been mounted on Poligon. This is due largely to asynchronous side effects. A system with better formal properties would be less error prone in this respect but might well make much less efficient use of the hardware. These difficulties could also be caused by an insufficiency of mechanisms to control coherency in Poligon. . In order to produce a reliable program it is necessary to write code which makes no assumptions about anything that any other part of the system might be doing. Failure to do so results in brittle systems. . In order to achieve a coherent solution it was found to be necessary to develop a number of programming methodologies. For example, the creation of blackboard Nodes is tricky. Because each element is likely to represent some real-world object, it is important either to provide a mechanism for resolving the conflict caused by multiple asynchronous requests to create an element that represents the same thing or to provide a mechanism for managing the creation of Nodes. Poligon opts for the latter approach. 2.3.4 - CAGE Problem Solving Framework CAGE [KSL 86-41, KSL 88-021 is a framework for building and executing applications as a cor.current blackboard system. CAGE is based on the AGE [KSL 80-291 serial blackboard framework. It includes mechanisms for the concurrent execution of knowledge sources, rules and parts of rules. The CAGE user has complete control over which of these mechanisms are used. CAGE is designed to execute on a shared-memory, multiprocessor system with tens to hundreds of processors. It is implemented using Qlisp, a concurrent dialect of Lisp designed for multiprocessors with a single, shared address space. CAGE currently executes on a shared-memory variant of CARE simulated using the SIMPLE simulation system. We are nearing completion of a series of end-to-end experiments for evaluating the utility and performance of the CAGE concurrent blackboard framework. During the coming year we intend to complete these experiments and disseminate the results. 2.3.5 - CAGE, Poligon and LAMINA Comparative Experiments During the past two years we have been developing application software and machine architecture models to supp.ort a series of end-to-end experiments comparing various concurrent programming systems for knowledge-based applications. The goals of these experiments are to: . Obtain quantitative comparisons of the performance of the programming systems. . Gain insights into how different concurrent programming models lead to different (or similar) application decomposition and organization. . Force the refinement of the concurrent programming systems so as to better support application development. . Gain insights into the ease or difficulty of writing application code in each of the programming systems. E. H. Shortliffe 32 5P41 -RR00785- 15 Details of Technical Progress The common application for these experiments is Elint [KSL SS-691, a real-time, knowledge-based system for integrating pre-processed, passively acquired radar emissions from aircraft. This Elint application has been implemented in three different concurrent programming systems: LAMINA, Poligon and CAGE. Each of the implemented applications are executed and evaluated using various input data sets and varying numbers of processors. Application code written in either LAMINA or Poligon compiles to code which executes on the CARE architecture. CAGE, however, is targeted toward a single address space, shared variable multiprocessor architecture. CAGE is implemented in QLisp, a concurrent Lisp for shared variable multiprocessors. To support CAGE we had to develop a multiprocessor "blackboard machine" variant of CARE. This blackboard machine models a shared variable architecture and includes the mechanisms and instruments necessary to manage and study memory contention. The architecture implements the blackboard and the control data structures in global, shared memory. It directly supports the CAGE system and application code written in QLisp. During the past year we have: . Completed the implementation of the the Elint application in each of the three concurrent programming systems. . Completed the development of the blackboard machine variant of CARE. . Developed an experiment plan fdr the comparative studies. . Developed a new measure of speedup as a function of the number of processors in a multiprocessor system. This measure is useful for evaluating system performance of real time applications and is based on the concept of maximum sustainable input data rate. . Completed the first set of experiments for each of the three programming systems. 2.3.6 - The AIRTRAC Application AIRTRAC [KSL 86-201 is the primary application driving our development of concurrent knowledge-based system programming methodologies. Also, it is one of the basic applications used for our multiprocessor architecture performance experiments. AIRTRAC is a knowledge-based signal interpretation and information fusion system. The system attempts. to identify, track, and predict the future behavior of aircraft. In particular, it attempts to recognize aircraft which might be engaged in covert activity, for example, smuggling. The inputs to AIRTRAC are periodic radar tracking system reports, a priori, filed flight plans for some aircraft, and occasional intelligence reports about suspected covert activity. AIRTRAC is designed to be sufficiently complex and realistic to adequately test various ideas about concurrent problem solving on multiprocessor machine architectures. The AIRTRAC application involves continuous input data streams, typical of real-time signal interpretation problems. Such problems often require a level of computational power two to three orders of magnitude beyond what is currently available. Moreover, the application uses data-driven, expectation-driven and model-driven styles of reasoning. These reasoning styles encompass a wide range of paradigms in artificial intelligence. 33 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 The AIRTRAC Data Association Module and associated experiments were completed as of summer, 1987 [KSL 87-34-j. The experiments were performed using the SIMPLE/CARE multiprocessor simulation system. They demonstrated that almost linear speedup as a function of the number of processors can be achieved (at least up to 100 processors) for a periodic data-driven knowledge-based system such as the Data Association Module. During the past year, the design and knowledge acquisition for the Path Association Module was completed. Over one half of the LAMINA code for this module has been implemented and debugged. The completed AIRTRAC application will provide an end-to-end example of a concurrent, knowledge-based signal interpretation system. It will demonstrate the benefits and costs of implementing and executing such systems on multiprocessor architectures. Also, the application is sufficiently complex that it will serve as important test case for evaluating multiprocessor architectures for knowledge-based systems and "tuning" the engineering parameters for such systems. 2.4 - Knowledge Acquisition and Machine Learning Our research in machine learning has focused on several distinct problem domains including medical (NEOMYClN/HERACLES), physics (ABLE), and biochemical (PROTEAN) in addition to domain-independent investigations. We also are motivated by the need for effective tools for knowledge- acquisition and maintenance of knowledge bases (IMPULSE and STROBE for FRM, BBEDIT, KSEDIT with BBl). 2.4.1 - Learning by Chunking Chunking is a learning mechanism that acquires rules from goal-based experience. SOAR is a general problem-solving architecture with a rule-based memory that can use the learning capabilities of chunking for the acquisition and use of macro- operators. Rosenbloom et al. are investigating chunking in SOAR and find that chunking obtains extra scope and generality from its intimate connection with the sophisticated problem solver (SOAR) and the memory organization of the production system. Another emphasis in SOAR is Explanation-Based Learning, a powerful technique that generalizes concepts learned from examples. In this past year, SOAR has demonstrated acquisition of diagnostic problem solving knowledge (similar to that in NEOMYCIN), learning from multiple examples and from analogy, and learning attribute-value information to acquire features of a single object incrementally, reusing known objects as values of attributes. The work on SOAR is continuing under Dr. Paul Rosenbloom at ISI in Los Angeles. 2.4.2 - Inductive Rule Learning In previous reports, we discussed the work of Buchanan, et al. on incremental learning process from examples with the rule-learning system RL (described in the 1986 SUMEX report). This work has continued, and has been applied to the domain of linear accelerator physics (see below). Results from the RL research indicate that intelligent selection of instances based upon knowledge of the state of the evolving theory results in a faster convergence of an evolving theory toward the target concept, requiring many fewer cases for learning. In the paper Simulation-Assisted Inductive Learning, to be presented at the 1988 AAAI conference in Minnesota, Buchanan, Clearwater, et al. describe the work done as a collaborative effort between the Rule Learner project and the Automated Beam E. H. Shortliffe 34 5P41 -RR00785-15 Details of Technical Progress Line Experiment project. The focus of this work is to show how RL can be used in a real-world domain with sparse data by working with a simulator which numerically models the domain. The domain is the classification and location of faults in a particle accelerator. This study demonstrates the effectiveness of RL even in this noisy, numerical domain. Some problems in generating the best examples from which to learn rules and how to learn the best rules from a given set of examples are explored. In addition, methods of weighing the evidence when several rules fire are being investigated. 2.4.3 - Learning Apprentice In addition, we continued several investigations of methods for bui!ding knowledge bases for knowledge-based programs. Knowledge engineering remains the "standard" method of building a knowledge base for commercial systems, so we have investigated ways of making that more efficient. Technical reports KSL 87-58 and KSL 87-62 describe some of the results, pertaining mostly to the principles of starting with a sound problem solving strategy and of exploratory programming. Reports KSL 87-60 and KSL 87-67 describe work on learning apprentice methods. In last year's SUMEX report, we reported results on the ODYSSEUS apprenticeship learning program, described by Wilkins in KSL 86-63, which is designed to refine and debug knowledge bases for the HERACLES expert system shell. ODYSSEUS analyzes the behavior of a human specialist using two underlying domain theories, a strategy theory for the problem solving method (heuristic classification), and an inductive theory based on past problem solving sessions. ODYSSEUS improves the knowledge base for the expert system shell, identifying bugs in the system's knowledge in the process of following the line-of-reasoning of an expert, serving as a knowledge acquisition subsystem. The system can also be used as part of an intelligent tutor, identifying problems in a novice's understanding and serving as student modeler for tutoring systems. Wilkins, et al. illustrate that an explicit representation of the problem solving method and underlying theories of the problem domain provide a powerful basis for automating learning for expert system shells [KSL 86-621. In the last year we began the creation of a case library from medical records for the NEOMYCIN domain. This library is essential for apprenticeship learning in ODYSSEUS in three ways. First, experiments have shown that the existing knowledge bases are too impoverished to follow the reasoning of an expert, and a case library will allow an induction system to automatically expand the domain knowledge to overcome this limitation. Second, when the learning system fails to explain an action of a student or expert correctly, the critic component of the system generates thousands of conjectures. To filter these requires a case library. Finally, the case library will allow us to demonstrate that apprenticeship learning can improve the performance of an expert system. 2.5 - Pragmatic Approaches to Reasoning Under Uncertainty The goal of this project is to investigate pragmatic approaches to computer-based probabilistic reasoning systems. In the past, artificial intelligence researchers have often avoided probability theory for reasoning with uncertainty because of the perception that the application of probability is invariably associated with a commitment to intractable algorithms and an inordinate amount of knowledge acquisition time. The development of efficient probability inference and assessment techniques will allow investigators to apply a theoretically justified theory of belief entailment to complex problems. Specifically, this project seeks to (1) develop 35 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 techniques for using knowledge about problem-solving tradeoffs to dynamicaliy optimize the value of computer performance to the user, (2) construct efficient algorithms for probabilistic reasoning, and (3) investigate pragmatic techniques fat the elicitation of knowledge from experts. 2.5.1 - Reasoning about inference tradeoffs Research on reasoning tradeoffs has focused on the time vs. quality-of-result tradeoff in several domains, Additionally, tradeoffs arising with the representation of knowledge and explanation of inference within a probabilistic framework have been identified [KSL 88-131 The growing perspective of this research is that problems traditionally ascribed to knowledge representation, inference, and explanation in probability-based reasoning systems have been encountered because of insufficient attention given to tradeoffs under bounded or varying resources available for engineering, computation, or cognition [KSL 88-13,87-281. Notable research on inference tradeoffs during this past year has been the implementation of a prototype strategic reasoner for the analysis of tradeoffs. Fundamental issues of control under varying resource limitations were explored with simple sorting algorithms [KSL 88-31. Some of this work will be reported in an article appearing in the AAAI conference this summer. Other work on inference tradeoffs has focused on applying a similar computational architecture to control the selection of alternative belief-network inference strategies [KSL 87-641. Work has been carried out on the implementation and characterization of useful classes of probabilistic approximation algorithms that contain explicit tradeoffs. There has been special interest in the development of flexible strategies for reasoning under uncertainty with varying resource limitations. Recent work along these lines has focused on the development of modified versions of a probabilistic inference technique developed by Pearl [KSL 88-271. The new approach allows the performance of the algorithm to be "gracefully degraded" under resource limitations, through pruning the consideration of terms in accordance with their expected effect on the final answer. Preliminary empirical analysis of the time/accuracy tradeoff for this algorithm has been carried out on a multiply-connected belief network. 25.2 - Efficient probabilistic inference algorithms During the past year, two of our primary goals have been to implement several known probabilistic inference algorithms and begin to test their efficiency. In particular, we have implemented Pearl's algorithm for multiply-connected belief networks and in the process we have gained important insights into the nature of the algorithm [KSL 88-271. These insights have allowed us to make design choices that yield an efficient implementation the algorithm. In particular, we have designed and developed a fast method for finding and using a cutset of nodes for inference in networks that contain complex loops. We also are working to improve the efficiency of updating belief networks that do not contain loops; our current algorithm is able to update a singly-connected network of propositional variables at a rate of approximately 50 nodes per second on a Macintosh II In Turbo Pascal. Our implementation of the Pearl algorithm has been successfully tested using a set of benchmarks of varying complexity that we have developed. These benchmarks soon will be used to test comparatively the computational time complexity of several additional inference algorithms. We also have begun a collaboration with a researchers at the University of Aalborg in Denmark who are leading an effort in Europe to develop expert systems based on belief networks. We have already E. H. Shortliffe 5P41 -RR00785-15 Details of Technical Progress shared some of our benchmarks with them. In the coming months we anticipate increased collaboration that will include comparison of benchmark timing results and the exchange of algorithms. The Pearl algorithm that we have implemented will serve as one type of exact algorithm. In addition, we are currently implementing a probabilistic inference algorithm by Lauritzen and Spiegelhalter. The Pearl and LauritzenISpiegelhalter algorithms will be our initial set of exact algorithms. We also plan to study approximation and heuristic techniques for probabilistic inference. This is particularly important in light of our proof that exact probabilistic inference using belief networks is NP-hard [KSL 87-271. We previously implemented a stochastic algorithm and will use this as our initial approximation algorithm. An neural-network algorithm will be used as an initial heuristic algorithm. We anticipate that by the end of the summer we can begin to compare these algorithms on sets of theoretical and real inference problems. These tests will provide important information for designing new algorithms in the coming year. Several other researchers in the Medical Computer Science Group are currently using our inference algorithms to develop expert systems based on belief networks. Systems that presently are being developed include 1) an intelligent anesthesia monitor, 2) a diagnostic system for the Intensive Care Unit, and 3) a system that assists in evaluating the statistical validity of a clinical drug trial report. These applications have given us practical feedback about the level of inference performance that is necessary in real domains. Although these applications are still in the early stages of development, they suggest that improved inference efficiency will be a critical issue in producing practical expert systems that are based on belief networks. During the past year we have developed a general knowledge engineering environment called KNET (Knowledge NETwork) on a Macintosh II in MPW Pascal [see Chavez article]. KNET is a flexible graphical interface system for entering a belief network [see Lehmann article] and running cases using a belief network. It provides a general software system foundation from which to experiment with different methods for pragmatic probabilistic reasoning. For example, a key feature is that KNET provides a modular environment in which different inference techniques can be tested. We recently were able quickly to install Pearl's inference algorithm into KNET. We soon plan to have several other inference algorithms running in KNET. Although the emphasis in the early stages of this work has been on developing an initial set of algorithms, we have also designed several new techniques. One technique uses dynamic programming to solve efficiently a large class of complex probabilistic inference problems. We have not yet implemented and tested this algorithm. However, it appears that for some types of complex belief network topologies this algorithm will be very fast relative to current techniques. Another method that was recently developed in our group allows any belief network algorithm to be used to solve decision problems (Le., influence diagram problems) [KSL 88-281. This general method significantly broadens the scope of application of our work on belief network algorithms. Of particular interest, it allows currently available exact, approximation, and heuristic belief network algorithms to be easily adapted to solve decision making problems. 37 E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 2.5.3 - Probability assessment During the past year, the majority of the research outlined for the reasoning-by-two assessment method, now called similarity networks, was completed. We began by implementing the similarity network approach in Turbo Pascal on the Macintosh II. We then evaluated the knowledge assessment tool by using the program to build a small component of the Pathfinder knowledge base, a module that helps a novice pathologist classify spherical structures seen in a lymph-node tissue section. Using the program, our expert was quickly able to identify the morphologic features relevant to the classification task and was able to specify the dependencies among these features. Upon using the program for probability assessment, it became clear that a generalization of the similarity network would be useful for reducing the number of assessments required. In particular, it became clear that arbitrary sets of hypotheses should be allowed to be clustered together and labeled as "similar." A pen and pencil approach to eliciting probabilities in this manner was developed and used to assess the probabilities required for the entire lymph-node pathology domain. In assessing the probabilities, it was assumed that all observations are conditionally independent on each hypothesis. The approach was quite successful as it reduced the number of probability assessments required of the expert by a factor of approximately twenty (from 30,000 assessments to roughly 1,500). Finally, the performance of the knowledge base constructed in this manner was evaluated [KSL 88-381. In doing so, a new evaluation metric based in decision- theory was developed. The results of the evaluation demonstrated that the performance of the knowledge base was close to that of the expert. However, the results also showed that there is still room for improving the knowledge base through the representation of dependencies among features. 2.5.4 - Collaborations During the past year we have continued to maintain contact with a number of Stanford faculty who are interested in the research goals of this grant. We have also collaborated with visitors from outside of Stanford. In particular, Prof. Max Henrion visited our group during the Autumn quarter. We continue to communicate frequently with him about our common research interests. We also have benefited from visits by Dr. David Spiegelhalter (a statistician from the Medical Research Council in England), Dr. Stig Andersen (a computer scientist from Aalborg University in Denmark), and Prof. Ross Shachter (a Stanford faculty member on sabbatical at Duke University). Most of the members of our group were able to attend the recent conference on Influence Diagrams in Berkeley. We have submitted several papers to the AAAI Uncertainty Workshop this year and most of us plan to attend this workshop in August. E. H. Shortliffe 38 5P41 -RR00785- 15 Details of Technical Progress lll.A.2.4. Core System Development 1 - Introduction In this section we describe progress on our core system development and work toward a distributed AIM community. In last year's report, we discussed the motivations and plans for our core system development work along four dimensions: 1) the motivation for the shift of the SUMEX-AIM community from a central mainframe-based computing resource model to a largely distributed workstation- based model; 2) the prospects for workstation technology and vendor support for a diverse distributed AIM community; 3) the projected core SUMEX-AIM systems tasks needed to complement vendor developments to realize distributed community operation: and 4) the integration, dissemination, and management of the shift of the AIM community from a centralized to a more distributed operation, including the remaining central resource functions. These were expanded still further at a site visit held in August 1987 in response to our request to the National Advisory Research Resources Council to restore the final 2 years of our grant award. Following a special study section review and reconsideration by the Council, our plan and the full 5-year grant award were approved. The review group's concluding recommendations included the following guideiines: "Consistent with its charter as a national resource, SUMEX should focus its systems activities on producing a distributed medical research environment that can be easily reproduced at other sites. It should also continue to play the important role it plays today as a repository of systems information and expertise for the medical Al research community, as well as the larger computer science community. However, it should avoid trying to be all things to all people and should focus its attention on a small number of standardized hardware and software configurations. A strong effort should be made to acquire information about related systems activities at other sites and to avoid duplication of effort. These guidelines should be used to establish priorities among the proposed set of system activities and to apply effort appropriately." The review committee's recommendation was very much in line with our own goals to more sharply focus our development resources and much of our effort over the past year has been devoted to that end. Since our 1980 renewal proposal in which our move to distributed workstation technologies began, we had taken on the development and support of a wide array of systems including mainframes (DEC 2060 and 2020) network servers (2 DEC VAX 1 l/750 UNIX file servers, a SUN 31180 UNIX file server, a Xerox file server, 7 network laser printers, Ethernet gateways and TIP's, and ARPANET and TELENET wide-area access systems), and workstations (50 Xerox D-machines, 20 TI Explorers, 6 Symbolics machines, 5 Hewlett-Packard 9836 workstations, and 3 SUN 3175 workstations). Whereas we cannot drop support for these systems irresponsibly, we resolved to pick a much more limited environment on which to focus our long-term systems development efforts and to phase out support for the other systems as quickly as possible. In summary, we have chosen Apple Macintosh II workstations as the general computing environment for researchers and staff, TI Explorer Lisp machines (including the microExplorer Macintosh coprocessor) as the near-term high-performance Lisp research environment, and a SUN-4 as the central system network server (wide- and local-area network interfaces, file services, printing services, etc.). 39 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 2 - Distributed System Evaluation and Selection 2.7 - Design Goals In planning for AIM community computing needs for the next few years to replace and upgrade the powerful and easy-to-use general computing tools and network services of the 2060, several goals were identified: . The work environment should be modern and combine graphics, pointing, and traditional keyboard modalities of interaction, as it is expected to be the primary work environment for some years to come. . The system should support the most powerful Al research and Lisp development environment available today, possibly involving special-purpose hardware. . The system should support small-to-medium-size Al and Lisp-based research work without requiring special hardware. . The cost per person should be low enough as to permit putting a machine on or near every desk and to consider the system as a potential Al delivery environment. . The system should integrate well into a heterogeneous computing environment typical of AIM research work. . The system should be capable of editing, organizing, and printing large documents, such as theses and books. . The system should be capable of generating and editing state-of the art graphics. . The design should be incrementally extendable and augmentable as new hardware and software technologies appear and as the number of users fluctuates. . The design should be simple enough as to refocus our systems work on a smaller number of machines and cost-effective enough as to be replicable at smaller AIM sites that wish to benefit from our experience. . The design should permit easy data sharing and exchange with collaborators at other sites and within Stanford University. In addition to user-related computing tools, other heavily-used network services traditionally provided by the shared 2060, must be replaced. These include wide- area network access (ARPANET and TELENET), electronic mail (transmission/routing, reception, and user access), community bboards, file service, and print spooling. 2.2 - Evaluation Results We examined many potential configurations before deciding on the solution involving Mac II's, microExplorer's, and the SUN-4 . Many of the considerations were technical in terms of the tools and services provided by the systems and many had to do with user preferences for interface style and environment. Timesharing machines were eliminated for their lack of modern interactive productivity tools. (The many reasons for the trend from timesharing to workstations E. H. Shortliffe 40 5P41 -RR00785- 15 Details of Technical Progress have been discussed in previous annual reports. The pressures behind this trend have grown stronger with time.) Xerox lisp machines were eliminated by virtue of their uniqueness and the questionable future of the hardware product line. Stand-alone TI (and other similar) Lisp machines were eliminated by virtue of their uniqueness and high cost and lack of general computing tools for mail, document preparation, etc. Sun workstations were eliminated by virtue of their relatively higher cost and engineering orientation and their dependence on the UNIX user interface which received almost uniformly negative comment in a KSL user questionnaire about computing environment preferences. IBM PC's were eliminated because of current limitations in their primitive operating system, window system, and interface style, when compared with the Macintosh. Of course, this kind of evaluation is very complex and the above reflects only a summary of key issues. There are many reasons for or against any of the above machines not fully enumerated here. In the end, we chose the Apple Macintosh II primarily because of the following considerations: . the Mac has a powerful, intuitive, and consistently applied icon-based user interface that facilitates wide use and effectiveness. o The Mac II is a powerful machine (Motorola 68020-based) with an open architecture that provides long-term configuration flexibility (e.g., for color, coprocessing units like the microExplorer, memory, i/o devices, etc.). . The Macintosh is popular and is used by a growing number of our collaborators. Many members of the AIM community specifically endorsed the Macintosh as their machine of choice and many already have Macintoshes at home. . Apple gives educational institutions a substantial discount. This and the low price of third-party disks meant we could put a Mac on almost every desk. . There is a large variety of third-party hardware and software available. Competition and volume mean lower prices; especially when compared to Sun or DEC third-party offerings. . Texas Instruments announced the microExplorer, a board-level product which gives the user a custom VLSI lisp machine inside his Mac, for a fraction of the cost of Tl's stand-alone Explorer workstation product. To replace the network service functions of the SUMEX-AIM 2060, we chose a Sun 41280 system. We investigated competitive systems, for example, the DEC VAX and Pyramid Technology 9000 series product line. Configurations are available from 3.5 to 25 MIPS and with individual I/O channel speeds up to 11 MBytes/Set. However, we decided that the SUN-4 was more cost effective, had more popularity at Stanford and at other universities, and offered outstanding research network service software. This configuration is at the beginning of its product cycle and can be expected to serve for many years to come. The SUN 41280 approach was made even more attractive by the opportunity to have it equipped with state-of-the-art 900 megabyte disks. This option was presented to us after a review of second-source disks indicated that SUN's then offered Fujitsu Double Eagle units (575 megabytes) were not the optimum in cost-effectiveness. 41 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 2.3 - Configuration The working plan we eventually settled on called for a Sun 4/280, 69 Macintosh II workstations, 10 TI microExplorer upgrades, 7 Kinetics FastPath gateways, and a combination of upgrade packages yielding 2 20 page/minute PostScript laser printers. The Sun 4/280 is configured with 32 megabytes of memory and 1.8 gigabytes of disk space. By choosing this package we were able to purchase the system for 40% off list price. An ARPANET interface for the Sun server is available and will be purchased in the near future. This will make the new server more readily available to AIM users outside Stanford. This server is currently up and undergoing test and configuration. By a special arrangement, we purchased 66 Macintosh II computers directly from Apple, each with 1 MB of memory (no disk, no display). At this writing, almost all of the the Macintoshes have been installed. Three units have proven defective and are being repaired under warranty. A package of 10 microExplorer upgrades was ordered from Texas Instruments. They arrived during the preparation of this report and will be installed shortly. (The features of the microExplorer are described in the Explorers section of this report.) 10 100 MB disks were purchased from Rodime and installed in the Macintoshes receiving the microExplorer upgrades. (A large paging disk is required by the microExplorer's use of virtual memory.) The balance of the Macintoshes were outfitted with 20 MB disks, also purchased from Rodime. The choice of Rodime disks was suggested by their low price (30% less than Apple's higher-education discount price) and long warranty (12 months). All disks have been installed and so far none have failed. Small disks were deemed sufficient, as users are encouraged to keep their files on the Sun file server (using the AppleShare filing protocol) for reasons of data backup and security. Our experience with large displays on other workstations suggested that we wanted the largest displays the market could offer at a reasonable price. We chose 33 Moniterm 19" displays and 33 Moniterm 24" displays, which we were able to obtain in a package at 40% off list price. Our experience with other third-party Macintosh displays told us that a resolution of no greater than 72 dots/inch is easiest on the eyes. Both of the models we purchased conform to this resolution. Also from our previous experience with Macintoshes, we knew that many applications require more than 1 MB of memory. In our initial purchase, we specified 12 4 MB memory upgrades. These were installed in 12 Macintoshes used primarily by staff and student developers of Mac software. The original 1 MB of memory was removed from each of these machines and added to 12 other machines, making a total of 2 MB in each of those machines. (10 of the 12 were the microExplorers.) We have already concluded that our memory demands require that we do the same with the remaining 1 MB Mac's Apple's memory orders are now backlogged 5 months, so we have ordered from National Semiconductor instead. To network the majority of the Macintoshes in the near term, we chose the Farallon Phone/Vet system which enabled us to reuse terminal wiring we had previously installed in all offices and student areas. In addition to this reason, we chose PhoneNet over Apple's LocalTalk wiring system because PhoneNet permits nets 3-4 times larger (by reason of different shielding and impedance characteristics). To connect the PhoneNet networks to the SUMEX-AIM Ethernet, we chose to install 7 Kinetics FastPath gateways. The FastPath is a commercial spin off resulting from E. H. Shortliffe 42 5P41 -RR00785-15 Details of Technical Progress the SUMEX work on the SEAGATE gateway. Owing to an earlier royalty payment agreement with Kinetics, we were able to procure the FastPath gateways at no cost. The number of gateways was chosen primarily because of the limited throughput characteristics of PhoneNet (230.4 Kb), but also to provide for hardware redundancy. To provide printing services for this number of Macintoshes, it was necessary to procure additional PostScript printers. Although Apple offers a substantial discount on its LaserWriter products to its higher-education customers, the 8 page per minute maximum speed (typically less) was deemed too low for our demand printing needs. (Other complaints about the design of the LaserWriter concerned the small paper tray and toner capacity.) As noted in the Printing Services section of this report, we obtained a no-cost PostScript upgrade for our 20 page per minute lmagen 3320 printer in consideration of our having beta tested the upgrade product. Our experience with the basic 3320 product over the past year has been positive. Our positive reaction to the PostScript upgrade convinced us that duplicating the configuration in our other offices was a good idea. We were able to do this by upgrading an lmagen 12/300 to a 3320 PostScript product at a 30% discount. The 3320 is appealing mostly for its print quality, speed, and minimal maintenance requirements. It holds a ream of paper and can print on 11" x 17" paper. 2.4 - More Details about the Transition Plan Having selected the Macintosh and microExplorer systems for our work, many additional decisions remained to select, configure, and integrate the routine computing environments for our users. The following summarizes this work. Text Processing - Editing There are many criteria for a system text editor including: . Easy to learn . Available from various contexts so that similar techniques can be used in editing mail, reports, and code. . Powerful manipulation facilities allowing structures such as words, lines, paragraphs, pages, expressions, code blocks, etc. to be selected, moved, copied, reformatted, transposed, etc. . Interchange ability allowing at least plain text to be imported from other systems and exported back to them. o Extensibility in the form of keystroke macros and, ideally, customization libraries allowing us to write packages that make the editor "understand" a new kind of document structure. Most of the commercially available Macintosh editors are targeted to desk-top publishing and so are fairly easy to use, but have manipulation facilities only for words, lines, paragraphs, sections, and pages. They are sadly lacking in understanding of other types of document structures such as programming languages or electronic messages. These editors typically offer interchange of "plain text" ASCII-only documents. They uniformly offer negligible extensibility. Of the systems we've looked at, Microsoft Word has proven most useful of those currently on the market, so we are using it in the meantime. Early demonstrations and tests of FullWrite Professional (marketed by Ashton-Tate) indicate that it may be E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 superior to Word but its commercial release is just taking place. The non- commercial GnuEmacs might offer a complimentary solution as it is an outgrowth of the Emacs editor widely used in the AIM community. It offers familiarity and powerful extensibility, but it does not offer the easy-to-use interface and multi-font display expected on the Macintosh, and having two different editors would complicate matters. In the coming year we plan to: o Track new editor programs, seeking one that better meets our criteria . Talk with editor vendors to encourage the addition of the desired features . Further investigate GnuEmacs for the Macintosh II Text Processing - Aids Most of the commercial text processing (TP) packages that we`ve looked at have built-in spelling checkers, sorting, hyphenation, forms filling, etc. Text Processing - Graphics MacDraw, MacPaint, and the other Macintosh drawing programs offer state-of-the-art TP graphics capabilities. Pictures from these programs can almost always be integrated into a document being prepared with one of the commercial TP systems. Text Processing - Formatting Most of the commercial TP systems are "What You See Is What You Get (WYSIWYG) editors, giving an on-screen representation of the final document during editing. The formatting quality and style control is quite good in the better systems, but we have found that some documents still require the extremely precise control offered by TeX and so are also using it, with it's embedded commands" style interface. "compile the manuscript with Text Processing - Bibliographic References Another major shortcoming of the TP systems is the lack of automatic bibliographic reference generation and formatting. In writing scientific papers it is important to cite relevant work, and we have found it extremely useful to be able to extract the significant information from a large bibliographic database by placing a reference key where the citation should appear in the text. We are pursuing TP vendors in the hopes that they will implement this facility as well as investigating development of an auxiliary program to handle bibliography generation. Printing Macintosh printing is fairly well developed in that most programs utilize the system- defined routines to print. We have installed PostScript on lmagen printers as well as Apple LaserWriters and are in the process of bringing up a spooling system on the file server. E. H. Shortliffe 44 5P41-RR00785-15 Details of Technical Progress Help Facilities Most programs have built-in help on the Macintosh, as well as reasonably consistent interfaces, but this is not enough. We find that users are still confused so we are undertaking to produce short introductory documents to help users get started, and to point them in the right direction. We will investigate using HyperCard to organize this data. System Information As the system configuration has gelled we have begun thinking more and more about tools and protocols for getting information about the status of the overall system to the users and maintainers. We will need to address network loading, user location, resource usage (file space, printing, computing cycles), and status information for individual elements of the distributed computing environment. We also need access to personnel information, bulletin boards, and other shared databases. Interpersonal Communication See the section on electronic mail development Systems Building Tools We are developing expertise in the Macintosh Programmer's Workbench (MPW) environment, including with C and PASCAL, and HyperCard. We are also tracking Allegro Common Lisp, and Neuron Data's NExpert. Filing We plan to stay with the strategy of a few centrally located file servers, but the local disks on the Mac's complicate the system. The foremost concern is data backup. We have sketched out a design that would automatically copy new versions of documents (files) created on the Macintosh to a reliable file server (i.e., one that is backed up to tape). This backup program will allow for exclusion of some files (e.g., temporary files) and will make an effort to not have multiple copies of the same file on the server. Also of significance to users who keep files on the Mac rather than a file server is the resultant inaccessibility from other computers. The proposed backup scheme would alleviate this problem as well. A full UNIX-based file backup and archival system is under consideration for the servers. 3 - R&D Task Plan -- Update and Progress In the presentation to the site visit review team and Council, we layed out a detailed plan for our developments (see Figure 1). The following summarizes our pruning and reprioritization of those goals, based on the Council review, and progress this past year. In general, wherever we showed parallel developments to maintain capabilities among Xerox, Symbolics, TI, and other workstations, we have restricted our efforts to the Mac and Explorer environments, in accord with the Council recommendation for a focus of effort. While we continue to stay abreast of new workstation hardware and software, we have concentrated our system development work in the following areas for the Mac/Explorer environments. Progress in some areas has been limited by the reductions in systems manpower necessitated by NIH cuts in our award funding. 45 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 3.7 - Remote Workstation Access . TIP TCP-IP support: We now have TCP-IP software running in our EtherTIPs. . Workstation TCP-IP access: Each vendor has supplied the requisite software. . TELENET X.25/TCP-IP Ethernet Gateway: The DEVELCON gateway has been installed and is used by the SUMEX-AIM community for TELENET access to the DEC 2060. . TELENET TELNET access (TCP-IP): The DEVELCON gateway is a bi- directional protocol translating gateway between X.25 and TCP-IP, and thus, fulfills this requirement. 3.2 - Remote Virtual Graphics . X Common Lisp client & server: A Common Lisp X (CLX) client has been released for TI Explorers, SUN Workstations, and Symbolics. An alpha release of a CLX server is expected from TI this Summer. Since Xerox is moving its lisp environment to SUN workstations, and CLX runs on SUNS, we are not going to port CLX to Xerox D-machines. X runs under Apple UNIX (AUX) on MAC II's but is not implemented for the Apple Operating System (This latter operating system has its own graphics protocol, MacWorkstation, which we are experimenting with). . Common Window Application standard/Implementation: We have not been able to give adequate attention to this item because of staff and budgetary constraints. It is worth noting that the Common Lisp User Environment (CLUE) is a window system defined on top of CLX and is currently in use in the KSL. . Develop/Extend Virtual Graphics applications: Very little progress has been made in this area because of staff and budgetary constraints. We intend to emphasize the development of virtual graphics applications beginning this summer. 3.3 - Distributed Mail System . InterLisp mail reader/composer: This software is completed, and is now part of the Lyric TCP-full-sysout. No further Xerox work is planned. . Redesign IMAP protocol (IMAP-2): This has been completed. o 2060 IMAP- Server: This has been completed. . UNIX IMAP- Server: This has been completed, and is currently being alpha tested. o Common Lisp mail reader/composer: The TI version should be completed and in initial testing by the end of June 1988. No work is planned on a Symbolics version. There are similarly no plans to translate the Xerox InterLisp client into Xerox Common Lisp given Xerox's plans to move to SUN's, and the existence of InterLisp within the Xerox environment for the foreseeable future. E. H. Shortliffe 48 5P41 -RR00785-15 Details of Technical Progress . Update Common Lisp IMAP- Clients: This is completed. . UNIX mail client/reader/composer: This project is on the backburner because of staff limitations, and the small number of SUN clients in use at SUMEX-AIM (a Macintosh-II client is underway for the Apple operating system rather than AUX, since the former is the primary OS in use at SUMEX-AIM). We have imported a UNIX version of the 2060 mail reader/composer called MM-C which was written at Columbia University. It is not a distributed mail system in that the reader/composer and mail file are assumed to reside on the same machine. The MM-C system will be used to provide national community and home mail services on the SUN-4 until further versions of the IMAP- clients are available. . Enhance reader/composer tools: The reader/composer tools have undergone significant continual development since their release to our local user community. For example: message filtering was introduced earlier this year and one can now filter on free text searches: subject, from, to, cc and bee text searches; new recent and old messages; On, before or since a given date; messages that are Seen/Unseen, FlaggedIUnflagged, Answered/Unanswered, Deleted/Undeleted; and message keyword searches. 3.4 - General Computing Environment . Lisp and Al shell environments: Common Lisp now runs on Xerox systems using the Lyric sysout, and Lucid Common Lisp is on our SUN workstations. The Macintosh-II supports Coral Common Lisp. Finally, with the advent of the microExplorer co-processor on the Macintosh II, the entire Explorer Common Lisp environment is available on Macintosh II's configured with this board and an Ethernet interface. In addition, we have made considerable progress in analyzing the performance of Lisp systems on various kinds of hardware, with an eye toward guiding our work on future Lisp systems and the trade-off between specially microprogrammed Lisp machines and implementations on standard workstations. We have also defined the requirements for a powerful Lisp programming environment based on the key features of the Xerox, Symbolics, and TI environments that we use routinely in our Al research work. . Distributed File Support: The Xerox Common Lisp RPC/NFS implementation has been completed, and Xerox has a strong interest in acquiring this software from us and including and supporting an enhanced version as part of their standard system release. We would in turn receive all improvements to the code. Both TI and Symbolics have released an RPC/NFS implementation. Because of budgetary and staff limitations this year, we have be unable to make any progress on Network management, backup, and archiving tools. This area has been given a high priority and we will begin to work on it this summer. . Distributed information access: We have installed SUN UNIFY (a powerful relational data base system) on our SUN file server and implemented a Common Lisp remote procedure call/SQL query interface for Lisp machines to experiment with remote data base access. We have also been experimenting with the Apple HyperCard system for organizing and disseminating information in a distributed community. 47 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 . Operations management tools: We have made little progress in this area during this report period. 3.5 - Phasing of the transition to a distributed AIM community Experiments in the Stanford/AIM community: Each new piece of hardware or software has been tested initially by a selected subset of the Stanford/AIM community, e.g., members of the systems staff who are willing to put up with problematic software in the alpha test phase. It has then been beta-tested by a larger subset of the same community, and then released to any interested member of the community as a whole. A typical example is the Xerox IMAP Client, MM-D. After several months of alpha testing by two or three members of our systems staff, the software was distributed to other KSL research staff members for beta testing and suggestions for improvement. Finally, it became a part of the standard Xerox Lyric system used by the Stanford Knowledge Systems Laboratory. 4 - Remote Workstation Access, Virtual Graphics, and Windows 4.1 - Remote Access As we move towards a distributed workstation computing environment for Al research in the SUMEX-AIM community (and move away from the centralized, shared DEC 2060), a number of technical obstacles must be overcome. One of the most important is to eliminate the need for the user display to be situated close to the workstation computing engine. This is important in order to allow users to work on workstations over networks from any location -- at work, at home, or across the country. The first step has been getting reliable terminal access operational on all workstations. All workstations now have TCP/IP based terminal servers, and TCP/IP is being installed in the SUMEX network terminal concentrators. This allows primitive (non-graphical) access to the workstation's abilities. A more comprehensive access will be provided through our remote graphics work. 4.2 - Virtual Graphics In order to link the output of workstation displays across networks, it is necessary to capture and encode the many graphics operations involved so that they can be sent over a relatively low-speed network connection with the same interactive facility as if one had the display connected through the dedicated high-speed (30 Mhz) native vendor display/workstation connection. A mechanism for doing this is called a remote graphics protocol. As reported last year, we selected the MIT Project Athena X window system [4] as the remote graphics protocol standard for our work, and noted that X is a very complete protocol that has been developed over the past several years at MIT'. We also reported that an X client2 for Texas Instruments Explorers was being written here at SUMEX-AIM, and that TI in conjunction with MIT was developing a server `The X protocol was completely redefined last year. Its most recent version, X.1 1, is assumed in all of the discussion that follows. 2The client software runs on the Lisp machine and sends the graphics protocol commands to the remote user display system. The dual of the client is the X server software which runs on the user display system and translates the X protocol sent by a client Lisp machine into real graphics pictures and mouse actions. E. H. Shortliffe 48 5P41 -RR00785-15 Details of Technical Progress ~~~ to symbols: m Development activity a System beta testing/improvement 0 System dissemination/update/support V Vendor-supplied system U86 a7 El88 8/89 al90 8191 Remote workstation access TIP IP-TCP support (serial access) Workstation IP-TCP TELNET access Symbolics, Tl, SUN Xerox TELENET X.25/IP-TCP Ethernet gateway TELENET TELNET access (IP-TCP) Upgrade to IS0 protocol stds WorkstatIon services Network servers TELENET X.2511SO gateway Remote Virtual graphics Evaluate/select remote .window protocol CommonLisp client & server Tl Explorer client Tl Explorer server Symbolics Xerox UNIX client&server SUN Macintosh-It Other machines (to be announced) VG over low bandwidth connections Convert to IS0 transport protocols Common window application stdlimplement Develop/extend VG applications (personal links, new display primitives, intelligent compression, etc.) Distributed mail system IMAP design and definition (IMAP-1) Prototype 2060 IMAP- server InterLisp IMAP- client InterLisp mail reader/composer Figure 1: Core System Development Schedule 49 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 8187 aiaa 8189 0I90 8191 Distributed mail system (cont.) CommonLisp IMAP- client Tl Explorer Symbolics Xerox CommonLisp mail reader/composer Tl Explorer Symbolics Xerox Redesign IMAP protocol (IMAP-2) 2060 IMAP- server UNIX IMAP- server Update CommonLisp IMAP- clients UNIX mail client/reader/composer SUN Macintosh-II Other machines (to be announced) Convert to IS0 transport protocols Enhance reader/composer tools (message filtering and organization, multi-media messages, discussion group tools, etc.) Distributed processing Evaluate/select interprocess comm. protocol CommonLisp IPC Implement experimental services Distributed resource manager User interface to distributed services Refine and expand services available General computing environment lisp and Al shell environments Xerox CommonLisp SUN CommonLisp environment Macintosh-II Lisp and environment Other systems/machines (to be annour ccc Text processing tools (: al86 al07 &.3 8189 al90 8191 Figure 1: Core System Development Schedule, Continued E. H. Shortliffe 50 5P41 -RR00785-15 Details of Technical Progress General computing environment (cont.) Distributed file support CommonLisp RPCNFS Xerox Tl Explorer Symbolics Network directory mgmnt tools Network file backup &archiving Update archival file store hardware Distributed information access Operations management tools Other tools (intelligent spread sheets, intelligent command interfaces, evaluate new machines, etc.) Phasing of the transition to a distributed AIM community Experiments in Stanford/AIM community (10 General Stanford community transition National community transition Phase-out of BRTP subsidy of SUMEX 2060 8/86 8I87 8188 8189 al90 819 1 V 8186 8187 8188 8189 8l90 8191 (1) We expect IP-TCP use to be dropped in favor of IS0 protocols by late 1989. (2) Vendor support expected after initial development/beta test period. (3) Penodic review and evaluation of new vendors/workstations for use in the AIM communit). (4) Initial exploration of concepts followed by extensive development. testing. and disseminatron. (5) Convert IMAP- to IMAP- -- see below. (6) Use of InterLisp is expected to phase out by late 1988 in favor of X VOX CommonLisp -- see below. (7) We expect use of the DEC 2060 to phase out between the 4th and ith years. (8) This work will involve an extended period of developments testing. and dirr.zmination. the exact schedule of which cannot be predetermined because it will depend on the results of other developments and experiments in this ared. (9) There will be an ongoing involvement with the vendor to test and support new system developments. (10) We will use a small group of Stanford and AIM Al researchers and students to guide development and testing of distributed subsystems throughout the research period. Initially these WIII come mainly from the Stanford community which is easily accessible and already familiar with workstation use. (11) We will need to have enough of the systems R&O done to begin testing tools in a broader Stanford user community. (12) Dissemination to the general national AIM community will actually start earlier in an experimental mode (see 10) but full scale testing will begin in the 3rd year to prepare for a completa migration over the final 2 years. Figure 1: Core System Development Schedule, Concluded 51 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 implementation. We had our X client well underway, when we discovered that TI and MIT decided to jointly develop a Common Lisp X-client (CLX) instead of a server. Thus, in order to conserve our limited development resources, we chose to take a "wait and see" attitude, and redirect our programming efforts to the more immediate need of a distributed mail system. The time spent on our X-client was not wasted, since we gained a very deep insight into the protocol and its implementation, and planned to continue this project at a later date. In retrospect our choice was the correct one. CLX and a supporting CLX library for the creation of windows, menus, scroll-bars and other graphical objects is done, as is the implementation of a CLX- server which is due for alpha-release this June. Finally, The Common Lisp User Environment (CLUE) is now available from TI and is a window system on top of the CLX library which uses the Common Lisp Object System (CLOS). Thus, MIT and TI have provided a sufficient set of graphics primitives upon which one can build Al and systems tools in the X paradigm on TI Explorers. And, any other remote system which runs an X client and server will be able to have full duplex graphics communication with an Explorer. Currently, such a capability is available on Symbolics, SUNS, and VAXs. Also, since CLUE is a more general window-system/user-environment on top of the CLX protocol primitives, it satisfies the long standing need for a portable window system for developing Common Lisp Al applications and will certainly find uses here at SUMEX-AIM. Yet, much more work needs to be done in this area to fully develop and integrate remote graphics capabilities into Lisp machine systems in order to make low bandwidth connections a reality. We alluded to this topic in last year's report, and as yet, no one has begun to work in this area. In the coming year we want to insure that cross-country connections will indeed support remote graphics and give usable response time. Success of this work will mean that one can use LISP machine systems from TELENET, ARPANET, or an EtherTIP connection throughout the SUMEX-AIM community. 4.3 - Remote Graphics Applications Our emphasis in the area of remote access/graphics has evolved from proving the concept of remote windows and remote tools to building real systems on top of remote access capabilities for routine use. TALK We've added a new protocol and neti service to the TALK (intra-machine user communication tool which employs both text and graphics) system we described previously. We added the `Sketch' service type as well as the ability to communicate using the IP/TCP protocols over the Ethernet. Several groups in the national Xerox Lisp machine community who used the TALK program we distributed requested the TCP/IP capability. In addition to implementing the TCP/IP layer, since most BSD 4.* UNIX systems have a TALK facility, we wanted to make the TTY mode of our TALK compatible with the UNIX implementation. This would have allowed us to TALK to users on any VAX or SUN; we actually got to the point (using shortcuts for testing) where we could TALK from a VAX to a Xerox Lisp machine. Unfortunately, we ran into a problem in that the packets that SUN UNIX generates for TALK are different than those generated by a VAX due to differences in word alignment. This is true of UNIX running on other architectures as well as the E. H. Shortliffe 52 5P41 -RR007851 5 Details of Technical Progress UNIX TALK program failed to make all of its data types network independent. We are, however, able to use all the TALK modes (TEdit, Sketch and TTY) to communicate between any Xerox Lisp machines available via the ARPA/Internet and have done a trans-continental TALK with a user at MIT. We implemented the new `Sketch' service for TALK which uses an object-based graphics editor as the basis for communication instead of a WYSIWYG text editor. The addition of this service was mostly a test to see if TALK was properly layered so that services could be added without modifying the TALK program itself, as we had designed. For the most part this was the case, the `Sketch' service was brought up without modifying TALK in any way. Later we made some minor modifications based on what we learned to further simplify the addition of new services. To put TALK into routine use in the KSL, we included it (and a few other Courier- based servers) in our default Lisp image so that it is always available to users in order to increase the use of sophisticated multi-machine tools. MacWorkstation We have started a project based on the MacWorkstation software licensed by APPLE. The MacWorkstation program is designed to allow the Macintosh to start up programs on mainframes and receive graphic and text output to the MAC in a seamless fashion. We plan to write a Common Lisp window package (based on Common Windows) that uses MacWorkstation so we can connect (via Ethernet, Applenet or RS-232CImodem) to any of our Common Lisp engines and run the same piece of code on any and/or all of them. The MacWorkstation program uses a high level protocol which handles windows, menus, files, events, text and graphics and provides higher level access to the Macintosh graphic facilities than the native system library routines. We do not plan to remote the existing window package of any given system; this interface will be written without regard to the window system on the machine on which it is being developed. We plan on providing a new library of window commands based on the emerging Common Windows standard. This will both simplify the design of the system and allow existing code that conforms to the CommonWindows standard to be used with little or no modification. This interface code can then be used on all Common Lisp engines (even mainframes) in our environment, including TI Explorers, SUN workstations, Symbolics and Xerox Lisp machines. A Macintosh II is being used to run MacWorkstation and a Xerox Lisp machine is being used for the Common Lisp development. The two are currently tied together using a serial line which will eventually be upgraded to an TCP/IP Ethernet connection. We expect the resulting interface to work with the full range of Macintosh systems including the smaller ones typically in home use. It is possible to extend the capabilities of the MacWorkstation software on the Macintosh side, so once the basic system is in place we can then add whatever features we feel are necessary (if anything) to handle real Lisp applications. We have already identified a group in our own laboratory that will be able to use the finished product. The SightPlan system needs to interface to BBl output on an Explorer so that a user can see graphically how the site is being laid out. That graphics system is being implemented on a Macintosh since they can not use the Explorer screen which is full of BBl data. We imagine systems based on the Common Windows/MacWorkstation interface that require the symbolic processing power of a dedicated lisp engine but also need to be available to a large number of 53 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 users. We have test software running now which allows the Lisp machine to open windows on the Macintosh and do some simple graphics. Other Remote Tools We finished a tool (MONITOR) that allows a Xerox Lisp machine to examine the display of another workstation. This tool shows a shrunken version of the entire remote machine's display in one window and allows you to examine a smaller portion of the display in full size in another. We plan to use this to remotely examine the display on the Oncology clinic machine (which runs the ONCOCIN expert system) when one of the physicians calls up with a problem. This tool is another built on top of the Courier server we described in previous years. 5 - File Access and Management A stable, efficient mechanism for storing and organizing data is central to any computing environment, and is one of the most challenging issues in the move to distributed, workstation-based computing. It is necessary to provide standard services, such as file backup, archival, a flexible, intuitive naming facility, and data interchange services (e.g., software distribution). Also, as the amount of data being manipulated grows, it will become more and more important to have powerful tools for managing hierarchies of files We plan to support the community with a number of UNIX-based file servers, like the VAX-based servers (see Figure 7) and the SUN- based server (see Figure 5) in use at SUMEX for several years. These will require continued SUMEX-AIM development, however. By keeping the number of servers small, the distributed namespace problem should be manageable in the near term. Current UNIX file servers are relatively cheap and fast. UNIX has many of the needed facilities, e.g., backup, long names, hierarchical directory structure, some file property attributes, data conversion, and limited archival tools. However, while general issues of networking, remote memory paging services, and flexible file access have received considerable attention in both the academic and commercial development of file servers, there seems to be only slow development of other critical operational needs. For instance, the much-used file archiving system of the DEC 2060 (sometimes called off-line cataloged storage) has no analog service in the UNIX systems. Perhaps this is the result of UNIX having its origin in the small computer world where the number of users and volume of data has traditionally been quite low. Our efforts are going into trying to adapt a commercial system developed by UniTech to allow users to manage large file systems by providing access to and from off-line tape storage as well as maintain a historical archive of files. We have had a well structured organization for managing disk usage and accounting on our 2060 system and we plan to duplicate some of these features on the SUN-4 such that we can manage its usage in much the same way. A UNIX accounting system will be put in place to allow cost accounting and provide a quota enforcement capability on the distributed file server(s). UNIX has always been weak in the management of large-scale file backup and with the advent of disks of near gigabyte size, it is clear that an organized approach to their usage is essential. In support of the long-term goals of the distributed community, we have been reviewing more advanced data storage methods. Optical disk systems are attractive but have not progressed as quickly as many had predicted. However, it seems likely that this sort of equipment would be ideally suited to satisfying our requirements for the archiving of files. Helical-scan magnetic tape equipment might also serve this function well. However, in both cases the absence of industry standards introduces some level of risk when planning to use these types of data storage equipment. We plan to continue our investigations of these technologies this coming year and to make a decision. E. H. Shortliffe 54 5P41 -RR00785-15 Details of Technical Progress The AppleTalk/UNIX File Service package (AUFS) developed at Columbia University provides file support on a UNIX server so that Macintosh workstations can connect and see representations of the files present in the same icon-based format as the file system on the native Macintosh. File transfers are invoked by moving icons around the Macintosh screen just as if the UNIX server were an extension to the internal Mac disk. AUFS functions will be expanded to provide better and more service to our large Macintosh community. 6 - Electronic Mail Electronic mail continues as a primary means of communication for the widely spread SUMEX-AIM community. As reported last year, the advent of workstations has forced a significant rethinking of the mechanisms employed to manage such mail in order to ensure reliable access, to make user addressing understandable and manageable, and to facilitate keeping the mail software distributed to workstations as simple, stable, and maintainable as possible. We are following a strategy of having a shared mail server machine which handles mail transactions with mail clients running on individual user workstations. The mail server can be used from clients at arbitrary locations, allowing users to read mail across campus, town, or country. The mail server acts as an interface among users, data storage, and other mailers. Users employ a mail access protocol (MAP) to retrieve messages, access and change properties of messages, manage mailboxes, and send mail. This protocol should be simple enough to implement on relatively inexpensive machines so that mail can be easily read remotely. This is distinct from some previous approaches since the mail access protocol is used for ali message manipulations, insulating the user client from all knowledge of how the mail is actually stored on the server. This means the the mail server can utilize whatever data storage and access methods are most efficient to organize the mail on a particular server system. The first prototype interim Mail Access Protocol (IMAP) was designed with this in mind. As noted in our previous report, we developed a prototype IMAP server on the DEC-20 and client on Xerox D-machines. The resulting mail environment proved to be quite usable and some D-machine users were able to use it as an alternative to the DEC-20 mail environment in their daily mail work. During implementation of the prototype client we observed that the protocol had several deficiencies which made certain mail concepts difficult or impossible to implement. For example, there was no way to notify the client of newly arrived mail and inadequate extensibility in the protocol made it difficult to add such a functionality. Furthermore, it was a "lock-step" protocol with no mechanism for multiplexed operation, leading to synchronization problems. Finally, we rethought parts of our design based on our examination of several related projects, including MIT's PCMAIL and various POP2-based systems. Our second-generation Interactive Mail Access Protocol (IMAP2) addresses these concerns. Instead of the lock-step query/response model of IMAP, IMAP uses tagged commands and data and explicitly allows unsolicited data to be sent from the server to the client. IMAP introduced a more formal structure to server-to-client path: in particular, all data is now identified unambiguously. IMAP also addressed various performance issues, such as allowing the fetching of multiple types of data items simultaneously. Furthermore, IMAP introduced server-based message searching and the concept of a structured "envelope" representing the information stored in the header of an RFC822 message. An IMAP server manipulates the actual file store copy of the user's incoming electronic mail under direction from the IMAP client. As noted above, the client has E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 no knowledge of the (possibly operating system dependent) format of mail on the server's file store; the IMAP protocol provides its own representation of mail and the server translates between this and its host system file store conventions. The IMAP client issues a series of fetch commands to retrieve data from the server. A fetch command has two arguments: a message sequence and the name of the data item to be fetched. A message sequence can be a single message index, a range of message indices, or a list of numbers or ranges. A message index is one of a set of consecutive numbers which provide a quick pointer to that particular message in the mailbox. A mailbox with 20 messages uses indices 1 through 20. As an example, a typical FETCH command might be A0001 FETCH 2:7,10 ENVELOPE meaning "fetch the envelope data for messages 2 through 7 and message 10". "AOOOl" is a random tag generated by the client, and used by the server in the completion response to an IMAP command. An envelope is in a. format very much like a Lisp S-expression, and represents, in a structured fashion, data such as the From/To/cc/bee lists, message Subject, Message-ID, etc. There is presently no structured representation for the text of the message itself; the client must fetch the "RFC822" data attribute which is returned as a text string of the message in traditional RFC822 format. This is due to our immediate need for a system operationally usable in today's environment and also our feeling that the requirements for such a structured representation are not well-enough understood at this time. There are other message attributes which can be fetched, the most interesting of which is the "FLAGS' attribute which contains various status flags associated with the message. The operation of removing a message from the mailbox consists of setting the system YDELETED" flag for a message via the "STORE" command, and subsequently doing an "EXPUNGE" command. Other operations in IMAP include LOGINILOGOUT for user authentication, SELECT for mailbox access, STORE to update/alter attributes associated with a message, EXPUNGE to permanently remove deleted messages from the mailbox, and SEARCH to do remote searches for messages based on various attributes. The SEARCH command is quite sophisticated: for example the command A0021 SEARCH FROM "SMITH" SUBJECT "REPORT" SINCE "20~MAR-88" requests the server to return a list of message indices of all messages from SMITH with a Subject that includes the string "REPORT" that were placed in the mailbox after March 20, 1988. An elaborate set of remote search criteria have already been defined and implemented in the IMAP servers. The server on the DEC-20 and Xerox Lisp client were upgraded to IMAP2. We have also implemented an IMAP server for UNIX, and have demonstrated a prototype client in Common Lisp for the Texas Instruments Explorer. The main thrust of our effort in the coming year, however, will be a client for the Macintosh. 6.1 - Xerox Lisp client An IMAP2-based client program for the Xerox Lisp environment was useable by mid- year. However, it was not in wide-spread use and lacked the necessary final polishing and debugging needed for release to a large community of users. A major E. H. Shortliffe 56 5P41 -RR00785-15 Details of Technical Progress push was made to make the necessary changes and the program was shuttled back and forth between members of the programming staff, who are specialists in different areas, in order to make the program as well-rounded as possible. Once the improved versions started to become available, the user community was steadily increased and where possible the resulting feedback was immediately reflected the next release. Along with the polishing of the client program, the documentation was completely revised and extended and the servers programs were further debugged. Some of the specific changes to the Xerox Lisp client included the fixing of storage leaks, code optimization, basic window management and the splitting of the various command menus into several smaller menus and submenus, based on functionality. The major changes included the addition of the text compression and breaking features of the `WEDIT' program (described in earlier reports) to the composition window to allow the mailer to freely convert between unbroken, streams of text and fixed length lines. Another major change was the addition of `Zoom' mode for pulling together selected messages in the mail header browsing window. This allowed users to take full advantage of both the existing and extended text search capabilities that were added to the program without paying a penalty for retrieving excess mail headers from the server or doing manual searches via scrolling. A feature to search for mail files and allow the names of those files to be used as input to other commands with the click of a button was also added. The Xerox Lisp client was further integrated into the KSL environment by adding it to one of the standard Lisp images (TCPFULL.SYSOUT), making it readily available to a larger community of users. The Xerox Lisp client is not going to be the mail client that the entire laboratory uses in the long term. The final push to finish the program was done in order to make available to a larger portion of the KSL a mail system based on the client and server model that everyone will be using eventually on other hardware. We wanted experience in running such a system and to allow users to have early feedback in the design of the ultimate system The Xerox Lisp client will be maintained only as is reasonable as the mail system evolves so it can be used as a fast prototyping system for new ideas. The Interlisp-based IMAP layer of this client may be replaced by the Common Lisp one under development in order to redlJCe the maintenance costs of the various systems. The Xerox Lisp client is currently the only fully functional, active client in our distributed mail system but probably will remain so for only a short while longer. Soon, we plan to make this client available to the Xerox Lisp user community along with the DEC-20 and UNIX IMAP servers. 6.2 - Texas Instruments Explorer client We are now writing an IMAP client for the TI Explorer system using Common Lisp. Although this project is not yet completed, it already can read mail, set message flags, move/copy messages between mailboxes and much more. Still to do is the message composing and sending. Except for the window system which is Explorer dependent, we expect much of this code can be exported to other Common Lisp workstations allowing us to develop mail systems for these workstations too. 6.3 - DEC-20 /MAP2 server The DEC-20 IMAP server was the first implementation of IMAP (and its predecessor, IMAP), and many of the ideas of our distributed mail environment were explored there first. This was due at least in part to the familiarity of the programmer with that traditional environment. The last major functionality addition to the DEC-20 server was the addition of the SEARCH facility last year. The current 57 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 version of the DEC-20 server is a complete and robust implementation of IMAP and we do not anticipate any further work on it. 6.4 - UNIX /MAP2 server In our distributed mail system we expect to rely heavily on UNIX engines as servers. To this end we have written a UNIX IMAP Server (UIMS) for remote mail access. UIMS handles the full complement of IMAP commands and has been in alpha-test mode for the past month. On a UNIX server a mailbox is represented by two files. The first is the actual text file containing the mail, and the second is an index file that caches a description of the text file's format. The index file is used to minimize UIMS disk i/o by keeping enough information about the text file and each message in that file to, first, speed random access to the text file, and second, make file access unnecessary for many of the IMAP protocol commands. For example, a binary representation of a message's date, flags and keywords is cached for each message. Thus queries on the state of these fields require no disk file access since a copy of the index file is kept in memory while a mailbox is selected. The index file also allows UIMS to implement a demand-paging algorithm to manage the core memory used for storing the text of each message as it is read or searched. As a consequence, only messages in which a user shows an interest need be kept in memory. In most instances this is only a small percentage of the number of messages in the text file. Such an algorithm conserves the server's resources, and is especially important under the UNIX operating system since it does not implement shared memory pages between processes. For compatibility with MM-C, the Columbia University UNIX MM program (see below), UIMS and MM-C use the same text file for mail. MM-C has no knowledge of the index file associated with this file, and thus, UIMS will recreate the index file if the mail text file has been modified since UIMS last read it. Also, a locking mechanism is used to prevent simultaneous writes to the same mail text file. Unfortunately, locking under UNIX is not implemented in the operating system itself, and thus can be violated if a program not respecting file system locks attempts to write to a file that is open and locked by a program that does. All of the programs we have written and intend to write will respect these locks, as does MM-C. UIMS also parses the MM-C init file, that describes a user's mail reading configuration, when this file exists. UIMS and MM-C have proved to be very successful when used in tandem for remote, and local mail reading, respectively. 6.5 - Transition Strategy and P/an While in transition from our mainframe-oriented paradigm to a distributed computing environment, the ability to read mail locally, from home or from geographically distant locations is clearly necessary at all points in this process. To make this move as easy as possible we hoped to be able to find a mail program that ran under UNIX to be used as a replacement for MM-20, the mail program now used at SUMEX-AIM by the majority of our community members. We were fortunate to discover MM-C, an MM-20 clone written at Columbia University in C to run on UNIX-based systems. We became an alpha test site for MM-C, and brought it up on both a VAX 1 l/750 and a SUN-31180. It has been used extensively by members of our staff, and is purported to be an excellent mail program. By June of this year we should have installed MM- C on our new SUN-4 server. As local community members move from the DEC-20 to the SUN-4, they will initially E. H. Shortliffe 58 5P41 -RR00785-15 Details of Technical Progress read their mail in the same mode as they did on the DEC-20, using MM-C if they do not have an Explorer or Xerox D-Machine. In the latter instances, they will use their respective clients in conjunction with the UNIX IMAP server discussed in the previous section. We anticipate that the SUN-4 will be used as a time-shared system for reading mail for the short term while the Mac client is being completed. When this is done, all such mail reading will be accomplished using the IMAP server/client paradigm. In order to read mail from home one will dial-in to one of our EtherTIPs, TELNET to the appropriate server and run MM-C. The UNIX IMAP server is designed to be compatible with MM-C so that one can read their mail both locally and remotely without conflict. It is also important to mention that Columbia University distributes MM-C with the statement "Permission is granted to any individual or institution to use, copy, or redistribute this software so long as it is not sold for profit, provided this copyright notice is retained". This makes MM-C an ideal mail reading program for the SUMEX- AIM community as we move away from the DEC-20, and towards our distributed environment, since SUMEX-AIM can freely distribute all of its mail-related software to its national community. 7 - System Building Tools 7.7 - Lisp System Performance One of the key issues in selecting the systems for our distributed computing environment was the performance of Common Lisp. In order to assist us in evaluating the performance of Lisp systems, we undertook an informal survey of Common Lisp environments using two KSL Al software packages, SOAR and BBl. The data collection for this evaluation is close to complete but we have only been able to do preliminary data analysis. The results are presented in Appendix B. In this survey we have focused on execution speed for simulated system runs and for system compilation. However, we emphasize that execution speed benchmarks are only one aspect of the system performance evaluation, especially for Lisp systems. Other crucial issues like programming and usage environments, compatibility with other systems, ability to handle "large" problems, and cost must also be considered. Both SOAR and BB1 were chosen because they are implemented in pure Common Lisp, making them extremely portable. SOAR is a heuristic search based general problem solving architecture developed by Paul Rosenbloom and BBl is a blackboard problem solving architecture developed by Barbara Hayes-Roth. Neither of these systems is an intensive user of numeric computation. These systems were initially developed in environments other than those tested and no attempt was made to optimize their performance for any of these tests. The workstation systems to be tested were chosen based on their availability as well as projected applicability in AIM community environments. Since we were interested in "real world" results, we ran the tests on each machine in what seemed to be its standard operating mode. The machines tested include: . Apple Mac II . DEC (uVAX II and Ill) . HP 9000/350 . IBM (Compaq 386 "PS/2 model 80" and RTIAPC) . SUN (3/75, 3160, 3/260, 4/260, 41280) 59 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 . Symbolics 3645 . Ti (Exp 1, Exp 2, uExp) . Xerox 1186 and the Lisp systems running on these machines include: . Explorer Lisp 3.0+ (4.0) . Franz Extended Common Lisp 2.0 . HP Common Lisp 1.0 . Kyoto Common Lisp . Lucid Common Lisp 2.x . Symbolics Lisp 6.1 . VAXLisp . Xerox Common Lisp (Lyric) Analysis of the data that were collected is still underway but several conclusions are evident at a high level: . The fastest machines for running BBl (the microprogrammed TI Explorer 2 and microExplorer Lisp machines) are not the same machines that run SOAR fastest (SUN-4 RISC machines). o Within a factor of two of the best performance, a considerable range of workstations based on stock microprocessor chips as well as specially microprogrammed Lisp chips have comparable performance. Thus, while the "jury has been out" on the issue of special microprogramming versus stock instruction sets for Lisp systems, it now appears that stock systems have closed the performance gap. Unless substantial performance improvements are forthcoming from the specially microprogrammed architectures, it appears that Lisp- based research can just as well be done in "standard" workstation environments which would ease many issues of program export and integration. 7.2 - Lisp Programming Environments Even though performance gaps between microprogrammed Lisp systems and stock workstation implementations are narrowing, there still remains a significant difference in the quality of the development environments. The power of the development tools and environment is what has been the primary strength of Lisp machines, allowing rapid design, implementation, and debugging of complex programs. We believe the key to good development tools is integration, both in terms of consistency of interface, and in the ability to move seamlessly from tool to tool, carrying along appropriate data and state information. These qualities must be manifest in any KSL research computing system. Over the years, KSL and AIM community Al systems have been implemented predominantly in the InterLisp, MACLisp, ZetaLisp, and more recently, Common Lisp dialects. Beginning in the early 1960's, our work moved from mainframe Lisp environments to workstation environments for many reasons, principally involving powerful tools for system development and debugging and graphical interfaces. Commercial versions of these tools, that evolved over many years in the Xerox D- Machine, Symbolics 36xx, LMI, and TI Explorer systems, have become an indispensable part of our work environment. Newer Lisp systems for workstations not specifically developed for Lisp have lacked many important features of these environments. Thus, in light of the runtime performance advances of stock workstations, we have attempted to summarize the key features of the Lisp machine E. H. Shortliffe 60 5P41 -RR00785-15 Details of Technical Progress environments that would be needed in stock machine implementations in order to make them attractive in a development setting. The full version of this draft specification can be found in Appendix C. The following summarizes some of the key points about our effort to define the central environmental issues. We are continuing to seek comments from others in the AIM and broader Al communities as well as work with vendors and sponsors to try to get these feature integrated into commercially available Lisp systems. . These requirements represent a snapshot of the tools and technology available on today's machines. Al has historically and will continue to ride the crest of the wave of new computing technologies for the foreseeable future, which enable ever more complex systems. Thus, these are not static requirements and we expect to be able to take advantage of the future improvements in hardware, graphics, and software as they are generated by computer science research and industry. . We have tried to sort out the key features of current systems that are important to our research work. Except where explicitly stated, everything in the write-up (Appendix C) describes this "core" of functionality. Some items are clearly more important than others, but all represent needs that really guide our decisions about which new systems can be broadly used in the KSL. . We have two overriding goals in adopting future computing environments (which may seem to be or actually are in some conflict). We want the most powerful development environments we can get to facilitate the building of complex Al programs. But at the same time, we want to be able to share (import and export) research results and tools with colleagues in other labs and so must maximize the portability of code among systems. We believe that these goals can be approached jointly by the establishment and careful adherence to standards where possible, while continuing systems development where necessary. The requirements discussion is organized according to a "layered" view of Lisp environments, beginning with the upper levels. This organization is a conceptual framework within which to describe the various parts of the environment but may not correspond in full detail to the way system modules are actually organized. The elements of our description include: . Program Development Tools and Environment o Editor o Debugger 0 Inspector o Software Management Tools o Performance Monitoring and Analysis 0 Lisp Listener . Languages and Utilities o Windowing o Multiple Processes 0 Common Lisp 0 Compilation 0 Input/Output 61 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 0 Utilities 0 Interface Toolkit 0 Help System 0 Status Information 0 Printing 0 Pretty Printing . Lower Level Issues o Address Space o Memory Management o Dedicated Versus Shared Systems o Hardware Capabilities o Overall System Integration in the KSL 7.3 - General System Building Environment Traditionally, a large set of languages and programming environments has been supported on the 2060 in order to encourage experimentation and development. We now believe that the experience gained in those years of broad experimentation can be distilled into a fairly small set of languages and tools, relieving the researcher of the need to learn many programming languages, while still providing the needed facilities to allow the experimentation to move further into the higher levels of knowledge representation systems and problem solving architectures. As we move to the workstation-based environment, we plan to phase out support for many of the languages we have offered in the past and concentrate on the most relevant languages for Al research and applications: C, Apple PASCAL, FORTRAN, InterLisp-D, ZetaLisp, and Common Lisp. Common Lisp has already achieved popularity as a standard (see page 74), and many projects are already using it. We expect to press for further adoption of Common Lisp as a community symbolic computing standard, consistent with prior investments in large software systems such as those which exist for on-going AIM projects. In addition, we will support important higher-level knowledge representation and problem solving architectures (e.g., S.l, KEE, Strobe, and others) as appropriate for community research and dissemination activities. 8- Text Editing -- TMAX One of the biggest obstacles in reducing our dependence on the 2060 is large document preparation, including sectioning, cross-referencing, and flexible bibliographic referencing. At the KSL this sort of processing has been done using SCRIBE (a registered trademark of Unilogic Ltd.), TeX, or LaTeX. There have been over fifty Xerox workstations (1108/9s. and 1186s) in the KSL and as part of one experiment to move users off of the 2060, we worked to develop a system called TMAX (TEdit Macros And extensions) with the equivalent functionality of SCRIBE on these machines. This work is now finished and the following reports on the results of the experiment. The system has proven quite powerful for complex documents -- we know of two Masters theses that have been written using TMAX. We have turned the TMAX system over to Xerox where is now part of their standard working environment. The Xerox AIS group may use TMAX for their next set of InterLisp-D manual releases. Besides Xerox and Stanford, TMAX is in use at SRI International and Ford Aerospace. The Xerox workstations come with a text editor called TEdit. TEdit is a WYSIWYG "text formatter" (which is part of what SCRIBE does) but it lacks the ability to do more complex operations. TMAX does not try to mimic SCRIBE; this would be a monumental task. Rather it extends TEdit by implementing some of the more E. H. Shortliffe 62 5P41 -RR00785-15 Details of Technical Progress commonly used features of SCRIBE. Currently there are five main areas of TMAX; numbering, table of contents, endnotes (similar to footnotes), forward and backward referencing, and indexing. There are three important points about the TMAX experiment. First it is completely menu-driven. To invoke any feature, the user simply moves the mouse to the appropriate item in the TMAX menu and buttons it. Secondly TMAX never changes the user's text. All the features are merely additions to the text. These additions may appear to be strings but they are really structured objects that are treated as single characters. This means users can delete any feature they add in the same way they would delete any other character. Finally, all the inserted features are shaded gray so users can distinguish them from the normal text. We have written a twelve page help file on TMAX that uses all of the TMAX features. This document is both a description and demonstration of how to use TMAX. Below is a brief description of each of the TMAX main features. 8.7 - Numbering TMAX has an extremely flexible numbering facility that allows the user to number arbitrary objects such as chapters, sections, figures, examples, etc. The numbering scheme is completely up to the user. TMAX allows the user to create and edit a tree of "number nodes" where each node defines a level of numbering. The user also has the ability to define initial number values, the format of each number, text before and after the number (useful for headings), and the font of the number. To insert a number, the user simply buttons the appropriate node of the tree and TMAX figures out what the number should be and inserts it. Suppose for example the user is writing a book. The user could define the top node as Chapter with subnodes Section, Subsection, etc. Each time the user buttons Chapter in the number tree, the next monotonically increasing chapter number will be inserted in the document. It will also reset all Chapter's subnodes to their initial values. To insert a Section number, the user buttons Section in the tree and so on. The numbers can be ordinary Arabic numbers or upper and lowercase letters or Roman numerals. In the example above, the chapters could be "l.", the sections "1.A" and the subsections "1 .A.i". Each value depends on the values of its immediately superior node in the tree. Finally, the numbering does not have to be inserted consecutively. If a user inserts a new number between other numbers, TMAX will automatically adjust all the numbers after the new one. This numbering mechanism insures uniform numbering throughout the document. If at any time the user wishes to change the numbering format, he just edits the definition in the number tree and TMAX takes care of all the rest. 8.2 - Table of Contents Closely related to numbering is the "Table of Contents" facility. When this item is buttoned, TMAX creates a file containing all the numbers it has inserted along with any text the user specified before and/or after the number followed by a dotted leader and the page each number appears on. The file is sorted by page number and the user has the ability to exclude certain numbers from the file. 8.3 - Endnotes Endnotes are like footnotes in that they insert a superscript number but the text associated with the number is appended to the end of the document rather than the bottom of the page. When a user buttons the Endnote item, TMAX inserts the next monotonically increasing number and then prompts the user for the text string 63 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 associated this the number. Like numbers endnotes need not be inserted consecutively. If an endnote is inserted between other endnotes, TMAX automatically adjusts ail the endnotes after the new one. Finally, there is no correspondence between numbering and endnotes numbers. 8.4 - Forward and Backward Referencing Whenever a number or endnote is inserted, the user has the option of "tagging" it. At any point in the document, the user can reference this tag in one of two ways. The reference can be by the value of the number (e.g. "see chapter 5") or by the page the number is on (e.g. "see page 7"). Of course the user could reference the same number both ways and get something like "see chapter 5 on page 7". Whenever a tag is specified, it is added to a reference menu. If the user is referencing the same tag is several places, he can simply insert the reference from the menu rather than retyping the tag each time. 8.5 - Indexing TMAX supports two styles of indexing; simple and extended. With simple indexing, the user just specifies the phrase to be indexed. With extended indexing the user can 1) specify the phrase that will appear in the index, 2) specify the font of this phrase, 3) have the phrase sorted by some other phrase, and 4) optionally exclude the page number from the index (useful for index headings). Whenever an index is first specified, it is added to an index menu. If the user is indexing the same phrase is several places, he can simply insert the index from the menu rather than retyping the phrase each time. At any time the user can create an index file. This files contains the alphabetically sorted indices followed by the page numbers on which they appear. Optionally the user can request "manual style" indexing in which the page numbers are replaced by selected numbers. the chapter, section, etc. in which the index appears. In this case the index specifies These are only the main TMAX features. There are several other minor features such as the ability to insert the date and/or time in several different formats. 9 - Distributed Information Resources and Access There are many user needs for getting information from and about the computing environment, ranging from help with command syntax to sophisticated database queries. A distributed computing environment adds new complexities in making such information accessible and also new requirements for information about the distributed environment itself. We are adapting the many workstation-specific information tools to include distributed environment information such as workstation and server availability, network connectivity, "Finger" information about user locations and system loads, and other information of interest to users in designing approaches to carrying our their research tasks. In addition, we will have to develop general systems tools for monitoring and debugging distributed system performance to identify workstation and network problems. Finally, we must adapt and develop distributed system tools for remote database queries and organize the diverse sources of information of interest to AIM community members to facilitate remote workstation access to community, project, and personal information that has traditionally existed in ad hoc files on mainframe systems. The Macintosh HyperCard tool provides a very powerful environment in which to hierarchically organize and provide access to information in the form of text, graphics, pictures, and even sound. We have begun development of a "KSL stack" for HyperCard that will initially include descriptions of KSL and AIM personnel, E. H. Shortiiffe 64 5P41 -RR00785- 15 Details of Technical Progress SUMEX-AIM projects, SUMEX-AIM computing systems, KSL building maps, the KSL bibliography, etc. While the current version of HyperCard is a good environment for a prototype distributed information access system, several developments are needed from Apple for full utility. These include remote network access to stacks, allowing multiple simultaneous readers of stacks, and more flexibility of how stack "cards" are presented on workstation screens. On another front, in conjunction with the SUN file server we have been integrating, we have mounted an experimental database system for remote information access using the commercial UNIFY database product. Our goal is to make access to the database information possible from a distributed workstation environment through network query transactions, as opposed to asking the user to log into the database system as a separate job and type in queries directly. This will facilitate remote information access from within programs, including expert systems, where the information can be filtered, integrated with other information, and presented to the user. The system will provide multi-user, multi-database access capability; that is, several users will be able to have access to a single database at the same time, and a single user will be able to have access to several databases at the same time. The initial implementation of the remote query system was done on a Ti Explorer and then adapted to the Xerox D-machine Common Lisp environment. The query interface on the Lisp machine communicates with the Sun UNIFY database system via the Remote Procedure Call (RPC) mechanism which underlays the NFS remote file access system. The Explorer calls a server on the SUN and sends an SQL/DML query command as an argument to a remote query procedure, and receives the retrieved data and/or a message sent back from the server. SUN UNIFY can already manage multiple databases, so a client can have several databases open at the same time. The operations on the database are transaction-oriented, and therefore the concept of a database access session is applicable. The access functions currently implemented are open a database, close a database, retrieve data from a database, insert records into a database, delete records from a database, update the database, lock a database, and unlock a database. 10 - Distributed system operation and management The primary requirements in this area are user accounting (including authorization and billing), data backup, resource allocations (including disk space, console time, printing access, CPU time, etc.), and maintenance of community data bases about users and projects. Our accounting needs are a function of system reporting and cost recovery requirements. The distributed environment presents additional problems for tracking resource usage and will require developing protocols for recording various kinds of usage in central data base logs and programs for analyzing and extracting appropriate reports and billing information. We are now involved in analyzing the kinds of resource usage that can be reasonably accounted for in a distributed environment (e.g., printing, file storage, network usage, console time, processor usage, server access), and investigating what facilities vendors have provided for keeping such accounts. Data backup is, of course, closely related to the filing issue. We continue to use and improve network based file backup for many of our file servers. 11 - Mainframe, Server, and Workstation System Environments The various parts of the SUMEX-AIM computing environment require development and support of the operating systems that provide the interface between user software and the raw computing capacity. This includes the mainframe systems, network servers, and the workstation systems. Following are some highlights of recent system software environment developments. 65 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 77.7 - TOPS-20 Our long-term plan to phase out the 2060 mainframe system has continued as scheduled. Development efforts on the 2060 have ceased, except where needed to keep the machine operational in the evolving distributed environment. This involves considerable work in areas such as file system archiving, retrieval, and backups; periodic updating, checkout, and installation of new versions of system softtiare; the regular maintenance and updating of system host and network tables; and monitoring of and recovery from system failures, both hardware and software. year, the main areas of activity include: Over the past . TOPS-20 Domain Name Resolver Software -- The Internet community has been in the process of converting to a domain naming scheme, to replace the flat address space of the old exhaustive host tables prepared by the Network Information Center. This past year, we worked very closely with a sister group at MIT- tc `ntegrate a version of the TOPS-20 Domain Name Resolver softwar with the TOPS-20 Mail System. In addition, several other network utilities, including TELNET and FTP were updated to use the Domain Resolver, rather than the old HOSTS.TXT table. Most other TOPS-20's are still running an ISI version of the DOMAIN resolver, which is inferior in several respects, so this represents a significant effort at SSRG to maintain network communication with the national Internet community. . Long-term access to SUMEX data files -- The TOP-20 approach to providing references to off-line (archived) files presents substantial overhead to the active file system. Since SUMEX has always had an interest in preserving the history of the intellectual achievement resulting from our project, we are quite concerned with access to even very old data files. Thus we have prepared a staged scheme for retaining access to the files of former users. Though access is not as convenient to these files as it is to those archived by current users, there is, nonetheless, the ability to retrieve files of former users of our Tops-20 system and its predecessor, the PDP-10 based TENEX system. o Cost Center accounting -- During the past year, the 2060 accounting programs were further updated to reflect the SUMEX Cost Center structure (see Page 121). Monthly reports have been promptly generated to reflect on-going usage. As part of the cost center management, a concerted effort has been made periodically to review all of the SUMEX accounts, and remove those that were no longer appropriate. 77.2 - UMX We run the Stanford (SULTRIX) version of the UNIX system that is distributed by the University of California at Berkeley on our shared VAX 1 l/750 file servers. SULTRIX adds support for Sun Micro Systems Network File Service (NFS), and the PARC Universal Packet (PUP) protocol that is necessary for our D-machines. The two VAX systems are used extensively as file servers, and minimally as tirne-sharing systems. One of these systems (ARDVAX) is used for systems development and the UNIX version of the IMAP server has been written and debugged on it. Little system development effort is done on these beyond staying current with operating system releases and useful SULTRIX community developments. E. H. Shortliffe 66 5P41 -RR00785 15 Details of Technical Progress 7 7.3 - Sun File Server The SUN-3/180 file server (KNIFE), briefly mentioned in last year's report has been fully integrated into our distributed environment, In addition to providing file storage for about 50 users, it represents a prototype for the recently delivered SUN-41280 in terms of implementation issues. lmagen Host Software was installed to allow print spooling from the file server or any of its clients to any of the several lmagen Printers in the lab. This software was then updated to take advantage of the UltraScript (PostScript like document description language) on the lmagen 3320. The InterLeaf Technical Publishing Software, and Frame Maker Software were installed for evaluation purposes. The Unify Database Software! installed last year, continued to be used to administrative database applications. The Columbia AppleTalk Package (CAP) was installed to allow local Macintoshes to access Apple-UNIX File Servers (AUFS) through Kinetics FastPath AppleTalk/Ethernet gateways. A copy of the UNITECH UNIX file system backup/archival software was acquired for evaluation. The limited evaluation period ended before we could complete our evaluation, however, but we will probably try it again for our Sun-41280 system, as an archival system is very important, and this package, although not perfect, looks like a good approach. A 6250 BPI l/2" magnetic tape Unit was added to the Sun-31180 file server, and operational procedures put in place for regular file system backups. 7 7.4 - Xerox D-Machines A considerable, but shrinking part of the SUMEX-AIM community continues to use Xerox Lisp; primarily on Dandetigers and Doves (Xerox 7709, 7786). A donation of hardware from an industrial source increased the number of Dorados (7 732) from 1 to 2. The Xerox workstations proved reliable and economical to repair, averaging under $150 in repair costs per workstation for the year. The Lyric release of the Xerox implementation of Common Lisp (which was beta- tested at SUMEX in March of `87 and saw general release in the Fall) proved so stable that it is still in use here a year later. The previous release, Koto, is still in use (by users of the commercial KEE and Strobe packages), but does not require or receive much support time from the staff. We will shortly beta-test the next release of Xerox Lisp--Medley. We are also involved in the testing of a possible new hardware platform for the Xerox Lisp environment. The ONCOCIN system was used as a test program and the results of the experiments are encouraging. (A fringe benefit of the test is that research ONCOCIN has been upgraded from Koto to Lyric sooner than had been planned.) The Lyric release required the reorganization of our font directories, containing several thousand files. (The font file names were changed to speed up lisp's search mechanism.) We deviated from the Xerox approach by further subdividing font directories according to font family name. The change required patching the lisp system software to exploit the new organization. This resulted in directories that are 67 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 much easier to maintain and increased speed in font searches -- in some situations nearly twenty times faster. As with any new lisp release we had to update the code and documentation for our over twenty Lispusers packages. This effort ranged from simple recompilation to serious rewriting to account for changes in the system and to add new features based on accumulated user feedback. New Lispusers packages we made available to the national community this past year included TCP and UDP time packages with both client and server routines, a new device driver for a color film recorder, client implementations of the Sun RPC and NFS protocols, the MONITOR package for viewing remote displays, and the SKETCHTALK package for remote graphic interaction between workstation users.' We wrote an experimental Xerox Lisp imagestream driver for a Bell 8 Howell CDI-IV film recorder. This device was connected to a public 1108 via a 19.2 Kb serial line. In its HPGL (Hewlett Packard Graphics Language) emulator mode, the device was capable of drawing only single width straight lines in several colors. We used some bitmap-to-line drawing routines from our (previously implemented) plotter driver and a scheme using intermediate bitmaps (for the color planes) to produce a driver that could accurately render any image from the Xerox workstation on the film recorder (in color). This allows us to make photographic prints and slides of a fairly high quality. Patches to the Xerox SKETCH drawing program were necessary to make it work correctly in color mode. (These patches are applicable to other color devices.) The film recorder itself was donated by Bell 8 Howell and is on loan from the Computer Science Department. We completed the Common Lisp implementation of Sun RPC (Remote Procedure Call) and NFS (Network File System) begun last Summer. The package was then used as part of a prototype system in which a database was accessible over the network to several types of workstation. Xerox has taken over responsibility for maintenance, improvement, and distribution of the package. The Info-l 100 discussion list which we sponsor continued to serve its readership on the ARPA Internet, Usenet, Bitnet, and CSNet. New subscriptions roughly equalled cancellations in number. Among the beneficiaries are other NIH-sponsored projects at Ohio State University and the University of Maryland. The Xerox NS file server was reconfigured to run automatic nightly incremental backups from three of the T-300 disk drives to the file system on the fourth drive. Although this backup to disk does not give us the ability to retrieve files from the distant past (as tape backups do) it does give us the ability to restore a damaged file system to within a day of the damage. With three disk packs per drive and monthly full backups we are able to retrieve files from the previous 60 to 90 days. One problem with the previous system was automatic backup of Lisp sysouts (large binary images on the order of 3MB to 10MB in size), which tended to fill the incremental disk too quickly. To solve this, we reallocated disk space, moving sysouts to a directory on the backup drive itself. This kept the sysouts from being backed up incrementally. (They are still dumped monthly with the other drives and made available from a less active unit, reducing disk contention.) The removal of the sysouts from the user file system freed up large amounts of space and the user community on this server has increased in size and activity. We moved the Xerox Communications server (which provides Clearinghouse service) to the MSOB machine room and upgraded its disk drive from 10MB to 40MB (using a `The last two packages are described in the Remote Workstation Access section of this report. E. H. Shortliffe 68 5P41 -RR007851 5 Details of Technical Progress disk from our spare parts supply). We did this for several reasons. By making it our NS boot server (instead of the Xerox printer) we were able to free up memory and disk storage on the printer so that we could turn on the the reprinting feature of the new server software. (This allows the printing of some Interpress documents that were too complex to print before the change.) We installed file service on the new boot server so that we could store the system installation files on it. Previously the boot and installation services were split between the printer and file server. Now we are able to boot the main file server over the network from software on the Communications server rather than using the slower, less efficient floppy-based system we were limited to previously. Finally, this gave us a Clearinghouse server on the MSOB network which makes authentication activities (like logins) much faster and more reliable (independent of network/gateway irregularities). The Xerox University Grants Program (UGP) server maintenance agreement expires at the end of this reporting year (May 31st). This program covers the file server, 15 workstations and the print server provided in the original grant. The program has been funded to extend the warranties, but the corporation headquarters hasn't given the final go-ahead. If this does not happen, we will have to cover the cost of maintenance of the file server. In this contingency, we will maintain the workstations and printer ourselves (as with our other systems). As reported last year, the Xerox NS file server software was upgraded to support random access of data. Although we had successfully written some patches to exploit this capability, we were fortunate to be asked by Xerox to beta-test an upgrade to Xerox Lisp's NS file system interface (to make use of the new features). This code is now part of our standard system and the file server is providing services such as hashed database access and on-line manual lookups (that were not possible before). We also beta-tested the ROOMS multiple virtual display window environment (developed by Xerox PARC). This environment allows the user to have any number of virtual screens at his disposal. He can switch from one to another with ease and share windows among any subset he chooses. ROOMS may solve the display problems of having the ONCOCIN, OPAL and OPUS systems all in the same Lisp image. For easy access in the clinic, all three systems could be left set up (on their own screens) and, using a mouse click, the user could move between them. 7 1.5 - Texas instruments Explorers The twenty Texas Instruments Explorers have enjoyed an increasing popularity as more projects have developed a need for the combination of execution speed, full Common Lisp, and sophisticated development facilities offered by the Explorer. Explorers have come into use in other parts of the national biomedical community as well, such as Ohio State University, MIT and the University of Maryland. However, the Explorer is still maturing as an Al workstation. Thus, our efforts have been directed at improving the environment of the Explorer by developing software, organizing user interest activities, and advising Texas Instruments. Previous experience has shown that the greatest source of advancement for a particular computing environment is the user community. They are the most in touch with the deficiencies of the system, and thus uniquely positioned to address them, as well as to utilize the strengths of the system. The product developers of the system are frequently too involved in the lower levels of detail to produce general, effective solutions to problems, as well as being hampered by limited manpower resources. However, a significant amount of time and effort is required to organize this effort. This task has traditionally fallen to a user-run organization, such as DECUS or Usenix. E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 We have spearheaded an effort to organize an international users' group for the Explorer. The slightly misleading name for the group is NEXUS, standing for National Explorer Users' Group. The goals of this undertaking are to: . facilitate dissemination of information by organizing meetings where presentations and discussions can be used to make little-known techniques and facilities more widely known, as well as feeding back information on needs and wants to developers, . allow more immediate communication via electronic mailing lists, which are used for distribution of important software fixes and discussion of items of general interest, such as new software tools, or proposed changes to the system, . publish a periodic newsletter containing usage tips, salient extracts from the electronic mailing lists, and announcements, . and, perhaps most importantly, establish and maintain a library of public domain, user supplied software. A steering committee has been formed and set about forming a charter for the group, consolidating membership information, arranging a meeting in conjunction with AAAI `88, and settling the legal issues involved with setting up a code library. There are already many entries ready for the library, most of which have been developed locally. We have maintained the software tools that were produced previously by fixing bugs, making improvements, and porting to new releases. Some of these have remained essentially the same, including: . Analog Clock . Backgrounds . Backup To File System . Batch Processor . Choice Facility Enhancements . Deexposed Mouse . DEFSTRUCT Type Checking . Development Tool Consistency Enhancements . Finger Via TCP . General Named Structure Message Handler . Graph Viewer . Graphical Value Monitors . lmagen Via TCP . Inspector Enhancements . Map Over Files . Rubber Band Rectangles . Single Window VT100 . Snapshot Windows . Soft Keys . Source Code Controller . Structure Enhancements . Symbolics 36xx to Explorer compatibility package , Transparent Windows . Vertically Ordered Menu Columns . Window Manager System Menu Many of the tools have been enhanced or newly written this year. E. H. Shortliffe 70 5P41 -RR00785 15 Details of Technical Progress General Inspector This new tool integrates the three inspectors (data, Flavor, and Class) into one tool. Search And Replace This new tool allows looking for a pattern in a set of files, optionally replacing occurrences of the pattern with a new string. Source Code Debugger This new tool allows the user to compile code with "back pointers" so that the point of execution in the source can be shown in the debugger. This tool is a non-trivial achievement. Spelling Checker This tool is based on Tl's spelling checker, but with extensions allowing user-defined dictionaries and other more minor enhancements. Window Debugger Enhancements -- Additions to this tool include displaying SELF in the Locals pane even when it isn't explicitly referenced and allowing items in the Inspect pane to be modified. Window Dragging This new tool allows many windows to be "grabbed" by holding down the middle mouse button and dragged to new locations on the screen. Window Icons This new tools provides the infra-structure to allow shrinking temporarily unused windows to iconic representations. Zmacs Enhancements New features in this tool include: o Commands to manipulate tag tables . Commands to load tools and show their documentation . Commands to talk to the compiler allowing one to see the result of optimizing code or to check the args in an expression . A facility to spawn commands into a background process o Commands to load and compile systems Of course, all of these will be provided to the user's library, and many of them have already been given to other sites, including IntelliCorp, Berkeley, ISI, University of Maryland, and Ohio State. Third party software is less utilized, but we stay abreast of the latest releases of the expert system shell KEE. We have installed a DVI previewing system from MIT allowing TeX and LaTeX output to be viewed on the Explorer screen. We have available several other imported tools such as a Common Lisp LOOP package and Macintosh style scrolling. We have been following the development of the Common Lisp Object System CLOS closely. Several projects have already begun using Portable Common Loops (PCL), the public domain near-implementation of CLOS. We have done a number of things to make PCL easier to use on the Explorer, including making the debugger understand PCL forms and display them legibly, making the editor understand PCL forms and manipulate them properly, constructing a PCL class inspector similar to the Flavor inspector, and fixing bugs. 71 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 In addition to producing and maintaining these software tools, we attempt to provide extensive testing and evaluation of Explorer hardware and software products in a sophisticated university research environment in order that these products work more effectively when they are distributed to the national community. This testing is critical to the development of the computing environment since the combination of concentrated in-house expertise and close links to the product developers allows a turnaround on problem fixes unavailable in the broader scope. This year we have participated in testing Tl's high-end product, the Explorer II, Releases 3.1 and 3.2, Release 2.0 of the Network File System protocol implementation, and the new microExplorer system. The microExplorer is an add-in co-processor board for the Apple Macintosh II based on Tl's VLSI Lisp chip. After initial presentations from TI we were concerned that the microExplorer software would be too limited to allow many of the applications we envisioned for the system so we had a design review session with Tl's developers before the final design of the system was frozen and as a result many aspects of the system are quite a bit more amenable to future extension. We were given one of the first systems outside of TI to beta test. During this testing dozens of problems were reported and addressed. It is our hope that this work will result in high-performance, low-cost sophisticated Lisp availability, allowing greater dissemination of Al software to the national community. In addition to specific testing and evaluation, we are constantly finding, tracking, fixing, and reporting software bugs. This year we submitted twenty-nine new bug reports on Explorer system software, twenty of which had fixes included. All of these fixes have been made available to the national community in a patch file. As well as working on these specific problems, we have had many meetings with Texas Instruments representatives wherein we have attempted to present the needs of the national community for short- and long-term Al workstation products, covering issues including the desirability of specialized hardware, address space, programming environment versus execution speed, and the ability to utilize the Al workstation's power for routine tasks. Of course, there is also a large number of day-to-day activities needed to keep the computing environment pleasant, including resource management (e.g., disk space allocation, printer management), assistance with file backup and magnetic tape usage, and introducing new users to the system. We have produced documents targeted at complete novice users, users of InterLisp-D machines, and users of Symbolics machines in order to facilitate user education. These documents have been used as examples at various places in the national community. For the coming year we plan to continue development and maintenance of the software tools, perhaps adding tools such as a DARPA Internet Domain Resolver, UNIX print spooling, better IP access control, CLX and CLUE packages, automatic backup, better documentation, better software management facilities, KSL specific NEW-USER facility, a Zmacs novice mode, and an Explorer version of the TALK program, as well as aiding the growth of the users' group. 7 7.6 - Symbolics Symbolics Our work with Symbolics equipment has continued at a low level pending resolution of long-standing maintenance issues. As has been stated previously, in order for workstations to be competitive with time-shared mainframe computing resources, they must not only have a low purchase price, but must be cost-effective to E. H. Shortliffe 72 5P41 -RR00785- 15 Details of Technical Progress maintain. This goal is normally achieved due to the economies of scale associated with having a large number of identical parts in an installation, as well as amortizing the cost of software development over many machines. We have come to reasonable agreements with all of the workstation vendors except for Symbolics. The high costs of service, the exceptionally high price of mail-in board repair, and the lack of a reasonable self-service alternative has left us unable to justify continued support of these machines unless a workable agreement can be reached. We have been trying a self-maintenance contract which is supposed to supply parts while we do the diagnosis and replacement work. This arrangement has not worked out well due to inadequate diagnosis software and procedures and delays in getting parts from Symbolics. In the coming year, we plan to try a standard Symbolics maintenance contract. We plan to bring up Symbolics Genera 7.2 this summer. We have installed the REFINE system as well as KEE and FORTRAN. 7 7.7 - SUN Workstations and Lisp Environment Due to the high performance relative to purchase cost, Sun workstations drawn strong interest as Lisp engines. For the past year we have had three SUN 3175 workstations in experimental use in the KSL. Because these were purchased for LISP work, we have added a 24 megabyte memory board (from Parity Systems Inc) to each of these. Also added were 70 megabyte SCSI disks. The systems have been set up to swap the large virtual memory to the local SCSI disk, rather than over the Ethernet to the server. One of the systems is being used in the "Very Large Reusable Knowledge Base" project. A speech recognition box has been connected to another of the clients, and an interface to a Xerox InterLisp workstation running ONCOCIN developed to study the use of speech input for medical information. An evaluation of SUN workstations was made in terms of their suitability as a platform for a general physician's workstation which would support data management, analysis and display, and consultative software. For now, the SUN/UNIX environment was judged not to be competitive, in terms of cost or user interface technology, with other workstations environments for this purpose. The vendor plans for LISP support on SUN workstations has be in a state of flux this past year. Currently their Lisp supplier is Lucid but there have been many rumors about a shift to Franz, Inc. as a replacement supplier. It appears that this uncertainty has delayed SUN's plan for improvements in their current package. However, since Franz Inc. is closely involved with CORAL in the LISP package we prefer to use with our MAC-II's, we might see a benefit in SUN making this change is suppliers. Overall, Sun's Lisp still does not provide the programming environment power and maturity of the implementations for microprogrammed Lisp machines (see Appendix C). Sun seems to be devoting a good deal of effort to it's Symbolic Programming Environment (SPE) product to alleviate this situation. We examined an early version of SPE and found it very similar to it's foundation, the HyperClass system from Schlumberger. We are given to understand that SPE has made significant progress since those early versions, so we intend to re-evaluate it soon. We have been using the above mentioned HyperClass system as well as keeping new releases installed. 73 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 12 - Workstation Standards and Access 7 2.7 - Computing Environment Standards In a heterogeneous computing environment, such as Al research inevitably involves, the issue of cross-system compatibility is a central one. Users of various machines want to be able to share software, as well as be able to use various machines with a minimum of overhead in learning the operating procedures and programming languages of new systems. Thus, it is crucial to specify and propagate powerful, flexible standards for various aspects of the computing environment so that it is possible to transfer both skills and information among machines. In order to improve the inter-machine compatibility of our software, we have been encouraging all users to use the Common Lisp programming language [5], as well as pressing vendors to provide more complete and efficient implementations of this language. We have already served as beta test sites for Xerox, Texas Instruments, and Lucid Common Lisp implementations. The Common Lisp language, however, is only a subset of the software needed for our research. Research projects need higher-level powerful facilities, such as an object-oriented programming system and sophisticated error handling. Therefore we have been supporting and following the development of the Common Lisp Object System (CLOS) via membership in the electronic discussion group, technical contributions, and porting of Portable Common Loops (PCL), a predecessor of CLOS, to the TI Explorer. We are now encouraging vendors to produce efficient implementations of the system, and users to familiarize themselves with it. We are also encouraging vendors to adopt the proposed Common Lisp error system. Other features of the computing environment also need to be standardized to be useful on more than one machine at a time. Another of the most important of these is the keyboard and display interface, often referred to as the "window system". See the virtual graphics section (page 48) for further discussion of window systems. There are also many other areas which could benefit greatly from standardization, including document page description languages, text and graphics representations, and more networking protocols. However, it is important that standards not be entered into hastily, as an insufficient standard can often be worse than no standard at all. We intend to continue working to develop standards for these and other computing needs as the understanding of the issues involved matures. 72.2 - Protocol Standards Another important area of standardization has been inter-machine communication, or networking. Underlying all network I/O must be a network protocol for packet transfer between cooperating hosts. At SUMEX we have had long term experience with several such protocols; PARC Universal Packet protocol (PUP), TCP/IP, ChaosNet, and XNSISPP. These have been used to implement higher level services such as remote booting, file transfers and access, TELNET access, electronic mail and bulletin boards, remote procedure call interfaces, remote graphics interfaces, and numerous utility services for locating network hosts, addresses, and the like. In the past we have elected to write servers for each new protocol in order to accommodate both vendor hardware and systems software requirements. This was necessary because no one protocol has been supported on all such systems. With others in the computer science research community, we have pressed vendors to supply implementations of the DARPA standard TCP/IP communications protocols. We are pleased that the IP protocol family is now supported on all hardware and E. H. Shortliffe 5P41 -RR00785-15 Details of Technical Progress operating system configurations currently at SUMEX. And we expect to have IP support on any new systems we purchase in the future. Similarly, IP is supported on all of our `JNIX based file servers, and the SUNet gateways route all IP datagrams. There has been a great deal of effort at Stanford and SUMEX to enforce IP as a standard protocol for new software development. This was motivated by its broad acceptance and the growing number implementations throughout the networking and vendor communities. This does not imply that we will abandon the other protocols but rather, since we are seeking to have uniformity across all vendors with the proposed Stanford distributed environment, we are choosing to limit new implementations to the IP protocol family. We are also currently working to provide improved support for TCP/IP in our Terminal Interface Processors (TIP's), having already implemented TCP/IP routing service. Such standardization has a price, however, in that observed network communications speeds are often higher between equipment "tuned" to individual vendor protocols. We continue our efforts to assess performance bottlenecks and to refine system implementations to achieve acceptable throughput. 13 - Network Services A highly important aspect of the SUMEX system is effective communication within our growing distributed computing environment and with remote users. In addition to the economic arguments for terminal access, networking offers other advantages for shared computing. These include improved inter-user communications, more effective software sharing, uniform user access to multiple machines and special purpose resources, convenient file transfers, more effective backup, and co- processing between remote machines. Networks are crucial for maintaining the collaborative scientific and software contacts within the SUMEX-AIM community. 73.7 - National and Wide-Area Networks 13.1.1 - General National Networking Issues A 2-day conference on national research computer network issues was held in Washington, DC during April 1988. It was sponsored by The National Association of State Universities and Land Grant Colleges, The EDUCOM Networking and Telecommunications Task Force, and The New York State Education and Research Network (NYSERNet) with support from Apple Computer, IBM, and NYNEX. The meeting was well organized and encompassed views from academia, federal and state government, National Academy of Science/National Research Council (NASINRC) and White House Office of Science and Technology Policy (OSTP) study groups, industry, and Congress. There were discussions about needs and goals, NSFNet status and plans (including the Michigan MERIT/IBM/MCI implementation of the NSFNet backbone and regional network experiences, especially NYSERNet), DARPA's viewpoint, high bandwidth network research issues, a congressional view, and many funding/legal/technical issues. The meeting had over 300 attendees representing most of the major players. No one was at the meeting from NIH. Also, NIH has apparently not been very active on the FRICC (Federal Research Internet Coordinating Committee) set up by the OSTP for interagency coordination. The FRICC includes DARPA, NSF, DOE, NASA, and HHS. Outside of DOD activities, almost all of the justification for the national NSFNet effort has come from the supercomputer initiative. Even most of the regional networks have been set up to facilitate access to supercomputers -- e.g., NYSERNet and BARRNet. The FCCSET (Federal Coordinating Council for Science, Engineering, and 75 E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 Technology) report from OSTP is titled "A Research and Development Strategy for High Performance Computing. The NSF management overview of NSFNet goals mentioned the "workplace without walls" but the main motivating thrusts were access to supercomputers, central data bases (e.g., GenBank), the MOSIS facility for telemanufacturing, etc. Other uses for general research communication, collaboration, routine resource sharing are mentioned but do not capture primary attention. This balance of emphasis was all reemphasized in discussions with congressional representatives. Current federal funding covers the supercomputer effort and the NSFNet backbone but additional funds (estimated variously at $25M - $lOOM per year) to support a general National Research Internet (NRI) have yet to be argued for. 13.1.2 - NSFNet Status The NSFNet consists of a national backbone network coupled to a number of regional networks (currently 11) that link research institutions with each other and with the backbone. The backbone is currently operating with 56 Kbit/sec lines and expects to move to full Tl (1.5 Mbit/set) links in July this summer. The experimental network is being managed by MERIT, Inc. (a nonprofit consortium of 8 Michigan universities set up long ago in conjunction with the development of the Michigan timesharing system for IBM mainframes). In fact, IBM and MCI are research partners in the NSFNet project. The backbone will link the 6 current NSF supercomputer sites and 7 regional academic computing networks. NSF has put up $14M over 5 years and the State of Michigan has put up $5M from a strategic fund. IBM and MCI are contributing "materials and services". There are longer term goals of making the configuration of the network more dynamically controllable depending on traffic needs and upgrading lines to DS3 service (45 Mbit/sac). The regional networks are not funded in this grant and many are facing funding crunches later this calendar year. Many regions are interstate and have bigger political barriers to cooperation, as compared to intrastate regional networks like BARRNet or NYSERNet -- although even these are coming under severe funding pressure. Some of the telephone companies (AT&T and MCI) think these services should be paid from from end user fees while many of the university people feel the regional intercampus links should be like the "national highway system" and supported out of general funds. An interestlng suggestion was made that an analog to COMSAT might be set up for the NRI, since it will involve a close (nonprofit?) cooperation between federal, state, industrial, and academic interests. 13.1.3 - ARPANET Link We continue our advantageous connection to the Department of Defense's ARPANET, managed by the Defense Communications Agency (DCA). A recent map of the ARPANET topology is shown in Figure 2. This connection has been possible because of the long-standing basic research effort in Al within the Knowledge Systems Laboratory that is funded by DARPA. Until the advent of NSFNet, ARPANET is the primary link between SUMEX and other university and AIM machine resources, including the large Al computer science community supported by DARPA. Major changes are underway in the allocation and management of ARPANET resources. DARPA is seeking to control the funding it allocates to network operations and does not see itself as having the mandate of organizing or running an NRI. Effective May 1 of this year, the whole southern ARPANET route (l/2 of the total line miles!) was eliminated, forcing 6 major Texas universities to seek an alternative link with the Texas regional network (SesquiNet) for national E. H. Shortliffe 76 ARPANET Geographic Map, 31 January 1988 lull1 USCIZ -- IYP w IUU 0 cmw A Cl30 r*c 1 Details of Technical Progress 5P41 -RR007851 5 communications through NSFNet. As a result of this cutback, the remaining 56 Kbit/sec ARPANET is becoming even more congested. By this summer, the ARPANET will move to a Ti backbone linking Boston, New York City, Washington DC, Pittsburgh, Chicago, Denver, San Francisco (NASA Ames), Los Angeles, and San Diego -- essentially 1 diagonal line across the US. Locations left off of this route will have to seek connectivity through NSFNet. The reduced coverage and higher speed but cheaper fiber optics lines will significantly reduce medium term network costs. "Multiple network connections" will be eliminated. In the longer term, DARPA's role will apparently be limited to DOD communications research and support of its contractor community. Thus, the ARPANET will once again become very restrictive in terms of access and use. DARPA expects to construct a Defense Research Internet (DRI) over the next 3 years (in cooperation with DCA and SDI), with the current ARPANET disappearing after that. There is an explicit assumption in what DARPA is doing that the NSFNet or some other NRI will fill in the gaps. Thus, another "lead agency" will be needed. For now, NSF is serving in that role but there is no commitment by them or others that that will continue. On the operational front, as a member of the ARPANET group, we have an obligation to help with certain network management tasks. For instance, we assisted ARPANET management with installing an additional 56 KB trunk line from our C/30 IMP to a new IMP at the nearby AMES Research Center (NASA). The addition provides an improved network topology in this area. We are working with the ARPANET group to reduce line costs and improve service in other respects also. We still depend heavily on the ARPANET to provide terminal communications service for many of our users as well as file transfer and electronic mail service. 13.1.4 - TELENET Link The TELENET remains the primary connection mode for those national users without access to ARPANET services. As discussed below, we now have a more general interface to this network that provides an additional password check and outbound as well as inbound services. Because TELENET, like other commercial networks, is oriented to "local-echo" schemes, they do not put a high priority on echo-response across their network. Thus remote users are substantially inconvenienced when using programs requiring close interaction. It appears that our switch from TYMNET to TELENET (made about two years ago) has improved service slightly in this respect. 13.1.5 - X.25IEthernet Link The SUMEX-AIM transition to a distributed computing environment and our heavy use of Ethernet motivated our need for a connection between our Ethernet system and the Public Data Networks (specifically TELENET). Because our need involves Ethernet connections to other X.25 based hosts and Packet Assemblers - Disassemblers (PADS), sometimes called Terminal Interface Processors, as opposed to connections between two Ethernets via an X.25 link, we needed a device that provides protocol translation. Such a package would provide SUMEX with both an inbound and outbound capability relative to TELENET and users on SUMEX would be able to access the large variety of hosts and services on the PDNs (such as NLM and Dialog) in a simple and reliable manner. Though the high level protocols for file transfer and mail exchange are developing slowly in the X.25 environment, some progress is being made, so a general purpose interface to these networks is an important asset. E. H. Shortliffe 78 5P41 -RROO785- 15 Details of Technical Progress A substantial amount of effort went in to selecting and implementing this equipment. While a large number of vendors claimed to have TCP/IP-to-X.25 interfacing unit, these were generally encapsulating devices intended to connect two TCP/IP networks via an X.25 link. Instead, we needed a device which would provide protocol translation between the X.3 style terminal sessions of the X.25 networks and TCP/IP TELNET support. After a long and frustrating search among potential vendors (e.g., SUN, Bridge, and ACC), we located a suitable system manufactured by Develcon. The experimental installation of the Develcon Network gateway has provided us with the capability to connect our Ethernet system to a variety of X.25 based networks (for terminal session support). This project has been pursued in conjunction with Stanford's Lane Medical Library and the Stanford University networking group (SUNet). We have implemented a four-way gateway, connecting our Ethernet system with two X.25 point-to-point lines and the TELENET network (see Figure 8). The result has been a more flexible connection to the TELENET system (at the X.25 level) which gives users the ability to choose a variety of destinations on our Ethernet system when coming into our facility via TELENET. Unlike the simple X.25 PAD which this equipment replaces, the Develcon unit is able to provide both inbound and outbound services. Our collaboration with Stanford's Lane Medical Library has resulted in this device providing access to the Tandem bas6d Lane On-line Information System (LOIS) via Tandem's X.25 package (no Ethernet package is available), and a Lane sponsored X.25 leased line to the Bibliographic Retrieval Service (BRS) Inc's medical information system (Colleague). Though this collaboration with the Lane Medical Library has been rather informal, it has proven valuable for both groups. As library science becomes heavily involved with computer based information services the interfacing of their systems with other computer based groups becomes very important. For instance, members of the SUMEX-AIM based Medical Information Sciences (MIS) program are able to make direct transfers of information from BRS to systems on our Ethernet. The system supplier, Develcon Inc., has recently announced a product based on the work we did with them on this gateway. Further vendor development is needed to support improved accounting for system access and use and for user authentication and access control mechanisms. 7 3.2 - Microcomputer Ne two&s We connected our Apple Macintosh computers in 2 buildings with AppleTalk and PhoneNet network products. More significantly, we integrated them with the rest of our equipment by connecting the microcomputer networks to the campus Ethernet networks using Kinetics f-z&Path gateways, a commercial spin off resulting from the SUMEX work on the SEAGATE gateway. Software written at Columbia University, Stanford, and elsewhere, makes it possible for a Macintosh to share a VAX file server with the Lisp machines and to access hosts on the ARPA internet as a first-class workstation. 79 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 13.3 - Local Area Networks - LAN's In the past, we have developed local area networking systems to enhance the facilities available to researchers. Much of this work has centered on the effective integration of distributed computing resources in the form of mainframes, workstations, and servers. Network gateways and terminal interface processors (TIP's) were developed and extended to link our environment together. We are purchasing special purpose gateways to interface other equipment as needed too. This year we added several Kinetics Applenet-to-Ethernet gateways to support our Mac II's, LaserWriters, and Micro Explorers, and to link them with the LAN. The original Applenet-to-Ethernet gateways were developed at SUMEX-AIM. Kinetics licensed this software/hardware from Stanford University and used it as a basis for its product. A diagram of our local area network system is shown in Figure 9 and the following summarizes our LAN-related development work. 13.3.1 - Ethernet Gateways In our heterogeneous network environment, in order to provide workstation access to file servers, mail servers, and other computers within the Stanford local area network, it is necessary to able to route multiple networking protocols through the network gateways. As reported last year, the SUMEX gateways support PUP, Xerox NS, Symbolics/Texas-Instrument CHAOSNET, and the TCP/IP protocols. This support not only provides the routers necessary to move such packets among the subnetworks, but also other miscellaneous services such as time, name/address lookup, host statistics, boot strap support, address resolution, and routing table broadcast and query information. This year a device driver for the Micom-lnterlan Ethernet interface was written. This interface has adequate on-board multibus memory to cache up to five 1514 byte input packets (the largest allowed on the network) and more than twenty small packets where "small" is less than 320 bytes. In contrast the older 3COM multibus interface has a fixed two buffer input cache. The larger packet cache minimizes dropped packets at the interface itself, and improves the performance of large file transfers that depend upon the ability to successfully send several back-to-back packets. In fact, we could not run NFS through our gateways with the 3COM interfaces unless we set the maximum number of back-to-back packets that the SUN-3 file server sends to two. With the Micom- lnterlan interface, we could change this number to the default which is six, run NFS through the gateways, and achieve maximal performance. 13.3.2 - Terminal Interface Processors SUMEX-AIM has five TIPS, and in previous years we spent a reasonable amount of time maintaining and augmenting this software. Over the past year, this software has remained stable, and we plan on using vendor supported EtherTIP software for future TIPS. This software originated at SUMEX-AIM and has been commercialized by Cisco Systems, Incorporated, which licensed the EtherTIP/Gateway code from Stanford University. Currently, one of our TIPS has ten dial-in ports and is used extensively by the local SUMEX-AIM community for dial-in access to the DEC 2060 and our VAX 1 I/750 servers from home during off hours. The four remaining TIPS are used to access various mainframes during work hours, and one of these runs the Cisco TCP/IP TIP code. As we begin to migrate from the 2060 to our distributed resource, we will in parallel replace our older PUP based TIP software with the above software. We currently run PUP in the majority of our TIPS because the 2060 more efficiently processes PUP, and most of our users login to the 2060 from these TIPS. E. H. Shortliffe 80 5P41 -RR00785- 15 Details of Technical Progress 14 - Printing Services Laser printers have become essential components of the work environment of the SUMEX-AIM community with applications ranging from scientific publications to hardcopy graphics output for ONCOCIN chemotherapy protocol patient charts. We have done much systems work to integrate laser printers into the SUMEX network environment so they would be routinely accessible from hosts and workstations alike. This expertise has been widely shared with other user groups in the AIM community and beyond. SUMEX operates 7 medium-speed (8-20 pages per minute) lmagen laser printers, 2 low-speed (~3 ppm) Apple laser printers, and 1 low-speed (~3 ppm) Xerox laser printer. Each of the lmagen printers includes an emulator for a line printer, a daisy wheel printer, a Tektronix plotter, and a typesetter (using the Impress language). Additionally, the two lmagen 3320 printers implement the PostScript typesetter language (also implemented by the Apple LaserWriters) required for printing Macintosh documents. The Xerox printer (an 8046) interprets the InterPress typesetter language. In total, the laser printers printed about half a million pages of output during the year. Most of the printout was simple text, followed in quantity by formatted text in Impress format, Impress-format drawings, and screen dumps. About 16,000 pages each of PostScript-format drawings and formatted text were printed on the Apple LaserWriters and lmagen 3320's (an eightfold increase over last year). We were one of a small number of sites to beta test Image&s implementation of the PostScript typesetter language. We submitted a number of bug reports and all the bugs we found were fixed in short order. (Imagen went on to become the first printer manufacturer to ship a non-Adobe implementation of PostScript.) In consideration of our help with the test, lmagen transferred ownership of the test hardware to our lab. Since the proliferation of Macintosh computers demands higher-speed printing than the Apple LaserWriter can provide, another lmagen 121300 was upgraded to model 3320 (configured for PostScript) and installed in the other building inhabited by SUMEX and KSL users. lmagen granted a special discount on this upgrade in consideration of our dissatisfaction with the longevity of the 12/300 printers purchased earlier. 15 - General User Software We have continued to assemble (develop where necessary) and maintain a broad range of user support software. These include such tools as language systems, statistics packages, vendor-supplied programs, text editors, text search programs, file space management programs, graphics support, a batch program execution monitor, text formatting and justification assistance, magnetic tape conversion aids, and user information/help assistance programs. A particularly important area of user software for our community effort is a set of tools for inter-user communications. We have built up a group of programs to facilitate many aspects of communications including interpersonal electronic mail, a "bulletin board" system for various special interest groups to bridge the gap between private mail and formal system documents, and tools for terminal connections and file transfers between SUMEX and various external hosts. Examples of work on these sorts of programs have already been mentioned in earlier sections, particularly as they relate to extensions for a distributed computing environment. 81 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 At SUMEX-AIM we are committed to importing rather than reinventing software where possible. As noted above, a number of the packages we have brought up are from outside groups. Many avenues exist for sharing between the system staff, various user projects, other facilities, and vendors. The availability of fast and convenient communication facilities coupling communities of computer facilities has made possible effective intergroup cooperation and decentralized maintenance of software packages. The many operating system and system software interest groups (e.g., TOPS-20, UNIX, D-Machines, network protocols, etc.) that have grown up by means of the ARPANET have been a good model for this kind of exchange. The other major advantage is that as a by-product of the constant communication about particular software, personal connections between staff members of the various sites develop. These connections serve to pass general information about software tools and to encourage the exchange of ideas among the sites and even vendors as appropriate to our research mission. We continue to import significant amounts of system software from other network sites, reciprocating with our own local developments. Interactions have included mutual backup support, experience with various hardware configurations, experience with new types of computers and operating systems, designs for local networks, operating system enhancements, utility or language software, and user project collaborations. We have assisted groups that have interacted with SUMEX user projects get access to software available in our community (for more details, see the section on Dissemination on page 123). E. H. Shortliffe 82 5P41 -RR00785-15 Details of Technical Progress lll.A.2.5. Relevant Core Research Publications The following is a list of new publications and reports that have come out of our core research and development efforts over the past year. In addition, we include references to earlier reports that are discussed in the text above. HPP 80-29 H. Penny Nii; An Introduction to Knowledge Engineering, Blackboard Model and AGE, March 1980, 45 pages KSL 86-19 J.P. Rice: Poligon, A System for Parallel Problem Solving, April 1986. To appear in: DARPA proceedings, Asilomar 7986. 16 pages KSL 86-20 JR. Delaney; Multi-System Report Integration Using Blackboards, March 1986. Submitted for publication to: 7986 American Control Conference. 12 pages KSL 86-38 STAN-CS-87-1147. Barbara Hayes-Roth, M. Vaughan Johnson Jr., Alan Garvey, and Micheal Hewett; A Modular and Layered Environment for Reasoning about Action, April 1987. To appear in: The Journal of Artificial Intelligence in Engineering, Special Issue on Blackboard Systems, October 7986. 63 pages KSL 86-41 H. Penny Nii; CAGE and POLIGON: Two Frameworks for Blackboard-based Concurrent Problem Solving, April 1986. To appear in: DARPA Proceedings, Asilomar 7986 8 pages [Superceded by KSL 87-711 KSL 86-62 STAN-CS-87-1175. David C. Wilkins, William J. Clancey, and Bruce G. Buchanan; Using and Evaluating Differential Modeling in Intelligent Tutoring and Apprentice Learning Systems, January 1987. To appear in: Intelligent Tutoring Systems: Lessons Learned, Lawrence Erlbaum Publishers, 7987. 37 pages K!X 86-63 David C. Wilkins; Knowledge Base Debugging Using Apprenticeship Learning Techniques, October 1986. 15 pages KSL 86-69 STAN-CS-86-1136. Harold Brown, Eric Schoen, and Bruce Delagi; An Experiment in Knowledge-Base Signal Understanding Using Parallel Architectures, October 1986. To appear in: Parallel Computation and Computers for Al, J.S. Kowalik Editor, Kluwer Publishers. 39 pages KSL 87-27 (Journal Memo) Gregory F. Cooper; Probabilistic Inference Using Belief Networks Is NP-Hard, August 1987. 23 pages KSL 87-28 (Journal Memo) Eric J. Horvitz; A Multiattribute Utility Approach to Inference Understandability and Explanation, September 1987. 34 pages KSL 87-34 (Working Paper) Russell T. Nakano; Experiments with a Knowledge-Based System on a Multiprocessor: Preliminary Airtrac-Lamina Qualitative Results, June 1987. 72 Pages 83 E. H. Shortliffe Details of Technical Progress 5P41-RR0078515 KSL 87-36 Lawrence J. Selig; An Expert System Using Numerical Simulation and Optimization To Find Particle Beam Line Errors, May 1987. 39 pages KSL 87-38 Naomi S. Rodolitz; Guidon Manage - Tutoring for Strategic Knowledge, June 1987. 43 pages KSL 87-39 (Journal Memo) Glenn D. Rennels, Edward H. Shortliffe, Frank E. Stockdale, Perry L. Miller; Updating an expert knowledge base as medical knowledge evolves: Examples from oncology management, June 1987. To appear in Proceedings of the AAMSI Congress 87. 6 pages KSL 87-40 Alan Garvey and Barbara Hayes-Roth; Implementing Diverse Forms of Control Knowledge in Multiple Control Architectures, June 1987. 32 pages KSL 87-41 Thierry Barsalou; An Object-Based Interface to a Relational Database System, June 1987. 10 pages KSL 87-43 STAN-CS-87-1166. Hirorhi G. Okuno and Anoop Gupta; Parallel Execution of OPS5 in QLISP, June 1987. Shorter version to appear in: Proceedings of the Fourth Conference on Artificial !ntelligence Applications (CA/A-881, IEEE, March 7988. 18 pages KSL 87-44 STAN-CS-87-1178. Gregory T. Byrd, Russell Nakano, and Bruce A. Delagi; A Dynamic, Cut-Through Communications Protocol with Multicast, August 1987. 23 pages KSL 87-45 (Journal Memo) David Heckerman and Holly Jimison; A Perspective on Confidence and Its Use in Focusing Attention during Knowledge Acquisition, July 1987. To appear in: Proceedings of AAAI 87. 10 pages KSL 87-46 (Journal Memo) Michael P. Wellman and David E. Heckerman; The Role of Calculi in Uncertain Reasoning, July 1987. To appear in: Proceedings of AAAI 87. 12 pages KSL 87-48 (Journal Memo) Gregory F. Cooper; Computer-Based Medical Diagnosis Using Belief Networks and Bounded Probabilities, February 1988. To appear in Topics in Medical Artificial Intelligence edited by Perry Miller. 17 pages KSL 87-49 (Journal Memo) Mark E. Frisse and Gio Wiederhold; Retrieving Information from Hypertext Systems, August 1987. 14 pages KSL 87-50 (Journal Memo) Mark E. Frisse; Searching for Information in a Hypertext Medical Handbook: The Washington University Dynamic Medical Handbook Project, August 1987. 19 pages KSL 87-51 (Journal Memo) Gregory F. Cooper; Expert Systems Based on Belief Networks - Current Research Directions, February 1988. 21 pages E. H. Shortliffe 84 5P41 -RR00785-15 Details of Technical Progress KSL 87-52 Curtis P. Langlotz; Advice Generation in an Axiomatically-Based Expert System, August 1987. To appear in: Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care, Washington, DC, 7987. 8 pages KSL 87-53 (Journal Memo) David Heckerman and Eric J. Horvitz; On the Expressiveness of Rule-Based Systems for Reasoning with Uncertainty, August 1987. To appear in: Proceedings of AAAI, Vol. 7, July, 7987. 7 pages KSL 87-54 Joshua Lederberg; How DENDRAL Was Conceived and Born, August 1987. To appear in: Proceedings of ACM Symposium on the History of Medical Informatics, National Library of Medicince, November 7987. 19 pages KSL 87-57 STAN-CS-87-1184. Hiroshi Okuno, Nobuyasu Osato and lkuo Takeuchi; Firmware Approach to Fast Lisp Interpreter, September 1987. To appear in: Proceedings of Twentieth Annual Workshop on Microprogramming (MICRO-201, ACM, December 7987. 25 pages KSL 87-58 William J. Clancey; Knowledge Engineering Methodology: An Annotated Bibliography of NEOMYCIN Research, September 1987. To appear in: R. Nossum (ed.), Lecture notes in Computer Science - ACAl'87, Springer-Verlag, 7988. 10 pages KSL 87-59 Janet McLaughlin; Utility-Directed Presentation of Simulation Results, December 1987. 57 pages KSL 87-60 Richard M. Keller; Defining Operationality for Explanation-Based Learning, October 1987. To appear in: Artificial Intelligence Journal. 19 pages KSL 87-61 STAN-CS-87-1188. Russell Nakano and Masafumi Minami; Experiments with a Knowledge-Based System on a Multiprocessor, October 1987. 47 pages KSL 87-62 John A. Brugge and Bruce G. Buchanan; Evolution of a Knowledge-Based System for Determining Structural Components of Proteins, October 1987. 26 pages KSL 87-63 (Journal Memo) Leslie Elaine Perreault; Automatic Test Case Generation by Modeling Patient States and Physician Actions, October 1987. 30 pages KSL 87-64 (Journal Memo) Eric J. Horvitz; Problem-Solving Design: Reasoning about Computational Value, Tradeoffs, and Resources, November 1987. To appear in: Proceedings of the National Aeronautics And Space Administration Artificial Intelligence Forum, Mountain View, CA. 19 pages KSL 87-65 STAN-CS-87-1189. Bruce A. Delagi, Nakul Saraiya, Sayuri Nishimura, and Greg Byrd: Instrumented Architectural Simulation, November 1987. 7 pages KSL 87-67 Barbara Hayes-Roth; Dynamic Control Planning in Adaptive Intelligent Systems, November 1987. 7 pages 85 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 KSL 87-73 Stephen Barnhouse; Knowledge Base Tours: Introducing a Knowledge Based System to a Novice User, December 1987. 15 pages KSL 88-01 M. Vaughan Johnson and Barbara Hayes-Roth; Learning to Solve Problems by Analogy, March 1988. 15 pages KSL 88-02 H. Penny Nii, Nelleke Aiello, and James Rice; Frameworks for Concurrent Problem Solving: A Report on Cage and Poligon, March 1988. 28 pages KSL 88-04 J. P. Rice: Problems with Problem-Solving in Parallel: The Poligon System, January 1988. 21 pages KSL 88-06 (Thesis) Mark Alan Musen; Generation of Model-Based Knowledge-Acquisition Tools for Clinical-Trial Advice Systems, January 1988. 293 pages [ONLY ABSTRACT AVAILABLE] KSL 88-09 (Working Paper) Bruce G. Buchanan and Reid G. Smith; Fundamentals of Expert Systems, March 1988. 33 pages KSL 88-10 Gregory T. Byrd and Bruce A. Delagi; A Performance Comparison of Shared Variables vs. Message Passing, February 1988. To appear in: May 7988 ISI Supercomputing Conference Proceedings. 15 pages KSL 88- 11 Richard M. Keller; Operationality and Generality in Explanation-Based Learning: Separate dimensions or opposite endpoints?, February 1988. To appear in: Proceedings of the AAAI Symposium on Explanation-Based Learning, March 7988, Stanford, CA. 5 pages KSL 88-12 Clifford E. Wulfman, Ellen A. Isaacs, Bonnie Lynn Webber, and Lawrence M. Fagan; Integration Discontinuity: Interfacing Users and Systems, February 1988. 12 pages KSL 88-13 Eric J. Horvitz, John S. Breese, and Max Henrion; Decision Theory in Expert Systems and Artificial Intelligence, February 1988. To appear in: Journal of Approximate Reasoning, Special Issue on Uncertainty in Artificial Intelligence, July 7988. 62 pages KSL 88-15 Tony Confrey and Fancois Daube; GS2D: A 2D Geometry Systems, March 1988. 15 pages KSL 88-16 Mark A. Musen; Conceptual Models of Interactive Knowledge-Acquisition Tools, March 1988. 9 pages KSL 88-19 Barbara Hayes-Roth, Micheal Hewett, M. Vaughan Johnson, and Alan Garvey; ACCORD: A Framework for a Class of Design Tasks, March 1988. 12 pages E. H. Shortliffe 86 5P41 -RR007851 5 Details of Technical Progress KSL 88-20 Barbara Hayes-Roth, Rattikorn Hewett, and Adam Seiver; Diagnostic Explanation using Generic Models, March 1988. IO pages KSL 88-21 Alan Garvey and Barbara Hayes-Roth; An Empirical Analysis of Explicit vs. Implicit Control Architectures, March 1988. 20 pages KSL 88-22 Micheal Hewett and Barbara Hayes-Roth; Real-Time I/O in Knowledge-Based Systems, March 1988. IO pages KSL 88-23 Robert Schulman and Barbara Hayes-Roth; Plan-Based Construction of Strategic Explanations, March 1988. 13 pages KSL 88-24 James F. Brinkley; The Potential for Intelligent Three-Dimensional Ultrasound, March 1988. To appear in: Chervenak, F.A., Isaacson, G., Campbell, S. teds.), Textbook of Ultrasound in Obstetrics and Gynecology, 1988. 28 pages KSL 88-25 (Working Paper) Greg Byrd: Modelling a Bus-Based Multiprocessor Using the CARE Simulation System, March 1988. 11 pages KSL 88-26 Mark A. Musen; Generation of Knowledge-Acquisition Tools from Clinical-Trial Models, March 1988. To appear in: Proceedings of Medical Informatics, Europe 1988. Oslo, Norway, August 1988. 6 pages KSL 88-27 (Working Paper) H. J. Suermondt and Gregory F. Cooper; Updating Probabilities in Multiply Connected Belief Networks, March 1988. 9 pages KSL 88-28 Cooper, G.F.; A method for using belief networks as influence diagrams, April 1988. KSL 88-34 (Thesis) lsabelle de Zegher-Geets; IDEFIX: Intelligent Summarization of a Time- Oriented Medical Database, June 1987. 99 pages KSL 88-38 Heckerman, D.E.; An empirical comparison of three scoring schemes, May 1988. KSL 88-TBA Chavez, R.M. and Cooper, G.F.; KNET: Integrating hypermedia and normative Bayesian modeling. KSL B8-TBA Lehmann, H., Knowledge acquisition for probability-based expert systems. Other Outside Articles: Cooper, G.F., Expert systems based on belief networks -- Current research directions, Journal of Applied Statistical Models and Data Analysis, 4, 1988. Cooper, G.F., invited commentary on: Lauritzen, S. and Spiegelhalter, D., Local computations with probabilities on graphical structures and their application to expert systems, Journal of the Royal Statistical Society, B, 50, 1988. 87 E. t-l. Shortliffe Details of Technical Progress 5P41 -RR00785-15 Horvitz, E.J., Reasoning under varying and uncertain resource limitations, in: Proceedings of the National Conference on Artificial Intelligence, Minneapolis, MN, August, 1988. Horvitz, E.J., Breese, J.S., and Henrion, M., Decision theory in expert systems and artificial intelligence, Journal of Approximate Reasoning, 1988. E. H. Shortliffe 88 5P41 -RR00785-15 Details of Technical Progress MA. 2.6. Resource Equipment The SUMEX-AIM resource is a complex, integrated facility comprised of mainframes, workstations, networks, and servers illustrated in Figures 3 - 9. A key role of the SUMEX-AIM resource is to continue to evaluate workstations as the technology is changing rapidly. This evaluation includes new hardware and software, 1) to provide superior development and execution platforms for Al research, and 2) to support the ancillary "office environment" (presently carried out on the DEC 2060, which is being phased out). Thus far no single workstation has materialized that provides all the services we would like to see in support of either or both of these missions. This means that for the foreseeable future, we will utilize a multiplicity of machines and software to address the needs of the projects. Systems based on the Motorola 68020 chip (e.g., SUN Microsystems or Apple Macintosh II workstations), the Intel 80286 and 80387 chips (e.g., IBM PS/l-4 machines), and other newer architectures such as reduced instruction set computer (RISC) chips, have Lisp benchmark data rivaling the performance of existing, specially microcoded Lisp machines (see Appendix B). It is still early to predict how this "race" will ultimately turn out and software environments play an equally important role with raw hardware speed in the decision. For now, the Lisp software environments on the "stock" machines are not nearly so extensively developed as on Lisp machines and conversely, the routine computing environments of Lisp machines (text processing, mail, spreadsheets, etc.) lag the tools available on stock UNIX machines. In earlier year's we experimentally tried increasing usage of TI and Xerox Lisp machines (purchased as Al research platforms) for text editing, electronic mail, and document formatting with considerable success (although many of these tools were only tested in a prototype form and were not widely distributed). In addition, the use of expensive Lisp machines for routine computing and office applications impacts their availability as research tools. We had been seeking an integration of both the Lisp machine and stock machine worlds. As discussed extensively in the progress section on Core Systems Research (see Page 39) these two capabilities came together as never before with the Macintosh II and microExplorer coprocessor systems. 1 - Purchases This Past Year The SUMEX 2060 hardware continues to be stable and the relatively small amount of SUMEX-AIM money for new purchases has been concentrated on experimental workstations and server equipment needed for distributed system development. These purchases are paced carefully with the developments of higher performing, more compact, and lower cost systems. The NIH-funded purchases this past year are summarized below. For the most part, these represent evaluation units used to review the suitability of particular types of equipment preparatory to a larger volume purchase from non-NIH funding sources, as detailed earlier starting on Page 39). 1. Apple Mac-II 2. Rodime Disk Drive 100 Megabyte Disk Drive (for MAC-II) 3. E-Machines Big Picture Display (for MAC-II) 4. U.S. Robotics 9600 Baud Modems (2 ea). 5. Develcon X.25IEthernet Gateway (shared cost with others) 89 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 DEC KLlO-E Central Processor 2M words of memory, Cache 01820 1 RH20 I I/O bus I DEC ARP Inter L. c I. jhortliffe MassBus RH20 I RH20 Mass&s MassBus and Controller and Controller and Controller RH20 I 1 l/40 Front End MassBus and Controller I Console lTY t- UNIBUS 4 DEC DH-11 Line Scanners 64 lines total Figure 3: SUMEX-AIM DEC 2060 Configuration 90 5P41 -RR007851 5 Sun 4/280 Central Processor 32 Mbytes Memory I VMEbus Details of Technical Progress I Hitachi I Disk Drive 1 Figure 4: SUMEX-AIM SUN-4 Configuration 91 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 Console Sun 31180 Central Processor 8 Mbytes Memory Ethernet - 10 Mbit Interface `/MEbus SCSI H Cartridge Controller Tape Drive I Tape Controller Fujitsu 6250 bpi Tape Drive Disk Controller Fujitsu Eagle 414 Mbyte Disk Drive Fujitsu Eagle 414 Mbyte Disk Drive Figure 5: SUMEX-AIM Sun-3 File Server Configuration E. H. Shortliffe 92 5P41 -RR00785-15 Details of Technical Progress X,Y busses Xerox 8000 Central Processor 1.5 Mbytes Memory IOP bus Console I/O Processor - 16 Kbytes Memory Shugart SAlOOO 1 Floppy Disk Drive I CDS Trident Disk Drive CDS Trident Disk Drive CDS Trident Disk Drive Figure 6: SUMEX-AIM Xerox File Server Configuration 93 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 MSSBIJS I UMBUS I Fujitsu Eagle 414 Mbyte Disk Drive Fujitsu Eagle 414 Mbyte Disk Drive Disk Controller Fujitsu Eagle 414 Mbyte Disk Drive CDC 256 Mbyte Removable-Media Disk Drive Emulex Controller KW-llV3dy - 600/16QO bpi Tape Drive DEC RK07 - Dlsk Controller and Drive Ethernet UNBUS Interface 10 Mbit DZ-11 Line Scanner 8 Lines Console DEC VAX 1 t/760 -l-l-Y Central Processor I I 2 Mbytes Memory 7-u 58 Tape Drive L- Massl Fujitsu Eagle 414 Mbyte Disk Drive 1 Emulex Disk Controller I I I I DZ-11 I l-l l-l Line Scanner I I I 8 Lines I Figure 7: SUMEX-AIM VAX File Server Configuration E. H. Shortliffe 94 5P41 -RR00785-15 Details of Technical Progress X.25 card (32 software ports] X.25 card (32 software ports) Develcon Gateway Core TCP/IP card (64 software ports) 10 Mbit Public Data Network Lane Medical Library BRS Inc. LOIS System Medline Service SUMEX-AIM Ethernet Figure 8: SUMEX-AIM Develcon X.25/TCP-IP Gateway Configuration 95 E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 Margaret Jacks Hall Xerox 1100's Xerox laser printers Other CSD Equipment Medlcal Center SUMEX Machine Room Electrical Engineering Pine Hall SUMEX 2060 SUMEX 2020 Xerox 1106 Xerox 8037 file server Vax 750 file server Sun file sewer IWT Ells's Xerox 1165 lmagen laser printer G w 1 I Medlcal School Office Building Symbolic Systems Resources Group Medical Computer Science Group Xerox 1100's G H-P 9636's n Explorers t-4 lmagen laser printers G Xerox laser printer Suns Sun file server q G Apple Macintoshes Apple laser printer Ether TlPs I - I I I P R Whelan Building (Welch Road) HPP and HELIX Campus "llnk net" /RJ Repeater O G Gateway 4 G 6- G Xerox 1100's n Explorers Symbolics 3600's NlT Elis's Sun t-m- Sillcon Graphics Iris Vax 750 file server Xerox 6033 file server lmagen laser printers Apple Macintoshes Apple laser printer Ether TIPS Xerox Alto Xerox 1132 E. H. Shortliffe Figure 9: SUMEX-AIM Ethernet Configuration 96 5P41-RR0078515 Details of Technical Progress WA. 2.7. Training Activities The SUMEX resource exists to facilitate biomedical artificial intelligence applications. This user orientation on the part of the facility and staff has been a unique feature of our resource and is responsible in large part for our success in community building. The resource staff has spent significant effort in assisting users to gain access to the SUMEX-AIM resources at Stanford and use it effectively as well as in assisting AIM projects in designing their own local computing resources based on SUMEX experience. We have also spent substantial effort to develop, maintain, and facilitate access to documentation and interactive help facilities. The HELP and Bulletin Board subsystems have been important in this effort to help users get familiar with the computing environment. We have regularly accepted a number of scientific visitors for periods of several months to a year, to work with us to learn the techniques of expert system definition and building and to collaborate with us on specific projects. Our ability to accommodate such visitors is severely limited by space, computing, and manpower resources to support such visitors within the demands of our on-going research. A tutorial was held in January 1988 on the Parallel Computing Architectures Project multiprocessor simulation system (see Page 29). This two-day session was attended by representatives from the DOD, NASA and Boeing, and included descriptions of the CARE/SIMPLE system, as well as the LAMINA programming interface. The attendees received instruction in use of the system for making measurements of the performance of various simulated multiprocessor applications. Finally, the training of graduate students is an essential part of the research and educational activities of the KSL. Based largely on the SUMEX-AIM community environment, we have initiated two unique, special academic degree programs at Stanford, the Medical Information Science program and the Masters of Science in Al, to increase the number of students we produce for research and industry. A number of students are pursuing interdisciplinary programs and come from the Departments of Engineering, Mathematics, Education, and Medicine. The Medical Information Sciences (MIS) program continues to be one of the most obvious signs of the local academic impact of the SUMEX-AIM resource. The MIS program received recent University approval (in October 1982) as an innovative training program that offers MS and PhD degrees to individuals with a career commitment to applying computers and decision sciences in the field of medicine. In Spring 1987, a University-appointed review group unanimously recommended that the degree program be continued for another five years. The MIS training program is based in the School of Medicine, directed by Dr. Shortliffe, co-directed by Dr. Fagan, and overseen by a group of six University faculty that includes two faculty from the Knowledge Systems Laboratory (Profs. Shortliffe and Buchanan). The specialized curriculum offered by the new program is intended to overcome the limitations of previous training options. It focuses on the development of a new generation of researchers with a commitment to developing new knowledge about optimal methods for developing practical computer-based solutions to biomedical needs. The program accepted its first class of four trainees in the summer of 1983 and has now reached its steady-state size of approximately twenty-two graduate students. The program encourages applications from any of the following: . medical students who wish to combine MD training with formal degree work and research experience in MIS; 97 E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 . physicians who wish to obtain formal MIS training after their MD or their residency, perhaps in conjunction with a clinical fellowship at Stanford Medical Center: . recent BA or BS graduates who have decided on a career applying computer science in the medical world; . current Stanford undergraduates who wish to extend their Stanford training an extra year in order to obtain a "co-terminus" MS in the MIS program; . recent PhD graduates who wish post-doctoral training, perhaps with the formal MS credential, to complement their primary field of training. In addition, a special one-year MS program is available for established academic medical researchers who may wish to augment their computing and statistical skills during a sabbatical break. As of Spring 1988, 55% of our trainees have previously received MD degrees and another 23% are medical students enrolled in joint degree programs. 27% are candidates for the MS degree, while the rest are doctoral students. The program has seven graduates to date, with several more expecting to complete degrees before the end of 1988. Except for the special one-year MS mentioned above, all students spend a minimum of two years at Stanford (four years for PhD students) and are expected to undertake significant research projects for either degree. Research opportunities abound, however, and they of course include the several Stanford AIM projects as well as research in psychological and formal statistical approaches to medical decision making, applied instrumentation, large medical databases, and a variety of other applications projects at the medical center and on the main campus. Several students are already contributing in major ways to the AIM projects and core research described elsewhere in this annual report. We are pleased that the program already has an excellent reputation and is attracting superb candidates for training positions. The program's visibility and reputation is due to a number of factors: . high quality students, many of whom publish their work in conference proceedings and refereed journals even before receiving their degrees: Stanford MIS students have won first prize in the student paper competition at the Symposium on Computer Applications in Medical Care (SCAMC) in 1985 and 1986, and have also received awards for their work at annual meetings of organizations such as the Society for Medical Decision Making, the American Association for Medical Systems and lnformatics (AAMSI), and the American Association for Artificial Intelligence (AAAI); . a rigorous curriculum that includes newly-developed course offerings that are available to the University's medical students, undergraduates, and computer science students as well as to the program's trainees; . excellent computing facilities combined with ample and diverse opportunities for medical computer science and medical decision science research: . the program's great potential for a beneficial impact upon health care delivery in the highly technologic but cost-sensitive era that lies ahead. E. H. Shortliffe 98 5P41 -RR00785-15 Details of Technical Progress The program has been successful in raising financial and equipment support from industry and foundations. It is also recipient of a training grant from the National Library of Medicine. The latter grant was recently renewed for another five years with a study section review that praised both the training and the positive contribution of the SUMEX-AIM environment. 99 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785 15 III .A. 2.8. Resource Operations and Usage 1 - Operations and Support The diverse computing environment that SUMEX-AIM provides requires a significant effort at operations and support to keep the resource responsive to community project needs. This includes the planning and management of physical facilities such as machine rooms and communications, system operations routine to backup and retrieve user files in a timely manner, and user support for communications, systems, and software advice. We spend significant time on new product review and evaluation such as Lisp workstations, terminals, communications equipment, network equipment, microprocessor systems, mainframe developments, and peripheral equipment. We also pay close attention to available video production and projection equipment, which has proved so useful in our dissemination efforts involving video tapes of our work. We continue to operate the primary elements of our equipment base in a generally unattended manner. Operations costs are kept to a minimum by utilizing a student staff for routine tasks. Senior members of this staff provide improvements to the operations procedures in addition to training and supervising new students. This has provided SUMEX with a cost effective operations scheme, contributed to the education of the students, and assisted students in meeting their obligations in undergraduate financial aid programs. While most of our equipment is concentrated in three computer equipment rooms, our move towards distributed computing has resulted in a substantial amount of equipment being installed in offices and student carrels. The planning of our project's area during the construction of the Medical School Office Building (described in last year's report) has made this distribution of equipment easier. 2 - Resource Usage Details The following data give an overview of various aspects of SUMEX-AIM central resource usage. There are 5 subsections containing data respectively for: 1. Overall resource loading data (page 101). 2. Relative system loading by community (page 102). 3. Individual project and community usage (page 105). 4. Network usage data (page 111). 5. System reliability data (page 113). For the most part, the data used for these plots cover the entire span of the SUMEX-AIM project. This includes data from both the KI-TENEX system and the current DECsystem 2060. At the point where the SUMEX-AIM community switched over to the 2060 (February, 1983), you will notice sharp changes in most of the graphs. This is due to differences in scheduling, accounting, and processor speed calculations between the systems. E. H. Shortliffe 100 5P41 -RR00785 15 2.1 - Overall Resource Loading Data Details of -Technical Progress 1974 1976 1976 1980 1982 1984 1986 1988 1990 Figure 10: Total CPU Hours Consumed by Month 101 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 2.2 - Relative System Loading by Community The SUMEX resource is divided, for administrative purposes, into three major communities: core ONCOCIN research, core Al research, and user projects based at the Stanford Medical School (Stanford Projects); user projects based outside of Stanford (National AIM Projects); and core system development efforts (System Staff). The initial resource management plan approved by the BRTP at the start of SUMEX specified that available system CPU capacity and file space resources were to be divided nominally between these communities in a 40:40:20 ratio. The "available" resources are those remaining after various monitor and community-wide functions (e.g., job scheduling, system overhead, network service, file space for subsystems, documentation, etc.) are accounted for. The monthly usage of CPU resources and terminal connect time for each of these three communities is shown in the plots in Figure 11 and Figure 12. Many of the national user projects have already moved to their own machines for intensive research computing and now use SUMEX mainly for communications and information access. Hence, one might expect the proportion of CPU and file space use by Stanford projects, as compared to non-Stanford groups, to continue to grow correspondingly, as has been the case in the past. However, this past year there has been a dramatic increase in the national use of SUMEX-AIM for remote communications and information access. Much of this has been through the "anonymous" file transfer mechanism for which we cannot identify the user by name. We will attempt to record more information about such information access connections in future years. E. H. Shortliffe 102 5P41 -RR00785-15 Details of Technical Progress 50 40 30 20 National Projects I I 8 I L , I 1974 1976 1978 I 980 t 982 i 984 1986 i 988 15 100 a0 - 60 - Stanford Projects 40 - 20 - 90 0 ; 1 I I I L I , 1974 1976 1978 1980 i 982 1984 1986 i 988 1990 60 - System Staff 30 - 20 - 10 - 1974 1976 1978 i 980 i 982 i 984 1986 t 988 1990 Figure 11: Monthly CPU Usage by Community 103 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 3000 2000 1000 0 1974 1976 1978 1980 1982 1984 1986 1988 1990 20000 Z E g 2 P 10000 0 + Stanford Projects 1974 1976 1978 1980 1982 1984 1986 1988 l! 8000 - 6000 - 30 1974 1976 1978 1980 1982 1984 1986 1988 1990 Figure 12: Monthly Terminal Connect Time by Community E. H. Shortliffe 104 5P41 -RR007851 5 Details of Technical Progress 2.3 - Individual Project and Community Usage The following histogram and table show cumulative resource usage by collaborative project and community during the past grant year. The histogram displays the project distribution of the total CPU time consumed between May 1, 1987 and April 30, 1988, on the SUMEX-AIM DECsystem 2060 system. Data include total CPU consumption by project (Hours), total terminal connect time by project (Hours), and average file space in use by project (Pages, 1 page = 512 computer words). These data were accumulated for each project for the months between May 1986 and April I 987. 105 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785-15 ATTENDING INTERNIST IlQMR MENTOR AIM Administratlon AIM Communications AIM Pilot Projects BB-ICU GUIDON/NEOMYCIN Med. Info. Scl. MOLGEN ONCOCIN PROTEAN RADIX/PENGUIN su Pilots SU Associates HELIX Al HPP Al KSL Associates Logic Al ONCOClN/MlS SUMEX Systems Stafl National AIM Collab Projects (Total 30.56%) * O.86 0.03 25.67 m 1.45 Stanford Collaborator Projects (Total 20.57%) IO.24 * 1.03 Core Al Research (Total 30.69%1 7.01 16.51 w 1.35 Core ONCOCIN Research (Total 8.94%) 0.94 Core Systems Research (Total 9.25%) 0.00 5.00 10.00 15.00 20.00 25.00 30.00 Percent CPU Tlme Used Figure 13: Cumulative CPU Usage Histogram by Project and Community E. H. Shortliffe 106 5P41 -RR00785-15 Details of Technical Progress Resource Use by individual Project - 5/87 through 4/88 National AIM Collaborator Community 1) ATTENDING "A Critiquing Approach to Expert Computer Advice" Perry L. Miller, M.D., Ph.D. Yale University School of Medicine CPU (Hours) 5.08 Connect File Space (Hours) Paws) 217.81 650 2) INTERNIST-QMR Project "Clinical Decision Systems Research Resource" Jack D. Myers, M.D. Randolph A. Miller, M.D. University of Pittsburgh 12.37 259.59 4693 3) MENTOR Project "Medical Evaluation of Therapeutic Orders" Stuart M. Speedie, Ph.D. University of Maryland Terrence F. Blaschke, M.D. Stanford University 9.62 4830.97 2000 4) AIM Pilot Projects PathFinder (Nathwani and Fagan) Dynamic Systems (Widman) Radiation Therapy (Kalet) 7.04 1002.52 1560 9.19 224.45 1433 0.02 1.42 4 5) AIM Communications AIM Mail-Only Users AAAI Management BIONET MCS Collaborators MOLGEN Collaborators Anonymous File/ Information Access Other 6.64 970.79 3567 5.50 1930.65 929 2.32 168.76 680 7.25 1796.62 1110 2.99 310.63 883 273.22 13593.75 233579 0.62 32.77 839 6) AIM Administration 0.32 44.53 2009 Community Totals 342.18 25385.27 253936 Figure 14: Table of Resource Use by Project 107 E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 Stanford Collaborator Community CPU (Hours) 2.65 Connect File Space (Hours) (Paws) 1512.22 237 1) 2) 3) 4) 5) 6) 7) 28.37 6359.48 1862 BBICU Project Lawrence M. Fagan, M.D., Ph.D. Department of Medicine Barbara Hayes-Roth, Ph.D. Computer Science Department GUIDON-NEOMYCIN Project William J. Clancey, Ph.D. Bruce G. Buchanan, Ph.D. Dept. Computer Science Medical Information Sciences Edward H. Shortliffe, M.D., Ph.D. Lawrence M. Fagan, M.D., Ph.D. Department of Medicine 45.76 MOLGEN Project 24.04 "Applications of Artificial Intelligence to Molecular Biology: Research in Theory Formation, Testing and Modification" Edward A. Feigenbaum, Ph.D. Peter Friedland, Ph.D. Charles Yanofsky, Ph.D. Depts. Computer Science/Biology ONCOCIN Project "Knowledge Engineering for Medical Consultation" Edward H. Shortliffe, M.D., Ph.D. Lawrence M. Fagan, M.D., Ph.D. Department of Medicine 54.31 PROTEAN Project Oleg Jardetzky School of Medicine Bruce Buchanan Computer Science Department 40.04 RADIX-PENGUIN Project Gio CM. Wiederhold, Ph.D. Depts. Computer Science/ Medicine 16.84 11821.74 4128 6523.52 7114 11759.34 7940 8795.76 4097 3150.80 9284 Figure 14: Table of Resource Use by Project, Continued E. l-i. Shortliffe 108 5P41 -RR00785-15 Details of Technical Progress 8) Stanford Pilot Projects REFEREE Project (Buchanan) 9) Stanford Associates Community Totals 6.72 11.55 --------- 230.29 1380.48 452 8064.25 3265 ----m--e_ --v--m- 59367.60 38380 Core Al Research CPU (Hours) 11.85 Connect File Space (Hours) (Pages) 1) 2) 3) 4) 5) 6) 7) 8) ABLE Proiect Robert S.-Engelmore, Ph.D. Computer Science Department Scott Cleat-water, Ph.D. Los Alamos National Laboratory Advanced Architectures Edward A. Feigenbaum, Ph.D. Computer Science Department Blackboard Architectures Barbara Hayes-Roth, Ph.D. Computer Science Department DART Project Michael R. Genesereth, Ph.D. Computer Science Department Financial Resource Management Bruce G. Buchanan, Ph.D. Thomas C. Rindfleisch Computer Science Department Intelligent Agents Project Michael R. Genesereth, Ph.D. Computer Science Department Knowledge Engineering Studies Bruce G. Buchanan, Ph.D. Dianna Forsythe, Ph.D. Computer Science Department Machine Learning Studies Bruce G. Buchanan, Ph.D. Computer Science Department 5372.31 570 105.23 32008.01 7284 53.33 10390.84 4636 6.10 3220.27 0 26.07 7449.24 2180 6.32 507.92 0 5.66 1313.59 236 34.23 9774.46 6134 Figure 14: Table of Resource Use by Project, Continued 109 E. H. Shortliffe Details of Technical Progress 5P41 -RR00785- 15 9) MRS Project Michael R. Genesereth, Ph.D. Computer Science Department 2.75 1336.60 0 10) SOAR Project Paul R. Rosenbloom, Ph.D. Information Sciences Institute University of Southern California 12.57 6979.07 907 11) Software Design Project H. Penny Nii Computer Science Department 1.13 124.99 12) Very Large Knowledge Bases Edward A. Feigenbaum, Ph.D. Richard Keller, Ph.D. Computer Science Department 13.33 3153.20 I 298 13) KSL Administration 55.15 13910.04 10090 14) KSL Associates 9.84 1645.64 1554 Community Totals 343.56 97186.17 35793 904 ------m-m ---------- ---a--- Core ONCOClN Research CPU Connect File Space (Hours) (Hours) Paws) 1) Core ONCOCIN and Medical Information Sciences Edward H. Shortliffe, M.D., Ph.D. Lawrence M. Fagan, M.D., Ph.D. Department of Medicine 100.07 23581.08 12068 Community Totals 100.07 23581.08 12068 Core Systems Research 1) SUMEX Staff R & D Thomas C. Rindfleisch Departments of Medicine and Computer Science CPU Connect File Space (Hours) (Hours) (Paws) 98.27 18547.71 10008 Figure 14: Table of Resource Use by Project, Continued E. H. Shortliffe 110 5P41 -RR00785-15 Details of Technical Progress 2) System Associates 5.24 252.56 ---w-m__- -----w____ 103.51 18800.29 1201 ----me_ 11209 Community Totals System Operations 1) System Operations 2) SUMEX Staff Opns & Mgmnt Community Totals 748.28 98864.23 Resource Grand Totals CPU Connect File Space (Hours) (Hours) Pages) 650.01 80316.52 3437 98.27 18547.7 1 I 0008 --------- ---------- ------- 13445 ----_--__ ---------- -----e--m ---------- 1867.89 323184.64 ------- ------- 364831 Figure 14: Table of Resource Use by Project, Concluded 2.4 - Network Usage Statistics The plots in Figures 15 and 16 show the monthly network terminal connect time for the public data networks and the INTERNET usage. The INTERNET is a broader term for what was previously referred to as ARPANET usage. Since many vendors now support the INTERNET protocols (TCP/IP) in addition to the ARPANET, which converted to TCP/IP in January of 1983, it is no longer possible to distinguish between ARPANET usage and Internet usage on our 2060 system. Similarly, after we switched to the Develcon gateway between the TELENET X.25 network and our TCP/IP Ethernet, we are not able to distinguish TELENET 2060 users from other Internet users. We are hoping to refine the accounting services available from the Develcon gateway so we will have a separate log of connection activity. 111 E. H. Shortliffe Details of Technical Progress 5P41 -RR007851 5 1400 , 1200 - 1000 - 800 - 600 - 400 - 200 - 0 ; I I I I I 1 I 1974 1976 1978 1980 1982 1984 1986 1980 1990 Figure 15: Public Data Network Terminal Connect Time 8000 1 1 6000 1 4000 1 2000 1 -- - i 1 i I 1 i I I 1974 1976 1978 1980 1982 1984 1986 1988 1990 E. H. Shortliffe 112 Figure 16: INTERNET Terminal Connect Time 5P41 -RR00785- 15 Details of Technical Progress 2.5 - System Reliability As in past years, the reliability of the 2060 system has been very high. The data below cover both harddare- and software-related system failures. The time listed under preventive maintenance (PM) downtime includes both the time required for hardware PM and the time required for installation of updated system software. The data cover the period of May 1, 1987 to April 30, 1988. May 1987 - April 1988: May Jun Jut Aug Sep Ott Nov Dee 14.2 3.2 5.0 0.1 24.0 10.9 2.6 4.0 Figure 17: 2060 Downtime Summary - Jan Feb Mar Apr 5.4 2.4 35.2 2.2 Hours per Month May 1987 - April 1988: Reporting period: 366 days, 0 hours, 12 min Total Uptime: 361 days, 10 hours, 56 min Uptime Percentage: 98.99% PM Downtime: 0 days, 20 hours, 9 min Actual Downtime: 3 days, 17 hours, 6 min Total Downtime: 4 days, 13 hours, 16 min MTBF: 1 5 days, 15 hours, 32 rnin Figure 18: Overall 2060 Reliability Summary `Mean Time Between Failures 113 E. H. Shortliffe Highlights 5P41 -RR007851 5 IILB. Highlights In this section we describe several research highlights from the past year's activities. These include notes on existing projects that have passed important milestones, new pilot projects that have shown progress in their initial stages, and other core research and special activities that reflect the progress, impact, and influence the SUMEX-AIM resource has had in the scientific and educational communities. E. H. Shortliffe 114 5P41 -RR00785-15 Highlights 1II.B. 1. PROTEGE - - Developing Knowledge Acquisition Tools for Clinical Trial Advice Systems Knowledge acquisition, the process whereby computer scientists (knowledge engineers) interview experts in a given application area and attempt to encode the experts' specialized knowledge in a computer program, is widely recognized as a principal obstacle in the development of knowledge-based systems. To ease these difficulties, workers in medical artificial intelligence have experimented with a number of computer-based tools designed to facilitate the construction of clinical advice systems. One such tool is OPAL, a program that allows physicians to enter descriptions of oncology treatment plans directly into the knowledge base of ONCOCIN, an expert system that offers advice concerning protocol-directed cancer therapy. Physicians who use OPAL do not have to understand the production rules and other data structures that are used internally by ONCOCIN to represent cancer- therapy knowledge. Rather, oncologists enter knowledge into OPAL by drawing flowcharts and by filling in the "blanks" of graphical forms that anticipate the concepts required to define cancer therapy protocols. OPAL automatically converts the physicians' specifications into the knowledge representations that ONCOCIN uses to generate its treatment advice. In 1986, system builders entered 36 oncology protocols into the ONCOCIN knowledge base using OPAL. Encoding these protocols via traditional knowledge engineering techniques might well have taken several person-years. Although OPAL clearly streamlines knowledge entry for the ONCOCIN system, OPAL itself required about 3.5 person-years of software development before it was ready for routine use. Furthermore, because OPAL takes advantage of a detailed model of therapy planning in oncology, the program is very much domain-dependent. OPAL is of no use, for example, to endocrinologists who desire to create knowledge bases for therapy planning for thyroid disease or to cardiologists interested in treating heart failure. PROTEGE is a system that allows expert system developers to create knowledge acquisition tools that are much like OPAL (i.e., have convenient and powerful user interfaces), but that are custom tailored for new application areas. PROTEGE permits knowledge engineers to define models of the kinds of clinical trials that occur in various areas of medicine. It then uses these models to produce domain-specific knowledge acquisition tools that allow physicians to define new protocols by filling in graphical forms and by drawing flowchart diagrams. To date, PROTEGE has been used to create two such tools: 1) p-OPAL -- a program that incorporates most of the functionality of OPAL and thus acquires knowledge concerning clinical trials in oncology and 2) HTN -- a knowledge acquisition tool for hypertension drug studies. Producing each of these knowledge acquisition tools required a knowledge engineer using PROTEGE to enter models for the relevant classes of clinical trials (oncology and hypertension, respectively), defining those clinical trial models in terms of a general model of treatment planning built into the PROTEGE system itself. The PROTEGE user fills in the blanks of various forms to define models for given types of clinical trial applications, much as the user of OPAL fills out graphical forms to define individual cancer protocols. The forms in PROTEGE directly reflect the general model of treatment planning. Once the clinical trial models had been specified, PROTEGE generates the corresponding knowledge acquisition tools (computer programs) automatically. 115 E. H. Shortliffe Highlights 5P41 -RR00785- 15 lll.B.2. A Speech Interface to ONCOCIN Motivated by a recurrent request from collaborating clinicians to augment the "classical" keyboard interface to expert systems with a speech-based interface and by recent technological advances in continuous-speech systems, we began a project to explore the integration of speech input with the ONCOCIN cancer therapy advisor system. The project uses a commercial continuous-speech system loaned to us by Speech Systems, Inc. (SSI) of Tarzana, California. The speech recognizer consists of a custom microelectronic processor and a suite of special speech decoding software modules. This experiment has taken advantage of our on-going work in distributed computing since the phonetic device, initial parsing software, and the ONCOCIN system all run on different pieces of hardware. Researchers have developed a prototype network connection and command interpreter between the speech module (running on a SUN workstation) and the Xerox 1186 computer that runs ONCOCIN and have designed a series of modifications to the ONCOCIN user interface has in turn been modified to accept verbal commands. The prototype interface system permits users to navigate the graphical ONCOCIN interface and enter clinical data using speech. The system uses the location of the cursor on the screen to provide a context for choosing candidate grammars with which to attempt to recognize the user's utterance. The system dynamically adjusts the list of candidate recognition grammars based on the on-going dialog and it is now possible to carry on most of the ONCOCIN data acquisition steps using speech alone or speech plus pointing with the mouse. In addition, some input data elements (such as the neural toxicities) can be entered as textual descriptions and automatically encoded on the l-4 point numerical scale used on oncology flowsheet forms. We are also performing experiments to enhance the system's grammars with a wider range of phrases clinicians actually use when talking to a computer and to gain insights into clinicians' models of spoken interaction with advice systems. This will allow us to ground our interface design better in observed practice. In order to assess how physicians would speak to a computer in an ideal situation, we are simulating fully functional continuous-speech understanding with a hidden computer operator generating the output of ONCOCIN as if it had the ability to understand all spoken input. A video camera records both audio and visual clues. The physicians use ONCOCIN in the same manner as it is used in the clinic when they see patients, but with the added capability of (simulated) speech input. These experiments enable us to build up both a basic vocabulary for the real speech system as well as examine subtle linguistic issues to guide future directions. E. H. Shortliffe 116 5P41 -RR00785-15 Highlights lll.B.3. SIMPLE/CARE -- Emulation of Parallel Computing Architectures Many applications require knowledge-based systems that can cope with large amounts of data and produce responses in real-time. The current hardware and software architectures for knowledge-based systems cannot support such requirements. The most promising approach for achieving orders of magnitude improvement in the quantitative performance of knowledge-based systems is by exploiting concurrency on multiprocessor systems. integrated circuit technologies, Based on projections for it is clear that highly parallel multiprocessor computers, consisting of 100's to 1000's of processors and realizing a variety of concurrent architectures, can be built. A major computer science issue is whether such computers can be used effectively to enhance the performance of knowledge- based systems. Since 1985, the Knowledge Systems Laboratory at Stanford University has been investigating these issues. The goals and technical approach of this project, largely supported by DARPA under the Strategic Computing Program, have been to achieve two to three orders of magnitude speed-up in the execution of knowledge-based systems, by identifying and exploiting sources of concurrency at all levels of system design: the application level, the problem-solving framework level, the programming language level and the hardware systems architecture level. Due to the inherent complexity of the task and the lack of theoretical foundations for parallel computation with ii!-structured problems, we have taken an empirical approach. Simulation of systems at an architectural level offers an effective way to study critical design choices. SIMPLE/CARE is a powerful simulation system that forms the foundation for our empirical investigations. SIMPLE is a CAD (Computer Aided Design) system for hierarchical, multiple-level specification of computer architectures and includes an associated mixed-mode, event-based simulator. CARE is a parameterized, multiprocessor array emulation defined in SIMPLE's specification languages and running on SIMPLE's simulator. Our simulation system has been used internally to make quantitative comparisons of the performance of various architectures and to gain insights into how different concurrent programming models support the development of concurrent applications decomposition and organization. The system is in use by several research groups at Stanford and it has been exported to several other sites, including NASA Ames Research Center. A tutorial was held in January 1988, attended by representatives from the DOD, NASA and Boeing, which described the CARE/SIMPLE system. The attendees received instruction in use of the system for making measurements of the performance of various simulated multiprocessor applications, A Stanford graduate course on these tools is currently in progress this spring quarter. 117 E. H. Shortliffe Highlights 5P41 -RROO785-15 lll.B.4. Toward the Distributed SUMEX-AIM Community We have made a key decision this past year on the core system definition that will support the first phase of the distributed AIM community. Guided by our requirements to provide powerful and widely-available tools for general computing and biomedical research and to sharply focus our limited development resources on a small number of standardized hardware and software configurations, we considered a wide range of alternative systems for AIM community computing needs to replace and upgrade the services of the 2060. Based on dominant user preferences for the icon-based interface, outstanding technical performance, very competitive academic pricing, and an already-growing group of national AIM users, we have chosen Apple Macintosh II workstations as the general computing environment for researchers and staff, TI Explorer Lisp machines (including the microExplorer Macintosh coprocessor) as the near-term high-performance Lisp research environment, and a SUN-4 as the central system network server (network services, file services, printing services, etc.). To actually implement a prototype of the planned distributed environment, a substantial quantity of hardware was purchased with DARPA research funds in the spring of 1988. We are now in the midst of the installation and integration process, concentrating initially on getting basic capabilities operational, such as for text processing, filing/archiving, printing, graphics, office management, system building tools, information resource access, and distributed system operation and management. Initial user response to the introduction of these systems has been overwhelmingly enthusiastic, even though there are many "rough edges" remaining to be smoothed out in the systems integration. Our core development work for the environment of the Mac II-, Explorer-, and SUN-based system has focussed on providing remote access between workstations themselves and with servers, integrating a solid support of the TCP-IP network protocol, and building a powerful distributed electronic mail system. The new Mac II mail system wil! be an adaptation of the prototype distributed system developed in recent years for the Xerox Lisp machine and which is in routine use by a number of people and is being ported to the Explorer. One of the key issues in selecting the systems for our distributed computing environment was the performance of Common Lisp and to help make this evaluation, we undertook an informal survey of the performance of two KSL Al software packages, SOAR and BBl, on a wide variety of machines. Within a factor of two of the best performance, a considerable range of workstations based on stock microprocessor chips (e.g., the Motorola 68Oxx and the Intel 80386) as well as specially microprogrammed Lisp chips have comparable performance. Even though performance gaps between microprogrammed Lisp systems and stock workstation implementations are narrowing, there still remains a significant difference in the quality of the development environments. We have attempted to distill and promote the commercial development of the key features of the Lisp machine environments that would be needed in stock machine implementations in order to make them attractive in a development setting. After the prototype distributed system is implemented and tested in the Stanford KSL environment, we will package and document its elements so that other sites in the AIM community and beyond can duplicate its capabilities. As this work progresses we will phase out the old DEC 2060 to be replaced by the SUN-4 as a general community communication and information server. E. H. Shortliffe 118 5P41 -RR007851 5 Administrative Changes IILC. Administrative Changes There have been few administrative changes within the project this past reporting year. Professor Shortliffe had been on sabbatical at the University of Pennsylvania the year before last and returned to Stanford in mid-July 1987, when he resumed his role as SUMEX Principal Investigator. We continue to operate the cost recovery system we reported on last year as part of phasing out BRTP subsidy of the DEC 2060 facility. The details of this system are discussed on page 121. In summary, we are successfully recovering the projected 40% of 2060 operations costs this year ($136,374) from Stanford users, with the declining component of NIH support (60% this year) used to protect national users from fees for service, including communications. This additional burden on Stanford projects continues to be absorbed almost entirely in existing direct cost budgets since no supplements for new computing costs were forthcoming in the middle of on- going grant and contract awards. This has affected staffing and student support directly in our labor-intensive research efforts. All of our new support applications are being written with requests for funds to cover projected computing charges. This next year we will increase the cost recovery goal to 60% of projected 2060 operations costs as scheduled in our grant application of June 1985. We also plan to physically phase out the 2060 and replace it with the new SUN-4 network server if technical development activities follow on schedule. The detailed interaction of this transition with our cost recovery procedures remain to be worked out during this coming year. 119 E. H. Shortliffe Resource Management and Allocation 5P41 -RR00785 15 1II.D. Resource Management and Allocation 1II.D. 1. Overall Management Plan Early in the design of the SUMEX-AIM resource, an effective management plan was worked out with the Biotechnology Resources Program (now Biomedical Research Technology Program) at NIH to assure fair administration of the resource for both Stanford and national users and to provide a framework for recruitment and development of a scientifically meritorious community of application projects. This structure has been described in some detail in earlier reports and is documented in our recent renewal application. It has continued to function effectively as summarized below. . The AIM Executive Committee meets periodically by teleconference to advise on new user applications, discuss resource management policies, plan workshop activities, and conduct other community business. The Advisory Group meets as needed to review project applications. (See Appendix D for a current listing of AIM committee membership). . We actively recruit new application projects and disseminate information about Al in biomedicine. With the development of more decentralized computing resources within the .AIM community outside of Stanford, the use of SUMEX resources by AIM members has shifted more and more toward communication with colleagues and access to information. . With the advice of the Executive Committee, we have opened SUMEX- AIM resources widely to biomedical users desiring electronic communications facilities. A list of current users who have used SUMEX for this purpose over this past year is given starting on Page IV-E. . We have carefully reviewed on-going projects with our management committees to maintain a high scientific quality and relevance to our biomedical Al goals. . We continue to provide active support for the AIM workshops. The most recent one was held this spring at Stanford University, under the auspices of the American Association for Artificial Intelligence (AAAI). . We have continued to provide systems advice to users attempting to set up computing resources at their own sites, based on the expertise developed in the SUMEX resource environment. . We have tailored resource policies to aid users whenever possible within our research mandate and available facilities. E. H. Shortliffe 120 5P41 -RR007851 5 Resource Management and Allocation lll.D.2. 2060 Cost Center General Cost Center Structure Our plan for the term of the current grant is a firm but responsible transition of the SUMEX-AIM resource to a distributed community model of operation. There has continued to be a group of national and local users -- particularly young projects needing seed support prior to obtaining major funding -- that depend on a central shared resource like the SUMEX mainframe. In addition, the 2060 has played a key role as a central server for intercommunity communication and shared information. Powerful and widely available workstation equipment is rapidly becoming accessible, however, at a cost that most projects can afford, even young ones. Thus, the period of critical dependence on the DEC 2060 for raw computing cycles is largely past and its role in supporting routine computing and communication services can also soon be replaced by other more cost effective means. We are in the process of phasing out the SUMEX 2060 machine over the next few years in favor of the new distributed workstation environment we are developing. This process is progressing gradually and responsibly so that our users can relocate to other facilities or move to workstation environments for their research without disruption. Specifically, our renewal proposal for the five-year period 8/l /86-7/31/91, submitted to the Division of Research Resources in June 1985, called for phasing out NIH support for DEC 2060 mainframe operations over the course of the grant period and the establishment of a cost center at Stanford to recover the unsubsidized costs of 2060 operations from the established Stanford user community. This phase-out process is taking place linearly over five years, with 80% of the 2060 costs charged to the resource budget in renewal year 1 (Grant Year 14), 60% in year 2 (current grant year), 40% in year 3, 20% in year 4, and 0% in year 5, when routine operations (even national user services) will be supported entirely by user revenues. Use of the 2060 by members of the national AIM community is still free of charge at this time, and we will continue to cover the total cost of national community 2060 usage from the NIH subsidy as long as funding permits. In keeping with this plan, during the summer of 1986, we requested and received the approval of the Government Cost and Rate Studies section of Stanford's Controller's Office to establish a 2060 cost center effective at the start of renewal year 1 (August 1, 1986) with a charge per CPU hour based on our projections of 2060 operations costs and anticipated billable CPU usage. In last year's annual report, we reported success in recovering the 20% of 2060 operations costs not subsidized by NIH from our Stanford user community during our first year of cost center operation. This year our objective was to recover 40% of 2060 costs from Stanford users with a corresponding increase in the rate charged per CPU hour as of August 1, 1987, the start of renewal year 2. We have been monitoring cost center expenses and revenues carefully this second year and again anticipate breaking even at the end of the cost center's fiscal year at the end of July. Figure 19 shows the cumulative user revenues collected by month for the period August 1987 through April 1988. 121 E. H. Shortliffe Resource Management and Allocation 5P41 ;RR00785- 15 E. H. Shortliffe Figure 19: 2060 Cost Center Performance 122 5P41 -RR00785-15 Dissemination of Resource Information 1II.E. Dissemination of Resource Information We are continuing our past practice of making a substantial effort to disseminate the Al technology developed here. This has taken the form of many publications -- over forty-five combined books and papers are published per year by the KSL; wide distribution of our software, including systems software and Al application and tool software, both to other research laboratories and for commercial development; production of films and video tapes depicting aspects of our work; and significant project efforts at studying the dissemination of individual applications systems such as the ONCOCIN resource-related research project (see 144). Software Distribution We have widely distributed both our system software and our Al tool software. Since much of our general system-level software is distributed via the ARPANET we do not have complete records of the extent of the distribution. Software such as TOPS-20 monitor enhancements, the Ethernet gateway and TIP programs, the SEAGATE AppleNet to Ethernet gateway, the PUP Leaf server, the SUMACC development system for Macintosh workstations, and our Lisp workstation programs are frequently distributed in this manner to the ARPANET community and beyond. Our primary distribution effort is directed towards the Al tools we have developed. In recent years, the volume of inquiries for this type of software and requests for tapes has been a substantial burden on the staff and so it was decided to turn over most of this type of software distribution to Stanford's Office of Technology Licensing (OTL). This organization handles software distribution and technology licensing matters for much of the Stanford community. Since there are several OTL staff members assigned to the distribution of Stanford software, requests for information and tapes are handled quickly and efficiently. Also, OTL's staff has the expertise needed to handle the legal questions that frequently arise in the distribution of software, and an established computerized record-keeping scheme. SUMEX staff continues to be available as needed to assist OTL with special administrative and technical matters. Specific software distribution events this past year include: . The Parallel Computing Architectures Project multiprocessor simulation system, CARE/SIMPLE, is in use by several research groups at Stanford and it has been ported to several external sites including NASA Ames Research Center. . Two (2) licenses were granted for the EMYCIN package and twenty-two (22) licenses were granted for the BBl package. . OTL has concluded a license arrangement with Cisco, Inc. for the commercial development and marketing of the SUMEX Ethernet gateway and TIP service software. . The agreement between Stanford University and Kinetics Inc. covering hardware and software technology for an Ethernet-to-AppleTalk gateway has been converted from an on-going royalty agreement to a fully paid license. . OTL reported the expiration of an exclusive license to Molecular Designs Ltd. covering some aspects of the software generated by the DENDRAL Project. The source code and binary versions of this software are now available to all users (commercial, government, and academic) through OTL. 123 E. H. Shortliffe Dissemination of Resource Information 5P41-RR00785-15 AIM Community Systems Support We continue to make a special effort to assist other members of the SUMEX-AIM community in integrating the technologies needed for biomedical Al research. This is often achieved through direct contact with staff members at these institutions, (e.g., with Professor Sticklen's group at Michigan State University after his move from Ohio State and with Professor Widman's group at the University of Texas), at meetings and workshops, or via electronic mailing lists. For example, the Info-MAC, Info- Explorer, and Info-l 100 mailing lists have hundreds of members and cover a broad range of equipment issues, software issues, and topics in artificial intelligence. Video Tapes and Films The KSL has continued to prepare video tapes that provide an overview of the research and research methodologies underlying our work and that demonstrate the capabilities of particular systems. These tapes are available through our groups, the Fleischmann Learning Center at the Stanford Medical Center, and the Stanford Computer Forum, and copies have been mailed to program offices of our various funding sponsors. In addition to the earlier tapes covering Knowledge Engineering in the KSL, ONCOCIN Overview, and ONCOCIN Demonstration, we have recent tapes on the PROTEAN project, the BBl project, and a one-day symposium on KSL research activities. E. H. Shortliffe 124 5P41 -RR00785-15 Suggestions and Comments 1II.F. Suggestions and Comments Resource Organization We continue to believe that the Biomedical Research Technology Program is one of the most effective vehicles for developing and disseminating technological tools for biomedical research. The goals and methods of the program are well-designed to encourage building of the necessary multi-disciplinary groups and merging of the appropriate technological and medical disciplines. Electronic Communications SUMEX-AIM has pioneered in developing more effective methods for facilitating scientific communication. Whereas face-to-face contacts continue to play a key role, in the longer-term computer-based communications will become increasingly important to the NIH and the distributed resources of the biomedical community. We would like to see the BRTP take a more active role in promoting these tools within the NIH and its grantee community. This is particularly important in the light of significant on-going changes to the national networking environment (see Page 75). 125 E. H. Shortliffe 5P41-RR00785-15 Description of Scientific Subprojects IV. Description of Scientific Subprojects The following subsections report on the AIM community of projects and "pilot" efforts including local and national users of the SUMEX-AIM facility at Stanford. In addition to these detailed progress reports, abstracts for each project and its individual users are submitted on a separate Scientific Subproject Form. However, we have included briefer summary abstracts of the fully-authorized projects in Appendix E on page 277. Those groups from the National AIM community which use the SUMEX-AIM resource solely for communication (i.e., electronic mail to and from colleagues or access to bulletin boards and other information resources at SUMEX) are listed starting on Page 220, without detailed reports on their research. The detailed collaborative project reports and comments are the result of a solicitation for contributions sent to each of the project Principal Investigators requesting the following information: I. SUMMARY OF RESEARCH PROGRAM A. Project rationale B. Medical relevance and collaboration C. Highlights of research progress --Accomplishments this past year --Research in progress D. List of relevant publications E. Funding support II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical collaborations and program dissemination via SUMEX B. Sharing and interactions with other SUMEX-AIM projects (via computing facilities, workshops, personal contacts, etc.) C. Critique of resource management (community facilitation, computer services, communications services, capacity, etc.) III. RESEARCH PLANS A. Project goals and plans --Near-term --Long-range B. Justification and requirements for continued SUMEX use C. Needs and plans for other computing resources beyond SUMEX-AIM D. Recommendations for future community and resource development We believe that the reports of the individual projects speak for themselves as rationales for participation. In any case, the reports are recorded as submitted and are the responsibility of the indicated project leaders. The only exceptions are the respective lists of relevant publications which have been uniformly formatted for parallel reporting on the Scientific Subproject Form. 127 E. H. Shortliffe Stanford Projects 5P41 -RR00785 15 1V.A. Stanford Projects The following group of projects is formally approved for access to the Stanford aliquot of the SUMEX-AIM resource. Their access is based on review by the Stanford Advisory Group and approval by Professor Shortliffe as Principal Investigator. In addition to the progress reports presented here, abstracts for each project and its individual users are submitted on a separate Scientific Subproject Form. E. H. Shortliffe 128 5P41-RR00785-15 BBICU Project 1V.A. 1. BBICU Project BBICU Project: Blackboard Applications in the Intensive Care Unit Adam Seiver, M.D. Department of Surgery Veterans Administration Hospital Palo Alto, California Lawrence Fagan, M.D., Ph.D. Department of Medicine Stanford University Barbara Hayes-Roth, Ph.D. Department of Computer Science Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale We are designing a data-interpretation and therapy-planning system for the intensive care unit (ICU). Fundamental research issues in temporal reasoning are associated with the ICU application area including assimilation of incoming data, representation of time-oriented intervals, and description of ongoing physiological processes [Fagan 841. In addition, in ICUs of the 1990s many more physiological measurements will need to be collected at frequent intervals, and increased access to the current medical record in coded format will be possible. Processing of incoming data will have to be opportunistic, selecting from a number of models that have different computational requirements and accuracy. We will use a blackboard architecture, known as BBl, that has evolved from earlier work on protein-structure elucidation and construction layout [Hayes-Roth 851. 661 is particularly .well suited for the ICU project because it maintains separate blackboards for domain and control knowledge. Although we have selected the blackboard structure as the organizing principle, many knowledge representation issues remain. First, we must represent the structure and function (anatomy and physiology) of the respiratory system. By characterizing the pathophysiology in terms of generic faults, we will create a more flexible means to diagnose problems in unusual situations -- in contrast to the phenomenological rules used in earlier systems. Second, we must coalesce quantitative and qualitative models. The physiology of the respiratory and cardiac systems have been modeled in detail, but it is impractical to base the entire reasoning process on complex mathematical equations. Instead, we are developing methods to transform quantitative models into simpler formulations. We must make explicit the simplifying assumptions and associate them with their corresponding models, so that we can select an applicable model for situations of varying complexity. Using the approach of our project for planning treatments for cancer patients [Langlotz 871, we will use strategic knowledge to create patient-specific specializations of standard treatment plans. We will use decision analytic methods to 129 E. H. Shortliffe BBICU Project 5P41 -RR00785-I 5 evaluate and explain the various treatment options available at any point in time. The long-term goal of this project is to embed the decision-making components within the data management tasks of the ICU. B. Medical Relevance and Collaboration The problem of too much data being generated in the ICU is well recognized. Originally, monitors were designed to provide more objective assessments of the physiology of the patients in life-threatening situations. However, as more and more measurements became available, the ability of clinicians to assimilate the data began to drop. Expert systems can be designed to sort through the data, recognize untoward events in context and help with therapy selection. An early version of this was Fagan's VM system, which was based on extensions to the production rule framework. The current research has far broader goals, including real-time response using multiple methods for reasoning, reasoning from anatomy and physiology, and the use of integrated mathematical models. This has led to a three way collaboration between the VA hospital which is installing a data management system for a new Surgical ICU (Seiver), the Computer Science Department where blackboard research for real-time systems is in progress (Hayes-Roth), and the Medical Computer Science Group where investigations in qualitative-quantitative reasoning is taking place (Fagan). C. Highlights of Research Progress This project is in its early stages, but we are beginning to see progress in both research directions. The BBI group has demonstrated a system that can reason by analogy from a set of symptoms corresponding to a physical "blockage" back to generic knowledge about structures involved in a flow process. This system has been developed in the BBI architecture on the Explorer Lisp machine. This group has developed a prototype that can: I. monitor a few types of electronically sensed data in real time, 2. dynamically focus attention on different types of data depending on the current situation, 3. incrementally classify asynchronously arriving data into temporal episodes of normal/abnormal physiological parameters, 4. dynamically compute conditional probabilities of alternative diagnoses as newly arriving data are classified, and 5. create alternative hypotheses when compiled clinical knowledge fails to explain the situation by reformulating the problem in terms of an underlying model of the structure and function of the body. To date, these goals have been carried out with a small number of data streams and recognized diagnoses. A major research goal of this project is to understand how the knowledge representations and processing techniques will have to adapt as the problem is scaled up. The quantitative/qualitative group has concentrated on building a variety of mathematical models based on the Dickenson model of oxygen transport through the circulatory system. These equations are at different levels of specification, and heuristics are used to select the most appropriate model at each point in time. Another active area of research is the relationship between the therapy planning module and the solution of mathematical models used for prediction. E. H. Shortliffe 5P41 -RR00785-I 5 BBICU Project D. Relevant Publications 1. Fagan, L., Kunz, J., Feigenbaum, E, and Osborn, J. Adapting a rule-based system for a monitoring task, in Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project, B. Buchanan and E. Shortliffe (eds.). Reading, MA: Addison-Wesley Publishing Co., 1984. 2. Hayes-Roth, 6. A Blackboard architecture for control. (Artificial Intelligence) 26:251-321, 1985. 3. Langlotz, C., Fagan, L., Tu, S., Sikic, B., and Shortliffe, E. A therapy planning architecture that combines decision theory and artificial intelligence techniques. Computers and Biomedical Research 20:279-303, 1987. II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations and Program Dissemination via SUMEX As described above, this project is a three-way collaboration between the Departments of Computer Science and Medicine, and Department of surgery at the VA Hospital. As, such we will need electronic mail and networking facilities. In addition, we imagine strong interactions with other projects around the world with similar research goals. We have already been contacted by research groups in Holland, Scotland, and Norway. In addition, similar research projects are underway at Yale, Berkeley, and Chicago. We expect that the networking facilities may allow us to share test cases, and possibly knowledge bases. B. Sharing and Interaction with Other SUMEX-AIM Projects The Yale project mentioned above is associated with Perry Miller's group. We also expect considerable interaction with the ONCOCIN and other parts of the Heuristic Programming Project at Stanford. The temporal issues involved with this project are relevant to Larry Widman. As the last AIM Workshop at Stanford in April, 1988, Larry demonstrated his programs on SSRG-based Explorers with the aid of SSRG staff member Rich Acuff. C. Critique of Resource Management The SUMEX staff have been quite helpful in the support of the various machines that have been used in this project so far. We anticipate that the project will migrate to Micro-Explorers and/or to Mac ils. We believe that the current efforts of the SUMEX staff are quite appropriate for our research needs. Ill. RESEARCH PLANS Our basic research agenda is described above. The basic research issues underlying this project will extend for several years, leading to an implementation in the Veterans Administration Hospital in Palo Alto. This research will continue to need help assistance with local area networking, file 131 E. H. Shortliffe BBICU Project 5P41 -RR00785-I5 service, and inter-network mail. We will need support for communications support within a project that is spread out over three geographical sites, and working with related but not identical hardware. Since this is a new project, we are only depending on the DEC-20 for mail, file service, and communication facilities. The SUN-4 arrangement should be able to provide these services if local area network mail is fully implemented. The SUMEX staff has been quite useful in providing support in other configurations of mainframe and workstations networked together. We anticipate that support for our unique collaborative arrangement will be equally superb. E. H. Shortliffe 132 5P41 -RR00785-I 5 GUIDON/NEOMYCIN Project IV.A.2. GUIDON/NEOMYCIN Project GUIDON/NEOMYCIN Project William J. Clancey, Ph.D. Department Computer Science Stanford University Bruce G. Buchanan, Ph.D. Computer Science Department Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale The GUIDON/NEOMYCIN Project is a research program devoted to the development of a knowledge-based tutoring system for application to medicine. The key issue for the GUIDON/NEOMYCIN project is to develop a program that can provide advice similar in quality to that given by human experts, modeling how they structure their knowledge as well as their problem-solving procedures. The consultation program using this knowledge is called NEOMYCIN. NEOMYCIN's knowledge base, designed for use in a teaching application, is the subject material used by a family of instructional programs referred to collectively as GUIDON2. The problem-solving procedures are developed by running test cases through NEOMYCIN and comparing them to expert behavior. Also, we use NEOMYCIN as a test bed for the explanation capabilities incorporated in our instructional programs. The purpose of the current contracts is to construct a knowledge-based tutoring system that teaches diagnostic strategies explicitly. By strategy, we mean plans for establishing a set of possible diagnoses, focusing on and confirming individual diagnoses, gathering data, and processing new data. The tutorial program has capabilities to recognize these plans, as well as to articulate strategies in explanations about how to do diagnosis. The strategies represented in the program, modeling techniques, and explanation techniques are wholly separate from the knowledge base, so that they can be used with many medical (and non-medical) domains. That is, the target program will be able to be tested with other knowledge bases, using system-building tools that we provide. B. Medical Relevance and Collaboration There is a growing realization that medical knowledge, originally codified for the purpose of computer-based consultations, may be used in additional ways that are medically relevant. Using the knowledge to teach medical students is perhaps foremost among these, and GUIDON2 focuses on methods for augmenting clinical knowledge in order to facilitate its use in a tutorial setting. A particularly important aspect of this work is the insight that has been gained regarding the need to structure knowledge differently, and in more detail, when it is being used for different purposes (e.g., teaching as opposed to clinical decision making). It was this aspect of the GUIDON research that led to the development of NEOMYCIN, which is an evolving computational model of medical diagnostic reasoning that we hope will enable us to better understand and teach diagnosis to students. An important additional realization is that these structuring methods are beneficial for improving 133 E. H. Shortliffe GUIDON/NEOMYCIN Project 5P41 -RR00785- 15 the problem-solving performance of consultation programs, providing more detailed and abstract explanations to consultation users, and making knowledge bases easier to maintain. As we move from technological development of explanation and student modeling capabilities, we are now collaborating closely with medical students and physicians to design an effective, useful tutoring program. In particular, medical students have served as research assistants, and a recent MSAI student is an experienced physician, John Sotos, from Johns Hopkins. The project has also collaborated with a community of researchers focusing on medical education, funded by the Josiah Macy, Jr. Foundation, C. Highlights of Research Progress Cl Accomplishments This Past Year C. I .I The GUIDON-MANAGE Tutoring Program This program teaches a student the language of diagnosis by having him or her enter all requests for patient information as an abstraction. Thus, the student issues "strategic commands" such as "test the hypothesis meningitis" or "ask a follow-up question about the headache," and the program (NEOMYCIN) carries out the tactics. By year end, this program was operational, with a complex interpreter for simulating NEOMYCIN to generate help, a feedback window to indicate what NEOMYCIN did when it carried out the commands, and many menus for making input to the program convenient. Research continues to focus on the assistance and feedback components of the program. C.I.2 Explanation Research Research in explanation is another major area. This year we adopted a new approach of printing the least possible information that would convey a line of reasoning, rather than generating text to describe everything the program did. The new explanation is based on the idea that the questioner seeking an explanation finds explanations more acceptable if it is necessary to fill in some gaps by his or her own effort. Our current approach is to print the domain relations (e.g., causes or subtype) that link foci to each other, leaving it to the questioner to infer the strategy behind each focus shift. This methodology will allow us to build up a more principled theory of explanation by determining what minimal information is acceptable and what causes an explanation to be inadequate. C.I.3 The ODYSSEUS Modeling Program Our third tutorial-related project involves continued development of a modeling program, OpYssEus. The purpose of ODYSSEUS is to discover domain knowledge discrepancies between an application domain knowledge base (e.g. the NEOMYCIN medical knowledge base) and a student or expert problem solver. IMAGE, an earlier modeling program developed in 1982, did not address this problem. The input to ODYSSEUS is the problem solver's patient data requests. When ODYSSEUS watches a student it functions as a student modeling program for GUIDON2 and when it watches an expert it functions as a knowledge acquisition program for HERACLES. During the past year, a dissertation describing the program has been completed. Cl.4 The HERACLES Expert System Shell The final major effort involves generalizing our expert system tool, HERACLES, so that it can be made available to other research groups who wish to develop knowledge E. H. Shortliffe 134 5P41 -RR00785-15 GUIDON/NEOMYCIN Project bases which can be tutored by GUIDON2. Several copies of HERACLES were shipped on floppies in the past year to users of Xerox D-Machines. C.l.5 Dissemination of Results Besides publications, a number of tutorials and invited talks by Dr. Clancey presented this work around the world: . Tutorial Speaker, Evaluation of Expert Systems, AAAI-87, July 1987. . Tutorial Speaker, Second Advanced Course on Al, Norway, August 1987, six hours of lectures. . Tutorial Speaker, Knowledge-Based Tutoring, IJCAI-87, Milan, August 1987. . Invited Speaker, Al-87, Osaka, Japan, October 1987. . Main Tutorial Speaker, "A Perspective on Knowledge Engineering," Inter Access, The Hague, Netherlands, February 1988, eight hours of lectures. . "Intelligent Tutoring Systems," panel of the Cognitive Science Society, Seattle, July 1987. . Nationai Space Sciences Educational Foundation, Stanford, July 1987. . Fujitsu, Tokyo Japan, CS Forum, November 1987. . CSK/CRI, Tokyo Japan, CS Forum, November 1987. C-2 Research in Progress As of March, 1988, Dr. William Clancey has moved to the Institute for Research on Learning in Palo Alto, California, where he continues his research on teaching and learning. His use of the SUMEX resource is reduced to accessing archived files. D. Publications Since January 1987 1. Clancey, W.J. Knowledge-Based Tutoring: The GUIDON Program, Cambridge: MIT Press. 2. Clancey, W.J. From Guidon to Neomycin and Heracles in twenty short lessons: ONR final report, 1979-1985. Current Issues in Expert Systems, 79-123, Academic Press, Inc., London. 3. Clancey, W.J. Intelligent tutoring systems: A tutorial survey. Current Issues in Expert Systems, 39-78, Academic Press, Inc., London. 4. Clancey, W.J. The knowledge engineer as student: Metacognitive bases for asking good questions. In Learning Issues in Intelligent Tutoring Systems, eds. H. Mandl and A. Lesgold, Springer-Verlag, in press. Also KSL 87-l 2. 5. Wilkins, D.C., Clancey, W.J., and Buchanan, B.G., Knowledge Base Refinement Using Abstract Control Knowledge. January, KSL-87-01. 6. Wilkins, D.C., Buchanan, B.G., and Clancey, W.J., The Global Credit Assignment Problem and Apprenticeship Learning. January, KSL-87-04. 135 E. H. Shortliffe GUIDON/NEOMYCIN Project 5P41 -RR00785-15 E. Funding Support Contract Title: "A Family of Intelligent Tutoring Programs for Medical Diagnosis" Principal Investigator: Bruce G. Buchanan, Prof. Computer Science, Research Associate Investigator: William J. Clancey, Research Assoc. Computer Science Agency: Josiah Macy, Jr. Foundation Term: March 1985 to March 1988 Total award: $503,415 direct costs Contract Title: "Computer-Based Tutors for Explaining and Managing the Process of Diagnostic Reasoning" Principal Investigator: Bruce G. Buchanan, Prof. Computer Science, Research Associate Investigator: William J. Clancey, Research Assoc. Computer Science Agency: Office of Naval Research ID number: N00014-85-K-0305 Term: March 1985 to November 1989 Total award: $712,411 total II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations and Program Dissemination via SUMEX We are frequently asked to demonstrate GUIDON-MANAGE, GUIDON-WATCH, and NEOMYCIN to Stanford visitors or at meetings in this country or abroad. Physicians have generally been enthusiastic about the potential of these programs and what they reveal about current approaches to computer-based medical decision making. We use network e-mail through SUMEX to communicate with other researchers worldwide. B. Sharing and Interaction with Other SUMEX-AIM Projects We interact periodically with Paul Feltovich at Southern Illinois Medical School. In addition, the central SUMEX development group acts as an important clearing house for solving problems and distributing new methods. C. Critique of Resource Management The SUMEX resources group has provided exemplary service. We have no complaints or suggestions whatsoever. Ill. RESEARCH PLANS A. Project Goals and Plans This research project has now moved from Stanford University to the new Institute for Research on Learning (IRL). We will no longer be an active member of the SUMEX Resource. B. Requirements for Continued SUMEX Use We have arranged to have archival access to our code and research notes (via dump tapes), which we prepared and stored at SUMEX from 1974 through 1987. We hope to move our personal archived files to our own disks in the next year, but will benefit from continuing access for project files over the next few years. E. H. Shortliffe 136 5P41 -RR00785- 15 MOLGEN Project IV.A.3. MOLGEN Project MOLGEN - Applications of Artificial Intelligence to Molecular Biology: Research in Theory Formation, Testing, and Modification Prof. E. Feigenbaum Department of Computer Science Stanford University Dr. P. Friedland NASA-Ames Research Center Moffett Field, CA Prof. Charles Yanofsky Department of Biology Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale The MOLGEN project has focused on research into the applications of symbolic computation and inference to the field of molecular biology. This has taken the specific form of systems which provide assistance to the experimental scientist in various tasks, the most important of which have been the design of complex experiment plans and the analysis of nucleic acid sequences. Our current research concentrates on scientific discovery within the subdomain of regulatory genetics. We desire to explore the methodologies scientists use to modify, extend, and test theories of genetic regulation, and then emulate that process within a computational system. Theory or model formation is a fundamental part of scientific research. Scientists both use and form such models dynamically. They are used to predict results (and therefore to suggest experiments to test the model) and also to explain experimental results. Models are extended and revised both as a result of logical conclusions from existing premises and as a result of new experimental evidence. Theory formation is a difficult cognitive task, and one in which there is substantial scope for intelligent computational assistance. Our research is toward building a system which can form theories to explain experimental evidence, can interact with a scientist to help to suggest experiments to discriminate among competing hypotheses, and can then revise and extend the growing model based upon the results of the experiments. The MOLGEN project has continuing computer science goals of exploring issues of knowledge representation, problem-solving, discovery, and planning within a real and complex domain. The project operates in a framework of collaboration between the Heuristic Programming Project (HPP) in the Computer Science Department and various domain experts in the departments of Biochemistry, Medicine, and Biology. It draws from the experience of several other projects in the HPP which deal with applications of artificial intelligence to medicine, organic chemistry, and engineering. 137 E. H. Shortliffe MOLGEN Project 5P41 -RR00785-15 B. Medical Relevance and Collaboration The field of molecular biology is nearing the point where the results of current research will have immediate and important application to the pharmaceutical and chemical industries. Already, clinical testing has begun with synthetic interferon and human growth hormone produced by recombinant DNA technology. Governmental reports estimate that there are more than two hundred new and established `industrial firms already undertaking product development using these new genetic tools. The programs being developed in the MOLGEN project have already proven useful and important to a considerable number of molecular biologists. Currently several dozen researchers in various laboratories at Stanford (Prof. Paul Berg's, Prof. Stanley Cohen's, Prof. Laurence Kedes', Prof. Douglas Brutlag's, Prof. Henry Kaplan's, and Prof. Douglas Wallace's) and over four hundred others throughout the country have used MOLGEN programs over the SUMEX-AIM facility. We have exported some of our programs to users outside the range of our computer network (University of Geneva [Switzerland], Imperial Cancer Research Fund [England], and European Molecular Biology Institute [Heidelberg] are examples). The pioneering work on SUMEX has led to the establishment of a separate NIH-supported facility, BIONET, to serve the academic molecular biology research community with MOLGEN-like software. BIONET is now serving many of the computational needs of over two thousand academic molecular biologists in the United States. More generally, our work in qualitative simulation as applied to molecular biology is also relevant to building models of many other medical and biological systems. For example, one Artificial Intelligence researcher (Kuipers) has been applying these techniques to the domain of renal physiology. Other researchers within the KSL are using similar techniques to build models of cardio-pulmonary physiology. C. Highlights of Research Progress C. 1 Accomplishments During the past year we have constructed a second model of the tryptophan operon. The first model we built focused on developing qualitative descriptions of the state variables of the tryptophan operon; this second model focuses on the objects in this biological system, their internal structures, and the processes which modify these objects over time. In addition, we have begun to use this second model as the basis for scientific theory formation. The highlights of this work are summarized below. C. 1.1 Qualitative Modeling and Simulation Our work in qualitative simulation has been directed towards building a program which embodies a theory of the tryptophan system. The earlier model which we constructed of the system was successful in its ability to predict state variable values for the trp system. However, this system lacked flexibility: its fixed network of state variables is valid for only a limited set of experiments. Many experiments involve introducing new state variables, removing old ones, or modifying the interactions between variables. Thus our goal was to build a model of the system which would essentially derive the state variable network used by the earlier system, given a description of what objects were present in an experiment. In the newer model a gene regulation experiment is described by specifying what objects are present at the start of the experiment and what their properties and relationships are. These objects are represented as instances of prototypical biological objects which are described in a large knowledge base. The modeling system uses a knowledge base of biological processes to detect interactions E. H. Shortliffe 138 5P41 -RR00785-15 MOLGEN Project between the objects which exist in the current simulation. These interactions can result in the creation of new objects and the establishment of linkages between object state variables, and are used to predict future states of the gene regulation system. The object knowledge base (KB) is a taxonomic hierarchy of biological objects such as genes, proteins, and chemical binding sites. This KB describes object properties, states, and their decomposition into component objects. The object KB can be viewed as a library of prototypical objects. Modeling of a specific biological experiment begins with a specification of what actual objects (as opposed to prototypes) are present in the experiment. In this context we developed techniques for representing the decomposition of complex objects such as proteins into their components, and for instantiating these prototypical descriptions. The process knowledge base describes the behaviors of the objects in the trp system. For example, processes encode chemical binding, re-arrangement, and dissociation events which are involved in such biological processes as transcription and biochemical pathways. When the modeling system is called upon to predict the outcome of an experiment, a process interpreter is responsible for employing processes in the process library to detect interactions between objects in the current experiment. As noted above, these interactions lead to effects such as the creation of new objects in the current experiment, the modification of old objects, and the establishment of linkages between object state variables such as object concentrations, This model of the trp system has been fully implemented as a working computer program and includes approximately 200 objects and 35 processes. It covers the important components of the trp operon as known in the early 1960s including transcription, translation, and the biosynthetic pathway for tryptophan, and can thus serve as a starting point for the generation of improved theories of the trp operon. C. 1.2 Theory Formation We have come to view the overall theory formation problem within a machine learning paradigm. Theory formation is considered to be a machine learning problem, which implies that any theory formation program should have two components: a performance element and a learning element. The performance element is the model of the trp operon described above. It contains knowledge of the objects and processes within the trp system, and inference mechanisms for using that knowledge base to predict experimental outcomes. The learning element is used when the performance element makes an incorrect prediction. The learning element must modify the performance element to increase the quality of its predictions. It appears to be productive to view the theory formation problem as a design problem. This allows us to apply knowledge of design within Artificial Intelligence to the problem of theory formation, which is less well understood. Design is a creative activity in which a designer constructs an entity which satisfies a set of constraints. This entity might be an object (e.g., a circuit), or a plan of action (e.g., a robot path plan). The design constraints specify predicates which the design process and the designed entity must satisfy. The entity is constructed from a set of primitives. Design operators specify all possible ways in which new primitives may be added to an entity under design. For example, circuits are constructed from transistors and other electronic devices; the design operators describe how these components may be wired together. Al has approached the problem of design through its central paradigm of search. That is, Al considers the design process to be a kind of search. Search problems 139 E. H. Shortliffe MOLGEN Project 5P41 -RR00785-15 have two important aspects: the space to be searched, and the means by which the search is accomplished. In design problems, the design space to be searched is the set of all legal configurations in which the design primitives may be combined to produce designed entities. The task of theory formation also involves the construction of an entity which satisfies a set of constraints. The entity to be designed is a theory. The primitives from which a theory is designed are the objects and processes within the domain. The constraints on the designed theory include: (a) it must account for, or predict, the observed phenomena, (b) it must be testable, (c) it must be confirmed by the tests, (d) it must conform to a large degree with other established knowledge, (e) it must satisfy certain constraints of form, e.g., it should be as simple as possible. Our earlier historical study of the discovery of attenuation directed us towards this view of theory formation, and it also provided the design operators needed to make it work. In addition to reconstructing the different theories of the trp operon which the biologists possessed at different times, we also compared consecutive theories to determine the differences between them, which tells us what modifications the biologists applied to existing theories to produce new theories. These theory modification operators are used to design new theories from old. Examples of these operators include postulating the presence of a previously known type of object within an experiment, postulating the presence of a previously unknown type of object, and postulating an interaction between existing objects which were not previously thought to interact (a new process). We have begun implementation of a theory formation program based on the above approach. The inputs to this program are (a) the current theory of the trp operon, (b) a description of an experiment, (c) the outcome of the experiment predicted by the theory, and (d) a description of how the prediction in (c) is incorrect. The output of the program is a set of possible modifications to the theory which cause it to make the correct prediction rather than the prediction in (c). For example, the initial theory might not predict that a certain chemical X is produced in a given experiment, when in fact this chemical is empirically observed to be present. The theory formation program is implemented as an Al planner: in the above example the planner is given the goal of predicting the presence of X. To achieve this goal it can use a number of different theory formation operators, such as postulating the existence of other chemicals in the experiment, and modifying the processes in the theory such that they would cause X to be produced from the chemicals present in the experiment. The current implementation consists of an agenda-based planner with an incomplete set of these theory formation operators. D. Publications 1. Bach, R., Friedland, P., Brutlag, D., and Kedes, L.: MAXIMIZE, a DNA sequencing strategy advisor. Nucleic Acids Res. 1 O( 1):295-304, January, 1982 2. Bach, R., Friedland, P., and Iwasaki, Y.: Intelligent computational assistance for experiment design. Nucleic Acids Res. 12(1):11-29, January, 1984. 3. Brutlag, D., Clayton, J., Friedland, P. and Kedes, L.: SEQ: A nucleotide sequence analysis and recombination system. Nucleic Acids Res. 10( 1):279-294, January, 1982. 4. Clayton, J. and Kedes, L.: GEL, a DNA sequencing project management system. Nucleic Acids Res. 10( 1):305-321, January, 1982. E. H. Shortliffe 140 5P41 -RR00785 15 MOLGEN Project Feitelson, J. and Stefik, M.J.: A case study of the reasoning in a genetics experiment. Heuristic Programming Project Report HPP-77-18 (working paper), May, 1977. 5. Friedland, P.: Knowledge-based experiment design in molecular genetics. Proc. Sixth IJCAI, August, 1979, pp. 285-287. 6. Friedland P.: Knowledge-based experiment design in molecular genetics. Stanford Computer Science Report STAN-CS-79-760 (Ph.D. thesis), December, 1979. 7. Friedland, P., Kedes, L. and Brutlag D.: MOLGEW-Applications of symbolic computation and artificial intelligence to molecular biology. Proc. Battelle Conference on Genetic Engineering, April, 1981. 8. Friedland, P.: Acquisition of procedural knowledge from domain experts. Proc. Seventh IJCAI, August, 1981, pp. 856-861. 9. Friedland, P., Kedes, L., Brutlag, D., Iwasaki, Y. and Bach R.: GENESIS, a knowledge-based gene tic engineering simulation sys tern for representation of genetic data and experiment planning. Nucleic Acids Res. 10( 1):323-340, January, 1982. 10. Friedland, P., and Kedes, L.: Discovering the secrets of DNA. Communications of the ACM, 28(! I):1 164-l 186, November, 1985, and IEEE/Computer, 18( 11):49:69, November, 1985. 1 I. Friedland, P. and lwasaki Y.: The concept and implementation of skeletal plans. Journal of Automated Reasoning, l(2): 161-208, 1985. 12. Friedland, P., Armstrong, P., and Kehler, T.: The role of computers in biotechnology. BIO\TECHNOLOGY 565-575, September, 1983. 13. Iwasaki, Y. and Friedland, P.: SPEX: A second-generation experiment design system. Proc. of Second National Conference on Artificial Intelligence, August, 1982, pp. 341-344. 14. Martin, N., Friedland, P., King, J. and Stefik, M.J.: Knowledge base management for experiment planning in molecular genetics. Proc. Fifth IJCAI, August, 1977, pp. 882-887. 15. Meyers, S. and Friedland, P.: Knowledge-based simulation of regulatory genetics in bacteriophage Lambda. Nucleic Acids Res. 12(1):1-g, January, 1984. 16. Stefik, M. and Friedland, P.: Machine inference for molecular genetics: Methods and applications. Proc. of NCC, June, 1978. 17. Stefik, M.J. and Martin N.: A review of knowledge based problem solving as a basis for a genetics experiment designing system. Stanford Computer Science Report STAN-CS-77-596, March, 1977. 18. Stefik, M.: Inferring DNA structures from segmentation data: A case study. Artificial Intelligence 11:85-l 14, December, 1977. 19. Stefik, M.: An examination of a frame-structured representation system. Proc. Sixth IJCAI, August, 1979, pp. 844-852. 20. Stefik, M.: Planning with constraints. Stanford Computer Science Report STAN-CS-80-784 (Ph.D. thesis), March, 1980. 141 E. H. Shortliffe MOLGEN Project 5P41 -RR00785-15 21. Karp, P., and D. Wilkins: An Analysis of the Deep/Shallow Distinction for Expert Sys terns. To be published in International Journal of Expert Systems, 1988. 22. Karp, P., and P. Friedland: Coordinating the Use of Qualitative and Quantitative Know/edge in Declarative Device Modeling, in Artificial Intelligence, Simulation, and Modeling edited by L. Widman and K. Loparo, 1988. 23. Karp, P.: A Process-Oriented Model of Bacterial Gene Regulation, Stanford University Knowledge Systems Laboratory Technical Report KSL-88- 18. 24. Round, A.: QSOPS: A Workbench Environment for the Qualitative Simulation of Physical Processes. Stanford University Knowledge Systems Laboratory Report KSL-87-37, 1987. 25. Karp, P., and P. Friedland, A Conceptual Reconstruct/on of the Discovery of Attenuation. In Preparation. E. funding Support The MOLGEN grant, which has supported the bulk of this research, is titled: MOLGEN: Applications of Artificial Intelligence to Molecular Biology: Research in Theory Formation, Testing, and Modification. This NSF Grant number MCS-8310236, expired on 1 O/31 186. The Principal Investigators were Edward A. Feigenbaum, Professor of Computer Science and Charles Yanofsky, Professor of Biology. Additional support for this research has been provided by the Defense Advanced Research Projects Agency, under contract N00039-86C-0033. II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE SUMEX-AIM continues to serve as the nucleus of our computing resources. The facility has not only provided excellent support for our programming efforts but has served as a major communication link among members of the project. Systems available on SUMEX-AIM such as EMACS, MM, Scribe and BULLETIN BOARD have made possible the project's documentation and communication efforts. The interactive environment of the facility is especially important in this type of project development. We strongly approve of the network-oriented approach to a programming environment into which SUMEX has evolved. The ability to utilize Lisp workstations for intensive computing while still communicate with all of the other SUMEX resources has been indispensable to our work. We currently have a satisfactory mode of operation where essentially all programming takes place on the workstations and most electronic communications, information sharing, and document preparation takes place within the TOPS-20 environment. The evolution of SUMEX has alleviated most of our previous problems with resource loading and file space. Our current Lisp workstations are not quite fast enough, but we are encouraged by the progress that has been made. We have taken advantage of the collective expertise on medically-oriented knowledge-based systems of the other SUMEX-AIM projects. In addition to especially close ties with other projects at Stanford, we have greatly benefited by interaction with other projects at yearly meetings and through exchange of working papers and ideas over the system. E. H. Shortliffe 142 5P41 -RR007851 5 MOLGEN Project III. RESEARCH PLANS A. Project Goals And Plans Our current work has the following major goals: 1. The abilities of the theory formation system will be extended by implementing additional theory formation operators to allow it to generate new classes of theories. 2. A mechanism for evaluating alternative theories will be constructed. This mechanism will be guide the planner's search towards more plausible theories, and will allow the system to present only the most credible theories it finds to the user. 3. Test the entire approach on the evolving theory of the trp operon regulatory system. Experiment with different initial knowledge bases to see how the discovery process is altered by the availability of new techniques, analogous systems, and so forth. B. Justification and Requirements for Continued SUMEX Use The MOLGEN project depends heavily on the SUMEX facility. We have already developed several useful tools on the facility and are continuing research toward applying the methods of artificial intelligence to the field of molecular biology. The community of potential users is growing nearly exponentially as researchers from most of the biomedical-medical fields become interested in the technology of recombinant DNA. We believe the MOLGEN work is already important to this growing community and will continue to be important. The evidence for this is an already large list of pilot exo-MOLGEN users on SUMEX. 143 E. H. Shortliffe ONCOCIN Project 5P41 -RR00785-15 IV.A.4. ONCOCIN Project ONCOCIN Project Edward H. Shortliffe, M.D., Ph.D. Departments of Medicine and Computer Science Stanford University Project Director: Lawrence M. Fagan, M.D., Ph.D Department of Medicine Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale The ONCOCIN Project is one of many Stanford research programs devoted to the development of knowledge-based expert systems for application to medicine and the allied sciences. The central issue in this work has been to develop a program that can provide advice similar in quality to that given by human experts, and to ensure that the system is easy to use and acceptable to physicians. The work seeks to improve the interactive process, both for the developer of a knowledge-based system, and for the intended end user. In addition, we have emphasized clinical implementation of the developing tool so that we can ascertain the effectiveness of the program's interactive capabilities when it is used by physicians who are caring for patients and are uninvolved in the computer-based research activity. 6. Medical Relevance and Collaboration The lessons learned in building prior production rule systems have allowed us to create a large oncology protocol management system much more rapidly than was the case when we started to build MYCIN. We introduced ONCOCIN for use by Stanford oncologists in May 1981. This would not have been possible without the active collaboration of Stanford oncologists who helped with the construction of the knowledge base and also kept project computer scientists aware of the psychological and logistical issues related to the operation of a busy outpatient clinic. C. Highlights of Research Progress C.l.A Background and Overview of Accomplishments The ONCOCIN Project is a large interdisciplinary effort that has involved over 35 individuals since the project's inception in July 1979. The work is currently in its ninth year; we summarize here the milestones that have occurred in the research to date: . Year 7: The project began with two programmers (Carli `Scott and Miriam Bischoff), a Clinical Specialist (Dr. Bruce Campbell) and students under the direction of Dr. Shortliffe and Dr. Charlotte Jacobs from the Division of Oncology. During the first year of this research (1979-1980) we developed a prototype of the ONCOCIN consultation system, drawing from programs and E. H. Shortliffe 144 5P41 -RR00785-15 ONCOCIN Project capabilities developed for the EMYCIN system-building project. During that year, we also undertook a detailed analysis of the day-to-day activities of the Stanford Oncology Clinic in order to determine how to introduce ONCOCIN with minimal disruption of an operation which is already running smoothly. We also spent much of our time in the first year giving careful consideration to the most appropriate mode of interaction with physicians in order to optimize the chances for ONCOCIN to become a useful and accepted tool in this specialized clinical environment. o Year 2: The following year (1980-i 981) we completed the development of a special interface program that responds to commands from a customized keypad. We also encoded the rules for one more chemotherapy protocol (oat cell carcinoma of the lung) and updated the Hodgkin's disease protocols when new versions of the documents were released late in 1980; these exercises demonstrated the generality and flexibility of the representation scheme we had devised. Software protocols were developed for achieving communication between the interface program and the reasoning program, and we coordinated the printing routines needed to produce hard copy flow sheets, patient summaries, and encounter sheets. Finally, lines were installed in the Stanford Oncology Day Care Center, and, beginning in May 1981, eight fellows in oncology began using the system three mornings per week for management of their patients enrolled in lymphoma chemotherapy protocols. . Year 3: During our third year (1981-1982) the results of our early experience with physician users guided both our basic and applied work. We designed and began to collect data for three formal studies to evaluate the impact of ONCOCIN in the clinic. This latter task required special software development to generate special flow sheets and to maintain the records needed for the data analysis. Towards the end of 1982 we also began new research into a critiquing model for ONCOCIN that involves "hypothesis assessment" rather than formal advice giving. Finally, in 1982 we began to develop a query system to allow system builders as well as end users to examine the growing complex knowledge base of the program. o Year 4: Our fourth year (I 982-1983) saw the departure of Carli Scott, a key figure in the initial design and implementation of ONCOCIN, the promotion of Miriam Bischoff to Chief Programmer, and the arrival of Christopher Lane as our second scientific programmer. At this time we began exploring the possibility of running ONCOCIN on a single-user professional workstation and experimented with different options for data-entry using a "mouse" pointing device. Christopher Lane became an expert on the Xerox workstations that we are using. In addition, since ONCOCIN had grown to such a large program with many different facets, we spent much of our fourth year documenting the system. During that year we also modified the clinic system based upon feedback from the physician-users, made some modifications to the rules for Hodgkin's disease based upon changes to the protocols, and completed several evaluation studies. . Year 5: The project's fifth year (1983-l 984) was characterized by growth in the size of our staff (three new full-time staff members and a new oncologist joined the group). The increased size resulted from a DRR grant that permitted us to begin a major effort to rewrite ONCOCIN to run on professional workstations. Dr. Robert Carlson, who had been our Clinical Specialist for the previous two years, was replaced by Dr. Joel Bernstein, while Dr. Carlson assumed a position with the nearby Northern California Oncology Group; this appointment permitted him to continue his affiliation both 145 E. H. Shortliffe ONCOCIN Project 5P41 -RR00785- 15 with Stanford and with our research group. In August of 1983, Larry Fagan joined the project to take over the duties of the ONCOCIN Project Director while also becoming the Co-Director of the newly formed Medical Information Sciences Program. Dr. Fagan continues to be in charge of the day-to-day efforts of our research. An additional programmer, Jay Ferguson, joined the group in the fall to assist with the effort required to transfer ONCOCIN from SUMEX to the 1108 workstation. A fourth programmer, Joan Differding, joined the staff to work on our protocol acquisition effort (OPAL). o Year 6: During our sixth year (1984-1985) we further increased the size of our programming staff to help in the major workstation conversion effort. The ONCOCIN and OPAL efforts were greatly facilitated by a successful application for an equipment grant from Xerox Corporation. With a total of 15 Xerox LISP machines now available for our group's research, all full-time programmers have dedicated machines, as do several of the senior graduate students working on the project. Christopher Lane took on full-time responsibility for the integration and maintenance of the group's equipment and associated software. Two of our programming staff moved on to jobs in industry (Bischoff and Ferguson) and three new programmers (David Combs, Cliff Wulfman, and Samson Tu) were hired to fill the void created by their departure and by the reassignment of Christopher Lane. In addition to funding from DRR for the workstation conversion effort, we have support from the National Library of Medicine which supports our more basic research activities regarding biomedical knowledge representation, knowledge acquisition, therapy planning, and explanation as it relates to the ONCOCIN task domain. We have continued to study the therapy planning process under support from the NLM. This research is led by Dr. Fagan and has concentrated on how to represent the therapy-planning strategies used to decide treatment for patients who run into serious problems while on protocol-described treatment. The physicians who treat these patients often seek out a consultation with the protocol study chairman. Dr. Branimir Sikic, a faculty member from the Stanford University Department of Medicine, and the Study Chairman for the oat cell protocol, collaborated on this project. Janice Rohn joined the ONCOCIN project as data manager and to assist in the knowledge entry process. . Year 7: The seventh year (1985-86) marked several milestones in our research on workstation-based programming. The OPAL knowledge acquisition system became operational, and several new oncology protocols were entered using this system. David Combs was primarily responsible for creating the operational version of OPAL (based on the initial prototype by Joan Differding Walton). As anticipated, we increased the speed and ease with which protocols can be added to the ONCOCIN knowledge base. Based on the protocols entered through OPAL, we began experimental testing of the workstation version of ONCOCIN in the Stanford oncology clinic. Clifford Wulfman developed the user interface (based on an initial prototype designed by Christopher Lane). Samson Tu developed the reasoning component (designed originally by Jay Ferguson). Much of their work is built upon an object-oriented system developed for our group by Christopher Lane. We connected the various parts of the system, and demonstrated that we have the capability to run ONCOCIN with the reasoning program and interface program on different machines in the communication network. The current version of the program is currently run on a single workstation, but future versions may take advantage of the multiple machine option. To increase the E. H. Shortliffe 146 5P41-RR00785-15 ONCOCIN Project speed at which we are able to test protocols entered into ONCOCIN, we developed additional programs to test real and synthetic cases without user interaction; these are then reviewed by our collaborating clinicians. We also developed a workstation-based program, OPUS, to help clinicians determine which protocols are appropriate for specific patients, OPUS was designed and implemented by Janice Rohn with the assistance of Christopher Lane. We have been using it in the clinic setting since the end of 1985. Thus, in addition to providing an information resource about protocols, the use of a graphically-oriented program provided a way to learn about the software style and hardware used in the workstation version of ONCOCIN. We discontinued the mainframe version of ONCOCIN, and began using the workstation version exclusively. The performance of the mainframe version of ONCOCIN was documented in two evaluation papers that appeared in clinical journals (see Hickam and Kent's papers). We continued our basic research in the design of advanced therapy-planning programs: the ONYX project. We developed a model for planning which includes techniques from the fields of artificial intelligence, simulation, and decision analysis. Artificial intelligence techniques are used to create a small number of possible plans given the ideal therapy and the patient's past treatment history. Simulation techniques and decision analysis are used to examine and order the most promising plans. Our goal is to allow ONCOCIN to give advice in a wider range of situations; in particular, the system should be able to recommend plans for patients who have an unusual response to chemotherapy. During this year, Stephen Rappapor-t, M.D. joined us as a programmer on the therapy planning research. Clinical expertise for ONCOCIN was provided by Richard Lenon, M.D. and Robert Carlson, M.D. . Year 8: The eight year (1986-87) concentrated on two diverse tasks: 1) scaling up the use of the workstation version of ONCOCIN in the clinic, and 2) generalization of each of the components. The latter task is described in the core research sections of this report(see page 19). In 1986, we placed the workstation version of ONCOCIN into the Oncology Day Care clinic. This version is a completely different program from the version of ONCOCIN that ran on the DECsystem 20--using protocols entered through the OPAL program, with a new graphical data entry interface, and a revised knowledge representation and reasoning component. One of the Oncology Clinical Fellows (Andy Zelenetz) became responsible for verifying how well our design goals for ONCOCIN had been accomplished. His suggestions have included the addition of key protocols and the ability to have the program used as a data management tool if the complete treatment protocol had not yet been entered into the system. Both of these suggestions were carried out during this year, and the program has achieved wider use in the clinic setting. In addition, laser-printed flowsheets and progress notes have been added to the clinic system. The process of entering a large number of treatment protocols in a short period of time led to other research topics including: design of an automated system for producing meaningful test cases for each knowledge base, modification of the design and access methods for the time-oriented database, and the development of methods for graphically viewing multiple protocols that are combined into one large knowledge base. These research 147 E. H. Shortliffe ONCOCIN Project 5P41 -RR00785-15 efforts will continue into the next year. In addition, some of the treatment regimens developed for the original mainframe version are still in use and can be transferred to the version of ONCOCIN. As the knowledge base grows, additional mech; ns will be needed for the incremental update and retraction of protocols. Additional changes in the reasoning and interface components of the system are described below. A new research project related to ONCOCIN was started this last year. We are exploring the use of continuous speech recognition as an alternate entry method for communicating with ONCOCIN. This project requires the connection of speech recognition equipment produced by Speech Systems, Inc. of Tarzana to the ONCOCIN interface module. Christopher Lane has developed a prototype network connection and command interpreter between the speech module (running on a Sun with special hardware added) and the Xerox 1186 computer that runs ONCOCIN. Clifford Wulfman has designed a series of modifications to the ONCOCIN user interface to allow for verbal commands. This work is described in more detail in the core ONCOCIN section. We continue to collaborate with Andy Zelenetz, Richard Lenon, Robert Carlson, and Charlotte Jacobs on the design and implementation of ONCOCIN in the clinic. Stephen Rappaport has started a residency program to continue his medical education. o Year 9: The majority of our effort this year has been to understand the limitations of the clinic version of ONCOCIN, and to concentrate on the generalization of these techniques to other application areas besides oncology. The majority of this research is thus described as part of the core research discussion on ONCOCIN. Highlights of this year include: (1) development of a general knowledge acquisition tool (PROTEGE) designed to handle skeletal planning applications for clinical trials in any area of medicine, (2) demonstration that the therapy planning and knowledge acquisition tools for ONCOCIN can be closely integrated, and (3) development of a speech input system for ONCOCIN. As a demonstration of the capabilities of the project to date, we undertook an experiment to see how difficult and time-consuming it is to bring up a new treatment protocol. A summary of a recent colon protocol was downloaded from the PDQ protocol database. Approximately 60% of the knowledge of the protocol summary fit easily into the OPAL high level description. Additional rules were entered using lower level editors. A limited consultation was run after about 4 hours of work. Although this is only one data point, we believe that it validates the generality of the knowledge acquisition and therapy planning approach that we have pursued for nearly a decade. Work continues on extending the knowledge acquisition and therapy planning tools to allow for a higher percentage of concepts that can be entered with the smallest possible amount of low level Lisp changes, Although we have completed the transfer of ONCOCIN into a stable and useful system on the Xerox Lisp workstations, it is now clear that this type of machine will not provide the type of dissemination hardware we would like to see. There are no planned additions to increase the speed, decrease the cost, or increase the integration capabilities of these workstations. Although there may be other solutions that will allow us to port ONCOCIN directly to alternative hardware platforms, we may need to move away from Xerox workstations and InterLisp language upon which most of our software is based. We are particularly interested in exploring the Mac II hardware recently purchased for the KSL. E. H. Shortliffe 148 5P41 -RR00785- 15 ONCOCIN Project C.7.6 Review of Research Issues in ONCOCIN and OPAL Our work to refine the clinic versions of ONCOCIN and OPAL reached a mature stage during this last research year. As our attention has moved to the generalization of these tasks (E-ONCOCIN and PROTEGE) it seems appropriate to describe the range of research issues that we have examined during the development of the ONCOCIN system. Research Issues in the Development of the ONCOClN Reasoner and Interviewer o Redesign of the reasoning component. A major impetus for the redesign of the system was to develop more efficient methods to search the knowledge base during the running of a case. We have implemented a reasoning program that uses a discrimination network to process the cancer protocols. This network provides for a compact representation of information which is common to many protocols but does not require the program to consider and then disregard information related to protocols that are irrelevant to a particular patient. We continue to improve portions of the reasoning component that are associated with reasoning over time; e.g., modeling the appropriate timing for ordering tests and identifying the information which needs to be gathered before the next clinic visit. In general, we are concentrating on improving the representation of the knowledge regarding sequences of therapy actions specified by the protocol. Our experience with adding a large number of protocols has led to the evaluation of the design of the internal structure of the knowledge base (e.g., the way we describe the relationships between chemotherapies, drugs, and treatment visits). We will continue to improve the method for traversing the plan structure in the knowledge base, and consider alternative arrangements for representing the structure of chemotherapy plans. Currently, the knowledge base of treatment guidelines and the patient database are separated. We propose to tie these two structures cioser together. Additional work is anticipated on turning ONCOCIN into a critiquing system, where the physician enters their therapy and ONCOCIN provides suggestions about possible alternatives to the entered therapy. Although we have concentrated our review of the ONCOCIN design primarily on the data provided by additional protocols, we know that non-cancer therapy problems may also raise similar issues. The E-ONCOCIN effort is designed to produce a domain-independent therapy planning system that includes the lessons learned from our oncology research. Samson Tu is primarily responsible for continued improvement of the reasoning component of ONCOCIN. o Development of a temporal network. The ability to represent temporal information is a key element of programs that must reason about treatment protocols. The earlier version of the ONCOCIN system did not have an explicit structure for reasoning about time-oriented events. We are experimenting with different configurations of the temporal network, and with the syntax for querying the network. We are also adapting this network so that it can interface with the ONYX therapy-planning systems. This research on temporal reasoning is part of Michael Kahn's Ph.D. thesis. Michael is a student in the Medical Information Sciences Program at University of California at San Francisco. . Extensions to the user interface. We continue to experiment with various configurations of the user interface. Many of the changes have been in response to requests for a more flexible data management environment. We are occasionally faced with data that becomes available corresponding to a E. H. Shortliffe ONCOCIN Project 5P41 -RR00785- 15 time before the current visit. This can happen if a laboratory result is delayed, or .a patient's electronic flowsheet is started in the middle of the treatment. We have added the ability to create new columns of data, and are designing the changes to the temporal processing components of ONCOCIN to allow for data that is inserted out of order. We have also extended the flowsheet to allow for patient specific parameters (e.g., special test results or symptoms) that the physician wishes to follow over time. The flowsheet layouts have been modified to create protocol specific flowsheets, e.g., lymphoma flowsheets have a different configuration than lung cancer flowsheets. The basic structure of the interface has been modified to use object-oriented methods, which allows for more flexible interaction between different components of the flowsheet and the operations performed on the flowsheet. A continuing area of research concerns how to guide the user to the most appropriate items to enter (based on the needs of the reasoning program) without disrupting the fixed layout of the flowsheet. The mainframe version of ONCOCIN modified the order of items on the flowsheet to extract necessary information from the user. In the workstation version, we have developed a guidance mechanism which alerts the user to items that are needed by the reasoning program. The user is not required to deviate from a preferred order of entry nor required to respond to a question for which no current answer is available. Cliff Wulfman is orimarily responsible for improvements to the user interface of ONCOCIN. o System support for the reorganization. The LISP language, which we used to build the first version of ONCOCIN, does not explicitly support basic knowledge manipulation techniques (such as message passing, inheritance techniques, or other object-oriented programming structures). These facilities are available in some commercial products, but none of the existing commercial implementations provide the reliability, speed, size, or special memory-manipulation techniques that are needed for our project. We have therefore developed a "minimal" object-oriented system to meet our specifications. The object system is currently in use by each component of the new version of ONCOCIN and in the software used to connect these components. In addition, all ONCOCIN student projects are now based on this programming environment. Christopher Lane created and is responsible for modifications to the object-oriented system. Interactive Entry of Chemotherapy Protocols by Oncologists (OPAL) We continue to refine the software (the OPAL system) that permits physicians who are not computer programmers to enter protocol information on a structured set of forms presented on a graphics display. Most expert systems require tedious entry of the system's knowledge. In many other medical expert systems, each segment of knowledge is transferred from the physician to the programmer, who then enters the knowledge into the expert system. We have taken advantage of the generally well- structured nature of cancer treatment plans to design a knowledge entry program that can be used directly by clinicians. The structure of cancer treatment plans includes: . choosing among multiple protocols (that may be related to each other); . describing experimental research arms in each protocol; . specifying individual drugs and drug combinations; E. H. Shortliffe 150 5P41 -RR00785- 15 ONCOCIN Project . setting the drug dosage level; . and modifying either the choice of drugs or their dosage. Using the graphics-oriented workstations, this information is presented to the user as computer-generated forms which appear on the screen. After the user fills in the blanks on the forms, the program generates the rules used to drive the reasoning process. As the user describes more detailed aspects of the protocol, new forms are added to the computer display; these allow the user to specify the special cases that make the protocols so complicated. Although the user is unaware of the creation of the knowledge base from the interaction with OPAL, a complex set of translations are taking place. The user's entries are mapped into an intermediate data structure (IDS) that is common for all protocols. From the IDS, a translation program generates rules for creating and modifying treatment, and integrates them with the existing ONCOCIN knowledge base. Considerable effort has been expended on producing a standard relational database as the appropriate data structure to underlie the OPAL IDS. The PROTEGE system described in the core ONCOCIN section was built upon this relational database. Although the "forms" were specifically designed for cancer treatment plans, the techniques used to organize data can be extended to other clinical trials, and eventually to other structured decision tasks. The key factor is to exploit the regularities in the structure of the task (e.g., this interface has an extensive notion of how chemotherapy regimens are constructed) rather than to try to build a knowledge-entry system that can accept any possible problem specification. The OPAL program is based upon a domain-independent forms creation package designed and implemented by David Combs. This program will provide the basis for our extension of OPAL to other application areas. We have now entered thirty-five protocols covering many different organ systems and styles of protocol design. Based on this experience, we continue to explore ways to modify OPAL to increase the percentage of the protocol that can be entered directly by our clinical collaborators. One direction in which we have extended the OPAL program is in providing a graphical interface of nodes and arcs to specify the procedural knowledge about the order of treatments and important decision points within the treatments. This work is described in several papers by Musen. C.2 Research in Progress The major thrusts in speech input and generalized knowledge acquisition are described in the core research description of ONCOCIN. We will describe here our research in complex therapy planning and it's spin-offs in temporal representations and summarization of patient records. C.2.7 Strategic Therapy Planning (ONYX) As mentioned above, we have continued our research project (ONYX) to study the therapy-planning process and to determine how clinical strategies are used to plan therapy in unusual situations. Our goals for ONYX are: (1) to conduct basic research into the possible representations of the therapy-planning process, (2) to develop a computer program to represent this process, and (3) eventually to interface the planning program with ONCOCIN. We have worked with our clinical collaborators to determine how to create therapy plans for patients whose special clinical situation preclude following the standard therapeutic plan described in the protocol document. The prototype program design has four components: (1) to review the patient's past record and recognize emerging problems, (2) to formulate a small number of revised 151 E. H. Shortliffe ONCOCIN Project 5P41 -RR00785- 15 therapy plans based on existing problems, (3) to determine the results of the generated plans by. using simulation, and (4) to weight the results of the simulation and rank order the plans by performing decision analysis. This model is described in the papers by Langlotz. We have built an expert system based on decision analytic techniques as part of the solution to the fourth step of the ONYX planning problem. The program carries out a dialogue with the user concerning the particular treatment choices to be compared, potential problems with the treatments, and the patient-specific utilities corresponding to the possible outcomes. A decision tree is automatically created, displayed on the screen, and solved. The solution is presented to the user, and is compatible with a explanation program for decision trees being developed as part of the Ph.D. research of Curtis Langlotz. A major spin-off from our ONYX work is a program that can summarize temporal trends in patient visits during chemotherapy and produce a summary of the patient's course using both data from the flowsheet and an underlying model of bone marrc;w physiology. This work has led to major improvements in the temporal representation and in the integrate of mathematical and symbolic models. This work is part of Michael Kahn's Ph.D. thesis. Summarization is defined as the task of combining multiple observations or features into a more general statement and abstraction as the task of selecting a subset of available features considered most relevant to answering a particular question. Both tasks require a model of the underlying system that encodes extensive knowledge about the entities and relationships that cause the system behavior and result in the observations. In the setting of a dynamic system, the model must be capable of representing temporal relationships between entities. This work proposes that the combination of mathematical and symbolic techniques can be used to construct useful summaries of complex time-ordered data. In particular, mathematical models are used to capture the knowledge about the physiological processes that are responsible for the patient's clinical findings. Model parameters represent physiological concepts that are clinically relevant for medical problem solving. Prior to any patient-specific observations, the model parameters are set to population-based estimates. Standard curve-fitting techniques using a Bayesian updating scheme adjust model parameters to new observations. As more patient-specific observations are obtained, the set of estimated model parameters move further away from the population estimates. Symbolic models are used to augment the mathematical model parameter and state estimates. As the patient's clinical course evolves, the symbolic model captures the concurrent contexts that affect the interpretation of the physiological model results. For example, a heart rate of 120 is considered abnormally high in the context of a resting person but may be inappropriately low in the context of a treadmill stress test. A key feature of the combined mathematical and symbolic approach is that the physiological model changes over time as additional data are obtained and the symbolic model modifies the interpretation of these model changes in light of the clinical contexts present when the data was observed. The methodology for combining mathematical and symbolic models emphasizes four main elements in summarizing complex time-ordered data: 1. A mechanistically-motivated model (in medicine, a physiological model) forms the basis for converting raw observations into more meaningful concepts. However, the interpretation of these concepts requires additional knowledge such as the contextual information contained in a symbolic model. E. H. Shortliffe 152 5P41 -RR00785- 15 ONCOCIN Project 2. The initial model is based on general knowledge since no specific observations are available to alter the initial impression. New observations will change the initial model by incorporating the new information. The collection of altered models captures state changes that have evolved over time. 3. Differences in key model features or states form the basis for selective abstraction and effective summarization. A method for determining which features are pertinent to a user question or sufficiently "interesting" to warrant inclusion into a summarization requires additional domain-specific reasoning. 4. The construction of a concise and useful summarization requires the use of additional contextual and domain-specific information so that the generated summary text conforms to the user's expectations and requirements. These principles form the basis for a computer program designed to summarize the clinical course of individual patients receiving experimental cancer chemotherapy. In this setting, patients are often receiving more than one treatment that have overlapping schedules and durations of action. Thus our temporal model requires the representation and the reasoning with multiple, simultaneous contexts to ensure the proper interpretation of a given observation or model estimate. ONCOCIN uses a specialized structure called the temporal network to represent treatment contexts used in temporal queries into a time-oriented patient record. We have extended the temporal network concept to create a symbolic model of the patient's clinical course over time. This structure permits the representation of multiple, concurrent contexts over time and therefore can capture the complex temporal nature of our patient's clinical course. For the proper interpretation of the mathematical model output, the temporal network provides the set of contexts that existed when the observation and model estimates were obtained. In addition, the interpretation task requires complex context-sensitive reasoning. For example, the interpretation of a model parameter may be different if two contexts were present concurrently than if either context was present alone. The temporal network provides a mechanism for altering the available reasoning methods based on the set of current contexts. In this use of the temporal network, reasoning methods are associated with each context. When a context is present, a temporal network node representing that context is created and the reasoning methods are made available to the interpretation process. A temporal network node may also withdraw methods made available by other temporal network nodes. In this manner, a general rule or method can be suspended if it is not appropriate in particular context. We believe that the combination of mathematical models along with specialized symbolic structures results in more representational and inferencing power than either method alone. Well established mathematical techniques convert observations into underlying system concepts while symbolic techniques interpret the mathematical results using additional domain knowledge. Although some of these features could possibly be represented using either mathematical or symbolic techniques alone, we believe the resulting "pure" models would be too complex to be useful. In combining the two approaches, concepts best modeled in a physiological paradigm can be expressed within the mathematical model while concepts best modeled symbolically can be represented within the temporal network. C.2.2 Documentation In 1986, we videotaped a lecture and demonstration of the ONCOCIN and OPAL systems at the XEROX Palo Alto Research Center. This videotape is available for E. H. Shortliffe ONCOCIN Project 5P41 -RR00785-15 loan from our offices. Our previous videotapes have been shown at scientific meetings and have, been distributed to many researchers in other countries. The publications described below further document our recent work on ONCOCIN. We have sent copies of our system to the University of Pittsburgh, and will distribute it to the National Library of Medicine. We have developed a user manual, description of sample interaction, reference card, and graphical flowchart to help with training in the use of ONCOCIN. D. Publications Since January, 7987 1. 2. 3. 4. 5. 6. 7. 8. 9. IO. Walton, J.D., Musen, M.A., Combs, D., Lane, CD., Shortliffe, E.H., and Fagan, L.M. Graphical access to medical expert systems: III. Design of a knowledge-acquisition environment. Memo KSL-85-30. Methods of - Information @ Medicine, 26:78-88, 1987. Musen, M.A., Fagan, L.M., and Shortliffe, E.H. Graphical specification of procedural knowledge for an expert system. Memo KSL-85-53. Presented at the Second IEEE Computer Society Workshop on Visual Languages, pp. 167-178, Dallas, TX, June 1986. Reprinted in Expert Systems: The User Interface (J. Hendler, ed.), pp. 15-35 (Chapter 2), Nor-wood, NJ: Ablex Publishing Company, 1988. Langlotz, C.P., Fagan, L.M., Tu, SW., Sikic, B.I., and Shortliffe, E.H. A therapy planning architecture that combines decision theory and artificial intelligence techniques. KSL-85-55. Computers B Biomedical Research, 20:279-303, 1987. Musen, M.A., Fagan, L.M., Combs, D.M., and Shortliffe, E.H. Use of a domain model to drive an interactive knowledge-editing tool (Memo KSL-86-24). International Journal of Man-Machine Studies - 26(1):105-121, 1987. Shortliffe, E.H. Artificial Intelligence in Management Decisions: ONCOCIN. Memo KSL-86-39. Proceedings of a Conference on Medical -- Information Sciences, University of Texas Health Sciences Get%- at San Antonio, July 1985. Also to appear in Frontiers of Medical Information - Sciences, Praeger Publishing, 1988. Langlotz, C.P., Shortliffe, E.H., and Fagan, L.M. A methodology for generating computer-based explanations of decision-theoretic advice. Technical report, KSL-86-57, July 1987. To appear in Medical Decision Making. Langlotz, C.P. and Shortliffe, E.H. An analysis of logic and decision- theoretic methods for planning under uncertainty. Report KSL-87-17, 1987. Shortliffe, E.H. Computer programs to support clinical decision making. Memo KSL-87-30. Journal of the American Medical Association, -- 258:6 l-67, 1987. Rennels, G.D. and Shortliffe, E.H. Advanced computing for medicine (Memo KSL-87-33). Scientific American, pp. 154-l 61, October 1987. Langlotz, C.P. Advice generation in an axiomatically-based expert system. Proceedings of the Eleventh Annual Symposium on -- Applications b Medical Care, pp. 49-55, Washington, D.C. 19m. Computer E. H. Shortliffe 154 5P41 -RR00785- 15 ONCOCIN Project 11. Shortliffe, E.H. and Hubbard, S.M. Information systems in oncology. Memo KSL-87-66, November 1987. To appear as a chapter in Cancer: Principles and Practice a Oncology (V.T. DeVita, S. Hellman, and S.A. Rosenberg, eds.), 1988. 12. Musen, M.A. Generation of Model-Based Knowledge-Acquisition Tools for Clinical-Trial Advice Systems. KSL-88-06. Doctoral dissertation, Medical Information Sciences Program, Medical Computer Science Group, Stanford University, January 1988. 13. Wulfman, C.E., Isaacs, E.A., Webber, B.L., and Fagan, L.M. Integration discontinuity: Interfacing users and systems. Memo KSL-88- 12, February 1988. Proceedings of Architectures for Intelligent Interfaces: Elements and Prototypes, pp. 57-68, MontereyxA, March 29-April 1, 1988. - 14. Musen, M.A. Conceptual models of interactive knowledge-acquisition tools. Report KSL-88-16, March 1988. 15. Musen, M.A. Generation of knowledge-acquisition tools from clinical-trial models. Report KSL-88-26, March 1988. To be published in The Proceedings &f Medical Informatics, Europe 1988, Oslo, Norway, August 1988. E. Funding Support Grant Title: "Therapy-planning strategies for consultation by computer" Principal Investigator: Edward H. Shortliffe Project Management: Lawrence M. Fagan Agency: National Library of Medicine ID Number: LM-04136 Term: April 1987 to March 1990 Total award: $380,123 Grant Title: "Knowledge Management for Clinical Trial Advice Systems" Principal Investigator: Edward H. Shortliffe Project Management: Lawrence M. Fagan Agency: National Library of Medicine ID Number: 1 R01 LM04420-01 Term: September 1985 through August 1988 Total award: $314,707 Grant Title: Postdoctoral Training in Medical Information Science Principal Investigator: Edward H. Shortliffe Project Management: Edward H. Shortliffe Agency: National Library of Medicine ID Number: 1 T32 LM07033 Term: July 1, 1984 - June 30, 1989 Total award: $903,718 Grant Title: Henry J. Kaiser Faculty Scholar in General Internal Medicine Principal Investigator: Edward H. Shortliffe Agency: Henry J. Kaiser Family Foundation Term: July 1983 to June 1988 Total award: $250,000 ($50,000 annually). 155 E. H. Shortliffe ONCOCIN Project 5P41 -RR00785- 15 Grant Title: Explanation of Computer-assisted therapy plans Principal Investigator: Lawrence M. Fagan Agency: National Institutes of Health ID Number: 1 R23 LM04316 Term: 2/l 985-l/1 988 Total award: $107,441 II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations and Program Dissemination via SUMEX A great deal of interest in ONCOCIN has been shown by the medical, computer science, and lay communities. We are frequently asked to demonstrate the program to Stanford visitors. We also demonstrated our developing workstation code in the Xerox exhibit in the trade show associated with AAAI-84 in Austin, Texas, IJCAI-85 in Los Angeles, AAAI-86 in Philadelphia, and Medinfo 86. Physicians have generally been enthusiastic about ONCOCIN's potential. The interest of the lay community is reflected in the frequent requests for magazine interviews and television coverage of the work. Articles about MYCIN and ONCOCIN have appeared in such diverse publications as Time and Fortune, and ONCOCIN has been featured on the "NBC Nightly News," the PBS "Health Notes" series, and "The MacNeil-Lehrer Report." Most recently it appeared in a special on Artificial Intelligence for TV Ontario (Canadian PBS station) and "Physician's Journal Update" on the Lifetime Cable Network. Due to the frequent requests for ONCOCIN demonstrations, we have produced a videotape about the ONCOCIN research which includes demonstrations ONCOCIN and OPAL. The tape has also been shown to both national and international researchers in biomedical computing. describes our speech research. We are producing a tape that Our group also continues to oversee the MYCIN program (not an active research project since 1978) and the EMYCIN program. Both systems continue to be in demand as demonstrations of expert systems technology. MYCIN has been demonstrated via networks at both national and international meetings in the past, and several medical school and computer science teachers continue to use the program in their computer science or medical computing courses. Researchers who visit our laboratory often begin their introduction by experimenting with the MYCINIEMYCIN systems. We also have made the MYCIN program available to researchers around the world who access SUMEX using the GUEST account. EMYCIN has been made available to interested researchers developing expert systems who access SUMEX via the CONSULT account. One such consultation system for psychopharmacological treatment of depression, called Blue-Box (developed by two French medical students, Benoit Mulsant and David Servan- Schreiber), was reported in July of 1983 in Computers and Biomedical Research. The EMYCIN experience is now well disseminated via commercial products. We may be able to demonstrate MYCIN from the 2020 after the 2060 is turned off, but this is not a major concern in the transition effort. 6. Sharing and Interaction with Other SUMEX-AIM Projects The community created on the SUMEX resource has other benefits which go beyond actual shared computing. Because we are able to experiment with other developing systems, such as INTERNISTICADUCEUS, and because we frequently interact with other workers (at AIM Workshops or at other meetings), many of us have found the E. H. Shortliffe 156 5P41 -RR00785-15 ONCOCIN Project scientific exchange and stimulation to be heightened. Several of us have visited workers at other sites, sometimes for extended periods, in order to pursue further issues which have arisen thrcugh SUMEX- or workshop-based interactions. In this regard, the ability to exchange messages with other workers, both on SUMEX and at other sites, has been crucial to rapid and efficient dissemination of ideas. Certainly it is unusual for a small community of researchers with similar scholarly interests to have at their disposal such powerful and efficient communication mechanisms, even among those researchers on opposite coasts of the country. During this past three years, we have had extensive interactions with Randy Miller at Pittsburgh. Via floppy disks and SUMEX, we have experimented with several versions of the QMR program. The interaction was very much facilitated by the availability of SUMEX for communication and data transmission. Several recent papers have been written to describe collaborations between students in our training program and the group at the University of Pittsburgh and Carnegie-Mellon University. C. Critique of Resource Management Our community of researchers has been extremely fortunate to work on a facility that has continued to maintain the high standards that we have praised in the past. The staff members are always helpful and friendly, and work as diligently to please the SUMEX community as to please themselves. As a result, the computer is as accessible and easy-to-use as they can make it. More importantly, it is a reliable and convenient research tool. We extend special thanks to Tom Rindfleisch for maintaining such high professional standards. As our computing needs grow, we have increased our dependence on special SUMEX skills such as networking and communication protocols. As described above, we will eventually be moving our software development to a combination of Lisp machines and Mac II's. These will need to be easily networked together while still providing the communications resources of the DEC-20. Ill. RESEARCH PLANS A. Project Goals and Plans In the coming year, there are several areas in which we expect to expend our efforts on the ONCOCIN System: 1. To generalize the reasoning and interaction components of the ONCOCIN system for other applications. 2. Extensions and generalizations of the OPAL and PROTEGE knowledge acquisition experiments 3. Extensions to the strategic planning framework including research on temporal representations, mathematical modeling and summarization 4. To continue testing of the workstation version of ONCOCIN 6. Justification and Requirements for Continued SUMEX Use All of our research takes place in the context of the SUMEX resource. The development of our project will continue to take place on LISP machines which we have purchased or which have been donated by the XEROX Corporation, with a gradual transition to Mac II's. The word processing and communication requirements will be met by the Mac II's plus the excellent network services provided by SUMEX. 157 E. H. Shortliffe ONCOCIN Project 5P41 -RR007851 5 C. Requirements for Additional Computing Resources Most of our CPU needs are met with our current equipment. However, our requirements for high bandwidth communication facilities are increased as we have an increasingly distributed environment. We would appreciate support that provides equivalent resources to what we are used to on the DEC-20 including mail, support, and file servers. We will continue to need access to the international networks that we now use for much of our communication with colleagues. For example, Mark Musen (who recently finished his Ph.D., and will be an Assistant Professor in our department when he returns) is currently in Holland for 9 months. During this period, we have been in constant electronic communication with him over a set of networks. This has been a significant asset to our research project. D. Recommendations for Future Community and Resource Development The continuation of network support and file service are our major needs. Help with software selection and implementation for our Mac II's is also an important requirement to maintain research continuity as the DEC-20 is disconnected. Maintaining the excellent mail service and access to the international networks is also essential for our research. E. H. Shortliffe 158 5P41 -RR00785-15 PROTEAN Project IV.A.5. PROTEAN Project PROTEAN Project Oleg Jardetzky Nuclear Magnetic Resonance Lab, School of Medicine Stanford University Bruce Buchanan, Ph.D. Computer Science Department Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale The goals of this project are related both to biochemistry and artificial intelligence: (a) use existing Al methods to aid in the determination of the 3-dimensional structure of proteins in solution, and (b) use protein structure determination as a test problem for experiments with large scale constraint satisfaction, an area of increasing interest to artificial intelligence. Empirical data from nuclear magnetic resonance (NMR) and other sources may provide enough constraints on structural descriptions to allow protein chemists to bypass the laborious methods of crystallizing a protein and using X-ray crystallography to determine its structure. This problem exhibits considerable computational complexity, but by formulating it as search problem, it should be possible to utilize many of the knowledge-based techniques developed in Al for dealing with large search spaces. B. Medical Relevance The molecular structure of proteins is essential for understanding many problems of medicine at the molecular level, such as the mechanisms of drug action. Using NMR data from proteins in solution will allow the study of proteins whose structure cannot be determined with other techniques, and will decrease the time needed for the determination. C. Highlights of Progress Atomic level determination of the structure of cytochromeb562. During the past year -- we greatly expanded the functionality of PROTEAN towards obtaining meaningful biochemical results. In the first half of the year the primary outcome of this effort was the atomic-level assembly of the protein cytochromeb562 using simulated NMR constraints from the known crystal structure. The strategy for assembling cytochrome involved functionality at the atomic as well as the more abstract solid level, and demonstrated one possible application of our general approach, which we call heuristic refinement. The heuristic refinement method falls within an exclusion paradigm for interpreting protein structure. In this paradigm all the atoms in the protein are initially assumed to be everywhere with respect to some arbitrary coordinate system. As constraints are introduced these initially infinite accessible volumes are gradually reduced until the remaining accessible volumes are small enough so that a representative set of structures may be enumerated. 159 E. H. Shortliffe PROTEAN Project 5P41 -RR00785 15 The exclusion paradigm has not been tried previously because it appears to be computationally intractable for any reasonably sized protein. For this reason most current techniques (non Al-based) use an adjustment paradigm to optimize a single structure towards a global minimum. These techniques all share difficulties common to optimization, namely lack of convergence for large numbers of variables or for starting structures far from the global minimum. In addition, for reasons of computation time, they are not able to provide a representative sample of all structures compatible with the constraints. The exclusion paradigm, by systematically eliminating structures incompatible with the constraints, has the advantage that all, or at least a representative set, of structures compatible with the constraints are retained. In order to deal with the combinatorics the heuristic refinement method formulates the problem as one of search, which is the fundamental paradigm of artificial intelligence. In particular, the problem is formulated as a geometric constraint satisfaction problem (GCSP), which can be solved by backtrack search if the size of the atomic accessible volumes can be made small enough. The complexity of solving a GCSP depends on the number of objects and the number of possible locations that each object can have in space. Therefore, the heuristic refinement method utilizes four techniques to reduce these two numbers to the point that backtrack search may be applied. The effect of each of these techniques is to utilize knowledge of protein structure and of techniques for solving GCSPs to prune the search tree: 1. Problem decomposition. The protein is broken into logical subparts such as secondary structures and sidechains. Each of these is treated as a separate GCSP, which is partially solved. Solutions to the subproblems either determine constraints at more abstract levels or are combined into larger subproblems. 2. Problem abstraction. Groups of locally highly constrained atoms are represented as single abstract "solid" level objects. Solutions to atomic level GCSPs are used to create abstract "solid" level constraints, resulting in a solid level GCSP with far fewer objects. Solutions at the solid level are then expanded to the atomic level, resulting in fewer possible locations for the atoms than would have been present if the initial solid level processing had not occurred. Almost all the work reported in previous reports was at this abstract solid level, but it is the current refinement to the atomic level that is of most interest to biochemists. 3. Local satisfaction of constraints. In each local GCSP filtering operations called network consistency algorithms are applied, allowing the accessible volumes to be reduced by looking at constraints pairwise (or to higher order) rather than all at once. These operations, which were initially developed for computer vision, are becoming more popular in Al beCaUS8 of their utility in many forms of constraint satisfaction problems. 4. Heuristic control. At 8aCh point in the problem solving there are many possible constraint satisfaction operations that may be applied. If there is no bias then the order of operations should not matter, but the efficiency may vary greatly. Much of the previously reported work in PROTEAN went towards developing the BBI framework for using heuristics to opportunistically determine the best operation to apply at any given time. The BBl framework has been used to control the solid abstract level of problem solving, but it has not yet been integrated with the atomic level functionality, which is currently manually controlled. E. H. Shortliffe 160 5P41 -RR00785-15 PROTEAN Project These principals were used to determine the structure of cytochromeb562. The program was given. as input the primary and secondary structure as obtained from the known crystal struc!ure. It was also given a set of 729 distance ranges designed to simulate the expected type of data from NMR. Using these data, as well as knowledge of protein structure, the heuristic refinement method produced a set of IO atomic level structures, with an average root mean square deviation from the crystal structure of 4.1 angstroms when all backbone alpha-carbons were included, and 2.8 angstroms when random-coil segments were excluded. These deviations were on the order of those found by other methods. Because we systematically excluded structures we could be confident that we had obtained an upper bound on the set of structures compatible with the constraints. These structures could then be adjusted by any of the current techniques (although we didn't do that in this case). The cytochrome results were obtained over a period of about three to four months, using the previously-described geometry system and BBI to obtain the solid level results, and LISP and C code to produce the atomic level results. Intermediate results were written to files, and individual programs were written by three different people. Removing bias. During the latter part of this year we have concentrated on improving the functionality used to obtain the cytochrome results. In the heuristic refinement method an important tradeoff is efficiency versus the amount of bias introduced by the abstractions. A major source of bias is the assumption that secondary structures are in their ideal configuration. For example, in the cytochrome and all previously reported results alpha helices were modeled as cylinders. This approximation meant that some legal atomic level solutions are prematurely eliminated at the solid level. In order to reduce this bias we have looked at actual examples of helices in the crystal structure database, and have used these to form more realistic constraints at the solid level. This relaxation of the ideal helix assumption has allowed us to obtain more accurate solid level accessible volumes, but at the cost of less constraining power. In later versions of the program we plan to allow the user to specify how much ideality to assume. Atomic level constraint satisfaction operations. As part of his Ph.D. work Bruce Duncan has extended many of the solid level constraint satisfaction operations to the atomic level. He has developed programs, written in C, which allow atomic locations to be described at different levels of resolution, from a coarse grid initially to a finer grid as refinement proceeds. This approach has been used to reconstruct the crystal structure of a small protein called crambin given exact distances less than five angstroms. In this case no solid level preprocessing was done, and the entire process took about six weeks on three Suns running in parallel. Bruce is currently working to develop more intelligent control before trying his system on a more realistic simulated data set. Probabilistic operations at the atomic level. -- Russ Altman, as part of his Ph.D. thesis, has been working to develop an alternative method for representing the accessible volumes of atoms. Currently, we represent accessible volumes as lists of xyz locations sampled on a regular grid at a specified resolution. In Russ's approach accessible volumes are represented as probability density functions described by a mean and covariance matrix. Constraints are also represented by a mean and variance, and a Kalman filter, which is basically another form of Bayes Formula, is used to determine the reduction in accessible volume. This approach has been used to reconstruct the amino acid tyrosine, and is being extended to handle larger numbers of atoms. Development of an integrated software environment and general strategies. All our -- E. H. Shortliffe PROTEAN Project 5P41 -RR00785-15 current results were obtained with more or less independently developed software modules. No one person is able to run PROTEAN by himself. For this reason we have designed a framework that allows these modules to communicate via common files organized as a distributed object-oriented knowledge base. Parts of this framework are currently being implemented by a part time programmer. The framework will allow the existing solid level functionality to be integrated with the emerging atomic level, and will allow different assumptions, representations and algorithms to be experimented with. Porting of heuristic control & TI Explorer in Common Lisp. The reasoning component in BB1 is currently written in InterLxp and runs on Xerox D-machines. Since InterLisp and D-machines are becoming obsolete the code has now been partially ported to the TI Explorer in Common Lisp. Network software has been written that allows the Explorer to control the geometry and graphics programs over the network. Once the port has been completed additional heuristic control experiments will be performed. Secondary structure prediction. An important pre-processing step for PROTEAN is the determination of secondary structure from NMR. John Brugge, for his masters thesis, designed a program called ABC which emulates expert knowledge in assigning secondary structure from NMR. The program is written in BBl and uses symbolic patterns in the NMR to predict structure. Validation studies showed that the program did about as well as experts. Peak Assignment. When an adequate amount of purified protein is available, the routine determination of protein structure in solution by nuclear magnetic resonance would require efficient methods for: 1) collecting data, 2) extracting structural constraints from the data, and 3) generating protein structures which satisfy these constraints. Many two-dimensional nuclear magnetic resonance (2D-NMR) experiments now exist to carry out step 1 ), and a number of computer methods have been presented to deal with step 3). However, step 2) is a severe bottleneck. The main difficulty in that step lies in the assignment problem in which each peak from 2D-NMR spectra must be matched to a pair of specific atoms from specific residues. Typically, hundreds of resonances must be identified, a tedious, slow, and error prone task. It involves keeping track of a large number of peaks, each of which initially has a large number of possible assignments. This complexity, however, contrasts with the relatively small number of principles and pattern matching techniques that are actually needed to assign the peaks in a 2D-NMR protein spectra. Craig Cornelius, Guido Haymann-Haber, and Olivier Lichtarge have developed a prototype program Called PEAKS that uses constraint satisfaction methods to apply patterns expected from amino acid residues to actual spectra. The program has been tested on simulated data sets for small proteins of 20 to 30 residues. The side chains of most residues can be correctly identified and sometimes assigned. The program can also deal with slightly incomplete spectra with degenerate or missing peaks. D. Relevant Publications 1. Altman, R. and Jardetzky, 0.: New strategies for the determination of macromolecular structures in solution. Journal of Biochemistry (Tokyo), Vol. 100, No. 6, p. 1403-1423, 1986. 2. Altman, R. and Buchanan, B.G.: Partial Compilation of Control Know/edge. Proceedings of the AAAI, pp 399-404, 1987. E. H. Shortliffe 162 5P41 -RR00785-15 PROTEAN Project 3. Altman, R., Duncan, B., Brinkley, J., Buchanan, B., Jardetzky, 0.: Determination. of the spatial distribution of protein structure using solution data, Proceedings of the Alfred Benzon Symposium 26: NMR Spectroscopy and Drug Development, Copenhagen, 1987. 4. Brinkley, J., Cornelius, C., Altman, R., Hayes-Roth, B. Lichtarge, O., Duncan, B., Buchanan, B.G., Jardetzky, 0.: Application of Constraint Satisfaction Techniques to the Determination of Protein Tertiary Structure. Report KSL-86-28, Department of Computer Science, 1986. 5. Brinkley, James F., Buchanan, Bruce G., Altman, Russ B., Duncan, Bruce S., Cornelius, Craig W.: A Heuristic Refinement Method for Spatial Constraint Satisfaction Problems. Report KSL 87-05, Department of Computer Science. 6. Brinkley, J.F., Altman, R.B., Duncan, B.S., Buchanan, B.G., and Jardetzky, 0.: The heuristic refinement method for the derivation of protein solution structures: Validation on cytochrome-b562, KSL Technical Report 88-03, 1988. 7. Brugge, J.A., Buchanan, B.G., and Jardetzky, 0.: Toward automating the process of determining pofypeptide secondary structure from `I-/ NMR data, To be published in J. Computational Chemistry, 1988. 8. Buchanan, B.G., Hayes-Roth, B., Lichtarge, O., Altman, A., Brinkley, J., Hewett, M., Cornelius, C., Duncan, B., Jardetzky, O.:The Heuristic Refinement Method for Deriving Solution Structures of Proteins. Report KSL-85-41. October 1985. 9. Duncan, B., Buchanan, B., Hayes-Roth, B., Lichtarge, O., Altman, R., Brinkley, J., Hewett, M., Cornelius, C., and Jardetzky, 0.: PROTEAN: A new method for deriving solution structures of proteins, Bull. Mag. Res., 8:111-119, 1987. 10. Garvey, Alan, Cornelius, Craig, and Hayes-Roth, Barbara: Computational Costs versus Benefits of Control Reasoning. Report KSL 87-l 1, Department of Computer Science. 11. Hayes-Roth, B.: The Blackboard Architecture: A General framework for Problem Solving? Report HPP-83-30, Department of Computer Science, Stanford University, 1983. 12. Hayes-Roth, B.: BBI: An Environment for Building Blackboard Systems that Control, Explain, and Learn about their own 68haViOr. Report HPP-84-16, Department of Computer Science, Stanford University, 1984. 13. Hayes-Roth, B.: A Blackboard Architecture for Control. Artificial Intelligence 26:251-321, 1985. 14. Hayes-Roth, B. and Hewett, M.: Learning Control Heuristics in BBi. Report HPP-85-2, Department of Computer Science, 1985. 15. Hayes-Roth, B., Buchanan, B.G., Lichtarge, O., Hewett, M., Altman, R., Brinkley, J., Cornelius, C., Duncan, B., and Jardetzky, 0.: PROTEAN: Deriving protein structure from constraints. Proceedings of the AAAI, 1986, p. 904-909. 163 E. H. Shortliffe PROTEAN Project 5P41 -RR00785- 15 16. Jardetzky, 0.: A Method for the Definition of the Solution Structure of Proteins from NMR and Other Physical M8aSUrementS: The LAC- Repressor Headpiece. Proceedings of the International Conference on the Frontiers of Biochemistry and Molecular Biology, Alma Alta, June 17-24, 1984, October, 1984. 17. Lichtarge, Olivier: Structure determination of proteins in solution by NMR. Ph.D. Thesis, Stanford University, November, 1986. 18. Lichtarge, Olivier, Cornelius, Craig W., Buchanan, Bruce G., Jardetzky, Oleg: Validation of the First Step of the Heuristic Refinement Method for the Derjvation of Solution Structures of Proteins from NMR Data., Proteins: Structure, Function and Genetics, 2:340-358, 1987. E. Funding Support Title: Interpretation of NMR Data from Proteins Using Al Methods PI's: Oleg Jardetzky and Bruce G. Buchanan Agency: National Science Foundation Grant identification number: DMB-8402348 Total Award Period and Amount: 2/l/87 - g/30/89 $120,000 (includes direct and indirect costs) Current award period and amount: 2/l/87 - 9130189 $120,000 (includes direct and indirect costs) The following grants and contracts each provide partial funding for PROTEAN personnel. Title: Knowledge-Based Systems Research PI: Edward A. Feigenbaum Agency: Defense Advanced Projects Research Agency Grant identification number: N00039-86-0033 Total award period and amount: IO/l/85 - g/30/88 $4,130,230 (direct and indirect) Current award period and amount: 10/l/87 - g/30/88 $1,467,300 (direct and indirect) PROTEAN component is $72,956, or 5.0 % of grant total II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations Several members of Prof. Jardetzky's research group are involved in this research. B. Interactions with other SUMEX-AIM projects We are occasionally in contact with researchers at Robert Langridge's laboratory at the University of San Francisco. C. Critique of Resource Management The SUMEX staff has continued to be most cooperative in supporting PROTEAN research. The SUMEX computer facility is well maintained and managed for effective support of our work. The computer network and Lisp workstations are supported very effectively by the SUMEX staff. E. H. Shortliffe 164 5P41 -RR00785-15 PROTEAN Project Ill. RESEARCH PLANS A. Goals 8 Plans Our long range biochemical goal is to build a system that can automatically interpret NMR and other types of constraints, in order to produce the family of structures compatible with the constraints. In so doing we have encountered and will continue to encounter important and interesting artificial intelligence problems in large scale, knowledge-based constraint satisfaction, The major steps towards these goals are as follows, more or less in order of expected order, although the validations will proceed in parallel with the addition of new features. Several of these steps have been worked on within the past year (see Highlights of Research Progress). A.1. D8V8lOp an integrated system. In order for the heuristic refinement method to be widely used it must be integrated into a computer system that contains the required functionality, is reasonably efficient, is semi-automatically controlled, relatively easy to use, and extensible. Using our previous work as a basis we will continue to work towards this goal. The required substeps in order to achieve the goal of an integrated system are: o Develop improved functionality. This is our current major focus and includes efforts to relax the ideal helix assumption, to utilize proven techniques from other structure determination methods (such as the use of distance bound smoothing algorithms as a pre-processing step, and the use of adjustment procedures as a post-processing step), to develop better methods of representing the spatial locations of both three-dimensional (non-oriented) objects, and six-dimensional (oriented) objects, and to develop more efficient constraint satisfaction algorithms. Progress has been made in several of these areas over the past year, especially by the Ph.D. students as they develop their dissertations. o Develop system architecture. During this year we have also designed a basic architecture for a distributed system that can be extended as new modules and representations are developed. Much of the programmers effort this year will go towards implementing this system and towards incorporating some of the advances made by the Ph.D. students. o Develop control strategies and knowledge @. As the integrated functionality is developed it will be made available either as remote servers accessible over the network (as with our current geometry system), or as loadable modules. BBI, or another suitable object-oriented expert system shell, will be used to create the control modules. These modules will use heuristics to intelligently pick the most efficient constraint satisfaction operation(s) to perform next. Progress towards one such control module has been made with the porting of the BBl system to the TI Explorer. o Develop 5 interface and improve graphics. To be useful the system should be relatively easy to run by a non-programmer biochemist. We have not yet begun to explore this issue, but other Al in Medicine projects at Stanford and elsewhere have worked in this area. Examples at Stanford are Oncocin and its derivatives, and Apple's HyperCard as an interface to an object-oriented belief network. Once the programmer has created a rudimentary and extensible system he will begin to look at this issue. A.2. Extend the system. As the required functionality, framework and control 165 E. H. Shortliffe PROTEAN Project 5P41 -RR00785- 15 strategies become integrated a version of the system will be made available (at least at Stanford) for analyzing real data. At the same time we will use the integrated system as the basis for extending its capabilities. Developing these new capabilities will require solution to some important Al problems dealing with object representation and reasoning over time as well as space. These extensions include: . Studying larger proteins and protein complexes. Since our system is hierarchical there is in theory no reason why we can't move all the way up the anatomic spectrum from atoms to organs. . Add new types of constraints, including global constraints such as volume and surface, and probabilistic criteria. The current work of Russ Altman should be very useful in this regard. . Automate the earlier interpretation of NMR data. We have made good beginnings in this area with the ABC program for secondary structure prediction and the peak detection programs. We will also continue our development of the PEAKS system for interpretation of 2-D NMR spectra, to be integrated with ABC and the rest of the PROTEAN system. . Analyze structure. The purpose of determining structure is to understand function, and a component of this is analyzing the structures produced. We have already developed several modules for analyzing structure, such as programs for determining root mean square deviations and for calculating distance distributions. Russ Altman has also done some work in automatically extracting features of interest from a protein, and will continue this as part of his dissertation. . Study protein dynamics. In some cases proteins in solution may be moving, a fact which greatly complicates the analysis of NMR data. So far we have ignored this issue because it is hard enough to solve the problem with stationary structures, but as we gain experience we will begin to consider these issues. Many problems dealing with incompatible constraints will arise as we look at this problem. A.3. Validate on known crystal structures. As new functionality is developed, it will be verified by attempting to reconstruct proteins whose structures are known from crystallography. In this way we will at least know that our methods work, even if there is no gold standard for solution data. A.4. Use on actual NMR and theoretical constraints. As the system is developed and verified on crystal data it will be made available to biochemists. Only those capabilities that have been verified will be supported. Funds for actually using the system on real data will be expected to be provided in the context of specific research projects. 5. Justification for continued SUMEX use We will continue to use the SUMEX facilities for integrating and expanding the PROTEAN system. The 2060 (or whatever replaces it) will primarily be a focus for integration and communication among machines and humans. Most of the actual program development will be on workstations running C or Common Lisp, communicating via the excellent local area network maintained by SUMEX. The communication and clearinghouse capability provided by SUMEX and the SUMEX staff is essential for continued success of such a large and evolving distributed project. E. H. Shortliffe 166 5P41 -RR007851 5 PROTEAN Project C. Need for other computing resources At this time our computational resources are almost adequate as long as we can continue to use the SUMEX-supported workstations and local area network. We could always use faster machines since many of our computations are very numerically intensive, but our major need is for personnel and systems, support. Since PROTEAN is evolving into a distributed problem-solving system we actively support and require extensive networking and remote procedure access capabilities. We believe that the requirements of PROTEAN make it an ideal test problem for many of the distributed systems issues being considered by the SUMEX core research staff. 167 E. H. Shortliffe RADIX and PENGUIN Projects 5P41 -RR007851 5 IV.A.6. RADIX and PENGUIN Projects The RADIX Project: Deriving Medical Knowledge from Time-Oriented Clinical Databases The PENGUIN Project: Applying Database and Knowledge base Technology to Medical Instrumentation Gio Wiederhold, Ph.D. Departments of Computer Science and Medicine Stanford University Thierry Barsalou, M.D. Medical Computer Science Stanford University Robert L. Blum, M.D., Ph.D. Department of Computer Science Stanford University I. SUMMARY OF RESEARCH PROGRAM The RADIX research has been phased out during this year. Although proposals to NLM and NCHSR for continued research support were approved, they have not been funded. Some closeout funding enabled the project to be brought to an orderly conclusion, and publications continue to appear. The experience gained here and under the predecessor project, RX, continues to influence basic research directions. The problem of automated knowledge acquisition remains an important area of research, and we expect that the foundations laid here will influence work of others as well. Some of the basic research has influenced the KBMSVKSYS project. Specifically problems encountered in the management and structuring of large quantities of knowledge are now being addressed, primarily under DARPA sponsorship. A subproject, on knowledge validation is being supported by IBM and will draw directly on the data and knowledge bases established for RADIX in its initial phases. The KBMS/KSYS work, in turn, is leading to another important medical application: PENGUIN. This project is currently mainly supported by KBMS, but a substantial research grant has been approved by NLM, and given a high priority, so that we are confident that we will be able to soon move forward intensely in this project. We will, below, provide two distinct sections. The first one will provide the final report of achievements under RADIX and the second will outline the current state and plans for PENGUIN. RADIX PROJECT GOALS AND PROGRESS E. H. Shortliffe 168 5P41 -RR00785-15 RADIX and PENGUIN Projects A. RADIX -- Technical Goals The objectives of the RADIX project were 1) Discovery: to provide knowledgeable assistance to a research investigator in studying medical hypotheses on large databases, and to automate the process of hypothesis generation and exploratory confirmation, 2) Summarization: to develop a program and set of techniques for automated summarization of patient records, and 3) Peer Review: to develop a program to assist physician reviewers examine case databases for medical peer review and quality assurance. For system development we have used a subset of the ARAMIS database. We will first describe our work on discovery, followed by summarization and peer review. Research was performed on the first two objectives. RADlX Discovery Module: Computerized clinical databases and automated medical records systems are starting to contain valuable long-term records of medical practice and experience. We have utilized the data collected by the ARAMIS Project, (American Rheumatism Association Medical Information System) under development since 1969 in the Stanford Department of Medicine. ARAMIS contains records of over 17,000 patients with a variety of rheumatologic diagnoses. Over 62,000 patient visits have been recorded, accounting for 50,000 patient-years of observation. A fundamental objective of the ARAMIS Project and many other clinical database projects is to use the data that have been gathered by clinical observation in order to study the evolution and medical management of chronic diseases. Unfortunately, the process of reliably deriving knowledge has proven to be exceedingly difficult. Numerous problems arise stemming from the complexity of disease, therapy, and outcome definitions, from the complexity of causal relationships, from errors introduced by bias, and from frequently missing and outlying data. A major objective of the RADIX Project is to explore the utility of symbolic computational methods and knowledge-based techniques at solving some of these problems. The first phase of the RADIX computer program helps to examine the time-oriented ARAMIS database displays information which helps recognize possibly causal relationships. The important concept is here the mapping of base information to higher level abstractions which are meaningful to physicians. Instances of data, when mapped, can display a patient's course in a compact and meaningful fashion. The intent was also that knowledge acquisition from physician-researchers can then be brought to a level where the relationships are meaningful. Instances of such relationships were acquired in the RX projects, but that work was not yet based on the more general concepts envisaged for RADIX. RADIX Summarization Module: The management of inpatients and outpatients is often complicated by the size and disorganization of patient charts. Computerized patient records are becoming increasingly available. While computerization of records renders data, it worsens the problem of information overload. The ability to automatically create patient summaries represents a useful adjunct to a patient record for rapid review of a case, for clinical decision making and patient monitoring, and for surveillance of quality of care. The goal of the RADIX summarization program is to infer a summary of a patient's clinical history from lengthy on-line medical records. The RADIX summarization program is a knowledge-based sub-system which produces intelligent summaries from a time-oriented data base of Systemic Lupus Erythematosus patients. Medical concepts in the system are represented by three entities of increasing complexity: abnormal primary attributes, abnormal states and diseases. Abnormal states and diseases are derived from the abnormal primary attributes by the Reasoner using a combination of model-driven and data-driven 169 E. H. Shortliffe RADIX and PENGUIN Projects 5P41 -RR00785-15 algorithms. Uncertainty associated with the derived states is handled with a Bayesian approach supplemented by boolean predicates, using likelihood ratios obtained from a transformation of the INTERNIST knowledge base. After summarizing the data, the system generates interactive, graphical displays with optional explanation windows. The prototypes we have implemented have shown that intelligent summarization of medical records is feasible and that interactive graphical display is of great help in conveying complex medical information. We have reported on the results of this work. B. RADlX -- Medical Relevance and Collaboration As a test bed for system development, our focus of attention were the records of patients with systemic lupus erythematosus (SLE) contained in the Stanford portion of the ARAMIS Data Bank. SLE is a chronic rheumatologic disease with a broad spectrum of manifestations. Occasionally the disease can cause profound renal failure and lead to an early death. With many perplexing diagnostic and therapeutic dilemmas, it is a disease of considerable medical interest. The medical relevance of the automated summarization program is readily apparent. A practicing physician or medical researcher, faced with a patient chart, often with dozens of visits and scores of attributes, rarely has time to read the entire chart. He (or she) would like a succinct summary of the important events in that patient's record to assist his decision making. The use of computerized medical records improves the quality of information but does not solve the problem of information overload. For this reason, it would be useful to have the ability to automatically summarize patient records into meaningful clinical events. C. RADIX -- Highlights of Research Progress C.7 April 7987 to September 7988 Our primary accomplishments in this period have been the following: I. Implementation and demonstration of a second generation of the automated summarization program. 2. Application of algorithms for transforming the Internist knowledge base into standard Bayes form. 3. Publication of papers on automated summarization, and presentation of results at medical conferences. 4. Training post-doctoral researchers, participants in RADIX, in methods of medical artificial intelligence research. C.1.7 Design and implementation of a second generation of the prototype automated summarization program We have completed a second generation of our prototype automated summarization program. This work is described in DeZegher-Geets, 1987, noted in the publications section. It improves upon a prototype implemented by Downs (Downs 1986); the knowledge base has been substantially enlarged, the inference mechanisms refined and enhanced for temporal reasoning, and the graphical display capability has been expanded. The summarization program produces intelligent summaries from a time- oriented data base of Systemic Lupus Erythematosus patients. Medical concepts in the system are represented by three entities of increasing complexity: abnormal E. H. Shortliffe 170 5P41 -RR00785-15 RADIX and PENGUIN Projects primary attributes, abnormal states and diseases. Abnormal states and diseases are derived from the abnormal primary attributes by the Reasoner using a combination of model-driven and data-driven algorithms. Uncertainty associated with the derived states is handled with a Bayesian approach supplemented by boolean predicates, using likelihood ratios obtained from a transformation of the INTERNIST knowledge base. After summarizing the data, the system generates interactive, graphical displays with optional explanation windows. C.7.2 Algorithms for transforming the internist know/edge base into standard Bayes form INTERNIST-l is an expert system for diagnosis across a broad spectrum of disease. Over twenty man-years of effort have gone into the construction of its knowledge base which contains relationships between approximately 600 diseases and 4,000 manifestations of disease. A major limitation of INTERNIST-l is that the quantities used within the system to represent uncertainty, called evoking strengths and frequencies, are poorly defined. This makes it difficult to tune the method used by the program to assign likelihoods to diseases (the scoring scheme) and makes it difficult to transport knowledge contained in the program to other medical diagnostic systems. In collaboration with R. Miller and D. Heckerman we have transformed the values in the INTERNIST KB into a probabilistic form. Various combinations of multiple regressions were performed on the evoking strengths, frequencies, probabilities of disease, p(D), and probabilities of manifestation, p(M), versus the likelihood ratios L(DjM) and L(Djnot M). This process yielded some interesting and unexpected results. For example, the multiple regression of evoking strength AND p(M) vs. L(DjM) showed an r-squared of 0.84, significantly better than the r-squared value for evoking strength vs. L(DjM) alone. Also, the transformation from frequency, p(M), and p(D) into L(Djnot M) revealed a correlation coefficient of 0.58. These results suggest a low cost method for converting the knowledge in INTERN!ST-1 to a probabilistic form. In particular, assessments of p(D) and p(M) (only about 4500 numbers) can be used in conjunction with evoking strengths and frequencies in the KB (about 40,000 numbers) to construct likelihood ratios. C.7.3 Publication of papers on automated summarization, and presentation of results at medical conferences We have submitted and had accepted several papers, as noted in the section on publications. Some of these were presented at medical conferences. C. 7.4 Training Post-Doctoral researchers, participants in RADIX, in methods of medical artificial intelligence research We have been training three post-doctoral researchers on the project during the earlier part of the current reporting year; Andrew G. Freeman, M.D., lsabelle de Zegher-Geets, M.D., and Donald Rucker, M.D.. Andrew Freeman has developed the Internist transformation algorithms. lsabelle de Zegher-Geets completed a thesis on Automated Summarization as part of Stanford's Medical Information Sciences program. PENGUIN PROJECT GOALS AND PROGRESS A. PENGUIN Project -- Rationale Databases and expert systems share a common goal -- generating useful information for action -- but accomplish their tasks separately, using different principles. It is clear, however, that future information systems will require both the problem-solving capabilities of expert systems (ESs) and the data-handling capabilities of database 171 E. H. Shortliffe RADIX and PENGUIN Projects 5P41 -RR00785-15 management systems (DBMSs). Indeed, combining database and expert system technologies into expert database systems (EDSs) is an emerging research area. One can define an EDS as "a system for developing applications requiring knowledge-directed processing of shared information". From a perspective of developing advanced biomedical information systems, this definition conveys two precise scenarios: (1) enhancing DBMSs with structuring and manipulation tools that take more semantics into account; (2) allowing ESs to access and to handle efficiently information stored in database(s). The object-oriented paradigm has gained much attention in recent years. In the database field, object-oriented DBMSs have emerged. The concept of an entity is also widely used by database design tools. In the field of artificial intelligence, frames are a well-known knowledge representation scheme. Although frames were conceived separately from the object paradigm, the two are in fact consistent with each other. In this research project, called PENGUIN, we investigate the hypothesis that the object-oriented approach can also serve as a unifying scheme for developing EDSs. We have the opportunity to explore this hypothesis in a practical biomedical environment. Fluorescence Activated Cell Sorting (FACS) is emerging as a major source of information for biomedical research and clinical practice. Currently, achieving good FACS performance requires analyzing and integrating various complex data and knowledge sources. PENGUIN is thus aimed at developing methods for bringing together expert system and. database technologies in an integrated advice system that could fulfill the information needs of FACS investigators. Central to this work is a very close collaboration between researchers in the Medical Information Sciences Program and the Departments of Computer Science and Genetics. B. PENGUlN -- Medical Relevance and Collaboration FACS has already demonstrated significant promise in clinical as well as research- oriented areas. As indicated earlier, basic studies identifying lymphocyte subpopulations have now been translated into clinical applications including monitoring of AIDS patients. Similar FACS applications can now be found in such diverse specialties as Hematology, Oncology, Infectious Diseases and even Gynecology-Obstetrics where current studies are evaluating FACS analysis of maternal blood sample as a potential non-invasive method for prenatal diagnoses. The major block to proliferation of these kinds of valuable studies involves the requirement for greater skills in reagent selection and FACS machine operation than are typically available in basic and clinical research settings. Thus developing sophisticated computer tools that provide a new level of automatic analysis and control for FACS and greatly facilitates FACS use by reducing the need for on-site human expertise should significantly improve the potential for using this versatile methodology in biomedical research and clinical practice. C. PENGUIN -- Highlights of Research Progress Our current effort focuses on developing a computer-based advisory system to assist the FACS investigator in designing experiment protocols. As mentioned above, this system will combine database and artificial intelligence methodologies in an integrated framework, as to provide 1) several levels of abstraction for information management and retrieval from a relational database system, 2) access to and integration of the results of past similar experiments as additional units of information through an interface with existing databases of FACS data and 3) inference E. H. Shortliffe 172 5P41 -RR007851 5 RADIX and PENGUIN Projects capabilities coupled to the database for high-level interactions with scientists during the design process, Although this work is motivated by a specific application, the formal design of the system will be domain-independent; it is then our belief that ideas, principles and programs developed in this process will be applicable to other medical and non-medical areas. In the past year, we have developed the concept of an object-based interface on top of a relational database system and implemented an initial prototype of the interface. We have drawn an analogy between the notions of object and database view. Using this analogy, we have defined three components in the object interface: 1. The object generator maps relations into object templates where each template can be a complex combination of join (combining two relations through shared attributes) and projection (restricting the set of attributes of a relation) operations on the base relations. In addition, an object network groups together related templates, thereby identifying different object views of the same database. The whole process is knowledge- driven, using the semantics of the database structure. We define the object schema as the set of object networks constructed over a given database. Like the data schema for a relational database, the object schema represents the domain-specific information needed to gain access to PENGUIN's objects; this information enables us to combine well-organized, regular tabular structures -- the relations -- into complex, heterogeneous entities -- the objects. 2. The object in&anti&or provides nonprocedural access to the actual object instances. First, a declarative query (e.g., Select instances of template x where attribute y < 0.5) specifies the template of interest. Combining the database-access function (stored in the template), and the specific selection criteria, PENGUIN automatically generates the relational query and transmits it to the DBMS, which in turn transmits back the set of matching relational tuples. In addition to performing the database-access function, the object template specifies the structure and linkage of the data elements within the object. This information is necessary for the tuples to be correctly assembled into the desired instances. Those instances are then made available to the expert system directly, or to the user through a graphic interface. 3. The object decomposer implements the inverse function; that is, it maps the object instances back to the base relations. This component is invoked when changes to some object instances (e.g., deletion of an instance, update of some attributes) need to be made persistent at the database level. An object instance is generated by collapsing (potentially) many tuples from several relations. By the same token, one update operation on an object may result in a number of update operations that need to be performed on the base relations. Preliminary results of PENGUIN's implementation indicate that such an object-based architecture provides (1) efficient access to and manipulation of complex units of biomedical information while preserving the advantages associated with persistent storage of data in relational format, and (2) a domain-independent, bidirectional communication channel between relational database systems and expert systems. In parallel to developing the object interface, we are currently engaged in a knowledge acquisition process to define the structure of the database and of the knowledge base. Through interviews with our experts in the Genetics department, we are eliciting the structure of the various components (genetic, histological, and 173 E. H. Shortliffe RADIX and PENGUIN Projects 5P41 -RR00785- 15 serological information) of the FACS reagents database. Finally, we are exploring the use of hypertext (more specifically, the HyperCard program for the Macintosh) as a possible framework for developing the user interface component of PENGUIN. D. Publications of the RADIX and PENGUIN projects 1. Barsalou, Thierry and Gio Wiederhold: "Automating a Cell Counter"; to be published in The International Journal of Artificial Intelligence in Engineering, Computational Mechanics Publ., UK, 1988. 2. Barsalou, Thierry and Gio Wiederhold: "Applying a Semantic Model to an Immunology Database"; in W.W. Stead (editor), Proceedings of the Eleventh Symposium on Computer Applications in Medical Care, pages 871-877, IEEE Computer Society Press, Washington, D.C., November 1987. 3. Barsalou, Thierry, W.A. Moore, L.A. Herzenberg, and G. Wiederhold: "A Database System to Facilitate the Design of FACS Experiment Protocols" (abstract): Cytometry, Vol.97, August 1987. 4. Barsalou, T. and G. Wiederhold. Knowledge-based mapping of relations into objects. To appear in the International Journal of Al in Engineering, 1988. 5. Barsalou, T. An object-based architecture for biomedical expert database systems. Submitted to the IEEE Twelfth Symposium on Computer Applications in Medical Care, 1988. 6. Blum, Robert L, Computers and Artificial Intelligence in Clinical Medicine: Current Systems and Future Prospects, Computer News for Physicians, October, 1987. 7. Blum, R. L., and Walker, M.G.: LISP as an Environment for Software Design: Powerful and Perspicuous. In Proceedings of the Tenth Annual Symposium on Computer Applications in Medical Care, pages 326-331. IEEE Computer Society, October, 1986. 8. Blum, R. L., and Walker, M.G.: Automated Medical Discovery from Clinical Databases: An Overview of the RADIX Project. In Proceedings of the Fifth Toyobo Biotechnology Foundation Symposium: Artificial Intelligence in Medicine. The Toyobo Foundation, Tokyo, Japan, August, 1986. 9. Blum, R. L., and Walker, M. G.: Automated Medical Discovery from Clinical Databases: an Overview of the RADIX Project. In Proceedings of the First International Conference on Artificial Intelligence and Its Impacts in Biology and Medicine, pages 59-83. Groupement Scientifique pour le Developpement de I'lntelligence Artificielle en Languedoc- Roussillon, Montpellier, France, September, 1986. 10. Blum, Robert L. and Gio C.M. Wiederhold: Studying Hypotheses on a Time-Oriented Clinical Database: An Overview of the RX Project. In J.A. Reggia and S. Thurim: `Computer Assisted Medical Decision-Making'; Springer Verlag, 1985, pp.245-253. Il. Blum, R.L.: Two Stage Regression: Application to a Time-Oriented Clinical Database. Knowledge Systems Laboratory Technical Report. 1985. E. H. Shortliffe 174 5P41 -RR00785-15 RADIX and PENGUIN Projects 12. Blum, R.L.: Modeling and encoding clinical causal relationships. Proceedings of SCAMC, Baltimore, MD, October, 1983. 13. Blum, R.L.: Representation of emprrically derived causal relationships. IJCAI, Karlsruhe, West Germany, August, 1983. 14. Blum, R.L.: Machine representation of clinical causal relationships. MEDINFO 83, Amsterdam, August, 1983. 15. Blum, R.L.: Clinical decision making aboard the Starship Enterprise. Chairman's paper, Session on Artificial Intelligence and Clinical Decision Making, AAMSI, San Francisco, May, 1983. 16. Blum, R.L. and Wiederhold, G.: Studying hypotheses on a time-oriented database: An overview of the RX project. Proc. Sixth SCAMC, IEEE, Washington D.C., October, 1982. 17. Blum, R.L.: Induction of causal relationships from a time-oriented clinical database: An' overview of the RX project. Proc. AAAI, Pittsburgh, August, 1982. 18. Blum, R.L.: Automated induction of causal relationships from a time- oriented clinical database: The RX project. Proc. AMIA San Francisco, 1982. 19. Blum, R.L.: Discovery and Representation of Causal Relationships from a Large Time-oriented Clinical Database: The RX Project. In D.A.B. Lindberg and P.L. Reichertz (Eds.), LECTURE NOTES IN MEDICAL INFORMATICS, Springer-Verlag, 1982. 20. Blum, R.L.: Discovery, confirmation, and incorporation of causal relationships from a large time-oriented clinical database: The RX project. Computers and Biomed. Res. 15(2):164-187, April, 1982. 21. Blum, R.L.: Discovery and representation of causal relationships from a large time-oriented clinical database: The RX project (Ph.D. thesis). Computer Science and Biostatistics, Stanford University, 1982. 22. Blum, R.L.: Displaying clinical data from a time-oriented database. Computers in Biol. and Med. 11(4):197-210, 1981. 23. Blum, R.L.: Automating the study of clinical hypotheses on a time- oriented database: The RX project. Proc. MEDINFO 80, Tokyo, October, 1980, pp. 456-460. (Also STAN-CS-79-816) 24. Blum, R.L. and Wiederhold, G.: Inferring knowledge from clinical data banks utilizing techniques from artificial intelligence. Proc. Second SCAMC, IEEE, Washington, DC., November, 1978. 25. DeLaPaz, R., Hanson, W., Bernstein, R., and Walker, M.G. "Tissue Characterization of MRI Using Fuzzy C-means Cluster Analysis" In preparation for Radiology. 26. DeLaPaz, R., Hanson, W., Bernstein, R., and Walker, M.G. "Clinical Application of Automated MRI Tissue Classification" In preparation for Magnetic Resonance in Medicine. 27. DeLaPaz, R., Bernstein, R., Chang, P., Hanson, W., and Walker, M.G. 175 E. H. Shortliffe RADIX and PENGUIN Projects 5P41 -RR00785-15 "Approximate Fuzzy C-means (AFCM) Cluster Analysis of Medical Magnetic Resonance Image (MRI) Data" Submitted to the International Conference on Pattern Recognition, Special Session on Diagnostic Medical Image Processing, Beijing, China, October 1988. 28. DeZegher-Geets, I., Freeman, A.G., Walker, M.G., Blum, R.L., and Wiederhold, G. "Intelligent Summarization and Display of On-Line Medical Records" Accepted for publication in M.D. Computing. 29. DeZegher-Geets, I.M., Freeman, A.G., Walker, M.G., Blum, R.L., and Wiederhold, G.C.M.: "Computer-aided Summarization of a Time-oriented Medical Data Base"; in Proceedings of the Third Annual Conference on Computerization of Medical Records. Institute for Medical Record Economics, Chicago, April 1987. Also Stanford Knowledge Systems Laboratory (KSL) Report KSL-87-18. 30. DeZegher-Geets, I.: Intelligent Summarization of a Time-Oriented Medical Data Base. Master's thesis, Stanford University, 1987. 31. Downs, S., Walker, M.G., and Blum, R.L.: Automated Summarization of On-line Medical Records. In Proceedings of the Fifth World Congress on Medical lnformatics (Medinfo), pages 800-804. Elsevier Science Publishers, 1986. 32. Downs, S.M.: A Program for Automated Summarization of On-line Medical Records, Master's thesis, Stanford University, 1986. 33. Kuhn, I., Wiederhold, G., Rodnick, J.E., Ramsey-Klee, D.M., Benett, S., Beck, D.D.: Automated Ambulatory Medical Record Systems in the U.S., to be published by Springer-Verlag, 1983, in Information Systems for Patient Care, B. Blum (ed.), Section Ill, Chapter 14. 34. Shortliffe, E. H., Wiederhold, G., Fagan, L., and Perrault,L. : An Introduction To Medical Computer Science. Addison-Wesley, 1987. In preparation, several chapters completed. 35. Walker, M.G. "Expert Systems in Geologic Exploration: Can They Be Cost Effective?" Accepted for publication in Geobyte. 36. Walker, M.G. "Automated Recognition of Brain Anatomy and Pathology in MRI and CT: a Review". In preparation for IEEE Transactions on Medical Image Processing. 37. Walker, M.G., Blum, R.L., and Fag&, L.M. "Minimycin: A Miniature Rule- Based System" in "M.D.Computing Tutorials", C.J.McDonald (editor), Springer-Verlag, 1988. Reprinted from M.D.Computing, V.2,No.4, 1985. 38. Walker, M.G., and Blum, R.L. "An Introduction to LISP" in "M.D.Computing Tutorials", C.J.McDonald (editor), Springer-Verlag, 1988. Reprinted from M.D.Computing, V.2, No.1, 1985. 39. Walker, M.G. "How Feasible is Automated Discovery?" The Computer and Philosophy Newsletter, Vol.3, No.1, pp. l-37, Feb 1988, Center for Design of Educational Computing, Carnegie Mellon University. Reprinted from IEEE Expert, Vol.2, No.1, pp. 70-82, Spring 1987. 40. Walker, M.G., Clauer, R.C., Samadani, R., Craven, J., and Frank, L. "Automated Analysis of Aurora1 Images"; Abstract. EOS, Vol. 68, No. E. H. Shortliffe 176 5P41 -RR007851 5 RADIX and PENGUIN Projects 44, page 1436, November 1987. Poster session paper presented at American Geophysical Union Annual Meeting, San Francisco, December 1987. 41. Walker, M.G. "Expert Systems in Exploration: Can They Be Cost Effective?" American Association of Petroleum Geologists, Annual Convention, Los Angeles, California, June 1987. 42. Walker, M.G., and Blum, R.L.: Towards Automated Discovery from Clinical Databases: the RADIX Project In Proceedings of the Fifth World Congress on Medical lnformatics (Medinfo), pages 32-38. Elsevier Science Publishers, 1988. 43. Walker, M.G., Blum, R.L., and Fagan, L.M.: Minimycin: A Miniature Rufe- Based System. M.D.Computing, Vol. 2, No. 4., 1985. 44. Walker, M.G., and Blum, R.L.: An Introduction to LISP. M.D. Computing, Vol. 2, No. l., 1985. 45. Wiederhold, Gio: "Hospital Information Systems"; in Encyclopedia of Medical Devices and Instrumentation, ~01.3, Webster,John G.(ed), Wiley 1988, pp.1517--1542 48. Wiederhold, Gio and Paul D. Clayton: "Processing Biological Data in Real Time"; M.D. Computing, Springer Verlag, Vol.2 No.8, November 1985, pages 18-25, republished in `Images, Signals, and Devices', C.J. McDonald (editor), Springer Verlag, 1987, pp.1 07--l 18. 47. Wiederhold, G.: File Organization for Database Design. McGraw-Hill Book Company, 1987. 48. Wiederhold, Gio CM, Michael G. Walker, Robert L. Blum, Stephen M. Downs, and lsabelle dezegher-Geets: "Acquisition of Knowledge from Medical Records" abstract: in Namen, Daten Themen, Benutzer- gruppenseminar Medizin I, Systems 87 conference, Munich, FRG, Oct.1987, ~~8213-214. 49. Wiederhold, Gio CM, Michael G. Walker, Waqar Hasan, Surajit Chaudhuri, Arun Swami, Sang K. Cha, XiaoLei Qian, Marianne Winslett, Linda DeMichiel, and Peter K. Rathmann: "KSYS: An Architecture for Integrating Databases and Knowledge Bases"; Computer Science Department, Stanford University, May 1987; in Amar Gupta and Stuart Madnick (editors) `Technical Opinions Regarding Knowledge- Based Integrated Information Systems Engineering', MIT, 1987, 50. Wiederhold, Gio: "Knowledge versus Data": Chapter 8 of `On Knowledge Base Management Systems: Integrating Artificial Intelligence and Database Technologies' (Brodie, Mylopoulos, and Schmidt, eds.), Springer Verlag, June 1988, pages 77 to 82. 51. Wiederhold, Gio, Robert L. Blum, and Michael Walker: "An Integration of Knowledge and Data Representation": Proc. of lslamorada Workshop, Feb.1 985, Computer Corporation of America, Cambridge MA; Chapter 29 of `On Knowledge Base Management Systems: Integrating Artificial Intelligence and Database Technologies' (Brodie, Mylopoulos, and Schmidt, eds.), Springer Verlag, June 1988, pages 431 to 444. 177 E. H. Shortliffe RADIX and PENGUIN Projects 5P41 -RR00785 15 52. Wiederhold, G.: Views, Objects, and Databases. 19( 12):37-44,. December, 1988. IEEE Computer 53. Wiederhold, G.C.M., Walker, M. G., Blum, R.L., and Downs, S.M.: Acquisition of Knowledge from Data. In Ras, Z.W., and Zemankova, M. (editors), Proceedings of the International Symposium on Methodologies for Intelligent Systems, ACM, Oct.1 988, Knoxville TN. 54. Missikoff, Michele and Gio Wiederhold: Towards a Unified Approach for Expert and Database Systems. In `Expert Database Systems', Larry Kerschberg (editor), Benjamin/Cummings, 1988, pages 383-399; also In Proceedings of First Workshop on Expert Database Systems, Kiawah Island, South Carolina, Oct. 1984, ~01.1, pp.188-208. 55. Wiederhold, Gio: Knowledge Bases; Future Generations Computer Systems, North-Holland, vol.1 no.4; April 1985, pp.223--235. 58. Wiederhold, Gio: Disease Registers: Use of Databases to Generate New Medical Knowledge; in K.Abt, W.Giere and B.Leiber: `Krankendaten, Krankheitsregister, Datenschutz', ~01.58, Medizinische lnformatik und Statistik, Springer Verlag, 1985, pp.39-55. 57. Wiederhold, G.: Networking of Data Information, National Cancer Institute Workshop on the Role of Computers in Cancer Clinical Trials, National Institutes of Health, June 1983, pp.1 13-119. 58. Wiederhold, G.: Database Design (in the Computer Science Series); McGraw-Hill Book Company, New York, NY, May 1977, 878 pp. Second edition, Jan. 1983, 788 pp. 59. Wiederhold, G.: In D.A.B. Lindberg and P.L. Reichertz (Eds.), Databases for Health Care, Lecture Notes in Medical Informatics, Springer-Verlag, 1981. 80. Wiederhold, G.: "Database technology in health care": J. Medical Systems 5(3):175-198, 1981. E. funding Support Status 1987-l 988 Principal Investigator: Gio Wiederhold 5120% (8 months) DADAISM, Databases for Ada Information System Modules (SRI//NRL/DoD Stars: $178,931) 1987-l 991 Principal Investigator: Gio Wiederhold 10% Support for Parallel Design in an Engineering Information System (NSF DTMP 8819595 $153,788, $148,380 prop., $158,044 prop.) 1987-l 988 Principal Investigator: Gio Wiederhold 5% Validation of Knowledge from a Database (IBM KBS Menlo Park, $50,051) 1988-l 990 Principal Investigator: Gio Wiederhold 50% effort Knowledge Base Management Structures (ONR/SPAWAR/ARPA, N39-84-C-211, task 7: $1,758,410) E. H. Shortliffe 178 5P41-RR00785-15 RADIX and PENGUIN Projects 1987-1988 Principal Investigator: Gio Wiederhold 5% Reasoning about RI ME (Digital Equipment Corp: $131,337) Requests in process: 1988 (8 mo) Principal Investigator: Gio Wiederhold 20% summer, 10% academic year DADAISM: DBMS in and for ADA-supported Information System Management (SRI/NRL/STARS/DoD: $338,719) 1987-1990 Principal investigator: Gio Wiederhold 25% effort FACS-Penguin: An Expert Workstation for Flow Cytometry (NLM/NIH : $858,449 direct costs, approved at high priority) 1987-1990 Associate Investigator: Gio Wiederhold 10% effort Integrating Knowledge and DBMS (SRI AI/ONR/SPAWAR/ARPA, to be funded: $75,000 Stanford Subcontract) 1988-l 990 Principal Investigator: Gio Wiederhold 10% effort PARADATA: Databases on Parallel Computers (NSF/IRI/K&DSP, approved, not yet funded, $499,385 requested) 1988-l 989 Principal Investigator: Gio Wiederhold 5% effort RADIX Project: Software for Automated Peer Review (NCHSR/ DHHS HS5832, approved, unlikely to be funded: $231,998 direct) II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Collaborations SUMEX AIM provides the central communication node for our research. While individual experiments tend to take place on workstations, the availability of SUMEX to load and edit files, communicate with colleagues and peers is absolutely critical. B. interactions with Other SUMEX-AIM Projects During the current reporting year we have had frequent interaction with members of other SUMEX projects; for example, development of algorithms for transforming INTERNIST data to Bayes form, presentation of research results at Stanford Medical Information Science Colloquia, discussions of automated discovery and automated summarization, practical programming issues, and training of Medical Computer Science Students in the use of KEE, Lisp workstations, and so on. The SUMEX community is an invaluable resource for providing such interaction. C. Critique of Resource Management The DECSystem 20 continues to provide acceptable performance, but it is frequently over loaded at peak hours. The archiving facility (SAFE) seems to be very loaded, so that we are forced to make decisions on retention of past material more frequently than optimal. As users of large databases this is one of the resources upon we must draw frequently. The SUMEX resource management continues to be accessible and quite helpful. New networking links, for instance to the Genetics Department equipment, make new collaborations viable. 179 E. H. Shortliffe RADIX and PENGUIN Projects 5P41 -RR00785- 15 Ill. RESEARCH PLANS A. Project Goals and Plans The the RADIX Project is now inactive. We plan to keep the results and software produced accessible for educational and sharing objectives within the MIS program. The long-range goal of the PENGUIN project is to integrate our experiment design advisory system with the set of programs developed in the Genetics department for FACS operation and control, thereby defining a comprehensive "FACS Workstation" that should considerably facilitate effective utilization of FACS technology. In the shorter term, we will: 1. complete the development and testing of the object interface. The interface should then be made a fully domain-independent module that could be applied in different contexts. 2. validate the structure of the database and make it available to the scientists in the Genetics department, for further refinement as well as for routine data storage and retrieval. 3. move gradually toward a distributed environment with Macintoshes as local workstations connected to a central VAX computer, acting as a database server. 4. develop the expert system module which will be coupled to the database through the object interface. The expert system module is aimed at assisting people in an experimental design task. This can actually be done by taking two different approaches, one being the actual automatic design of the experiment (i.e., a planning task), the other being the critique of a protocol proposed by the user. Although we recognize both directions as important, we will focus on the first approach because knowing how to properly plan new experiments is a necessary step prior to any critiquing process. Although the problem solving strategies of our experts are not yet fully determined, we plan to investigate the general concept of hierarchical planning. B. Justification and Requirements for Use of SUMEX The PENGUIN project is dependent on new methods of communication among clinical and academic researchers. C. Recommendations for Resource Development The movement towards work-stations is obviously the direction of the future. We hope that the communication capability, now supported via the central SUMEX facility, the DEC 2080, will continue to be supported. We continue to make constant use of this communication with our colleagues within and outside of Stanford, preparation of large documents, file and database handling, and program demonstrations. E. H. Shortliffe 180 5P41 -RR00785-15 National AIM Projects W.B. National AIM Projects The following group of projects is formally approved for access to the AIM aliquot of the SUMEX-AIM resource. Their access is based on review by the AIM Advisory Group and approval by the AIM Executive Committee. In addition to the progress reports presented here, abstracts for each project and its individual users are submitted on a separate Scientific Subproject Form. 181 E. H. Shortliffe ATTENDING Project 5P41 -RR00785-15 MB. 1. ATTENDING Project ATTENDING Project - - Expert Critiquing Systems Perry L. Miller, M.D. Ph.D. Department of Anesthesiology Yale University School of Medicine New Haven, CT 08510 I. SUMMARY OF RESEARCH PROGRAM A. Project rationale Our project is exploring the "critiquing" approach to bringing computer-based advice to the practicing physician. Critiquing is a different approach to the design of artificial intelligence based expert systems. Most medical expert systems attempt to simulate a physician's decision- making process. As a result, they have the clinical effect of trying to tell a physician what to do: how to practice medicine. In contrast, a critiquing system first asks the physician how he contemplates approaching his patient's care, and then critiques that plan. In the critique, the system discusses any risks or benefits of the proposed approach, and of any other approaches which might be preferred. It is anticipated that the critiquing approach may be particularly well suited for domains, like medicine, where decisions involve a great deal of subjective judgment. To date, several prototype critiquing systems have been developed in different medical domains, including: 1. ATTENDING, the first system to implement the critiquing approach, critiques anesthetic management. 2. HT-ATTENDING critiques the pharmacologic management of essential hypertension. 3. W-ATTENDING critiques aspects of ventilator management. 4. PHEO-ATTENDING critiques the laboratory and radiologic workup of a patient for a suspected pheochromocytoma. In addition, a domain-independent system, ESSENTIAL-ATTENDING, has been developed to facilitate the implementation of critiquing systems in other domains. C. Highlights of Research Progress Current projects include the following: HT-ATTENDING -- The original prototype version of HT-ATTENDING has been converted to the ESSENTIAL-ATTENDING format, and updated to reflect current thinking in the field of hypertension management. A major priority is to subject this system to validation and clinical evaluation. In addition, we are currently exploring concretely how best to disseminate the system as a practical consultation tool. E. H. Shortliffe 182 5P41 -RR007851 5 ATTENDING Project ICON: Critiquing Radiological Differential Diagnosis -- Most existing diagnostic computer systems produce a ranked differential diagnosis as their output. In this process, the rich structure of the knowledge that went into developing the diagnoses may be lost to the user. ICON explores a different approach to diagnostic advice in the domain of radiology. To use ICON, a radiologist describes a set of findings seen on chest x-ray, together with a proposed diagnosis. ICON then produces a detailed analysis of why the observed findings serve to support or to rule out the diagnosis. It may also suggest further findings that might help refine the diagnosis, again explaining why the findings are important. In addition, we are currently exploring how images might be incorporated into ICON's knowledge base to enhance its consultative effectiveness. We see this project as an exploration of the question of how prose and images might best be indexed and organized for purposes of explanation in the context of a particular clinical case. D. Publications 1. Miller, P.L. (Ed.): Selected Topics in Medical Artificial Intelligence. New York: Springer-Verlag (in press). 2. Rennels, G.D., Miller, P.L.: Artificial intelligence research in anesthesia and intensive care. Journal of Clinical Monitoring (in press). 3. Miller, P.L., Rennels, G.D.: Prose generation from expert systems: An applied computational linguistics approach. Al Magazine (in press). 4. Rennels, G.D., Shortliffe, E.H., Stockdale, F.E., Miller, P.L.: A computational model of reasoning from the clinical literature. Computer Methods and Programs in Biomedicine 24:139-149, 1987. 5. Rennels, G.D., Shortliffe, E.H., Stockdale, F.E., Miller, P.L.: A structured representation of the clinical literature and its use in a medical management advice system. Bulletin du Cancer 74:215-220, 1987. 6. Miller, P.L., Bar-wick, K.W., Morrow, J.S., Powsner, S.M., Riely, CA.: Semantic relationships and medical bibliographic retrieval: A preliminary assessment. Computers and Biomedical Research 21:64-77, 1988. 7. Miller, P.L., Morrow, J.S., Powsner,. S.M., Riely, C.A.: Semantically assisted medical bibliographic retrieval: An experimental computer system. Bulletin of the Medical Library Association 76:131-136, 1988. 8. Miller, P.L.: Exploring the critiquing approach: Clinical practice-based feedback by computer. Biomedical Measurement, lnformatics and Control (in press). 9. Rennels, G.D., Shortliffe, E.H., Stockdale, F.E., Miller, P.L.: A computational model of reasoning from the clinical literature. The Al Magazine (accepted pending revision). IO. Miller, P.L., Fisher, P.R.: Causal models in medical artificial intelligence. Proceedings of the Eleventh Symposium on Computer Applications in Medical Care, Washington, D.C., November 1987, pp. 17-22. 11. Powsner, S.M., Barwick, K.W., Morrow, J.S., Riely, C.A., Miller, P.L.: Coding semantic relationships for medical bibliographic retrieval: A preliminary study. Proceedings of the Eleventh Symposium on Computer Applications in Medical Care, Washington, D.C., November 1987, pp. 108-112. 183 E. H. Shortliffe ATTENDING Project 5P41 -RR00785- 15 E. Funding Support EXPERT COMPUTER SYSTEMS WHICH CRITIQUE PHYSICIAN PLANS NIH Grant R01 LM04336 Principal Investigator: Perry L. Miller, M.D., Ph.D. Annual Direct Costs: approximately $95,000 Period of Support: 3/l /88-2/28/91 This grant supports the exploration of the critiquing approach to bringing computer-based advice to the physician, focusing especially on refining and evaluating the /-IT-ATTENDING system which critiques hypertension management, and on developing tools for know/edge base maintenance and updating. SUPPORT OF THE UNIFIED MEDICAL LANGUAGE PROGRAM NLM Contract NO1 LM63524 Principal Investigator: Perry L. Miller, M.D., Ph.D. Annual Direct Costs: approximately $100,000 Period of Support: 8/22/86-8/21/88 This research contract is part of the NLM Unified Medical language (UML) program. We are defining a set of semantic relationships which could be used to augment the UML, to facilitate such functions as medical bibliographic retrieval. SUPPORT FOR MEDICAL INFORMATICS AND ARTIFICIAL INTELLIGENCE Ira DeCamp Foundation Co-Principal Investigators: Henry A. Swett, M.D. Perry L. Miller, M.D., Ph.D. Annual Costs: $75,000 Period of Support: 7/l /86-6130190 This grant supports activities of our Medical lnformatics program. MEDICAL INFORMATICS RESEARCH TRAINING AT YALE Principal Investigator: Perry L. Miller, M.D., Ph.D. NLM Training Grant T15 LM07056 Period of Support: 7/l 187-6130192 This grant currently supports 3 postdoctoral trainees and three predoctoral trainees in Medical Informatics. Pending Support BIOTECHNOLOGY COMPUTER RESEARCH AT YALE Perry L. Miller, M.D., Ph.D. (PI) RFA 88-LM-01 This grant proposes computer-related research in molecular biology. II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE Until last year we were using the RUTGERS-AIM Resource. We used that facility to E. H. Shortliffe 184 5P41 -RR00785-15 ATTENDING Project implement all of our early critiquing systems. This year we moved the HT- ATTENDING project. to the SUMEX-AIM facility. Very recently, we have moved HT- ATTENDING to a local VAX station, and we are currently migrating entirely to local workstations for our research. This past year, our main use of SUMEX-AIM has been the following: 1. We used SUMEX-AIM to refine and to demonstrate HT-ATTENDING. 2. We used SUMEX-AIM for communication access to the national AIM community. We have found our use of the RUTGERS-AIM and SUMEX-AIM facilities to be extremely valuable. They provided us the resources needed to initiate our research and to continue several projects which are still active. They provided a natural vehicle to allow us to demonstrate the various systems easily, both in the United States and in Europe. Also, they enabled us to collaborate very closely with Dr. Glenn Rennels in his Stanford Medical Information Science thesis project on the Roundsman system. Via SUMEX-AIM and RUTGERS-AIM, Dr. Rennels and Dr. Miller maintained very close contact, typically with multiple messages each week, and sometimes within a single day. Ill. FUTURE PLANS We plan to continue our critiquing research as outlined above. As discussed previously, we have now moved all our research onto local workstations, and will continue the research using these resources. 185 E. H. Shortliffe INTERNIST-VQMR Project 5P41 -RR00785-15 1V.B. 2. INTERNIST-I/QMR Project INTERNIST-I/QMR (Quick Medical Reference) Jack. D. Myers, M.D. University Professor Emeritus (Medicine) Randolph A. Miller, M.D. Associate Professor of Medicine Chief, Section of Medical lnformatics University of Pittsburgh 1291 Scaife Hall Pittsburgh, Pa., 15261 I. SUMMARY OF RESEARCH PROGRAM A. Proiec t rationale The principal objective of this project is the development of a high-level computer diagnostic program in the broad field of internal medicine as an aid in the solution of complex and complicated diagnostic problems. To be effective, the program must be capable of multiple diagnoses (related or independent) in a given patient. A major achievement of this research undertaking has been the design of a program called INTERNIST-I, along with an extensive medical knowledge base. This program has been used over the past decade to analyze many hundreds of difficult diagnostic problems in the field of internal medicine. These problem cases have included cases published in medical journals (particularly Case Records of the Massachusetts General Hospital, in the New England Journal of Medicine), CPCs, and unusual problems of patients in our Medical Center. In most instances, but by no means all, INTERNIST-I has performed at the level of the skilled internist, but the experience has highlighted several areas for improvement. f3. Medical Relevance and Collaboration The program inherently has direct and substantial medical relevance. The development of the QUICK MEDICAL REFERENCE (QMR) under the leadership of Dr. Randolph A. Miller has allowed us to distribute the INTERNIST-I knowledge base in a modified format to over twenty other academic medical institutions. The knowledge base can thereby be used as an "electronic textbook" in medical education at all levels -- by medical students, residents and fellows, and faculty and staff physicians. This distribution is continuing to expand. The INTERNIST-I program has been used in recent years to develop patient management problems for the American College of Physician's Medical Knowledge Self-assessment Program. E. H. Shortliffe 186 5P41 -RR00785-15 INTERNIST-I/QMR Project C. Highlights of Research Progress Cl Accomplishments this past year For the record, it should be noted that grant support for the QMR project has come solely from the CAMDAT Foundation of Farmington, Conn., from the Department of Medicine of the University of Pittsburgh, and from Dr. Miller's NLM RCDA grant and NLM R01 grant. In the past year, the University of Pittsburgh was named recipient of a National Library of Medicine Medical lnformatics Training Grant Award. The group of us (Myers, Miller and Masarie) together with assigned residents in internal medicine and fellows in medical informatics are continuing to expand the knowledge base and to incorporate the diagnostic consultative program into QMR. The computer program for the interrogative part of the diagnostic program is the main remaining task. An editor for the QMR knowledge base, as modified from the INTERNIST-I knowledge base, has been written from scratch in Turbo Pascal by Dr. Masarie. The entire QMR program can be accommodated in, maintained (particularly edited) and operated on individual IBM PC-AT computers. Our group has incorporated into the QMR diagnostic consultant program modifications and embellishments of the INTERNIST-I knowledge base, and will continue to do so over the next year by adding "facets" of diseases or syndromes. This addition and modification is expected to improve the performance of the diagnostic consultant program. The medical knowledge base has continued to grow both in the incorporation of new diseases and the modification of diseases already profiled so as to include recent advances in medical knowledge. Several dozen new diseases have been profiled during the past year. The current number of diseases in the QMR knowledge base is 589, and 4237 possible patient findings are included. C.2 Research in progress There are four major components to the continuation of this research project: 1. The enlargement, continued updating, refinement and testing of the extensive medical knowledge base required for the operation of INTERNIST-I and the QMR modification. 2. Institution of field trials of QMR on the clinical services in internal medicine at the Health Center of the University of Pittsburgh. This has been accomplished in a limited fashion, which began in 1987; a "computer-based diagnostic consultation service" has been made available to attending physicians and house staff on the medical services of our two main teaching hospitals. Institutional Review Board (IRB) approval was granted to the service before it was initiated. 3. Expansion of the clinical field trials to other university health centers which have expressed interest in working with the system. 4. Adaptation of the diagnostic program and data base of INTERNIST-I and the QMR modification to subserve educational purposes and the evaluation of clinical performance and competence. 187 E. H. Shortliffe INTERNIST-I/QMR Project 5P41 -RR00785-15 Current activity is devoted mainly to the first two of these, namely, the continued development of the.medical knowledge base, and the implementation of the improved diagnostic consulting program, and preliminary evaluation of the diagnostic consultation service. D. List of relevant publications I. Myers JD. The Background of INTERNIST-I and QMR. In: Proceedings of the History of Medical lnformatics Conference. National Library of Medicine. pp. 195-197, November 1987. 2. Masarie, F.E., Miller, R.A. Medical Subject Headings and Medical Terminology: An analysis of terminology used in hospital charts. Bulletin of the Medical Library Association, 1987; 75:89-94. 3. Masarie FE, Miller RA. INTERNIST-I to Quick Medical Reference (QMR): Transition from a mainframe to a microcomputer. Proceedings Ninth Annual IEEE/Engineering in Biology and Biology Society. IEEE Press. pp. 1521- 1522, November 1987. 4. Parker, RC, Miller RA. Using causal knowledge to create simulated patient cases: The CPCS project as an extension of INTERNIST-I. Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care. pp. 473-480, November 1987. 5. Bankowitz RA, Blumenfeld BH, Miller RA, et al. User variability in abstracting and entering printed case histories with Quick Medical Reference (QMR). Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care. pp. 68-73, November 1987. 6. Masarie FE, Miller RA. Quick Medical Reference (QMR): An information management tool for clinical diagnosis. IN: Critical Reviews in Medical Informatics. Boca Raton, Florida, CRC Press, 1988. 7. Parker RC, Miller RA. Using causal knowledge to created simulated patient cases: The CPCS project as an extension of INTERNIST-I. IN: Miller PL (ed) Topics in Medical Artificial Intelligence. Computers and Medicine Series, Springer-Verlag, New York, 1988. 8. Challinor SM, McNeil MA, Bankowitz RA, et al. Evaluation of a Computer-Assisted General Medicine Diagnostic Consultation Service. Abstract presented at the 1 lth Annual SGIM Meeting, Washington, D.C., April 27-29, 1988. 9. Miller RA. From Automated Medical Records to Expert System Knowledge Bases: Common Problems in Representing and Processing Patient Data. Topics in Health Record Management. 75(3):23-36, 1987 IO. Miller RA. Computer-based Diagnostic Decision-making. Medical Care. 25(12):Sl48-SI52, 1987. E. funding support I. Diagnostic-Internist: A Computerized Medical Consultant Randolph A. Miller, M.D. E. H. Shortliffe 188 5P41 -RR00785- 15 INTERNIST-I/QMR Project Associate Professor of Medicine Chief, Section of Medical lnformatics University of Pittsburgh Department of Medicine National Library of Medicine - Development Award Research Career National Institutes of Health 5 K04 LM00084-03 09/30/85 - 09/29/86 - $55,296 09/30/86 - 09/29/87 - $55,296 09/30/87 - 09/29/88 - $54,648 Support recommended for 2 additional years ending 09/29/90, The Amounts to be determined annually. 2. Developing INTERNIST-I Knowledge Base into a Resource Randolph A. Miller, M.D. Associate Professor of Medicine Chief, Section of Medical lnformatics University of Pittsburgh Department of Medicine National Library of Medicine National Institutes of Health I ROI LM04622-01 09130187 through 09/29/90 09/30/87 - 09129188 - $71,892 09/30/88 - 09129189 - $112,938 09/30/89 - 09/29/90 - $112,580 3. Pittsburgh Medical Information Sciences Training Program Randolph A. Miller, M.D. Associate Professor of Medicine Chief, Section of Medical lnformatics University of Pittsburgh Department of Medicine National Library of Medicine National Institute of Health 07/01/87 through 06/30/92 07/01/87 - 06/30/88 - $153,454 07101188 - 06130189 - $199,038 07/01/89 - 06130190 - $217,572 07/01/90 - 06/30/91 - $278,092 07/01/91 - 06130192 - $276,596 II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A#. Medical Collaborations and Program Dissemination Via SUMEX INTERNIST-I and QMR remain in a stage of research and particularly development. As noted above, we are continuing to develop better computer programs to operate the diagnostic system, and the knowledge base cannot be used very effectively for collaborative purposes until it has reached a critical stage of completion. These factors have stifled collaboration via SUMEX up to this point and will continue to do so for the next year or two. In the meanwhile, through the SUMEX community there continues to be an exchange of information and states of progress. Such interactions particularly take place at the annual AIM Workshop. 189 E. H. Shortiiffe INTERNIST-VQMR Project 5P41 -RR00785-15 C. Critique of Resource Management SUMEX has been an excellent resource for the development of INTERNIST-I. Our large program is handled efficiently, effectively and accurately. The staff at SUMEX have been uniformly supportive, cooperative, and innovative in connection with our project's needs. Ill. RESEARCH PLANS A. Project Goals and Plans Continued effort to complete the medical knowledge base in internal medicine will be pursued including the incorporation of newly described diseases and new or altered medical information on "old" diseases. The latter two activities have proven to be more formidable than originally conceived. B. Justification and Requirements for Continued SUMEX Use Our use of SUMEX has declined with the adaptation of our programs to the IBM PC- AT. Nevertheless, the excellent facilities of SUMEX are expected to be used for certain developmental work. It is intended for the present to keep INTERNIST-I at SUMEX for comparative use as QMR is developed here. We will not need the DEC 2060 beyond its anticipated phase-out in early 1989, but will require access to its replacement for mailing purposes and to maintain contact with the national medical informatics community. C. Needs and Plans for Other Computing Resources Beyond SUMEX-AIM Our predictable needs in this area will be met by our recently acquired personal work stations. E. H. Shortliffe 190 5P41 -RR00785-I 5 MENTOR Project IV.B.3. MENTOR Project MENTOR Project -- Medical Evaluation of Therapeutic Orders Stuart M. Speedie, Ph.D. School of Pharmacy University of Maryland Terrence F. Blaschke, M.D. Department of Medicine Division of Clinical Pharmacology Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale The goal of the MENTOR (Medical EvaluatioN of Therapeutic ORders) project is to design and develop an expert system for monitoring drug therapy for hospitalized patients that will provide appropriate advice to physicians concerning the existence and management of adverse drug reactions. The computer as a record-keeping device is becoming increasingly common in hospital-based health care, but much of its potential remains unrealized. Furthermore, this information is provided to the physician in the form of raw data which is often difficult to interpret. The wealth of raw data may effectively hide important information about the patient from the physician. This is particularly true with respect to adverse reactions to drugs which can only be detected by simultaneous examinations of several different types of data including drug data, laboratory tests and clinical signs. In order to detect and appropriately manage adverse drug reactions, sophisticated medical knowledge and problem solving is required. Expert systems offer the possibility of embedding this expertise in a computer system. Such a system could automatically gather the appropriate information from existing record-keeping systems and continually monitor for the occurrence of adverse drug reactions. Based on a knowledge base of relevant data, it could analyze incoming data and inform physicians when adverse reactions are likely to occur or when they have occurred. The MENTOR project is an attempt to explore the problems associated with the development and implementation of such a system and to implement a prototype of a drug monitoring system in a hospital setting. B. Medical Relevance and Collaboration A number of independent studies have confirmed that the incidence of adverse reactions to drugs in hospitalized patients is significant and that they are for the most part preventable. Moreover, such statistics do not include instances of suboptimal drug therapy which may result in increased costs, extended length-of- stay, or ineffective therapy. Data in these areas are sparse, though medical care evaluations carried out as part of hospital quality assurance programs suggest that suboptimal therapy is common. Other computer systems have been developed to influence physician decision making by monitoring patient data and providing feedback. However, most of these 191 E. H. Shortliffe MENTOR Project 5P41 -RR00785- I5 systems suffer from a significant structural shortcoming. This shortcoming involves the evaluation rules that are used to generate feedback. In all cases, these criteria consist of discrete; independent rules, yet medical decision making is a complex process in which many factors are interrelated. Thus, attempting to represent medical decision-making as a discrete set of independent rules, no matter how complex, is a task that can, at best, result in a first-order approximation of the process. This places an inherent limitation on the quality of feedback that can be provided. As a consequence it is extremely difficult to develop feedback that explicitly takes into account all information available on the patient. One might speculate that the lack of widespread acceptance of such systems may be due to the fact that their recommendations are often rejected by physicians. These systems must be made more valid if they are to enjoy widespread acceptance among physicians. The MENTOR system is designed to address the significant problem of adverse drug reactions by means of a computer-based monitoring and feedback system to influence physician decision-making. It employs principles of artificial intelligence to create a more valid system for evaluating therapeutic decision-making. The work in the MENTOR project is a collaboration between Dr. Blaschke at Stanford University, Dr. Speedie at the University of Maryland, and Dr. Charles Friedman at the University of North Carolina. Dr. Speedie provides the expertise in the area of artificial intelligence programming. Dr. Blaschke provides the medical expertise. Dr. Friedman contributes expertise in the area of physician feedback design and system impact evaluation. The blend of previous experience, medical knowledge, computer science knowledge and evaluation design expertise they represent is vital to the successful completion of the activities in the MENTOR project. C. Highlights of Research Progress The MENTOR project was initiated in December, 1983. The project has been funded by the National Center for Health Services Research since January 1, 1985. Initial effort focused on exploration of the problem of designing the MENTOR system. As of June I, 1988, a working prototype system has been developed and is undergoing evaluation. The prototype consists of a Patient Data Base, an Inference Engine, an Advisory Module and a Medical Knowledge Base. The Medical Knowledge Base currently contains information related to aminoglycoside therapy, digoxin therapy, potassium supplementation, surgical prophylaxis, and microbiology lab reports. The system is currently implemented on a Xerox 1186 Al Workstation. Another version of the Patient Data Base has been developed for a VAX 750 that is connected via an asynchronous line to the 1186 running the inference engine. The project has received additional funding from the National Center for Health Services Research to install and evaluate the MENTOR system in a Veterans Administration Hospital. This effort began in June of 1988 and will continue for two additional years. The VA system will reside on an 1186 and a VAX Station II connected directly to the VA's Ethernet LAN, and accessing hospital data through the FILEMAN software. E. Funding Support Title: MENTOR: Monitoring Drug Therapy for Hospitalized Patients Principal Investigators: Terrence F. Blaschke, M.D. Division of Clinical Pharmacology E. H. Shortliffe 192 5P41 -RR00785-15 MENTOR Project Department of Medicine Stanford University Stuart M. Speedie, Ph.D. School of Pharmacy University of Maryland Funding Agency: National Center for Health Services Research Grant Identification Number: I RI8 HS05263 Total Award: January I, 1985 - May 31, 1990 Direct Costs $1,056,201 Total Current Period: June 1, 1988 - May 31, 1989 Direct Costs $396,569 Total II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations and Program Dissemination via SUMEX This project represents a collaboration between faculty at Stanford University Medical Center, the University of Maryland School of Pharmacy, and the University of North Carolina in exploring computer-based monitoring of drug therapy. SUMEX, through its communications capabilities, facilitates this collaboration of geographically separated project participants by providing electronic mail and file exchange between sites. B. Sharing and Interactions with Other SUMEX-AIM Projects Interactions with other SUMEX-AIM projects has been on an informal basis. Personal contacts have been made with individuals working on the ONCOCIN project concerning system development issues. Dr. Perry Miller has also been of assistance by providing software for advisory generation. Given the geographic separation of the investigators, the ability to exchange mail and programs via the SUMEX system as well as communicate with other SUMEX-AIM projects is vital to the success of the project. C. Critique of Resource Management To date, the resources of SUMEX have been fully adequate for the needs of this project. The staff have been most helpful with any problems we have had and we are quite satisfied with the current resource management. Ill. RESEARCH PLANS A. Project Goals and Plans The MENTOR project has the following goals: 1. Implement a prototype computer system to continuously monitor patient drug therapy in a hospital setting. This will be an expert system that will use a modular, frame-oriented form of medical knowledge, a separate inference engine for applying the knowledge to specific situations, and automated collection of data from hospital information systems to produce therapeutic advisories. 193 E. H. Shortliffe MENTOR Project 5P41-RR00785-15 2. Select a small number of important and frequently occurring medical settings (e.g.; combination therapy with cardiac glycosides and diuretics) that can lead to therapeutic misadventures, construct a comprehensive medical knowledge base necessary to detect these situations using the information typically found in a computerized hospital information system and generate timely advisories intended to alter behavior and avoid preventable drug reactions. 3. Design and begin to implement an evaluation of the impact of the prototype MENTOR system on physicians' therapeutic decision-making as well as on outcome measures related to patient health and costs of care. 1988 will be spent on continued prototype development in six content areas, refinement of the inference mechanisms, and installation of the system at the Palo Alto Veterans Administration Hospital. B. Justification and Requirements for Continued SUMEX Use This project needs continued use of the SUMEX facilities for one primary reason. Access to SUMEX is necessary to support the collaborative efforts of geographically separated development teams at Stanford and the University of Maryland. Furthermore, the MENTOR project is predicated on the access to the SUMEX resource free of charge over the next two years. Given the current restrictions on funding, the scope of the project would have to be greatly reduced if there were charges for use of SUMEX. C. Needs and Plans for Other Computing Resources Beyond SUMEX-AIM A major long-range goal of the MENTOR project is to implement this system on a independent hardware system of suitable architecture. It is recognized that the full monitoring system will require a large patient data base as well as a sizeable medical knowledge base and must operate on a close to real-time basis. Ultimately, the SUMEX facilities will not be suitable for these applications. Thus, we have transported the prototype system to a dedicated hardware system that can fully support the the planned system and which can be integrated into a Hospital Information System. For this purpose a VAX 750 and three Xerox 1186 workstations have been acquired and our development efforts have been transferred to them. D. Recommendations for Future Community and Resource Development In the time we have been associated with SUMEX, we have been generally pleased with the facilities and services. However, it is clearly evident that the users' almost insatiable demands for CPU cycles and disk space cannot be met by a single central machine. The best strategy would appear to be one of emphasizing powerful workstations or relatively small, multi-user machines linked together in a nationwide network with SUMEX serving as the its central hub. This would give the individual users much more control over the resources available for their needs, yet at the same time allow for the communications among users that have been one of SUMEX's strong points. For such a network to bc successful, further work needs to be done in improving the network capabilities of SUMEX to encourage users at sites other than Stanford. Further work is also needed in the area of personal workstations to link them to such a network. Given the successful completion of this work, it would be E. H. Shortliffe 194 5P41 -RR00785- 15 MENTOR Project reasonable to consider the phase-out of the central SUMEX machine years and its replacement by an efficient, high-speed communications server. 195 E. H. Shortliffe Pilot Stanford Projects 5P41 -RR00785-15 1V.C. Pilot Stanford Projects Following are descriptions of the informal pilot projects currently using the Stanford portion of the SUMEX-AIM resource, pending funding, full review, and authorization. In addition to the progress reports presented here, abstracts for each project are submitted on a separate Scientific Subproject Form. E. H. Shortliffe 196 5P41 -RR00785-15 REFEREE Project W.C. 1. REFEREE Project REFEREE Project Principal Investigator: Bruce G. Buchanan, Ph.D. Computer Science Department Stanford University Co-Principal Investigator: Byron W. Brown, Ph.D. Department of Medicine Stanford University Associate Investigator: Daniel E. Feldman, Ph.D., M.D. Department of Medicine Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale The goals of this project are related both to medical science and artificial intelligence: (a) use Al methods to allow the informed but non-expert reader of the medical literature to evaluate a randomized clinical trial, and (b) use the interpretation of the medical literature as a test problem for studies of knowledge acquisition and fusion of information from disparate sources. REFEREE and REVIEWER, a planned extension, will be used to evaluate the medical literature of clinical trials to determine the qualify of a clinical trial, make judgments on the efficacy of the treatment proposed, and synthesize rules of clinical practice. The research is an initial step toward a more general goal - building computer systems to help the clinician and medical scientist read the medical literature more critically and more rapidly for use in making clinical decisions. 6. Medical Relevance The explosive growth of the medical literature has created a severe information gap for the busy clinician. Most physicians can afford neither the time required to study all the pertinent journal articles in their field, nor the risk of ignoring potentially significant discoveries. The majority of clinicians, in fact, have little sophistication in epidemiology and statistics; they must nonetheless base their pragmatic decisions on a combination of clinical experience and published literature. The clinician's computerized assistant must ferret out useful maxims of clinical practice from the medical literature, pass judgment on the quality of medical reports, evaluate the efficacy of proposed treatments, and adjudicate the interpretation of conflicting and even contradictory studies. C. Highlights of Progress REFEREE presently encodes the methodological knowledge of a highly regarded biostatistician at Stanford (Dr. Bill Brown). The system allows the informed but non- expert reader of the medical literature to evaluate the credibility of a randomized clinical trial. In the future, REFEREE and its extensions will alleviate the knowledge-acquisition 197 E. H. Shortliffe REFEREE Project 5P41 -RR00785-15 bottleneck for an automated medical decision-maker: the program will evaluate the quality of a clinical trial, judge the efficacy of the treatment proposed therein, and synthesize rules of clinical practice. For the present, however, the fusion of knowledge from disparate sources remains a problem in pure Al. The efforts of the REFEREE team have instead focused their efforts on the refinement and deepening of REFEREE's biostatistical knowledge by applying effective knowledge acquisition and knowledge engineering techniques. Dr. Diana Forsythe and Dr. Harold Lehmann are developing and using interview methods to acquire this knowledge from Dr. Brown, and R. Martin Chavez is implementing this in the prototype REFEREE expert system. The REFEREE prototype is a consultant that evaluates the design and reporting of a single conclusion from randomized control trial for its credibility. It contains, in preliminary form, Professor Brown's expert knowledge of biostatistics. REFEREE evaluates each statistical procedure described by the authors of the paper. The automated consultant then determines the most appropriate method for the problem at hand, based on the design of the trial and the hypotheses to be tested. REFEREE checks critical assumptions, looks for possible statistical abuses, and verifies adjustments. The Knowledge Base: Randomized controlled trials are used to test hypotheses regarding the effectiveness of various kinds of medical interventions. Dr. Brown classifies studies on the basis of three major attributes: the type of intervention tested (e.g. drug, surgery, health process change, etc.); the type of endpoint against which that intervention was tested (e.g. mortality, objective morbidity, subjective morbidity, etc.); and the type of conclusion drawn by the investigator/author on the basis of the research (e.g. that different treatments do or do not produce different outcomes, that a particular treatment is or is not cost-effective, etc.). Following this classificatory scheme, we decided to begin by producing a prototype REFEREE system that would help the reader to evaluate a single published conclusion concerning the effect of a given drug treatment on mortality. Knowledge Acquisition: Having defined the scope of the initial knowledge base, we turned to the problem of collecting the information from Dr. Brown for inclusion in the system, i.e. knowledge acquisition. This task generally involves a relatively long- term process of face-to-face information gathering during sessions between the expert and one or more knowledge engineers. Dr. Diana Forsythe has noted a parallel between the communicative and analytical tasks involved in knowledge acquisition and those undertaken in ethnographic research. For this reason, we included an anthropologist in the research team and make use of ethnographic techniques in order to maximize the efficiency and quality of the data collection process. Dr. Lehmann and Dr. Forsythe have carried out several months of systematic interviews with Dr. Brown in order to begin the process of constructing and refining the knowledge base for the current REFEREE prototype. We have combined a case- based approach that allows us actively to observe Dr. Brown as he reads papers, with semi-directed interviewing oriented toward understanding his terminology and category system. We find that these techniques work very well: Dr. Brown's interest in the knowledge acquisition process has been sustained, and indeed has increased over time as the system based on his expertise has evolved. He is clearly comfortable with this approach, and notes that it has actually afforded him additional insight into the way he interprets the literature. Over the past year we have altered our knowledge representation from that of rules to that of an influence diagram. This is an acyclic directed graph of propositions or variables connected by links, where the absence of a link indicates conditional E. H. Shortliffe 198 5P41 -RR00785 15 REFEREE Project independence of the two variables. This formalism has been used in decision analysis to enable, experts to convey their knowledge of a domain, and, more recently, has been used in Al to represent that knowledge in expert systems. This shift in formalism significantly altered the knowledge acquisition process and the implementation of that knowledge in our program. Based on information from our expert, we have taken credibility as the goal parameter of the present system. This goal is defined operationally by Dr. Brown as "my odds that the conclusion of the paper would be replicated in an experiment based on the methods reported in the paper but without any of the flaws". In assessing credibility, for instance, Dr. Brown considers the blindedness of the randomization, the blindedness of the execution, the equivalence of the two groups at baseline, the equivalence in treatment of the two groups, the completeness of results reporting, and the propriety of the statistical analysis. We recognize that these variables are not all conditionally independent on credibility; work is in progress to assess as accurately as possible just what the conditional relationships are. Our use of influence diagrams has numerous advantages: the approach is acceptable to Dr. Brown, it is flexible, it can represent several aspects of the structure of the knowledge used by the expert, and the resultant data can be entered easily into the computer. Inference in REFEREE: REFEREE was originally built within EMYCIN, a backward- chaining rule-based Al environment developed from MYCIN at Stanford. This environment is ideally suited for ordered collection of evidence and a diagnosis of the goal state at the end of that process. The state of belief or knowledge in parameters not directly between the evidence and the goal state is irrelevant. Our present system focuses on maintaining consistency over the entire knowledge base as new evidence is incorporated into the system. The constraints implied by the new data and Dr. Brown's prior knowledge are propagated throughout the system by Judea Pearl's message-passing algorithm for belief networks. During the consultation with the program, questions are chosen by the user and answered at his or her discretion, and the state of belief in any parameter can be requested at any time. The odds of replicating the study, then, can be viewed at any point during evidence collection. The User Interface: REFEREE was initially run entirely on the SUMEX resource. Mr. Chavez reimplemented the program on a stand-alone workstation, the Xerox 1186 in the KEE commercial expert system shell. The availability of bit-mapped screens made us more sensitive to issues of the user interface, but the shell could not deal easily with the uncertainty inherent in our domain. Mr. Chavez then ported the system to a Texas Instrument Explorer work-station, for which he designed an entirely new knowledge engineering shell which integrated EMYCIN and influence diagrams. It was apparent, however, that to accommodate the multiple interface needs of our potential user community, we needed a graphics environment that would allow frequent changes and customization. Thus, we turned to a final environment custom-made for influence-diagram-based expert systems. The KNET system, also developed by Mr. Chavez, separates the inferencing capabilities and graphical manipulation of the knowledge base into MPW Object Pascal from the textual part of the knowledge base and the the evidence collection in HyperCard. This system runs on the Macintosh II with 4 MB of RAM. The program code is now entirely independent of the knowledge required for reading papers. REFEREE has a new interface that is intuitive and consistent. There is an innovative consultation mode in which questions are presented in free-format menus. The dialogues are mixed-initiative and of mixed levels, allowing the user such options as requesting more detailed questions or cutting off apparently fruitless lines of questioning. With the new REFEREE prototype, the user interacts with the 199 E. H. Shortliffe REFEREE Project 5P41 -RR00785-15 machine using a mouse-pointing device Finally, the screen enables the user to orient himself at all times, obviating the need for special commands to help the user "navigate" through the knowledge base. Our expert recently provided the best indication of the usability of this new system. After only a brief introduction to the new machine and interface, he was able - for the first time - to run an entire consultation by himself. Current Status: At this point, REFEREE is a prototype that enables the clinician to read clinical trials more critically. A number ot computational issues remain, such as the optimal representation of Dr. Brown's knowledge in our current formalism. Furthermore, REFEREE represents only the first step in a larger research plan, the automation of knowledge acquisition (see section on Research Plans, below). Current work in the restricted domain of clinical trials will, we hope, illustrate general principles in the design of decision makers that gather expertise from written text and multiple knowledge sources. D. Relevant Publications 1. Haggerty, J.: REFEREE and RULECRIT/C: Two prototypes for assessing the quality of a medical paper. REPORT KSL-84-49. Master's Thesis, Stanford University, May 1984. 2. *Chavez, R. Martin and Cooper, G. F.: KNET: integration Hypermedia and Normative Bayesian Modeling. REPORT KSL. Stanford University, March 1988. 3. *Lehmann, H. Knowledge Acquisition for Probabilistic Expert Systems. Submitted to Symposium on Computer Applications in Medical Care, 1988. E. Funding Support REFEREE currently receives only a small amount of funding. Most of the research is performed in time contributed by the researchers to this project. Title: Knowledge-Based Systems Research PI: Edward A. Feigenbaum Agency: Defense Advanced Projects Research Agency Grant identification number: N00039-86-0033 Total award period and amount: 1 O/l /85 - 9130188 $4,130,230 (direct and indirect) Current award period and amount: 10/l /87 - 9130188 $1,467,300 (direct and indirect) REFEREE component is $27,706, or 1.9 % of grant total. II. INTERACTIONS WlTH THE SUMEX-AIM RESOURCE A. Medical Collaborations Dr. Brown and Dr. Feldman of the Stanford University School of Medicine are actively involved in the REFEREE project and are the primary domain experts and critics for this project. E. H. Shortliffe 200 5P41 -RR00785- 15 REFEREE Project C. Critique of Resource Management The SUMEX computer resource and Lisp workstations have been very important for the work to date, and the SUMEX staff has continued to be very cooperative with the REFEREE project. Ill. RESEARCH PLANS A. Goals & Plans The overall objective of the REFEREE project is to use recent Artificial Intelligence techniques to build a system that helps the informed but statistically non-expert reader to evaluate critically the medical literature on randomized controlled trials (RCT'sj. This system will contain and be able to apply dynamically the detailed specialized knowledge of Dr. Byron W. Brown, a biostatistician expert in the design and evaluation of randomized controlled trials. We have divided our overall objective into two goals: . Goal 1 is the construction of an expert system to help readers (e.g. medical students, medical researchers, clinicians, journal editors, or editorial assistants) assess the credibility of a single conclusion drawn from a single journal report of .a randomized controlled trial. We have already made substantial progress toward this goal with the development of the prototype REFEREE system. o Goal 2 is the expansion of REFEREE to an expert system that can be used by a similar range of readers to facilitate the evaluation of multiple reports based on randomized controlled trials. This expanded system, to be known as the REVIEWER, will thus perform meta-analysis. The task of extending and refining the prototype REFEREE system in order to achieve these goals can be characterized in terms of three dimensions: . Making the system more accessible to a variety of people by improving the user interface, validating the system's performance with different types of users, and providing an explanatory capability . Expanding the knowledge base by continuing the knowledge acquisition process to cover additional types of RCT's . Improving the inference engine to ensure consistency of the knowledge base and to focus the consultation process on questions relevant to the situation and the individual user. The specific steps that are planned for the enhancement of the REFEREE system include the following: . Critique individual clinical trials according to the methodological quality of the trial; . Measure the efficacy of treatment as demonstrated in a randomized control trial; . Compare and contrast the credibility and efficacy of treatment reported by multiple journal articles; and 201 E. H. Shortliffe REFEREE Project 5P41 -RR00785-15 . Combine the qualitative techniques of heuristic reasoning and the quantitative methods of statistical meta-analysis to extract a consensus opinion from multiple knowledge sources. In addition, plans for Goal 2, the REVIEWER system to analyze multiple RCT's and form a consensus judgment, include: . Complete a review of the available literature on meta-analysis and augment the REFEREE prototype to produce estimators for meta-analysis and incorporate expert knowledge on the appropriateness of these methods. . Add explicit and heuristic knowledge needed for the calculation of robust, non-parametric estimators of effect size. . Construct a prototype of a system that builds categorical models in the domain of meta-analysis, to perform autonomous investigations in the domain of statistical model-building. The REVIEWER will utilize expert knowledge in biostatistics to guide its search for meaningful models. . Build a prototype of a system that can explore the domain of regression models for multiple RCT's that will use expert knowledge in its selection of predictor variables. . Package the REVIEWER in a form suitable for use by physicians and their assistants. . Verify the expertise of the REVIEWER system on a suite of papers drawn from clinical trials, similar to the validation of REFEREE above. 6. Justification for continued SUMEX use The local area network maintained by the SUMEX staff is essential to the effective development and use of the REFEREE system on Lisp workstations. The availability of the Xerox workstations makes possible the evaluation of prototypes in that environment, and also facilitates the development of good user interfaces. The connections through the 2060 to local and national computer networks such as ARPANET are important for sharing ideas and results with other medical researchers. C. Need for other computing resources We are exploring the implementation of REFEREE on personal computers such as the Macintosh II and other high performance machines. We anticipate the need for at least two of these machines for transporting our system and developing new modes of interaction with both naive and experienced users. E. H. Shortliffe 202 5P41 -RR00785-15 Pilot AIM Projects W.D. Pilot AIM Projects Following is a description of the informal pilot projects currently using the AIM portion of the SUMEX-AIM resource, pending funding, full review, and authorization. In addition to the progress report presented here, an abstract is submitted on a separate Scientific Subproject Form. 203 E. H. Shortliffe Dynamic Systems Project 5P41 -RR00785-15 MD. 1. Dynamic Systems Project Decision Support for Time-Varying Clinical Problems Lawrence E. Widman, M.D., Ph.D. Division of Cardiology University of Texas Health Science Center San Antonio, Texas 78284 I. SUMMARY OF RESEARCH PROGRAM A. PROJECT RATIONALE Time-varying systems, which include many areas of medicine, science, economics, and business, can be described mathematically by differential equations. They are distinct from the pattern-matching and logic-based domains dealt with so successfully by existing expert system methods, because they can include feedback relationships. It is generally felt that they are best approached by enhancement of existing methods for deep model-based reasoning. The goal of this project is to develop Al methods for capturing and using knowledge about time-varying systems. The strategy is to address general problems in model- based knowledge representation and reasoning. The intermediate objective is to develop methods which are powerful enough to work in selected realistic situations yet are general enough to be transportable to other, unrelated knowledge domains. Our approach is to work on well-defined yet complex and interesting problems in the medical domain. We have, therefore, selected the human cardiovascular system as our prototype of a time-varying system, and are developing methods for representing and reasoning about its mechanical and electrical activities in the normal and diseased states. A.1 Technical Goals This project presently has two distinct tracks: hemodynamic modeling and cardiac arrhythmia interpretation. 1. Hemodynamic Modeling The goals of this subproject are to develop: (a) a knowledge-representation method using symbolic modeling which captures the qualitative and, when possible, the quantitative behavior of systems with feedback relationships. Preferably, the symbolic model should be translatable into the differential equations which describe the behavior of the system being modeled. (b) a reasoning method based on the symbolic modeling tool created in subgoal (a) which permits the inference of differential diagnoses (a set of hypothesized diagnoses) from incomplete data. (c) a reasoning method based on subgoals (a) and (b) which permits inference of the state of the model for each hypothesized diagnosis. This subgoal would be satisfied by an algorithm which specifies a self-consistent set of values for all E. H. Shortliffe 204 5P41-RR00785-15 Dynamic Systems Project variables in the model, for a given hypothesis based on a given set of data. Such sets of data would constitute initial conditions for differential equations derived from the model. (d) a simulation method, based on the model and its equivalent differential equations together with the initial conditions derived from the differential diagnosis (steps a-c above), for predicting the expected time course of the system being modeled for each hypothesized diagnosis. This method could also be used to predict the effects of treatments being considered for recommendation by the program. (e) a reasoning method, based on domain-independent properties of the model, for shrinking and/or expanding the model automatically to use a minimal model configuration to account for normal and abnormal data. (f) an explanation facility for examining the model, the given data, the inferred hypothesized diagnoses, predicted behaviors, and modifications of the model, to answer user queries and to teach fundamental concepts. 2. Cardiac Arrhythmia Recognition The goals of this subproject are to develop: (a) a symbolic model of the electrical system of the human heart, including pertinent anatomic and electrophysiologic features of the normal and diseased heart. The electrophysiologic features would include deterministic characteristics (e.g., conduction velocities, refractory periods), stochastic features (e.g., behavior of automatic foci), and temporal interactions (e.g., competing pacemakers). (b) a symbolic/numeric representation of the observable features of the electrical activity of the heart, both surface ECG and intracardiac recordings, including noise. This representation would be intended to altow a feature extraction module working on actual patient data to communicate with a symbolic reasoning module, and would be translatable directly into waveform display format. (c) a reasoning method for extracting features (identifying waveforms) from raw, digitized signal data. This method would augment established signal processing techniques by using knowledge-based algorithms to improve detection of P and T-U waves and to improve rejection of noise. It should be noted that this is itself a major research undertaking in the signal processing domain. (d) a reasoning method for inferring the cardiac rhythms consistent with a given disease state in the model, similar to the prediction of consequences of the hemodynamic model in the first subproject. The output of this method would be in the symbolic/numerical representation of subgoal (b). (e) a reasoning method for inferring possible disease states in the model from a given feature-extracted recording of the electrical activity of the heart. This subgoal constitutes cardiac arrhythmia interpretation, and is itself a major research project. (f) a categorization method for inferring hierarchies of diagnoses from elementary abnormalities, For example, "periods of atrial fibrillation up to 30 minutes at up to 150 beats/min, supraventricular tachycardia of up to 10 beats length at a rate of 130 beats/min, and sinus bradycardia with a minimum rate of 45, all consistent with the sick sinus ("tachy-brady") syndrome" and "two QRS morphologies are present: they are narrow at rates less than 120 and are wide at rates above 120, consistent with a rate-dependent bundle branch block". (g) an explanation facility for examining the model, the input data, and the interpretations to answer user queries and to teach fundamental concepts. 205 E. H. Shortliffe Dynamic Systems Project 5P41-RR0078515 5. MEDICAL RELEVANCE AND COLLABORATIONS The two subprojects have related but separate medical goals: 1. Hemodynamic Modeling. There are two subgoals in this subproject: model-based sensor integration and model-based care-giver assistance. a. Model-based Sensor Integration The long-range application of this subproject is the integration of patient-related data in the intensive care environment. Model- based real-time systems would allow the system to share a global understanding of the patient's condition with the human care-givers. Thus, it could interpret significant trends in key parameters and could draw attention to relationships which might otherwise escape attention in the constant flood of data common to these environments. b. Model-based Care-giver Assistance It could also serve as an assistant to the care-giver. In this mode, the human care- giver could evaluate the merits of proposed diagnostic and therapeutic measures in light of available data on the patient's condition. Practical application of these concepts requires further development of the model and the reasoning algorithms, and extensive testing against real clinical scenarios. Refinement and quality control are presently the responsibility of the principal investigator, who is a board-certified cardiologist with subspecialty interest in cardiac electrophysiology. Practical application also awaits general acceptance of standardized hospital data buses for automatic acquisition of important parameters now stored primarily on paper or on computers outside the intensive care setting, such as fluid inputs and outputs, medications, and results of invasive and non-invasive tests. Further, improved user interfaces will require better graphics and increased computer literacy on the part of care-givers. 2. Cardiac Arrhythmia Recognition. The long-range application of this sub-project is in clinical devices such as intensive-care arrhythmia monitors, portable Holter monitors, and implantable cardioverter-defibrillators. There are two subgoals: recognition of surface electrocardiographic (ECG) recordings and recognition of intracardiac recordings. a. Recognition of surface electrocardiographic recordings. Substantial and well-recognized obstacles in signal processing will likely prevent non-Al algorithms from advancing beyond the current state of the art of interpretation of surface ECG recordings. These obstacles are primarily the problems of reliable detection of P and T-U waves, and rejection of noise. We hope further that, by mimicking the behavior of expert human cardiologists, these obstacles can be bypassed if they cannot be overcome. We have enlisted as consultants Dr. William Long of M.I.T., who has long experience with modeling of the cardiovascular system and with symbolic approaches to ECG interpretation, and Dr. Charles Mead, a consultant formerly with the Division of Cardiology at Washington University - St. Louis who has 15 years experience with the development of algorithms for clinical arrhythmia recognition. b. Recognition of intracardiac recordings. E. H. Shortliffe 206 5P41 -RR00785-15 Dynamic Systems Project Intracardiac recordings, which are taken from wires placed in the heart by percutaneous venous puncture or around the heart by surgery, are relatively free of P wave ambiguity and of noise. They are representative of the quality of signals available to implantable cardioverter-defibrillators. Cardioverter-defibrillators are devices like pacemakers in that they monitor the heart rhythm in a patient to determine if an abnormality exists. They are capable of taking action (electrical countershock) if an appropriate abnormality is detected. Unlike ordinary pacemakers, these devices detect abnormalities characterized by rapid rates of heart activity, rather than excessively slow rates. They have been shown to reduce one-year mortality in high-risk patients from 30% to 2%, and they are expected to play an increasingly large role in treatment of such patients. These relatively new devices currently use quite simple algorithms to detect abnormalities. The action they take consists of applying an electrical shock directly to the heart. This shock is frequently unpleasant to the patient. The problem is that the algorithms sometimes confuse innocent rapid heart rates, such as from exercise or atrial tachyarrhythmias, with lethal ventricular arrhythmias. This has proved troublesome enough to prompt repeated calls in the electrophysiology literature for improved algorithms for arrhythmia recognition in these devices. The algorithms developed in this subproject would be suitable for this application when the computer power in the devices improves. Because these devices require powerful energy sources to perform repeated shocks over their lifetimes of 2-3 years, the power drain of more sophisticated computer chips is less important than it would be in ordinary pacemakers. C. Highlights of Research Progress 1. Hemodynamic Modeling Subgoals (a) through (d) have been accomplished in prototype for,m. The approach relies on a semi-quantitative representation (subgoal (a)) which assigns values by default if the user does not specify more detailed information. Constraint propagation using a dynamically generated semi-quantitative quantity space is performed by interpreting the model as a set of constraint equations. Domain- independent heuristics which recognize morphological features of the model are used to further constrain the propagation of constraints and to generate hypotheses when ambiguities arise. These heuristics generate a set of self- consistent hypotheses, each of which is a hypothesized diagnosis (subgoal b). Each hypothesized diagnosis is then refined by mathematical relaxation, in which the propagated values are treated as initial guesses, and the values are refined iteratively, again by interpreting the model as a set of constraint equations (subgoal cl* In the several scenarios which have been examined, the value assignments achieved by hypothesis and iterative refinement have achieved correlation coefficients up to 0.90 with the values obtained by simulation of the same model. Phase (d) was accomplished by translating the model into a set of dynamical systems equations, which were then integrated in the standard manner. We did not anticipate continuing this subproject after beginning the cardiac arrhythmia subproject. However, an opportunity has arisen to collaborate with Professor Benjamin Kuipers at the University of Texas at Austin on the development of algorithms which combine the advantages of classical qualitative simulation and of our semi- quantitative simulation approach. Professor Kuipers and a graduate student will spend the summer of 1988 in San Antonio for this purpose and to develop a case-based reasoning knowledge base of cardiovascular disorders. 207 E. H. Shortliffe Dynamic Systems Project 5P41 -RR00785- 15 2. Cardiac Arrhythmia Recognition This subproject has been operational for almost one year. With support from the University of Texas Health Science Center at San Antonio and the American Heart Association, Texas Affiliate, we have acquired and integrated a Symbolics 3640 LISP workstation and an IBM- compatible microcomputer with attached optical scanner. Progress has been made in two subprojects of this project: a) Signal acquisition from paper records. Many standardized and interesting ECG recordings are available on paper but not in digitized form. To make these recordings machine-readable, we have developed software in TurboPascal (a popular microcomputer language) which digitizes the ECG signal on a paper ECG record by extracting the ECG line from the bit map of the image produced by the optical scanner. Algorithms have been developed with can extract the ECG line when grid lines are present and in the presence of varying intensity of the original image (as, for example, in Xerox copies of paper records). The output of this software is a sequence of numbers representing the ECG voltage at sequential instants of time, equivalent to the output of an analog-to-digital converter. This set of numbers can then be used for development of ECG recognition software. In comparison to conventional analog-to-digital recordings, the optical scanner output has a root- mean-square error of 3.5-4-O% (Widman and Freeman), or about 1.5 times the thickness of a typical pen line on paper. b) Identification of waveforms in raw signal data. Using digitized signals from electronic analog-to-digital conversion and the optical scanner conversion described above, we have implemented prototype software for waveform identification. It is organized by levels of abstraction. At the lowest level, conventional slope-analysis and filtering algorithms are used to identify the baseline and the QRS complexes. Knowledge-based slope-analysis algorithms then analyze the remaining signal to assign tentative labels of "P waves," "T waves," and "other" waves. Then, knowledge-based pattern recognition algorithms look for temporal and morphological relationships to confirm or correct the tentative labels. The prototype system is able to find P waves superimposed on T waves, a combination which cannot be detected by any commercial cardiac arrhythmia system. D. List of Relevant Publications 1. Widman, L.E. Knowledge-Based Fault Identification and "What If" Simulation in Symbolic Dynamic Systems Models. Proceedings of the 1988 Society for Computer Simulation Multiconference on Al and Simulation, pp. 89-94, 1988. 2. Widman, L.E., Lee, Y.-B., and Y.-H. Pao. Diagnosis of Causal Medical Models by Semi-Quantitative Reasoning. In: Miller, P.L. (ed.). Topics in Medical Artificial Intelligence, Springer-Verlag (in press) 3. Widman, L.E. Semi-Quantitative "Close Enough" Systems Dynamics Models: An Alternative to Qualitative Simulation. In: Widman, L.E., Loparo, K.A., and N.R. Nielsen (eds.). Artificial Intelligence, Simulation, and Modeling. John Wiley 8 Sons, New York (in press). 4. Widman, L.E., Loparo, K.A., and N.R. Nielsen (eds.). Artificial Intelligence, Simulation, and Modeling. John Wiley 8 Sons, New York (in press). 5. Widman, L.E. and G.L. Freeman. A-to-D Conversion from Paper Records with a Desktop Scanner and a Microcomputer. (submitted to the American Journal of Physiology). E. H. Shortliffe 208 5P41 -RR00785- 15 Dynamic Systems Project E. Funding Support 1. American Heart Association, Texas Affiliate Grant-in-Aid Award. Knowledge-Based Computer Algorithms for Arrhythmia Analysis. Principal Investigator: Lawrence E. Widman. Award period: July, 1987 - June, 1988. Level: $24,850 direct costs. 2. Medical Research Service, Department of Medicine and Surgery, Veterans Administration. Knowledge-Based Computer Algorithms for Cardiac Arrhythmia Interpretation. Principal Investigator: Lawrence E. Widman. Award period: July, 1988 - June, 1990. Level: $33,097, year 1; $26,647, year 2. (direct costs) II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Sharing and interactions with other SUMEX-AIM projects The major interaction with SUMEX-AIM this year has been communication with members of the AIM community, now that our work has been transferred to the Symbolics machine here. SUMEX-AIM allows ready Email access to users of ARPANET, Bitnet and CSNet. This access has proved invaluable in communicating rapidly and effectively with co- workers at other institutions. The value of this access has been demonstrated particularly during the preparation of the book, Artificial Intelligence and Simulation, which will go into production in June, 1988. Email was used to communicate with contributors, to receive electronic copies of their manuscripts, and to issue unobtrusive, gentle but effective reminders when the manuscripts were late. WithOfJt Email access, the editorial process would have been much more difficult. Review of the longer term history of this project shows that it would not exist had SUMEX-AIM not provided telecommunication support for the initial feasibility project in 1984-1985, which was carried out on the computers of the MIT Laboratory for Computer Science, Clinical Decision Making Group, and computational support during the development of the semi-quantitative approach in the years 1985-1987. C. Critique of Resource Management The service provided by SUMEX-AIM has been exemplary, largely because of prompt and effective response to difficulties as they arise. There has been a clear effort to assure that telecommunication access remained reliable during changes in commercial vendors, and the staff have responded to several technical questions promptly and accurately. Down-time has been minimal compared to that of other systems we have used, and is almost always scheduled several days in advance. The reason we work within the AIM community is that it is the natural niche for our research interests. There is no short-term prospect that this project will reach commercial maturity or that it will lose sight of fundamental Al issues, and so we feel that it still belongs in the scientific AIM framework. As noted in the previous section, the communication with other members of the AIM community has proved invaluable in the advancement of this project. 209 E. H. Shortliffe Dynamic Systems Project 5P41 -RR00785- 15 III. RESEARCH PLANS A. Project Goals and Plans The long range goals of this project are to develop intelligent comprehensive monitoring/alarm systems for intensive care unit settings: and intelligent arrhythmia recognition systems for monitors, Holter recorders, and implantable cardioverteri defibrillators. The short term strategies for achieving these goals are discussed above. The principle investigator is a staff cardiologist and Assistant Professor in the Division of Cardiology. He serves as an Attending Cardiologist during hemodynamic, angiographic and electrophysiological studies on selected patients. Substantial time is committed to research, and this project will constitute his major research emphasis. 6. Justification and requirements for continued SUMEX use The justification for this project is its potential for advancing the state of the art of expert system technology in the area of temporal reasoning and deep causal modeling, and for demonstrating practical use of expert symbolic computing in potentially life-saving, knowledge-intensive environments. The requirements for continued SUMEX-AIM use should be the same as currently: telecommunications support, ARPANET access, about 3 megabytes of disc space, and a small amount of CPU time. C. Needs and plans for Other Computing Resources Beyond SUMEX-AIM As predicted in last year's annual report, the main computational burden has been transferred to a local resource. SUMEX-AIM is needed primarily for communication and demonstration projects, as noted above. D. Recommendations for Future Community and Resource Development Our strong recommendation is that SUMEX-AIM be maintained as a national AIM resource for communication, development of software useful to the AIM community, and sharing of demonstration projects. SUMEX-AIM could also serve as a central source of advice for new workstation users who may be geographically isolated from experienced workstation users. Additionally, we would strongly support retention of the current telecommunication support and enough computing power to support promising young investigators who would otherwise not have access to symbolic computing power. E. H. Shortliffe 210 5P41 -RR00785 15 Pathfinder Project IV.D.2. Pathfinder Project Pathfinder Project Bharat Nathwani, M.D. Department of Pathology University of Southern California Lawrence M. Fagan, M.D., Ph.D. Department of Medicine Stanford University I. SUMMARY OF RESEARCH PROGRAM A. Project Rationale Our project addresses difficulties in the diagnosis of lymph node pathology. Five studies from cooperative oncology groups have documented that, while experts show agreement with one another, the diagnosis made by practicing pathologists may have to be changed by expert hematopathologists in as many as 50% of the cases. Precise diagnoses are crucial for the determination of optimal treatment. To make the knowledge and diagnostic reasoning capabilities of experts available to the practicing pathologist, we have developed a pilot computer-based diagnostic program called Pathfinder. The project is a collaborative effort of the University of Southern California and the Stanford University Medical Computer Science Group. A pilot version of the program provides diagnostic advice on 72 common benign and malignant diseases of the lymph node based on 110 histologic features. Our research plans are to develop a full-scale version of the computer program by substantially increasing the quantity and quality of knowledge and to develop techniques for knowledge representation and manipulation appropriate to this application area. The design of the program has been strongly influenced by the INTERNISTKADUCEUS program developed on the SUMEX resource. Pathfinder computer science research is focused on the exploration and extension of formal techniques for decision making under uncertainty. Research foci include (1) the assessment and representation of important probabilistic dependencies among morphologic features and diseases, (2) the representation of knowledge about the progression of disease over time, (3) the acquisition and use of independent expert knowledge bases, (4) the customization of the system's reasoning and explanation behaviors to reflect the expertise of the user, and, (5) the explanation of complex formal reasoning techniques. Toward the pragmatic goal of constructing a useful pathology teaching and decision support system, Pathfinder investigators are attempting to use intelligent computation to substantially increase the quantity and quality of pathology knowledge available to pathologists. Important areas of this knowledge integration task involve ongoing research on the crisp definition important morphologic features and feature severities, the synthesis of information from multiple experts, and the translation among multiple pathology classification schemes. A group of expert pathologists from several centers in the U.S. have showed interest in the program and helped to provide the structure of the knowledge base for the Pathfinder system. 211 E. H. Shortliffe Pathfinder Project 5P41 -RR00785- 15 6. Medical Relevance and Collaboration One of the most difficult areas in surgical pathology is the microscopic interpretation of lymph node biopsies. Most pathologists have difficulty in accurately classifying lymphomas. Several cooperative oncology group studies have documented that while experts show agreement with one another, the diagnosis rendered by a "local" pathologist may have to be changed by expert lymph node pathologists (expert hematopathologists) in as many as 50% of the cases. The National Cancer Institute recognized this problem in 1968 and created the Lymphoma Task Force which is now identified as the Repository Center and the Pathology Panel for Lymphoma Clinical Studies. The main function of this expert panel of pathologists is to confirm the diagnosis of the "local" pathologists and to ensure that the pathologic diagnosis is made uniform from one center to another so that the comparative results of clinical therapeutic trials on lymphoma patients are valid. An expert panel approach is only a partial answer to this problem. The panel is useful in only a small percentage (3%) of cases; the Pathology Panel annually reviews only 1,000 cases whereas more than 30,000 new cases of lymphomas are reported each year. A panel approach to diagnosis is not practical and lymph node pathology cannot be routinely practiced in this manner. We believe that practicing pathologists do not see enough case material to maintain a high level of diagnostic accuracy. The disparity between the experience of expert hematopathology teams and those in cpmmunity hospitals is striking. An experienced hematopathology team may review thousands of cases per year. In contrast, in a community hospital, an average of only ten new cases of malignant lymphomas are diagnosed each year. Even in a university hospital, only approximately 100 new patients are diagnosed every year. Because of the limited numbers of cases seen, pathologists may not be conversant with the differential diagnoses consistent with each of the histologic features of the lymph node; they may lack familiarity with the complete spectrum of the histologic findings associated with a wide range of diseases. In addition, pathologists may be unable to fully comprehend the conflicting concepts and terminology of the different classifications of non-Hodgkin's lymphomas, and may not be cognizant of the significance of the immunologic, cell kinetic, cytogenetic, and immunogenetic data associated with each of the subtypes of the non-Hodgkin's lymphomas. In order to promote the accuracy of the knowledge base development we will have participants for multiple institutions collaborating on the project. Dr. Nathwani will be joined by experts from Stanford (Dr. Dorfman), St. Jude's Children's Research Center -- Memphis (Dr. Berard) and City of Hope (Dr. Burke). C. Highlights of Research Progress C. 1 Previous Accomplishments Since the project's inception in September, 1983, we have constructed several versions of Pathfinder. The first several versions of the program were rule-based systems like MYCIN and ONCOCIN which were developed earlier by the Stanford group. We soon discovered, however, that the large number of overlapping features in diseases of the lymph node would make a rule-based system cumbersome to implement. We next considered the construction of a hybrid system, consisting of a rule-based algorithm that would pass control to an INTERNIST-like scoring algorithm if it could not confirm the existence of classical sets of features. We finally decided that a modified form of the INTERNIST program would be most appropriate. The original version of Pathfinder is written in the computer language MacLisp and runs E. H. Shortliffe 212 5P41 -RR00785- 15 Pathfinder Project on the SUMEX DEC-20. This was transferred to Portable Standard Lisp (PSL) on the DEC-20, and later transferred to PSL on the HP 9836 workstations. Two M.D./Ph.D (Stanford Medical Information Science Program) students, David Heckerman and Eric Horvitz, designed and implemented the program and are continuing to lead research on the project. The prototype knowledge base was constructed by Dr. Nathwani. During the early part of 1984, we organized two meetings of the entire team, including the pathology experts, to define the selection of diseases to be included in the system, and the choice of features to be used in the scoring process. During the last three years, we have focused on methodologies for more accurately representing expert beliefs. In particular, we have used influence diagrams to represent dependencies among features in the Pathfinder knowledge base. A great deal of effort has been devoted to assessing and representing the intricate relationships among features that exist in the domain. We believe that this process will help to overcome some of the limitations of medical diagnostic systems. We have also focused on the problem of complex information-theoretic inference. The explanation of a systems diagnostic behavior has been found to be of extreme importance to physicians. Unfortunately, it is often difficult to explain reasoning based on optimal models of inference. We have worked on the use of a set of alternative abstraction hierarchies to control inference. Our current techniques enable us to trade off optimality for the transparency of reasoning. We are now studying the control of this tradeoff to optimize inference. We are also currently developing specialized probabilistic inference techniques that take advantage of particular patterns of independence that occur among features in lymph node pathology. In addition, we are developing richer mechanisms for controlling uncertain reasoning within the Pathfinder expert system. C.2 The Pathfinder knowledge base The basic building block of the Pathfinder knowledge base is the disease profile or frame. Each disease frame consists of features useful for diagnosis of lymph node diseases. Currently these features include histopathologic findings seen in both low- and high-power magnifications. Each feature is associated with a list of exhaustive and mutually exclusive values. For example, the feature pseudofollicularity can take on any one of the values absent, slight, moderate, or prominent. These lists of values give the program access to severity information. In addition, these lists eliminate obvious interdependencies among the values for a given feature. For example, if pseudofollicularity is moderate, it cannot also be absent. Qualitative dependencies among features for each disease are represented using the influence diagram methodology mentioned above. An influence diagram contains nodes and arcs. Nodes represent features and arcs represent dependencies among features. In particular, an arc is drawn from one feature to another when an expert believes that knowing one feature can change his beliefs that another feature will take on its possible values even when the diagnosis is known. Probabilities are used to quantitate the beliefs asserted by the expert. C.3 Macintosh II Workstations We are currently moving the Pathfinder system into the Macintosh II environment. Thus the SUMEX-AIM resource will only be used for file archival and retrieval and for communication between the Stanford and USC Pathfinder research teams. 213 E. H. Shortliffe Pathfinder Project 5P41 -RR00785- 15 D. Publications Since January 7984 I. Horvitz, E.J., Heckerman, D.E., Nathwani, B.N. and Fagan, L.M.: Diagnostic Strategies in the Hypothesis-directed Pathfinder System, Node Pathology. HPP Memo 84-l 3. Proceedings of the First Conference on Artificial Intelligence Applications, Denver, Colorado, Dec., 1984. 2. Heckerman, D. E., and Horvitz, E. J., based Systems," in "The Myth of Modularity in Rule- Uncertainty in Artificial Intelligence, Vol. 2, J. Lemmer, L. Kanal, ed., North Holland, New York, 1987. 3. Horvitz, E.J., Heckerman, D.E., Nathwani, B.N. and Fagan, L.M.: The Use of a Heuristic Problem-solving Hierarchy to Facilitate the Explanation of Hypothesis-directed Reasoning. KSL Memo 86-2. Medlnfo, Washington D.C., October, 1986. Proceedings of 4. Horvitz, E. J., "Toward a Science of Expert Systems," Invited Paper, Computer Science and Statistics: Proceedings of the 18th Symposium on the Interface, American Statistical Association, March, 1986, pgs. 45-52. 5. Heckerman, D.E., "An Axiomatic Framework for Belief Updates," in Uncertainty in Artificial Intelligence, Vol. 2, J. Lemmer, L. Kanal, ed., North Holland, New York, 1987. 6. o Heckerman, D. E., and Horvitz, E. J., based Systems," "The Myth of Modularity in Rule- in Uncertainty in Artificial Intelligence, Vol. 2, J. Lemmer, L. Kanal, ed., North Holland, New York, 1987. 7. o Heckerman, D.E., and Horvitz, E.J., "On the expressiveness of rule- based systems for reasoning under uncertainty," Proceedings of the National Conference on Artificial Intelligence, Seattle, Washington, July, 1987. 8. ' Horvitz, E. J., Heckerman, D. E., Langlotz, C. P., "A framework for comparing alternative formalisms for plausible reasoning," Proceedings of the AAAI," August, 1986, Morgan Kaufman, Los Altos, CA, 1986, 9. * Horvitz, E.J., Breese, J.S., Henrion, M., Decision Theory in Expert Systems and Artificial Intelligence, International Journal of Approximate Reasoning, Elsevier, N.Y. July, 1988. 10. * Heckerman, D.E., An Evaluation of Three Scoring Schemes, Proceedings of the 4th AAAI Workshop on Uncertainty in Artificial Intelligence, Minneapolis, MN., (to appear August 1988). Il. * Horvitz, E.J., A Multiattribute Utility Approach to Inference Understandability and Explanation, Tech. Report, KSL-28-87, Knowledge Systems Laboratory, Stanford, California, March, 1987. E. Funding Support Research Grant submitted to National Institutes of Health Grant Title: "Computer-aided Diagnosis of Malignant Lymph Node Diseases" Principal Investigator: Bharat Nathwani Funding for three years from the National Library of Medicine 1 ROl LM 04529 E. H. Shortliffe 214 5P41 -RR00785- 15 Pathfinder Project $766,053 (direct and .indirect) Professional Staff Association, Los Angeles County Hospital, $10,000. University of Southern California, Comprehensive Cancer Center, $30,000. Project Socrates, Univ. of Southern Calif., Gift from IBM of IBM PC/XT. II. INTERACTIONS WITH THE SUMEX-AIM RESOURCE A. Medical Collaborations and Program Dissemination via SUMEX Because our team of experts are in different parts of the country and the computer scientists are not located at the USC, we envision a tremendous use of SUMEX for communication, demonstration of programs, and remote modification of the knowledge base. The proposal mentioned above was developed using the communication facilities of SUMEX. B. Sharing and interaction with Other SUMEX-AIM Projects Our project depends heavily on the techniques developed by the INTERNISTICADUCEUS project. We have been in electronic contact and have met with members of the INTERNIST/CADUCEUS project, as well as been able to utilize information and experience with the INTERNIST program gathered over the years through the AIM conferences and on-line interaction. Our experience with the extensive development of the pathology knowledge base utilizing multiple experts should provide for intense and helpful discussions between our two projects. The SUMEX pilot project, RXDX, designed to assist in the diagnosis of psychiatric disorders, is currently using a version of the Pathfinder program on the DEC-20 for the development of early prototypes of future systems. C. Critique of Resource Management The SUMEX resource has provided an excellent basis for the development of a pilot project. The availability of a pre-existing facility with appropriate computer languages, communication facilities (especially the TELENET network), and document preparation facilities allowed us to make good progress in a short period of time. The management has been very useful in assisting with our needs during the start of this project. Ill. RESEARCH PLANS A. Project Goals and Plans Collection and refinement of knowledge about lymph node pathology The knowledge base of the program is about to undergo revision by the experts, and then will be extensively tested. We will be extending the knowledge base to capture important probabilistic dependencies among features. We will also begin to evaluate the knowledge base and inference techniques. Other possible extensions include: developing techniques for simplifying the acquisition and verification of knowledge from experts, and creating mapping schemes that will facilitate the understanding of the many classifications of non- Hodgkin's lymphomas. We will also attempt to represent knowledge about special 215 E. H. Shortliffe Pathfinder Project 5P41 -RR007851 5 diagnostic entities, such as multiple discordant histologies and atypical proliferations, which do not fit into the classification methods we have utilized. Representation Research We have been working to enhance the knowledge base by structuring features so that overlapping features are not incorrectly weighted in the decision making process, implementing new methods for scoring hypotheses, and creating appropriate explanation capabilities. El. Requirements for Continued SUMEX Use We are currently dependent on the SUMEX computer for file storage and archival, and for communication. We have transferred the program over to Portable Standard Lisp which is used by several users on the SUMEX system. While the switch to workstations has lessened our requirements for computer time for the development of the algorithms, we will continue to need the SUMEX facility for the interaction with each of the research locations specified in our NIH proposal. The SUMEX community version of the program is stored on the SUMEX mainframe for use by non-Stanford users. C. Requirements for Additional Computing Resources Most of our computing resources will be met by the 2060 plus the use of the Macintosh II workstations. We will need additional file space on the 2060 as we quadruple the size of our knowledge base through the construction of multiple knowledge bases. We will continue to require access to the 2060 for communication purposes, access to other programs, and for file storage and archiving. D. Recommendations for Future Community and Resource Development We encourage the continued exploration by SUMEX of the interconnection of workstations within the mainframe computer setting. We will need to be able to quickly move a program from workstation to workstation, or from workstation back and forth to the mainframe. Software tools that would help the transfer of programs from one type of workstation to another would also be quite useful. Until the type of workstations that we are using in this research becomes inexpensive, we will continue to need a machine like SUMEX to provide others with a chance to experiment with our software. E. H. Shortliffe 216 5P41 -RR00785- 15 Knowledge Engineering for Radiation Therapy IV.D.3. Knowledge Engineering for Radiation Therapy KNOWLEDGE ENGINEERING FOR RADIATION THERAPY Ira J. Kalet, Ph.D. Witold Paluszinski University of Washington Seattle, Washington I. Summary of Research Program A. Project Rationale We are developing an expert system for planning of radiation therapy for head and neck cancers. The project will ultimately combine knowledge-based planning with numerical simulation of the radiation treatments. The numerical simuiation is needed in order to determine if the proposed treatment will conform to the goals of the plan (required tumor dose, limiting dose to critical organs). The space of possible radiation treatments is numerically very large, making traditional search techniques impractical. Yet, with modern radiation therapy equipment, the design of treatment plans might be significantly aided by automatically generating plans that meet the treatment constraints. The project will result in systematization of knowledge about radiation treatment design, and will also provide an example of how to represent and solve design problems with a knowledge based system. 5. Medical Relevance and Collaborations Radiation therapy has shown dramatic improvement in the cure rate for many tumor sites in the last two decades. Much of this can be attributed to the improved penetration capability of modern megavoltage X-ray machines. These high energy beams can deliver high tumor doses without overdosing surrounding tissue in many cases. However, they are typically used in very limited ways, because of the lack of suitable simulation systems to compute the dose distribution for any but a few narrow choices of treatment geometry. In the last few years these simulation systems have been extended to the full range of geometric treatment arrangement that any therapy machine is capable of. Thus it would be valuable to be able to generalize our knowledge of treatment technique by exploring these expanded possibilities. In addition, even treatments with standard geometries can be very complex, and it is tedious to explore all of them individually. A knowledge-based system can generate a few "best" plans which satisfy the constraints and allow more time for the physician to evaluate the options, or make minor adjustments for optimization. Since cancer treatment is a multi-disciplinary approach involving surgery and chemotherapy as well as radiation, it is important to coordinate this work with knowledge-based program projects in those areas. Most significant is the ONCOCIN project, which addresses management of patients on chemotherapy protocols. This project has some relevance to computer science as well, in that our approach, if successful, may contribute to a better understanding of design problem solving with knowledge-based systems. 217 E. H. Shortliffe Knowledge Engineering for Radiation Therapy 5P41 -RR00785-15 C. Highlights of Research Progress In the past year, we have made further additions to the rule database for details of head and neck cancer treatment. In addition we have examined the issues of control strategy associated with using prototypes in planning. The major effort has been to rewrite the entire program in COMMON LISP on a VAXstation II/GPX, and to use the Portable Common LOOPS (PCL) package, which appears to be on the way to becoming a standard (Common Lisp Object System - CLOS). Following the successful rewrite, we have added code to allow the lisp program to run the dose computation program as a subprocess, and exchange messages with that subprocess. The expert system can now create a plan, run the dose computation and receive results, analyzed into a form that will facilitate plan refinement. Our expert system now has about two hundred rules, a two-level (agenda-based) control strategy, and about ten prototypes for plan construction. It is now implemented in COMMON LISP, on a VAX running the VMS operating system. This environment was chosen because it is also the environment used for a graphic simulation system that does radiation dose calculations for arbitrary treatment plans. The dose calculation is needed to determine whether a plan meets the treatment goals set by the system in its early phases of planning. D. List of Relevant Publications 1. I. Kalet and W. Paluszynski: A Production Expert System for Radiation Therapy Planning. Proceedings of the AAMSI Congress 1985, May 20-22, 1985, San Francisco, California. Edited by Allan H. Levy and Ben T. Williams. American Association for Medical Systems and Informatics, Washington, D.C., 1985. 2. W. Paluszynski and I. Kalet: Radiation Therapy Planning: A Design Oriented Expert System. WESTEX-87 (Western Conference on Expert Systems), Anaheim, California, June 2-4, 1987. 3. I. Kalet and J. Jacky: Knowledge-based Computer Simulation for Radiation Therapy Planning. Proceedings of the Ninth International Conference on the use of Computers in Radiotherapy, Scheveningen, the Netherlands, June 1987. North Holland, 1987. 4. I. Kalet and W. Paluszynski: Knowledge-based Computer Systems for Radiotherapy Planning. American Journal of Clinical Oncology. In press. II. Interactions with the SUMEX-AIM Resource Our main use of the SUMEX-AIM resource has been as a means to be in contact with other researchers working on AIM projects. The existence of a mailbox at SUMEX-AIM has made it much easier for colleagues at other institutions to communicate with us, and has been valuable in assisting us with organizing the AIM Workshop for 1987. We have had a great deal of contact with members of the ONCOCIN project and other groups. This has been valuable to us in stimulating creative approaches to our project. E. H. Shortliffe 218 5P41 -RR00785- 15 Knowledge Engineering for Radiation Therapy Ill. Research Plan A. Project Goals and Plans We plan to continue to acquire rules and develop our current expert system. This includes solving problems of use of prototypes, satisfaction of constraints by some kind of backtracking search, and incorporating evaluation of plans by using the results of the dose computation. This last idea involves coupling the expert system with the dose computation system (written in PASCAL) in suitably efficient ways. We are implementing rules for plan improvement by modification of parameters, and by adding or deleting treatment plan components (treatment fields). Our long-term goal is to shape the user interface and improve the system performance to where it can provide assistance to clinicians in treatment design for patients ir: the normal course of treatment. B. Justification and Requirements for Continued SUMEX use We foresee continued need to be in touch with other members of the AIM community, particularly projects centered at SUMEX. While we do not expect to use the computing resources of SUMEX directly, some more extensive communication and involvement is likely to be useful. C. Plans For Other Computing Resources The main computing resources for our project will continue to be local. We continue to develop the expert system code in VAX Lisp, an implementation of Common Lisp on the DEC VAXstation. We are currently using a VAXstation II/GPX. This appears to be a good choice to satisfy our need for high performance graphic simulation and a reasonable Lisp system. However, the resources for the dose computation may not be adequate as we incorporate more sophisticated computation models. As this develops, we hope to experiment with distributed systems, in which the dose computation may run on a remote resource, which may or may not be at SUMEX. D. Recommendations for Future Community and Resource Development Two areas will be of increasing importance to us in the future: communication capabilities (electronic mail and file transfer) and centralized databases. By centralized databases, we refer to the need for better maintenance of mailing lists, information about projects, and possibly on-line reports. Dr. Kalet's experience in organizing the AIM Workshop for 1987 demonstrated that electronic communication is invaluable, even in its present state, but in order to create a list to send announcements to, we expended many hours of manually cutting and pasting messages containing past lists and searching for up-to-date electronic mail addresses. If fees for use of SUMEX resources were imposed, the main impact on our project would be one of increased isolation, unless we could find grant support for the fees. 219 E. H. Shortliffe AIM Communications Users 5P41 -RR00785- 15 W.E. AIM Communications Users One of the key functions of the SUMEX-AIM resource has been inter-community communications. As the CPU-cycle-provider part of SUMEX's role phases out according to our plan for the distributed AIM community, this communications role will become increasingly important. In a recent action, the AIM Executive Committee approved broad guidelines for admitting communications users to SUMEX-AIM and we are implementing that policy. Over the past year, several of the projects that had been using SUMEX-AIM as their primary source of computing resources have obtained local resources and now use SUMEX only for communications contact with others in the AIM community. Those projects include: . Hierarchical Mode/s of Human Cognition (CL/PI?), under Professors Walter Kintsch and Peter Polson at the University of Colorado. . Problem Solving Expertise (SOLVER), under Professors Paul Johnson and William Thompson at the University of Minnesota. . RXDX, under Professor Robert Lindsay at the University of Michigan. . Computer-Based Exercises in fathophysiologic Diagnosis, under Professor J. Robert Beck at Dartmouth College. . SOAR, under Dr. Paul Rosenbloom at USC/IS1 in Los Angeles (formerly Stanford University). . Logic Group projects (DART, intelligent Agents, and MRS), under Professor Michael Genesereth at Stanford University Other projects will be undergoing similar transitions this coming year and we will report these in the next annual report. The following is a list of users and groups from the National AIM community who use the SUMEX-AIM resource primarily for communication, All of the people listed have logged on to SUMEX over the past report year or have their mail forwarded to other sites. Many have been investigators on past projects that made use of SUMEX as a CPU provider and are now doing their research computing on machines at their home institutions. No detailed reports on their research are included. American Association for Artificial Intelligence (AAAI Hdqtrs.) ALTENBERG, Lee, Ph.D. Department of Biochemistry Stanford University Medical Center AMAREL, Saul, Ph.D. Department of Computer Science Rutgers University ANZALDI, Emmanuele lnstituto di lnformatica e Sistemistica Universita' di Pavia, Italy E. H. Shortliffe 220 5P41 -RR00785- 15 AIM Communications Users < BELLAIRS> BAKER, William R., Jr., Ph.D. Knowledge Research Associates Bradenton, FI. 34203-0035 BECK, J. Robert, M.D. Dartmouth College, School of Medicine BELLAIRS, Keith University of Minnesota BRAND, Tony Chemistry Department Trenton State College, NJ CHANDRASEKARAN, B., Ph.D. Department of Computer and Information Science Ohio State University DARDEN, Lindley, Ph.D. Committee on the History and Philosophy of Science llniversity of Maryland DAVIS, Randall, Ph.D. Massachusetts Institute of Technology DAVISON, Daniel, B., Ph.D. Department of Biochemical and Biophysical Sciences University of Houston DIETTERICH, Tom, Ph.D. Department of Computer Science Oregon State University ROGERS, David Research Institute for Advanced Computer Science NASA Ames Research Center DUMAS, Jean-Pierre, Ph.D. France FELTOVICH, Paul J. Southern Illinois University School Of Medicine FREKSA, Christian lnstitut fuer lnformatik Technische Universitaet Muenchen, West Germany FRISSE, Mark, M.D. Department of Internal Medicine Washington University School of Medicine GOERZ, Gunther Friedrich-Alexander-Universitat Erlangen, West Germany 221 E. H. Shortliffe AIM Communications Users 5P41 -RR00785-15 HAMM, Greg H. European Molecular Biology Labcratory Heidelberg, W. Germany HEDRICK, Charles L., Ph.D. Center for Computer and Information Services Rutgers University HENRION, Max Carnegie-Mellon University HORSWILL, Ian D. University of Minnesota HUMPHREY, Susanne M. National Library of Medicine EXOMOLGEN Project Directory c/o Walter BODMER, Ph.D. Imperial Cancer Research Fund Laboratories London, England ARONSON, Jules National Library of Medicine KAIHARA, Shigekoto, M.D. University of Tokyo Hospital Tokyo, JAPAN DUNCAN, Karen Health Information Systems Los Altos, California KERN, Kevin B. Lab for Computer Science Research Rutgers University KINGSLAND, Lawrence C., III, Ph.D. National Library of Medicine KULIKOWSKI, Casimir, Ph.D. Department of Computer Science Rutgers University LAGADIC Pascal Clinique Kerfriden Chateaulin, France LANGRIDGE, Robert, Ph.D. Computer Graphics Laboratory University of California at San Francisco LEDERBERG, Joshua, Ph.D. The Rockefeller University E. H. Shortliffe 222 5P41 -RR007851 5 AIM Communications Users LESGOLD, Alan M., Ph.D. Learning Research & Development Center University of Pittsburgh LINDBERG, Donald, M.D. National Library of Medicine MASINTER, Larry Xerox Corporation, Palo Alto Research Center Allan Maxam, Ph.D. Harvard Medical School MAZZETTI, Claudia Executive Director, American Association for Artificial Intelligence MCDONALD, Clement J. University of Indiana Medical Center KENT, Marty Learning Research & Development Center University of Pittsburgh MOEN, James Center for Research in Human Learning University of Minnesota NOVAK, Gordon, Ph.D. Computer Science Department University of Texas at Austin PEARSON, William R., Ph.D. Department of Biochemistry University of Virginia JOHNSON, Paul E., Ph.D. Center for Research in Human Learning University of Minnesota POLSON, Peter G., Ph.D. Department of Psychology University of Colorado GREENES, Robert, M.D. Department of Radiology Harvard Brigham and Women's Hospital SIEBER, Willi, Ph.D. Sandoz Ltd., Switzerland SLAGLE, James University of Minnesota 223 E. H. Shortliffe AIM Communications Users 5P41 -RR007851 5 SOTOS, John Division of Cardiology Johns Hopkins Hospital STEFIK, Mark, Ph.D. Xerox Palo Alto Research Center SUWA, Motoi Tsukuba, JAPAN SZOLOVITS, Peter Massachusetts Institute of Technology THOMPSON, William B., Ph.D. Computer Science Department University of Minnesota VAN BEMMEL, Jan Vrije Universiteit Amsterdam, Netherlands WEBBER, Bonnie Dept. of Computer and Information Sciences University of Pennsylvania WEISS, Sholom, Ph.D. Department of Computer Science Rutgers University WIPKE, W. Todd, Ph.D. Department of Chemistry University of California at Santa Cruz ZINK, Sandra, Ph.D. Lawrence Berkeley Labs ZUKER, Michael, Ph.D. National Research Council Ottawa, Canada E. H. Shortliffe 224 5P41 -RR00785-15 Knowledge Systems Laboratory Brochure Appendix A Knowledge Systems Laboratory Brochure ARTIFICIAL INTELLIGENCE RESEARCH IN THE KNOWLEDGE SYSTEMS LABORATORY Stanford University Department of Computer Science Department of Medicine January 1988 225 E. H. Shortliffe Knowledge Systems Laboratory Brochure 5P41 -RR00785-15 Introduction The Knowledge Systems Laboratory (KSL) is an artificial intelligence (AI) research laboratory of over 100 peopltfaculty, staff, and students -within the Departments of Computer Science and Medicine at Stanford University. XSL is the new name for the interdisciplinary AI research community that has evolved over the past two decades. Begun as the DENDRAL Project in 1965 and known as the Heuristic Progr amming Project from 1972 to 1984, the new organizhtion reflects the diversity of the research now under way. The KSL is a modular laboratory, consisting of four collaborating yet distinct groups with different research themes: o The Heuristic Programming Project (HPP), Professor Edward A. Feigenbaum, scienti& director--large, multi-use knowledge bases, blackboard systems, concurrent system architec- tures for AI, automated software design, expert systems for science and engineering. Executive director: Robert Engehnore. Research scientists: Harold Brown, Scott Clearwater, Bruce De- lagi, Barbara Hayes-Roth, Hirotoshi Maegawa, H. Penny Nii, and Hirosbi Okuno. o The HELIX Group, Professor Bruce G. Buchanan, scientific director-machine lesuning, transfer of expertise, and problem solving. Research scientists: James Brinkley, Wiiam J. Clancey, Craig Cornelius, Diana Forsythe, Barbara Hayes-Roth, Rich Keller, Catherine Man- ago. o The Medical Computer Science (MCS) Group, Associate Professor Edward H. Shortme, scientific director (Department of Medicine with courtesy appointment in Computer Science)- fundamental research and advanced biomedical applications in the area of AI and decision sciences; includes the Medical Information Sciences (MIS) program. Assistant Professor: Mark A. Musen; Research scientists: Gregory F. Cooper, Lawrence M. Fagan (Associate Director). o The Symbolic Systcrns Resources Group (SSRG), Thomas C. Rindfleisch, scientific director (joint appointment Departments of Computer Science and Medicine)-research on and operation of distributed computing resources for AI research, including the SUMEX-AIM facility. Assistant director: Wiiam J. Yeager. The KSL is guided by an Executive Committee consisting of the four sublaboratory directors. Tom Rindfieisch serves as overall KSL director. This brochure s ummarizes the goals and methodology of the KSL, its research and academic programs, its achievements, and the research environment of the laboratory. Basic Research Goals and Methodology Throughout a 20-year history, the KSL and its predecessors, DENDRAL and HPP, have con- centrated on research in expert spstems- that is, systems using symbolic reasoning and problem- solving processes that are based on extensive domain-specific knowledge. The KSL's approach has been to focus on applications that are themselves significant real-world problems, in domains such as science, medicine, engineering, and education, and that also expose key, underlying AI research issues. For the KSL, AI is largely an empiricical science. Resezuch problems are explored, not by eramining strictly theoretical questions, but by designing, building, and experimenting with programs that serve to test underlying theories. The basic research issues at the core of the XSL's interdisciplinary approach center on the computer representation and use of lasge amounts of domain-specific knowledge, both factual and heuristic (or judgmental). These questions have guided our work since the 1960s and are now of central importance in all of AI research: E. H. Shortliffe 226 5P41 -RR00785 15 Knowledge Systems Laboratory Brochure 1. Knowledge representation. How can the knowledge necessary for complex problem solving be represented for its most effective use in automatic inference processes? Often, the knowledge obtained from experts is heuristic knowledge, gamed from many years of experience. How can this knowledge, with its inherent vagueness and uncertainty, be represented and applied? How can knowledge be represented so that it can be used for many problem solving purposes? 2. Knowledge acquisition. How is knowledge acquired most efficiently-whether from human experts, from observed data, from experience, or by discovery? How can a program discover inconsistency and incompleteness in its knowledge base ? How can knowledge be added without perturbing the established knowledge base unnecessarily? 3. Use of knowledge. By what inference methods can many sources of knowledge of diverse types be made to contribute jointly and efficiently toward solutions? How can knowledge be used intelligently, especially in systems with large knowledge bases, so that it is applied in an appropriate manner at the appropriate time? 4. Explanation and tutoring. How can the knowledge base and the line of reasoning used in solving a particular problem be explained to users? What constitutes a sticient or an acceptable explanation for different classes of users? 5. System tools and architectures. What kinds of software tools and system architectures can be constructed to make it easier to implement expert programs with greater complexity and higher performance? What kinds of systems can serve as vehicles for the cumulation of knowledge of the field for the researchers? HEuRlSnC PRCGRAMMING PROJECT Feigenbaum, Engalmore, Brown, Hayes-Roth, Nii L HEUX GFOJP suchmlm, comelii, Brikley, Cmcey, .%myme. Hayes-Roth, Keller Knowledge Systems Laboratory Organization 227 E. H. Shortliffe Knowledge Systems Laboratory Brochure 5P41 -RR00785-15 Current Research Projects The following list of projects now under way within the four KSL research groups gives a brief summary of the major goals of each project and lists the personnel (staff and Ph.D. candidates) directly involved. More complete information on individual projects can be obtained from the person indicated as the project contact. Inquiries should be addressed in care of: Knowledge Systems Laboratory Department of Computer Science Stanford University 701 Welch Road, Building C Palo Alto, CA 94304 415-723-3444 The Heuristic Programming Project o Advanced Architectures Project -Design a new generation of computer architectures to exploit concurrency in blackboard-based signal understanding systems. PersonneE Edward A. Feigenbaum (contact), Nelleke Aiello, Harold Brown, Bruce Delagi (DEC), Robert Engelmore, Hirotoshi Maegawa (Sony), Penny Nii, Sayuri Nishimura, Hiroshi Okuno (NTT), James Rice, Nakul Saraiya. o Blackboard Architecture Project-Integrate current knowledge about blackboard frame- work problem-solving systems and develop a domain-independent model that includes knowledge-based control processes. PersonneZ: Barbara Hayes-Roth (contact), LMicheal Hewett, Penny Nii. o Large Multi-use Knowledge Bases (LMKB)-Develop a knowledge base of scientific and engineering facts, principles and methods, along with appropriate representations of the knowledge, for multiple uses, including diagnosis and monitoring, planning, configuration, and tutoring. PersonneE Edward Feigenbaum (contact), Richard Keller, Scott Clearwater (LANL), Robert Engelmore. o Automated Software Design-Assist software designers in designing new program modules via intelligent selection and modification from ? library of existing software modules. Personnel: Penny Nii(contact), Cordell Green (Kestrel Institute). The HELIX Group o PROTEAN-Study complex symbolic constraint-satisfaction problems in the blackboard framework with application to protein structure det ermination from nuclear magnetic reso- nance data. Personnel: Bruce Buchanan (contact), Oleg Jardetzky (Stanford Magnetic Resonance Labo- ratory), Russ Altman, Jim Brinkley, Enrico Carrara, Craig Cornelius, Bruce Duncan, Guido Haymann-Haber, Olivier Lichtarge. o NEOMYCIN/GUIDONZ-Develop knowledge representation and explanation capabilities for computer-aided teaching of diagnostic reasoning. This work is moving to the Xerox Institute for Research on Learning in Spring 1988. Personnel: Bill Clancey (contact), Stephen Barnhouse, Bob London, Steve Oliphaut. E. H. Shortliffe 228 5P41 -RR00785- 15 Knowledge Systems Laboratory Brochure . Knowledge Acquisition Studies-Study the processes for transferring knowledge into a computer program, including learning by induction, analogy, watching, chunking, reading, and discovery. Personnel: Bruce Buchanan (contact), 1Marti.n Chavez, Tze-Pin Cheng, Diana Forsythe, Haym Hirsh, Richard Keller, Harold Lehmann, Eric Schoen, John Sullivan. o Financial Resources Management -Develop a constraint-based expert system for financial resource planning. Personnel: Bruce Buchanan and Tom Rindfleisch (contacts), Craig Cornelius, Andy Gehnan, Catherine Manago. o Large Multi-use Knowledge Bases (LMKB)-See description under HPP. The Medical Computer Science Group . ONCOCIN-Develop knowledge-based systems for the administration of complex medical treatment protocols such as those encountered in cancer chemotherapy. Personnel: Ted ShortlilIe (contact), Charlotte Jacobs (Oncology), Larry Fagan, David Combs, Robert Carlson, Christopher Lane, Curt Langlotz, Rick Lenon, Mark Musen, Janice Rohn, Samson Tu, Cliff Wuliinan, Andrew Zelenetz. . OPAL/PROTEGE-Develop graphics-based knowledge acquisition tools for clinical trials. OPAL developed out of the ONCOCIN project to provide a method for specifying cancer treatment experiments. The PROTEGE program is capable of creating OPAL-like knowledge acquisition tools for various areas of medicine. Personnel: Mark Musen (contact), Larry Fagan, Ted ShortlifIe, David Combs, Eric Sherman. . Speech Input to Expert Systems-Develop multi-modal interface to expert systems, con- centrating on a connected speech input device. Primary application will be extension to the ONCOCIN graphical interface. Personnel: Larry Fagan (contact), Bonnie Webber (University of Pennsylvania), Ted Shortliffe, Ed Feigenbaum (HPP), Ellen Isaacs (Psycholinguistics), Clifford Wulfman. o Physician's Workstation-Develop advanced integrated workstation suitable for providing decision support functions to clinicians in both inpatient and outpatient settings; initial work in the area of cardiovascular disease prevention, with an emphasis on the management of lipid disorders. Personnel: Ted Shortliffe (contact), John Schroeder (Cardiology), David Maron (Heart Disease Prevention Center), Jonathan King, Tom Rind&&h, Don Rucker, Joan Walton. o Blackboard/Intensive Care Unit (BBICU)--Interpret data from the intensive care unit and suggest therapy plans for patients with mechanical breathing support. Two aspects of the project are: (1) representing the structure and function of the body and (2) combining qualitative and quantitative reasoning techniques. Personnel Larry Fagan (contact for qualitative/quantitative), Barbara Hayes-Roth (HPP - contact for structure/function), Adam Seiver (Palo Alto Veterans Hospital), Lewis Sheiner (University of California, San Francisco), Ingo Beinlich, Reed Hastings, Micheal Hewett, Noi Hewett, Michael Kahn (UCSF), Nick Parlante (Palo Alto VA Hospital), John Reed, George Thomsen, Rich Washington. o Probabilistic Expert Systems- Develop pragmatic and theoretically sound methods for the acquisition and computation of probabilistic information within medical expert systems. Persunnel: Greg Cooper (contact), Ted Shortliffe, David Heckerman, Eddie Herskovits, Eric Horvitz, Jaap Suermondt. 229 E. H. Shortliffe Knowledge Systems Laboratory Brochure 5P41 -RR00785-15 The Symbolic Systems Resources Group (SSRG) o SUMEX-AIM Resource-Develop and operate a national computing resource for biomed- ical applications of arti&ial intelligence in medicine and for basic research in AI at KSL. Personnek Tom Rindfleisch (contact), Rich AcufF, Mark Crispin, Frank Gilmurray, Michael Marria, Christopher Schmidt, Andrew Sweer, Bob Tucker, Nicholas Veizades, Bill Yeager. o AI Workstation and Network Systems -Develop network-based computing environments for Lisp workstations including remote graphics and distributed computing. Personnek SSRG staff o Financial Resources Management-See description under HELIX. Students and Special Degree Programs Graduate students are an essential part of the research productivity of the KSL. Currently 36 students are working with our projects centered in Computer Science and another 21 students are working with the MCS/MIS programs in Medicine. Of the 36 working in Computer Science, 16 are working toward Ph.D. degrees, and 20 are working toward M.S. degrees. A number of these students are pursuing interdisciplinary programs and come from the Departments of Engineering, Mathematics, Education, and Medicine. Of the 21 working in Medicine, 15 are working toward Ph.D. degrees, and 6 are working toward MS. degrees. Because of the highly interdisciplinary and experimental nature of KSL research, two special degree programs have been established: Medical Information Sciences (MIS)- an interdepartmental program approved by Stan- ford University in 1982. It offers instruction and research opportunities leading to the M.S. or Ph.D. degree in medical information sciences, with an emphasis on either medical computer sci- ence or medical decision science. The program, directed by Ted ShortlifIe and co-directed by Larry Fagan, is formally administered by the School of Medicine, but the curriculum and degree require- ments are coordinated with the Dean of Graduate Studies and the Graduate Studies Committee of the University. The program reflects our local interest in the interconnections between computer science, artificial intelligence, and medical problems. Emphasis is placed on providing trainees with a broad conceptual overview of the field and with an ability to create new theoretical and practical innovations of clinical relevance. Master of Science in Computer Science: Artificial Intelligence (MS:AI)- a termi- nal professional degree offered for students who wish to develop a competence in the design of substantial knowledge-based AI applications but who do not intend to obtain a Ph.D. degree. The MS:AI program is administered by the Committee for Applied Artificial Intelligence, composed of faculty and research staff of the Computer Science Department. Normally, students spend two years in the program with their time divided equally between course work and research. In the first year, the emphasis is on acquiring fundamental concepts and tools through course work and project involvement. During the second year, students implement and document a substantial AI application project. Academic and Research Achievements The primary products of our research are scientific publications on the basic research issues that motivate our work, computer software in the form of the expert systems and AI architectures we develop, and the students we graduate who continue AI research in other academic and industrial laboratories. E. H. Shortliffe 230 5P41 -RR00785-15 Knowledge Systems Laboratory Brochure The KSL has averaged publishing more than 45 research papers per year in the AI literature, including journal articles, theses, proceedings articles, and working papers.' In addition, many talks and invited lectures are given annually. In the past few years, 11 major books have been published by KSL faculty, staff, and former students, and several more are in progress. Those recently published include: . Heuristic Reasoning about Uncertainty: An AI Approach, Cohen, Pitman, 1985. . Readings in Medical Artificial Intelligence: The First Decade, Clancey and Shortliffe, Addison- Wesley, 1984. o Rule-Based Ezpert Systems: The MYCIN Ezpetiments of the Stanford Heuriafic Programming Project, Buchanan and Shortliffe, Addison-Wesley, 1984. o The Fifth Generation: Artificial Intelligence and Japan's Computer Challenge to the World, Feigenbaum and McCorduck, Addison-Wesley, 1983. o Building Ezpert Systems, F. Hayes-Roth, Waterman, and Lenat, eds., Addison-Wesley, 1983. o System Aids in Constructing Consultation Programa: EMYCIN, van Melle, UMI Research Press, 1982. o Knowledge-Based Systems in Artificial Intelligence: AM and TEIRESIAS, Davis and Lenat, McGraw-Hill, 1982. o The Handbook of Artificial Intelligence, Volume I, Barr and Feigenbaum, eds., 1981; Volume Ii, Barr and Feigenbaum, eds., 1982; Volume III, Cohen and Feigenbaum, eds., 1982; Kaufmann. o Applications of Artificial Intelligence for Organic Chemistry: The DENDRAL Project, Lindsay, Buchanan, Feigenbaum, and Lederberg, McGraw-Hill, 1980. Our laboratory has pioneered in the development and application of AI methods to produce high-performance knowledge-based programs. Programs have been developed in such diverse fields as analytical chemistry (DENDRAL), infectious disease diagnosis and treatment (MYCIN), cancer chemotherapy management (ONCOCIN), pulmonary function evaluation (PUFF), VLSI design (KBVLSI/PALLADIO), molecular biology (MOLGEN) , and parallel machine architecture simu- lation (CARE). Some of our systems and tools (e.g., UNITS, EMYCIN, and AGE) are now also being adapted for commercial development and use in the AI industry. Following our Iead in work on biomedical applications of AI and the development of the SUMEX-AIM computing resource, a nationally recognized community of academic projects on AI in medicine has grown up. Central to all KSL research are our faculty, staff, and students. These people have been recog- nized internationally for the quality of their work and for their continuing contributions to the field. KSL members participate extensively in professional organizations, government advisory commit- tees, and journal editorial boards. They have held managerial posts and conference chairmanships in both the berican Association for Artificial Intelligence (AAAI) and the International Joint Conference on Artificial Intelligence (IJCAI). ' Copies of individual KSL publications may be obtained through the Stanford Department of Computer Science publications office. The full collection of KSL reports is being published in microfiche by COMTEX Scientific Corporation. 231 E. H. Shortliffe Knowledge Systems Laboratory Brochure 5P41 -RR00785-15 Several KSL faculty and former students have received significant honors. In 1976, Ted Short- LifIe received the Association of Computing Machinery Grace Murray Hopper award. In 1977, Doug Lenat was given the IJCAI Computers and Thought award, and in 1978, Ed Feigenbaum received the National Computer Conference Most Outstanding Technical Contribution award. In 1979 and 1981, Ted ShortlifIe's book Computer-Based Medical Consultation: MYCIN was identified as the most frequently cited work in the IJCAI proceedings. In 1982, Doug Lenat won the Tioga prize for the best AAAI conference paper while Mike Genesereth received honorable mention. In 1983, Ted Shortliffe was named a Kaiser Foundation faculty scholar, and Tom Mitchell received the IJCAI Computers and Thought award. In 1984, Ed Feigenbaum was elected a fellow of the American As- sociation for the Advancement of Science (AAAS), and he and Ted Shortme were elected fellows of the American College of Medical Informatics. In 1986, Ed Feigenbaum was elected to the National Academy of Engineering and in 1987, Ted Shortme was elected to the Institute of Medicine of the National Academy of Sciences. KSL Research Environment Funding-The KSL is supported solely by sponsored research and gift funds. We have had funding from many sources, including DARPA, NIH/NLM, ONR, NSF, NASA, and private foun- dations and industry. Of these, DARPA and NIH have been the most substantial and long-standing sources of support. All, however, have made complementary contributions to establishing an effec- tive overall research environment that fosters interchanges at the intellectual and software levels and that provides the necessary physical computing resources for our work. Computing Resources-Under the Symbolic Systems Resources Group, the KSL develops and operates its own computing resources tailored to the needs of its individual research projects. Current computing resources are a networked mixture of mainframe host computers, Lisp work- stations, and network utility servers, reflecting the evolving hardware technology available for AI research. Our mainframe host is currently a DEC 2060 running TOPS-20 (this is the core of the national SUMEX biomedical computing resource). Its network service functions will be replaced shortly by a SUN-4 system running UNIX. Its routine computing functions (electronic mail, text processing, and information retrieval) will be replaced by distributed user workstations. Our Lisp workstations include 35 Xerox LlOO-series machines, 20 Texas Instruments Explorers, 6 Symbolics 3600-series machines, 3 SUN 31'75 workstations, and 5 Hewlett-Packard 9836 machines. We are in the process of acquiring a significant number of Apple Macintosh II workstations for routine computing support and many of these will also be configured to run Lisp programs. Network print- ing, file, gateway, and terminal interface services are provided by dedicated machines including 2 VAX 11/750's, a SUN 3/180;and numerous dedicated microprocessor systems. These facilities are integrated with other computer science resources at Stanford through an extensive Ethernet and to external resources through the ARPANET and TELENFT. Funding for these resources comes principally from DARPA and NIH and hardware vendor gifts. E. H. Shortliffe 232 5P41 -RR007851 5 Knowledge Systems Laboratory Brochure Margaret Jacks Hall. Electrical Engrneering Pine Hall Score 2060 Xerox 1100's Ether np Xerox laser printers Other CSD Equipment Medical Center SUMEX Machine Room SUMW 2060 SUMEX 2020 Xerox 1108 Xerox 6037 fife server Vax 750 tile server NIT Elis's Xerox 1185 lmagen laser printer I Campus "link net" cl R Repeater El G Gateway Medical School Office Building Symbolic Systems Resources Group Medical Computer Science Group I Xerox 1100's H-P 9636's n Explorers lmagen laser printers Xerox laser printer Suns Sun file server Apple Macintoshes Apple laser printer Ether TIPS ct+ G Fl is G Whelan Building (Welch Road) HPP and HELIX G e Silicon Graphics Iris Sun Vax 750 file server Xerox 8033 file serfer lmagen laser printers Apple Macintoshes Apple laser printer Ether TIPS . Xerox Alto Xerox 1132 SUMEX-AIM System and Local Area Network 233 E. H. Shortliffe 5P41 -RR00785 15 Appendix 6 Lisp Performance Studies Lisp Performance Studies Performance of Two Common Lisp Programs on Various Workstation Systems by Richard Acuff Knowledge Systems Laboratory Stanford University *** DRAFT *** 1 - Introduction In order to assist us in understanding performance of Lisp systems, we have undertaken an informal survey of Common Lisp environments using two KSL software packages. The data collection is close to complete but there has been very little data analysis. Thus the data is included here with very little in the way of observations or conclusions. In this survey we have focused on execution speed which has long been a differentiator among computer systems. The first comparison of two systems solving the same problem (benchmarking) was probably done shortly after the creation of the second computer, and benchmarking has been a primary differentiator among computers systems ever since. However, execution speed benchmarks are only one aspect of the systems, especially Lisp systems. Issues like programming and usage environments, compatibility with other systems, ability to handle "large" problems, and cost must also be considered. The test software we used was SOAR and the BBI blackboard core. Both systems were chosen primarily because they are implemented in pure Common Lisp, making them extremely portable. Both are systems in daily use in the KSL and represent two distinct research directions. SOAR is a heuristic search based general problem solving architecture developed by Paul Rosenbloom and BBl is a blackboard problem solving architecture developed by Barbara Hayes-Roth. Neither of these systems is an intensive user of numeric computation. These systems were initially developed in environments other than those tested and no attempt was made to optimize their performance for any of these tests. All runs of SOAR were done solving an eight-puzzle problem in one of three modes: 1. Mode "1,3" just solves the problem. 2. Mode "1,l" solves the problem while learning how to better solve it (this mode takes the most time). 3. Mode "3,3" solves the problem after learning (this mode takes the least time). SOAR's source code consisted of a single 280k character file, plus two small files containing the "rules" for the eight-puzzle problem: DEFAULT.SOAR at 24k characters and EIGHT.SOAR at 10k characters. The runs and compilation were done in separate instantiations of the Lisp environment. 237 E. H. Shortliffe Lisp Performance Studies 5P41 -RR007851 5 All runs of BBl went through three cycles of adding IO items to the blackboard, accessing those IO items, and then deleteing them. All references to BBI in this document refer only to the "core" blackboard parts of the system and does not include any other layers of the problem solving architecture, or the user interface. The BBl source code used for the testing is spread over 10 files containing 295k characters. Compilation, loading, and execution were all done in a single instantiation of the Lisp environment. 2 - Systems Under Test The systems to be tested were chosen based on their availability to the testers as well as suspected potential usefulness in future programming efforts. Since we were interested in "real world" results, we ran the tests on each machine in what seemed to be its standard operating mode. In particular, if there are typically "background" activities going on during normal operation, then those were allowed to continue during the taking of these measurements. If code is typically executed from within an editor or other special context, then that was done. No special process priority altering or other attempt to optimize the execution was made unless noted in the description of the systems. It is worth noting that on almost all of the systems tested, virtual memory paging was a neglibible part of the overall run time. Nor was it a significant factor during compilation. In the following descriptions "Code" refers to a short name used to indicate the systems under test. Usually it is the model of the machine except where there is more than one Lisp for a machine (as in the case of the Sun 3175) in which case a letter is prefixed to indicate the Lisp being used. "Timing Template" indicates how the information reported by the TIME function was recorded. "Elapsed" indicates the total elapsed time, "run" indicates CPU time used, "gc" indicates time spent in garbage collection, "user" and "system" distinguish between user mode and kernel mode time, and "paging" indicates time waiting for virtual memory disk operations. Though all of this information was recorded we have not reproduced it in this document. Code: 31260 Computer Type: Sun 31260 Operating System: Sun OS 3.4 Lisp: Lucid 2.0 Disk Configuration: 280MB Swapping Size: 60MB Memory Configuration: 8MB Display Configuration: Color in mono mode Other Configuration: Special Comments: used :EXPAND 130 :GROWTH-RATE 130 Timing Template: elapsed (user-run + system-run) Code: 3160 Computer Type: Sun 3/60 Operating System: Sun OS 3.4 Lisp: Lucid 2.1 Disk Configuration: SCSI 141 MB Swapping Size: unknown Memory Configuration: 24MB Display Configuration: Hi Res Color in mono mode Other Configuration: Special Comments: E. H. Shortliffe 238 5P41 -RR00785- 15 Appendix B Lisp Performance Studies Lisp Performance Studies Performance of Two Common Lisp Programs on Various Workstation Systems by Richard Acuff Knowledge Systems Laboratory Stanford University **I DRAFT *** 1 - Introduction In order to assist us in understanding performance of Lisp systems, we have undertaken an informal survey of Common Lisp environments using two KSL software packages. The data collection is close to complete but there has been very little data analysis. Thus the data is included here with very little in the way of observations or conclusions. In this survey we have focused on execution speed which has long been a differentiator among computer systems. The first comparison of two systems solving the same problem (benchmarking) was probably done shortly after the creation of the second computer, and benchmarking has been a primary differentiator among computers systems ever since. However, execution speed benchmarks are only one aspect of the systems, especially Lisp systems. Issues like programming and usage environments, compatibility with other systems, ability to handle "large" problems, and cost must also be considered. The test software we used was SOAR and the BBI blackboard core. Both systems were chosen primarily because they are implemented in pure Common Lisp, making them extremely portable. Both are systems in daily use in the KSL and represent two distinct research directions. SOAR is a heuristic search based general problem solving architecture developed by Paul Rosenbloom and BBl is a blackboard problem solving architecture developed by Barbara Hayes-Roth. Neither of these systems is an intensive user of numeric computation. These systems were initially developed in environments other than those tested and no attempt was made to optimize their performance for any of these tests. All runs of SOAR were done solving an eight-puzzle problem in one of three modes: 1. Mode "1,3" just solves the problem. 2. Mode "1,l" solves the problem while learning how to better solve it (this mode takes the most time). 3. Mode "3,3" solves the problem after learning (this mode takes the least time). SOAR's source code consisted of a single 280k character file, plus two small files containing the "rules" for the eight-puzzle problem: DEFAULTSOAR at 24k characters and EIGHT.SOAR at 10k characters. The runs and compilation were done in separate instantiations of the Lisp environment. 237 E. H. Shortliffe Lisp Performance Studies 5P41 -RR00785- 15 All runs of BBl went through three cycles of adding 10 items to the blackboard, accessing those 10 items, and then deleteing them. All references to BBl in this document refer only to the "core" blackboard parts of the system and does not include any other layers of the problem solving architecture, or the user interface. The BBl source code used for the testing is spread over 10 files containing 295k characters. Compilation, loading, and execution were all done in a single instantiation of the Lisp environment. 2 - Systems Under Test The systems to be tested were chosen based on their availability to the testers as well as suspected potential usefulness in future programming efforts. Since we were interested in "real world" results, we ran the tests on each machine in what seemed to be its standard operating mode. In particular, if there are typically "background" activities going on during normal operation, then those were allowed to continue during the taking of these measurements. If code is typically executed from within an editor or other special context, then that was done. No special process priority altering or other attempt to optimize the execution was made unless noted in the description of the systems. It is worth noting that on almost all of the systems tested, virtual memory paging was a neglibible part of the overall run time. Nor was it a significant factor during compilation. In the following descriptions "Code" refers to a short name used to indicate the systems under test. Usually it is the model of the machine except where there is more than one Lisp for a machine (as in the case of the Sun 3/75) in which case a letter is prefixed to indicate the Lisp being used. "Timing Template" indicates how the information reported by the TIME function was recorded. "Elapsed" indicates the total elapsed time, "run" indicates CPU time used, "gc" indicates time spent in garbage collection, "user" and "system" distinguish between user mode and kernel mode time, and "paging" indicates time waiting for virtual memory disk operations. Though all of this information was recorded we have not reproduced it in this document. Code: 3/260 Computer Type: Sun 31260 Operating System: Sun OS 3.4 Lisp: Lucid 2.0 Disk Configuration: 280MB Swapping Size: 60MB Memory Configuration: 8MB Display Configuration: Color in mono mode Other Configuration: Special Comments: used :EXPAND 130 :GROWTH-RATE 130 Timing Template: elapsed (user-run + system-run) Code: 3/60 Computer Type: Sun 3160 Operating System: Sun OS 3.4 Lisp: Lucid 2.1 Disk Configuration: SCSI 141 MB Swapping Size: unknown Memory Configuration: 24MB Display Configuration: Hi Res Color in mono mode Other Configuration: Special Comments: E. H. Shortliffe 238 5P41 -RR00785-15 Lisp Performance Studies Timing Template: elapsed (user-run + system-run) Code: 386 Computer Type: Compaq 386 (20Mhz 386) Operating System: 386/1X 5.3 rev level 1.01 (Unix) Lisp: Lucid 2.0 Disk Configuration: 134MB ESDI Swapping Size: unknown Memory Configuration: IOMB; 32kB 20ns cache Display Configuration: terminal Other Configuration: none Special Comments: none Timing Template: elapsed (run) Code: 386T Computer Type: Compaq 386 portable (Toaster) Operating System: 386/1X 5.3 rev level 1.01 (Unix) Lisp: Lucid 2.0 Disk Configuration: 40MB Swapping Size: unknown Memory Configuration: IOMB; no cache Display Configuration: tiny LCD Other Configuration: tiny display Special Comments: portable versio of "386" above Timing Template: elapsed (run) Code: 41260 Computer Type: Sun 41280 Operating System: SunOS 3.2 Gamma Lisp: Lucid 2.1 Disk Configuration: unknown Swapping Size: unknown Memory Configuration: 32MB Display Configuration: Hi Res color in mono Other Configuration: Special Comments: used :EXPAND 130 :GROWTH-RATE 130 Timing Template: elapsed (user-run + system-run) Code: 41280 Computer Type: Sun 41280 Operating System: SunOS 3.2 Gamma Lisp: Lucid 2.1 beta Disk Configuration: 417 (Eagle) Swapping Size: 60MB Memory Configuration: 8MB Display Configuration: Hi Res mono Other Configuration: Special Comments: Timing Template: elapsed (user-run + system-run) Code: DEC-II Computer Type: DEC MicroVax WGPX Operating System: VMS Lisp: VaxLisp Disk Configuration: 2 x 159MB Swapping Size: 3k pg page, 8k pg swap 239 E. H. Shortliffe Lisp Performance Studies 5P41 -RR00785-15 Memory Configuration: 16MB Display Configuration: GPX Other Configuration: Special Comments: Timing Template: elapsed - gc-elapsed (run - gc-run) Code: DEC-III Computer Type: DEC MicroVax III (3500) Operating System: VMS Lisp: VaxLisp Disk Configuration: (RD53) Swapping Size: unkown Memory Configuration: 16MB Display Configuration: Other Configuration: Special Comments: Timing Template: elapsed - gc-elapsed (run - gc-run) Code: E-3/75 Computer Type: Sun 3175 Operating System: SunOS 3.1 Lisp: Franz Extended Common Lisp 2.0 Disk Configuration: 70MB SCSI Swapping Size: 50MB local Memory Configuration: 28MB Display Confiyuration: standard resolution mono Other Configuration: Files on Sun 3/180 NFS server Special Comments: Under suntools Timing Template: elapsed (run + gc) Code: EXPl Computer Type: Texas Instruments Explorer I Operating System: Explorer Lisp Release 3.0+ Lisp: Explorer Lisp Release 3.0+ Disk Configuration: 2 x 140MB SCSI Swapping Size: 80MB Memory Configuration: 8MB Display Configuration: 1024 x 768 mono Other Configuration: Special Comments: TGC (incremental generation scavenging GC) on unless otherwise noted Timing Template: elapsed - paging Code: EXP2 Computer Type: Texas Instruments Explorer II Operating System: Explorer Lisp Release 3.0+ Lisp: Explorer Lisp Release 3.0+ Disk Configuration: 2 x 140MB SCSI Swapping Size: 80MB Memory Configuration: 16MB Display Configuration: 1024 x 768 mono Other Configuration: Special Comments: TGC (incremental generation scavenging GC) on unless otherwise noted Timing Template: elapsed - paging E. H. Shortliffe 240 5P41 -RR00785-15 Lisp Performance Studies Code: HP Computer Type: Hewlett .Packard 9000/350 Operating System: Unix Lisp: HP Lisp 1.0 Disk Configuration: 130MB (7958) Swapping Size: unknown Memory Configuration: 16MB Display Configuration: color Other Configuration: under gnuemacs Special Comments: Timing Template: elapsed - run Code: K-3/75 Computer Type: Sun 3/75 Operating System: SunOS 3.1 Lisp: Kyoto Common Lisp "September 16, 1986" Disk Configuration: 70MB SCSI Swapping Size: 50MB local Memory Configuration: 28MB Display Configuration: standard resolution mono Other Configuration: Files on Sun 31180 NFS server Special Comments: Under suntools Timing Template: elapsed - run Code: L-3175 Computer Type: Sun 3/75 Operating System: SunOS 3.1 Lisp: Lucid 2.0 Disk Configuration: 70MB SCSI Swapping Size: 50MB local Memory Configuration: 28MB Display Configuration: standard resolution mono Other Configuration: Files on Sun 3/180 NFS server Special Comments: used :EXPAND 90 :GROWTH-RATE 90 Timing Template: elapsed (user-run + system-run) Code: mX Computer Type: Texas Instruments microExplorer Operating System: Explorer Lisp 4.0 beta Lisp: Explorer Lisp 4.0 beta Disk Configuration: 1 OOMB Rodime Swapping Size: 60MB Memory Configuration: 12MB mX processor; 2MB Mac II Display Configuration: 19" (1024 x 768) Moniterm Viking Other Configuration: Apple EtherTalk Special Comments: Timing Template: elapsed - paging Code: RT Computer Type: IBM RT/APC Operating System: AIX 2.1.2 (Unix) Lisp: 2.0.5 (Lucid 1.01) Disk Configuration: "Fast" EESDI controller; 3 x 70MB Swapping Size: 80k x 512kB blocks (40,960MB) Memory Configuration: 16MB of "fast" memory Display Configuration: Moniterm 1024 x 768 mono 241 E. H. Shortliffe Lisp Performance Studies 5P41 -RR00785-15 Other Configuration: AFT floating point unit; GSL windows Special Comments:. Used :EXPAND 69 to get 6MB semispace; This should be the fastest RT version now available Timing Template: elapsed (user-run + system-run) Code: Sym Computer Type: Symbolics 3645 Operating System: Symbolics Release 6.1 Lisp: Symbolics Release 6.1 Disk Configuration: 368MB Swapping Size: 200MB Memory Configuration: Display Configuration: 8MB Other Configuration: FPA, no color Special Comments: EGC on Timing Template: elapsed - paging Code: XCL Computer Type: Xerox 1186 Operating System: Xerox Lisp, Lyric release Lisp: Xerox Lisp, Lyric release Disk Configuration: 40MB Swapping Size: 16MB Memory Configuration: 3.5MB Display Configuration: 19" mono Other Configuration: Special Comments: Timing Template: elapsed - gc - paging 3 - Compilation and Execution For both BBI and SOAR the time taken to compile the system and make a standard run was measured. Here are those results (all numbers are seconds): BBl SOAR Code Compile Run Compile Run 31260 540 ;5 687 171 3/60 551 569 218 386 355 386 142 386T 416 z: 479 175 41260 324 5: 307 70 41280 482 523 105 DEC-II 1774 207 1227 1908 DEC-III 633 63 423 476 E-3175 444 211 450 500 Expl 327 87 520 400 Exp2 96 29 162 146 HP 235 115 237 229 L-3175 919 1365 756 K-3/75 1234 :: 1040 297 Mac11 349 254 Not available1 mX 242 31 242 219 RT 586 75 574 206 Sym 257 111 252 210 XCL 1927 559 1800 1613 `We were not able to get SOAR to run properly in Allegro Common Lisp 1.0 or 1 .l. E. H. Shortliffe 242 5P41 -RR00785-15 Lisp Performance Studies For convenience of viewing, these numbers are graphed in Figures 20 and 21. The system types are sot-fed into order of best run time for SOAR and BBl separately to facilitate comparative observation. 4 - Effect of Compiler Optimize Settings on BBl We were also interested in the effect of two of Common Lisp's compiler optimizer settings, SPEED and SAFETY. These switches have four settings, 0 through 3, with 0 being the highest priority. Thus settings of SAFETY 0 and SPEED 3 should allow the compiler to produce the fastest code, while SAFETY 3 and SPEED 0 would result in conservative code, perhaps with more type checking, etc. We comipiled and ran BBI with 4 settings of these switches: 1. The system default 3. (PROCLAIM `(OPTIMIZE (SAFETY 0) (SPEED 3))) 3. (PROCLAIM `(OPTIMIZE (SAFETY 3) (SPEED 0))) 4. (PROCLAIM `(OPTIMIZE (SAFETY 2) (SPEED 3))) Figures 22 and 23 show the effect of these settings on the compilation and run times of the BBl test. Here are the numbers: Default Safe 0, Spd 3 Safe 3, SDd 0 Safe 2. Spd 3 Code Comp 3/260 540 3160 551 386 355 386T 416 41260 324 41280 482 DEC-II 1774 DEC-III 633 E-3/75 444 Expl 327 Exp2 96 HP 235 K-3175 1234 L-3/75 919 Mac11 349 mX 242 RT 586 Sym 257 XCL 1927 Run 62 73 ii z: 207 63 211 87 29 115 96 90 254 %! 111 559 Comp 524 537 332 408 318 483 1987 635 403 318 117 250 1815 1056 365 249 587 281 2022 Run 62 72 47 54 46 34 206 z5 E 113 165 90 258 28 76 109 543 Comp 532 444 271 341 229 385 2245 732 469 314 111 235 1379 1054 354 232 508 256 2230 Rbn 69 %YP 76 540 El 339 410 :; 309 492 231 3094 ::6 990 443 90 357 28 121 141 247 147 1171 127 910 261 363 32 239 77 595 110 262 559 2020 Run 62 72 ti 3: 236 70 206 83 26 118 iii 259 35 75 111 556 5 - Effect of Output Reduction on SOAR We had previously noted that some systems were able to run the eight-puzzle benchmark much faster when the volumous typeout produced was reduced. Figure 24 shows the difference between run times of the 1,3 mode solution of the eight puzzle with full tracing versus no tracing. The numerical data follows: 243 E. H. Shortliffe Lisp Performance Studies 5P41 -RR00785-15 Code Normal Reduced 3/260 49 3160 ii 386 :: 386T 52 :; 4/260 23 41280 35 :: DEC-II 351 283 DEC-III 95 76 E-3/75 124 109 Expl 90 63 ExpP HP i': :: K-3/75 186 136 L-3/75 82 67 mX RT 2 ii Sym 55 40 XCL 473 390 6- Future Work Now that this data is collected it is our intention to write it up in a technical report, including comoarisons with the Gabriel Benchmarks and remarks on certain surprising facts that this work has turned up. This report will be made widely available to the AIM community to assist future descisions in the use of Lisp systems. E. H. Shortliffe 244 5P41 -RR00785-15 Lisp Performance Studies Exp2 mX 41280 386 3861 4/260 o 31260 : DEC-III : 3160 % RT `; Expl 4 L-3/75 $ K-3/75 Sym I-P DEC-II E-3175 Macll xa 0 50 100 150 200 250 300 350 400 450 500 550 BBl Run Tlms (see) 41260 41280 386 * Mode 1,3 Exp2 3Q60 386T d syI g 3/60 g mX 5 I+ $ L-3l75 Expl DEC-III E-3175 K-3n5 xa DEC-II 0 200 400 600 800 1000 1200 1400 1600 1800 2000 BOAR Total Run Time (sac) Figure 20: Run Times for BBl and SOAR 245 E. H. Shortliffe Lisp Performance Studies 5P41 -RR00785-15 ExpP mX 41280 386 386T 4l260 * 3l260 3 " DEC-III `E Expl a L-3/75 m" K-3175 Sym I-P DEC-II E-3I75 Mad xa BBl Complle Time (sac) 41260 4i280 386 Exp2 31260 386T :: FIT s g Sym 3160 ' mX 5 I-P g L-3ff5 Expl DEC-III E-3175 K-3175 xa DEC-II 0 200 400 600 800 1000 1200 1400 16bO lab0 2000 BOAR Compile Time (set) E. H. Shortliffe Figure 21: Compilation Times for BBl and SOAR 246 5P41 -RR007851 5 Lisp Performance Studies Exp2 mX 306 386T 4/260 31260 8 DEC-III 0" B 3160 I-" E RT 2 P v, Expt L-3/75 K-3175 DEC-II E-3175 Macll xa * Default Run O Safe 0, Spd 3 Safe 3, Spd 0 El Safe 2, Spd 3 0 50 100 150 200 250 so0 3iu 400 450 500 550 961 Run Tlmas (90~) Figure 22: Run Times for BBl under Various Compiler Settings 247 E. H. Shortliffe Lisp Performance Studies 5P41 -RR00785-15 mX 366 366T 41260 31260 DEC-III 4 * Default ,Compile O Safe 0, Spd 3 Safe 3, Spd 0 O Safe 2, Spd 3 Sym DEC-II 1000 1500 2000 2500 3000 BBl Compllrtlon Time (see) Figure 23: Compilation Times for BBl under Various Compiler Settings E. H. Shortliffe 248 5P41 -RR00785-15 Lisp Performance Studies 366 Exp2 31260 366T p" Sym 0" g 3160 F I 5 mX s I-P L-3ff5 4/260 * Mode 1,3 41200 a Mode 1,3 nt Expl DEC-III K-3/75 xa DEC-II I SOAR Run Time (see) Figure 24: Differences Between Normal and Reduced Ouput SOAR Run 249 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 Appendix C Lisp Environment Specification KSL Lisp Environment Requirements *** D R A F T *** Richard Acuff Knowledge Systems Laboratory Computer Science Department, Stanford University May 11, 1988 25 1 E. H. Shortliffe 5P41-RR00785-15 Draft Lisp Environment Spec 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .._._..............._.................................. 1 2. Program Development Tools & Environment ...................................................... .3 2.1. Editor ..................................................................................................... 3 2.2. Debugger.. .............................................................................................. 4 2.3. Inspector ................................................................................................ 5 2.4. Software Management Tools.. ................................................................. 6 2.5. Performance Monitoring and Analysis .................................................. 6 2.6. Lisp Listener ......................................................................................... 7 3. Languages and Utilities ......................................................................................... 7 3.1. Windowing ............................................................................................. 7 3.2. Multiple Processes.. .............................................................................. 1 0 3.3. Common Lisp.. ....................................................................................... .l 1 3.4. Compilation.. .......................................................................................... 1 2 3.5. Input/Output ......................................................................................... .l 3 3.6. Utilities ................................................................................................. 13 3.7. Interface Toolkit .................................................................................... 14 3.8. Help System .......................................................................................... .l 4 3.9. Status Information.. ............................................................................... 1 4 3.10. Printing ............................................................................................... 15 3.11. Pretty Printing ................................................................................... 15 4. Lower Level Issues ............................................................................................... .l 5 4.1. Address Space.. ...................................................................................... -1 5 4.2. Memory Management ............................................................................. 16 4.3. Dedicated Versus Shared Systems.. ........................................................ 1 7 4.4. Hardware Capabilities.. ......................................................................... 1 7 4.5. Overall System Integration in the KSL.. ................................................ 1 7 5. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .._................................................... 18 E. H. Shortliffe 252 5P41-RR00785-15 Draft Lisp Environment Spec 1. Introduction The Knowledge Systems Laboratory (KSL) is an artificial intelligence research laboratory at Stanford University that has been developing systems and tools in Lisp environments for more than 20 years. These systems have been implemented in the Interlisp, MACLisp, ZetaLisp, and more recently, CommonLisp dialects predominantly. Beginning in the early 1980's, our work moved from mainframe Lisp environments to workstation environments for many reasons, principally involving powerful tools for system development and debugging and graphical interfaces. Commercial versions of these tools, that evolved over many years in the Xerox D-Machine, Symbolics 36xx, LMI, and TI Explorer systems, have become an indispensible part of our work environment. Newer Lisp systems for workstations not specifically developed for Lisp have lacked many important features of these environments. This document attempts to summarize the key features of the Lisp machine environments that would be needed in "stock" machine implementations in order to make them attractive in a development setting. There are several overall points to be emphasized about this write-up: 1 ) These requirements represent a snapshot of the tools and technology available on today's machines. Al has historically and will continue to ride the crest of the wave of new computing technologies for the forseeable future, which enable ever more complex systems. Thus, these are not static requirements and we expect to be able to take advantage of the future improvements in hardware, graphics, and software as they are generated by computer science research and industry. 2) It has been hard to describe concisely many aspects of the Lisp environments because they involve visual interactions and the "feel" of the way systems are organized and interconnected. The write-up assumes a general familiarity and experience with the Lisp environments Xerox, Symbolics, TI Explorer systems. 3 ) We have tried to sort out the key features of current systems that are important to our research work. Except where explicitly stated, everything in this document describes this "core" of functionality. Some items are clearly more important than others, but all represent needs that really guide our decisions about which new systems can be broadly used in the KSL. 4 ) The discussion is organized according to a "layered" view of Lisp environments shown in Figure 1, beginning with the upper levels. This organization is a conceptual framework within which to describe the various parts of the environment but may not correspond in full detail to the way system modules are actually organized. While most of the discussion focuses on the higher layers in this diagram, unavoidably some issues involving lower level or more general issues such as address space, system speed, and graphics facilities have to be mentioned. 5 ) While this description is based on what we know, use, and understand today, we have attempted to allow for innovation by describing the functionality that we require in a fairly abstract way wherever possible, rather than specifying, for example, the "Symbolics XYZ feature". This may result in 253 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 some ambiguities that will need to be addressed by appropriate discussion and iteration. 6 ) We have two overriding goals in adopting future computing environments (which may seem to be or actually are in some conflict). We want the most powerful development environments we can get to facilitate the building of complex Al programs. But at the same time, we want to be able to share (import and export) research results and tools with colleagues in other labs and so must maximize the portability of code among systems. We believe that these goals can be approached jointly by the establishment and careful adherence to standards where possible, while continuing systems development where necessary. 7 ) Because of the evolving nature of these research environments, no vendor's system can ever be "finished". While we expect reasonably professional standards of robustness and reliability in the systems we use, we also expect to have special needs and to work closely with the vendors of products we use to adapt, extend, and debug the environment and tools. Our experience has been that in order to do this effectively, it is essential that we have broad access to system source codes. Program Development Tools & Environment Languages and Utilities Lower Level Issues i l&lDebugger I Performance I LisD I~;~gnement(lnspectorI Tools / Listener Windows Help and UI Networks Utilities Substrate x.1 1 1 1 aos 1 Common Lisp 1 Conditions1 Processes 1 I I I Host Machine hardware and software I Figure I 2. Program Development Tools & Environment The quality of the development tools and environment is what has been the primary strength of Lisp machines, allowing rapid design, implementation, and debugging of complex programs. We believe the key to good development tools is integration, both in E. H. Shottliffe 254 5P41-RR00785-15 Draft Lisp Environment Spec terms of consistency of interface, and in the ability to move seamlessly from tool to tool, carrying along appropriate data and state information. These qualities must be manifest in any KSL research`computing system. 2.1. Editor The hands of the development environment is the editor. There has been a great deal of experimentation with various styles of editing, most significantly text-based, as in Zmacs on the TI Explorer and Symbolics machines, versus structure-based, as in Xerox Lisp. We are inclined to believe that an editor rooted in a text-based approach but having understanding of the structural content of code being edited is the best approach as it allows full base-level generality for dealing with all kinds of text but. if well implemented, can be specialized for various types of editing. Given this, we feel the editor in the Lisp systems should have the following features: . since it is common to build tools that utilize editing it is important to have a complete programatic interface to the editor . able to use Emacs-like commands for hands-on-keyboard control as in Zmacs . also uses pointer for moving editing focus, selection, some command selection, etc. o fully extensible in Lisp . uses the pretty printer described above to allow code reformatting (eg. for narrower/wider windows) . minimally includes some source libraries to be used as examples . supports keyboard macros . integrated with Lisp such that edit definition, incremental compilation, documentation string viewing, argument list viewing, macro expansion, evaluation, etc. are available easily from the editing environment o knows lisp syntax such that users can manipulate Lisp expressions (eg. move forward on s-expression, select an s-expression, etc.) o has a complete edit definition facility such that the source of a DEFSTRUCT, DEFUN, DEFCMSS, or other definition of symbol will be automatically loaded if available without the need for explicit cross referencing . allows user-defined modes for non-Lisp . if text can be selected and operated on the selected text should be highlighted as in region marking in Zmacs on the Tl Explorer . if the matching grouping character is visible when the editing focus is on a grouping character, it should be indicated; a "grouping character" is a parentheses, curly brace, square brace, angle bracket, double quote, or other user-specified character; a "match" is found when a symmetric character is found in a syntactically legal place (ie. not in a different context than the focus character, such as in a comment or literal string when the focus 255 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 character is in code), thus requiring that the editor have "understanding" of the syntax being used . allows multi-fonting and font shift stripping (eg., writes #2\a to files) . has the ability to automatically place code into different fonts depending on context . if the editor allows code with unbalanced parentheses to be entered, it should be possible to check the code for unbalanced parentheses, and such a check should be done when the code is saved in a file (eg. x-X Find Unbalanced Parentheses in Tl and Symbolic9 Zmacs) . can be instantiated multiple times (multi-window and multi-process) . completion of commands and file names o hooks for buffer switching, buffer creation, mode changes o per buffer/window editory control variable bindings . ZetaLisp style attribute lists or some other mechanism for telling the editor what packages, fonts, base, etc. are used with data 2.2. Debugger The interactive debugger is also a critical part of the Lisp environment. In many ways, it can be viewed as an extension of the inspector. It should have the following features: . "Terminal" based and window based versions for times when the window system fails . ability to force entry to debugger from keyboard, or abort execution of running program . ability to see and modify arguments and locals of active stack frames and closures, as well as evaluating expressions in the lexical context of stack frames running compiled code o ability to peruse the stack easily and quickly . ability to return from or restart an arbitrary stack frame . able to have multiple concurrent instantiations . must be robust in the face of errors that occur during the operation of the debugger (eg. can't print some datum) . arglists and docstrings must be around and accessible . fast startup E. H. Shortliffe 256 5P41-RR00785-15 Draft Lisp Environment Spec One of the most difficult aspects of debugging is understanding where in a program the error is occurring. To. assist with this, we require a good disassembler and some level of source code debugging. The disassembler must be able to give information about operations being performed including names of variables, indication of arguments being set up and functions being called, and current execution location. The source code debugger should indicate either the current source form being executed in a given stack frame or the most recently exited form, and allow entering the editor on that source code. It is acceptable to have to compile code with a special flag on or with special declarations in effect in order to achieve source code debugging, and a moderate (approximately twofold at the most) performance penalty in code compiled with source code debugging in effect is acceptable. We have implemented such a system at the KSL by storing the program location counter (PC) before and after the execution of each form and using this to index back to the source code. It is not required that the source code debugger (SCD) work with all optimizations, but it should be possible to disable those optimizers to use the SCD and still run effectively. 2.3. Inspector The backbone of the development environment is the Inspector. This tool must be quite flexible and user customizable. Given a datum the Inspector should display it in a window in such a way as to make the structure and contents quickly apparent to the user. For instance, a DEFSTRUCT structure might be displayed with a column with field names on the left and values on the right. It should further be possible to ask for alternative perspectives so that the user could view a list as a simple list of items, an ALIST, or a PLIST, and similarly for structures. The mouse should be used to traverse data structures by further inspecting. The display should not be strictly tabular to admit to nested data, graphs, etc.. The Inspector should also have the following attributes: . there should be a well-defined protocol to allow instances to display themselves and have non-standard mouse sensitivity in the inspector . able to work with all CL types, CLOS objects, compiled code, stack groups, and other objects that can be found in the system. . startup quickly o format only what is visible so that users don't have to wait for formatting of large data structures of which they wish to view only a small part . allow fields to be modified easily . handle circular data structures, preferably using the Common Lisp #l=(a b #l#) notation; (a b . ..) is not acceptable. . use multiple windows (one per datum) with a separate history window . support concise and verbose modes so that, for instance, an instance being viewed as part of another structure could display itself briefly, but be more detailed when being viewed directly 2.4. Software Management Tools An often neglected component of the development environment is a tool to manage software systems, keeping track of versions, patches, compilation and load dependencies, 257 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 etc. This has frequently been handled in the past with simple command procedures and the file system. These primitive mechanisms fail to handle many cases that are becoming more and more important such as version checkpointing and multiple programmers, as well as needing version numbers in the host file system to support backup versions. We have explicitly not required version numbers on files, but only under the condition that the usefulness of the version numbers is addressed in the software management system (SMS). The SMS should have the following features: . in the SMS, even more than usual, quick response and non-intrusive function are critical so that users aren't tempted to circumvent the system, thus ruining its integrity . allow multiple versions of objects to be kept for both backup and for release cycling . allow for patches to "released" systems . allow partial or complete recompilation of systems, automatically taking care of dependencies . allow transitive dependencies so that if system A depends on system B and system C depends on system A, manipulations of C cause both A and B to be affected appropriately . support team programming via object or module "check out" (ie. only one programmer is able to write a module, and ideally audit trails are kept); the the smaller the module size, the less chance for two programmers requiring write access to a module at the same time . can be either file or object based 2.5. Performance Monitoring and Analysis An important aspect of writing software is the ability to find out where programs are spending their time so that tuning work can be applied appropriately to sluggish programs. Thus we require the following performance measurement facilities: . stack sampling wherein a record of what functions are active on the stack is recorded at small intervals o function entry counting . accurate meters; microsecond precision desired 2.6. Lisp Listener There must be a "listener" or "top-level" which is how the user interacts with the read-eval-print loop of Lisp. Along with the terminal oriented listener there should be a window based implementation (ideally based on the system editor) with the following features: . a history mechanism allowing access to past typein and results (in at least a text form) . editing ability, including using the pointer E. H. Shortliffe 258 5P41-RR00785- 15 Draft Lisp Environment Spec 3. Languages and Utilities 3.1. Windowing Currently, the best way for a computer to present information to an interactive user seems to be via digital images presented on CRT displays. The display is divided into sub-displays called "windows" allowing various pieces of information to be presented at once. The programmatic and user interfaces to this mechanism is called the "window system". It is very difficult to fully specify a window system complete, flexible, and efficient enough to be what everybody needs and will need. Therefore, the most important aspect of the desired window system is that it be able to evolve as more is learned about user interface and data presentation. Therefore the window system must be well layered and modular to support incremental mutation and experimentation. In particular, we expect to see tools on top of an application toolbox, on top of window system primitives, on top of a window transport protocol (such as CLX with X.11), with the inter-layer communication passing through well-defined CLOS protocols such that new layers can be implemented with a minimum of trouble. In particular, we also expect to routinely use remote windowing capabilities, so it's very important that the windowing protocol be a standardized one accessible from many machines, such as X.1 1. Standards in other layers should be used as they become available and appropriate. The layering approach also allow flexibility in display devices, and if well implemented, would allow easy redirection of output to a "display" device that happens to be a printer to give high-resolution hard copy, such as seen in Xerox Lisp's ImageOp facility, as well as color displays, higher or lower resolution displays, files, etc. We expect people to be developing tools under at least two different windowing paradigms, including the "messy desk" metaphor which is characterized by many small (relative to the display size) windows each of which is an application or piece thereof, like a desk with many papers on it, and the "display swapping" metaphor in which there are a few applications, each of which typically takes the whole screen when active, though the individual applications usually have smaller "panes" in the display-filling "frame", and the user swaps which application is on the display via some keystroke sequence. There are arguments for both styles, and we would like our next generation system to support both styles to the extent possible. Generally, we feel the "messy desk" approach is the more general of the two, and the more sought after, and so should receive the most attention. With that general framework, here are some specific requirements for the window system (Note: the "as in" comments below are intended to give examples of current systems with the type of functionality we wish to describe, and are not indended as strict specification of how the functionality should be implemented): . since we can't predict the needs of our programmers in this time of rapidly changing user interface technology, we have to insist on access to the source code for at least the higher levels of the window system, and that the inter- layer protocols be well documented and flexible . while we expect almost all interactions to be window-based, it will be necessary at times to access the Lisp from a non-windowing system or terminal so we need terminal-like interfacing to be available on at least a 259 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 rudimentary basis making it possible, for instance, to check on a long- running program from a remote location over a crude link o provide programmer specifiable on-screen mini-dot about available mouse actions, and use this facility in system tools o there must be a way to reestablish connections to the window system from remote hosts without restarting Lisp so that if, say, a network connection fails the user can reconnect and reattach to the Lisp session if the operating system hasn't killed it . horizontal and vertical scrolling based on customizable redisplay (wherein the user programs how to fill in the newly exposed window area) such as in Symbolics Lisp . hierarchical (nested) window structuring as in TI Explorer Lisp o high tolerance for "logical" errors, so that minor operational errors, such as incorrectly reshaping a window, don't cause the window system to crash . non-restrictive parameters such the maximum number of windows, depth of the window hierarchy, size of a window, etc.. . possible to write to non-exposed windows as in the Apple Macintosh . a window title mechanism (with option of attaching mouse handlers to titles) as in TI Explorer Lisp . customizable scroll bars (eg. size, location, color, pattern, pop-up, mouse- capturing, mouse-click actions, extra "buttons", etc.) . constraint frames that allow automatic configuration of inferior windows when the superior is altered as in TI Explorer Lisp . fast opening windows, with no more than fraction of a second delay for typical windows in typical circumstances . scrolling and character drawing rate of at least 1,500 char/set in a full- screen window . operations that work in color and BAN . optional color . use n-dimensional abstract positions (not X and Y coordinates), even if the window system only uses 2-d points in order to allow for future display devices that may well be able to deal in 3-d o documented font formats so users may define new fonts . ability to use any font, any time without using a "font map" (requiring separate operations to ensure proper baseline calculations is ok) E. H. Shortliffe 260 5P41-RR00785- 15 Draft Lisp Environment Spec . customizable, titled pop-up menu styles with 2 standard styles: roll-out (the menu stays i,n place with no buttons down until an item is selected or the mouse is moved away from the menu as in TI Explorer Lisp) and button-up (the menu stays up while a mouse button is held down, selecting the item the mouse is over when the button is released or none if the mouse is outside the menu as in Xerox Lisp); "pull down" menus should be implementable . a "snapshot" facility to allow a section of the screen image to be recorded in another window for later viewing, printing, or saving on a file in a published raster format . ability to shrink application windows into smaller icons as in Xerox Lisp . customizable event distribution (mouse clicks, etc.) . user-extensible pop-up windows used for entering data ("dialog box") such as seen on the Apple Macintosh dialog boxes or the TI Explorer Choose Variable Values menus. . allow CLOS instances to display themselves, allowing graphical menu items, etc. o there must be a rich set of facilities for running user specified code when the pointing device enters or exits a region, and when a region is "touched" by the pointing device; these "active regions" must: be able to be non-rectangular, though they may be constrained to be rectilinear and congruent to the X and Y axis of the window system O have built in ways to work in scrolling windows have built in ways to be highlighted via boxing, inverting, and color washing on color displays when the pointer is inside the region o the ability to channel mouse events and keystrokes through streams . a modular way to add items to menus in tools . the mechanism for redisplaying windows that are being uncovered or moved should be flexible and programmable so that, for instance, windows need not have a data structure that stores the pixels associated with the window when they are not visible, a technique that can be expensive when there are many large windows or many bits per pixel . mechanism for synchronizing keyboard and mouse so switching windows works smoothly o have available variable width fonts The following items are features which would be very useful, but are not required: . optional larger or multiple displays as in the Macintosh II 261 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 . have available primitive drawing operations that are very low overhead (non-consing and f.ast) . coordinate transforms . allow access to low level color/frame buffer control for some applications o kerning (changing the position a character is drawn in when it is next to certain other characters) of certain character combinations or at least pseudo-kerning (offsetting certain characters in a font by a small amount to improve the aesthetics of the resulting text 3.2. Multiple Processes A critical part of any powerful software development environment is the ability to run multiple tasks simultaneously. In Lisp it is particularly important that one have access to multi-tasking within the single address space of the running Lisp environment so that cooperating agents can freely access shared data. These "processes" should have the following properties: . inexpensive in terms of system resources and time to create/use (lightweight) (to assist this, it's advisable that a process not have things like an I/O window until it's needed) o processes should be preemptable and should be scheduled under a priority and quantum based scheme . locks and events should be available to control synchronization of cooperating agents so that, for instance, a data-producer could signal an event that would re-activate a consumer process o the scheduler should maintain queues rather than typically calling a "are you runnable" routine to avoid high overhead when there are many processes and events should be used to move processes onto run queues . a complete set of operations to control processes (eg. [unlarrest, kill, change priority/quantum, inspection of statistics/top-level functionsietc., and interrupt) o the keyboard attaches to processes, not windows . the notion of a stack or stack group should be separate from the actual process . processes should be CLOS objects It is conspicuous that unless process switching is a very, very low overhead operation, of the order of 2-4 function calls, the scheduler shouldn't run in it's own stack, but run on the last running processs' stack so that undue overhead isn't introduced. It is desirable to have but potentially difficult to implement a system wherein when one process waits for an event like I/O or a page fault other processes are able to continue. We would very much like this feature, but do not require it. 3.3. Common Lisp E. H. Shortliffe 262 5P41-RR00785-15 Draft Lisp Environtnent Spec Lisp is the computing language of choice in the KSL and is likely to remain so due to it's utility in the type of programming required for the KSL's research, as well as the tremendous amount .of available experience in building strong computing environments in Lisp and the strong commitment already in place. We feel it is critical that the Lisp be Common Lisp, as described in Guy Steele's "Common Lisp the Language" today, and as specified by ANSI's X3J13 in the future. It is essential that with the Lisp there is a strong object oriented programming system. In particular, the Common Lisp Object System {ret) (CLOS) should be fully supported, being well integrated with the Lisp support environment, and used where appropriate in system software. The CLOS specification has not yet been completed, but major parts of it are essentially complete and have been accepted by X3J13 and so, given the critical role of the object system, we feel that even an implementation of the partial specification is important, along with further implementation as the specification matures. Similarly, we require a condition handling system, and in particular the Common Lisp Condition System {rer>, under conditions similar to CLOS. The condition system should also provide a mechanism to catch and abort "trivial" errors committed during top-level typein such as unbound variables and undefined functions and a mechanism allowing searching for functions or symbols in other packages when they are undefined (package DWIM). The system must also support the following: o Large FlXNUMs (at least 24 bits) . IEEE Floating Point Numbers {what's the IEEE spec name?} 3.4. Compilation Mondern Lisp systems almost always "compile" the Lisp source code into instructions more suited to the architecture of the machine in use. As execution speeds increase and the size of problems being tackled increases it's important that compilation time not introduce painful delays into the development loop, ruining the quick edit- compile-run cycle characteristic of Lisp. The compilation delay for "small" code units (approximately 10 to 50 lines of code not involving heavy macro expansion), should be negligible, while mechanisms should be available for larger "batch" compilations as part of the software management system. It is acceptable to have mutiple ways to execute the same source code, such as an interpreter, a compiler that executes quickly but produces slower code, and a compiler that executes slowly but produces faster code. However, it is absolutely essential that all of these have indistinguishable semantics. The following features should be present with compilation: . Compilation of individual top-level forms (incremental compilation) o Complete compilation of all forms, including closures and other lexical functions o Ability to cause code to be compiled in-line via the use of declarations 263 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 o Ability to do unboxed floating point operations if appropriate declarations are present . Documentation on built-in compiler optimizers . The ability for the user to define new compiler optimizers o Optimization of tail recursive calls o Automatic compilation of "encapsulation" code such as ADVISE or TRACE (described elsewhere) so that at the time of the encapsulation the associated code is compiled To achieve further portability, we would like to see cross compilation or multi- targeting capability, though this is not strictly required. 3.5. Input/Output Moving data into and out of Lisp is something that is done quite a bit during the normal operation of the machine, so I/O performance must be on par with the rest of the system. In particular, loading compiled files (FASLOAD), reading source files into the editor, loading source files (READ), file probes (OPEN and FILE-WRITE-DATE) (typically done in the software management system) must be quick. As a guideline, we would expect read/write speeds into and out of the Lisp world of close to 40 kilobytes (kB) per second to a network file server, or 150 kB/sec to a local disk, as seen on the Explorer II. Additionally, the system should support a large number of simultaneously open files (certainly 30 or more), as well as multiple streams (input and output) to the same file. Access to remote files should be transparent to the Lisp user (ie. no special "copy to the local system" step should be needed to access data available via filing protocols including at least NFS.) 3.6. Utilities There are a number of environmental utilities needed to use the system effectively, including: . a way to save a lisp image for later reuse, as in Tl's DISK-SAVE and Xerox's . a mechanism for "advising"; wrapptng code around entry points to affect the behavior of code as in TI Explorer Lisp's ADVISE o routines for accessing network facilities (TCP, etc.) . a WITH-TIMEOUT routine that would allow execution of some code to be aborted if it does not complete within the alloted time . trace, including internals (FOO-in-BAR, LABELS, FLET, closures) o a facility for searching the world for symbols (APROPOS) . ways to determine what functions call another function or use a particular special variable, as well as to determine what functions are called by a (and specials used by) a particular function E. H. Shortliffe 264 5P41-RR00785-15 Draft Lisp Environment Spec . routines for laying out and drawing hierarchies and graphs where the nodes and edges can be instances that draw themselves and define their own mouse sensitivity . a foreign language interface on machines which support non-Lisp languages . remote procedure call (Sun RPC) . stream interfaces to various facilities such as networks, windows, and printers . a single stepper, probably only on interpreted code o file properties including write-date, author, security status, locking, and properties native to the operating system; ideally the user would be able to define and use arbitrary file properties . ability to restart Lisp in the same address space, allowing one to reinitialize windows, processes, etc. without losing edits and other work . routines for manipulating time values o a mechanism for reconstituting structure definitions (DEFSTRUCT) 3.7. Interface Toolkit While there is a good deal of disagreement about what user interfaces should look like it is generally accepted that consistency within a system is worth working for when other considerations aren't overriding. Thus, to encourage overall consistency, we require that the higher level programming tools be built on a single interface subtrate such that they have consistent use of menus, typein, display format, etc., and that this tool be available to the user. An examples of this type of tool is Tl's Universal Command Loop. 3.8. Help System Important to the overall usability of the system is good "novice" support in the form of some combination of on-line help (eg. general help files), on-line tutorials (eg. the Machintosh Guided Tour), context sensitive help (including menus of commonly used commands and completion) (eg. Tl's Su.ggestions Menus), on-line documentation (eg. dot strings in functions and variables), primer documentation, and informative error messages. The help system should be consistent, used in all the system tools, and useable in user written programs. 3.9. Status Information It's important for the user, and especially, the programmer to get good information about the current operational status of the machine. The TI and Symbolics WHO-LINE and Sun's Perfmeters are examples of this sort of facility. We feel that some information about the following should be includable on the screen at all times: . gc activity . cpu used by lisp 265 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00785-15 o paging . consing . system load . current package . status of process owning the keyboard . what process owns the keyboard . lisp file activity 3.10. Printing While the capabilities of the system being specified will encourage a paperless office, hardcopy printing is still an important part of our activities for debugging, passing along information, and keeping records. The printing system should: . use generic operations as in Xerox Lisp (see window system discussion) . allow users to add new printer types/drivers . support at least PostScript initially . allow printing of unformatted files, formatted files, and window images 3.11. Pretty Printing The ability of the system to format output, especially in a window based environment, is important to the user's ability to understand the data being displayed. Thus, a "pretty printing" facility must be included with the following features: . user customizable . has a protocol to interact with instances so that they can make formatting decisions . interprets arg lists for macros and formats accordingly so that, for instance, &BODY arguments get formatted as code . works with intermediate data structures so that entire expressions need not be printed for efficiency with scrolling windows, especially in the inspector . can format user-defined mouse sensitive data 4. Lower Level Issues 4.1. Address Space As the size of problems being addressed increases so does the need for address space in Lisp systems. It is difficult to quantify address space requirements as they are affected by other facets of the system including effectiveness of the memory management system and the space required by data strucutures, but we can say that we need the E. H. Shortliffe 5P41-RR00785-15 Draft Lisp Environment Spec potential for a very large address space, such that the address space is typically limited by how much disk it's feasible to have rather than the number of bits used in addresses. Good examples of today's systems are the Symbolics 3600 {number} or the TI Explorer with Extended Address Spaces {number}. Note that systems that migrate unused objects out of the primary address space should allow a primary address space of at least 100 megabytes. We expect this requirement to expand in the future. Conditions should be signalled when address space gets to a user-definable minimum, with default handlers that will notify the user of the low address space condition. A parallel to address space is stack size. Recursive or other deeply nested programs must work without modification and so we require that the execution stack be expandable at run time as with the TI Explorer and Symbolics systems, or very, very large. Minimally the stack should be large enough to run 5,000 function call levels with an average of 4 arguments and 4 locals per level in the most stack-hungry execution mode (usually interpreted). 4.2. Memory Management A frequent thorn in the side of Lisp programmers is the reclamation of allocated but no longer used memory, or garbage collection (GC). Therefore, we require that: . in general, GC take no more than 10% total overhead, with less being very desirable . no programmer/user intervention be required in normal operation . the amount of time that the machine is made unavailable to the user is limited to a few seconds at a time either by time limiting the amount of work done at once or by using a concurrent system, with either solution implying a dynamic algorithm o the working set not be unduly expanded by GC operation to avoid thrashing o there be controls available to the programmer to tune the GC to a particular program, or to inhibit it at times for real-time program segments or for timing . there be finalization code associated with some objects like CLOS instances for cases like the need to release resources that aren't resident in the Lisp address space In addition to the automatic memory management software there should be tools that allow a programmer explicit control over storage allocation with notions similar to "area? for new allocation which can be declared exempt from GC or to be deallocated in bulk. Also allocation aids such as RESOURCE structures should be provided. 4.3. Dedicated Versus Shared Systems Timesharing, as opposed to having a processor dedicated to a single user, is acceptable in principle, though it is important that it be well done in the sense that users not step on each others' toes by doing simple things like running programs. In particular the scheduler and paging algorithms should be such that if the system is claimed to support N users, all N users should simultaneously see the kind of minimum responsiveness and performance we require. 267 E. H. Shortliffe Draft Lisp Environment Spec 5P41-RR00783-15 4.4. Hardware Capabilities The display should be able show approximately 70 lines of monochrome text with 130 columns each and still be read comfortably, such as the approximately 1024 x 768, 72 dot/inch black and white displays found on TI Explorers and Symbolics 3600 class machines, with a strong desire for being able to display two 80 character wide windows side by side, as Sun workstations with hi-resolution displays and Xerox 1186's with 19" displays. The display must be stable and crisp, which means it should probably be non-interlaced. It must also be possible to get a video output for demonstrating software to large audiences. The data input mechanism should support a rate at least equal to that found in accomplished touch typists (approximately 70 words per minute), such as a keyboard with 2 key or more rollover, as well as a fast pointing device equipped with at least two, and preferably three kinds of "touches", such as a 3 button mouse, a way of programming idioms into short cut sequences, such as programmable function keys on a keyboard, and a way of sending non-text commands which would mean at least two (Control and Meta) modifier keys on a keyboard. 4.5. Overall System Integration in the KSL Any new computing systems in the KSL must be able to fit in to the existing environment and interact with pre-existing systems to facilitate sharing data, moving users from system to system, and system administration. Incoming systems should have the following properties to integrate well into the KSL environment: . good networking including filing (Sun IP/UDP/NFS and IP/TCP/FTP minimally), virtual terminal service (lP/TCP/TELNET), remote procedure call (Sun IP/UDP/RPC), and name service (lP/UDP/DOMAIN) . provisions for file backup, possibly via NFS . a large limit, if any, on file names, with 40 characters per field minimum . if the system is a workstation, the user must be able to reboot from the console and be able to run first-order diagnostics to determine in most cases what major component is responsible for any failure . the element of the system that sits in offices should have minimal power requirements, thus requiring no additional air conditioning capacity, and should not generate distracting noise; if the system is a workstation unit, it is probably necessary to remote the processor from the display to achieve this requirement and we would need at least 500 feet of potential separation to reach from our office spaces to our machine room spaces; for example, we consider the TI Explorer too noisy and hot to have the system unit in most offices, and consider most Apple Macintosh II's to be just under the acceptable noise level 5. Acknowledgement The summary presented here is the result of extensive discussions among the following group: Bruce Delagi, Christopher Lane, James Rice, Thomas Rindfleisch, and Eric Schoen. E. H. Shortliffe 268 5P41 -RR007851 5 AIM Management Committee Membership Appendix D AIM Management Committee Membership Following are the current membership lists of the various SUMEX-AIM management committees: AIM Executive Committee: SHORTLIFFE, Edward H., M.D., Ph.D. (Chairman) Principal Investigator - SUMEX Medical School Office Building, Rm. X271 Stanford University Medical Center Stanford, California 94305 (415) 723-6970 FEIGENBAUM, Edward A., Ph.D. Co-Principal Investigator - SUMEX Heuristic Programming Project Department of Computer Science 701 Welch Road, Building C Stanford University Stanford, California 94305 (415) 723-4879 KULIKOWSKI, Casimir, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-2006 LEDERBERG, Joshua, Ph.D. President The Rockefeller University 1230 York Avenue New York, New York 10021 (212) 570-8080, 570-8000 LINDBERG, Donald A.B., M.D. (Past Adv Group Chrmn) Director, National Library of Medicine 8600 Rockville Pike Bethesda, Maryland 20814 (301)496-6221 MYERS, Jack D., M.D. School of Medicine Scaife Hall, 1291 University of Pittsburgh Pittsburgh, Pennsylvania 15261 (412) 648-9933 273 E. H. Shortliffe AIM Management Committee Membership AIM Advisory Group: MYERS, Jack D., M.D. School of Medicine Scaife Hall, I291 University of Pittsburgh Pittsburgh, Pennsylvania 15261 (412) 648-9933 5P41-RR00785-15 (Chairman) AMAREL, Saul, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-3546 COULTER, Charles L., Ph.D. (Exec. Secretary) Bldg 31, Room 5841 Biomedical Research Technology Program National Institutes of Health 9000 Rockville Pike Bethesda, Maryland 20892 (301) 496-5411 FEIGENBAUM, Edward A., Ph.D. (Ex-officio) Co-Principal Investigator - SUMEX Heuristic Programming Project Department of Computer Science 701 Welch Road, Building C Stanford University Palo Alto, California 94305 (415) 723-4879 KULIKOWSKI, Casimir, Ph.D. Department of Computer Science Hill Center Busch Campus Rutgers University New Brunswick, New Jersey 08903 (201) 932-2006 LEDERBERG, Joshua, Ph.D. President The Rockefeller University 1230 York Avenue New York, New York 10021 (212) 570-8080, 570-8000 LINDBERG, Donald A.B., M.D. Director, National Library of Medicine Building 38, Rm. 2E-17B 8600 Rockville Pike Bethesda, Maryland 20814 (301) 496-6221 E. H. Shortliffe 274 5P41 -RR00785- 15 AIM Management Committee Membership MINSKY, Marvin, Ph.D. Artificial Intelligence Laboratory Massachusetts Institute of Technology 545 Technology Square Cambridge, Massachusetts 02139 (617) 253-5864 MOHLER, William C., M.D. Associate Director Division of Computer Research and Technology National Institutes of Health Building 12A, Room 3033 9000 Rockville Pike Bethesda, Maryland 20892 (301) 496-l 168 PAUKER, Stephen G., M.D. Department of Medicine - Cardiology Tufts New England Medical Center Hospital 171 Harrison Avenue Boston, Massachusetts 02111 (617) 956-5910. SHORTLIFFE, Edward H., M.D., Ph.D. (Ex-officio) Principal Investigator - SUMEX Medical School Office Building, Rm. X271 Stanford University Medical Center Stanford, California 94305 (415) 723-6979 SIMON, Herbert A., Ph.D. Department of Psychology Baker Hall, 339 Carnegie-Mellon University Schenley Park Pittsburgh, Pennsylvania 15213 (412) 578-2787, 578-2000 275 E. H. Shortliffe AIM Management Committee Membership 5P41 -RR00785-15 Stanford Community Advisory Committee: SHORTLIFFE, Edward H., M.D., Ph.D. (Chairman) Principal Investigator - SUMEX Medical School Office Building, Rm. X271 Stanford University Medical Center Stanford, California 94305 (415) 723-6979 FEIGENBAUM, Edward A., Ph.D. Heuristic Programming Project Department of Computer Science Margaret Jacks Hall Stanford University Stanford, California 94305 (415) 723-4879 LEVINTHAL, Elliott C., Ph.D. Departments of Mechanical and Electrical Engineering Building 530 Stanford University Stanford, California 94305 (415) 723-9037 E. H. Shortliffe 276 5P41-RR00785-15 Scientific Subproject Abstracts Appendix E Scientific Subproject Abstracts The following are brief abstracts of our collaborative research projects. 277 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785- 15 National AIM Project: ATTENDING Project: A Critiquing Approach to Expert Computer Advice Principal Investigator: Perry L. Miller, M.D., Ph.D. Department of Anesthesiology Yale University School of Medicine New Haven, CT 06510 (203) 785-2802 Our project is exploring the "critiquing" approach to bringing computer-based advice to the practicing physician. Critiquing is a different approach to the design of artificial intelligence based expert systems. Most medical expert systems attempt to simulate a physician's decision- making process. As a result, they have the clinical effect of trying to tell a physician what to do: how to practice medicine. In contrast, a critiquing system first asks the physician how he contemplates approaching his patient's care, and then critiques that plan. In the critique, the system discusses any risks or benefits of the proposed approach, and of any other approaches which might be preferred. It is anticipated that the critiquing approach may be particularly well suited for domains, like medicine, where decisions involve a great deal of subjective judgment. To date, severa; prototype critiquing systems have been developed in different medical domains: 1. ATTENDING, the first system to implement the critiquing approach, critiques anesthetic management. 2. HT-ATTENDING critiques the pharmacologic management of essential hypertension. 3. W-ATTENDING critiques aspects of ventilator management. 4. PHEO-ATTENDING critiques the laboratory and radiologic workup of a patient for a suspected pheochromocytoma. In addition, a domain-independent system, ESSENTIAL-ATTENDING, has been developed to facilitate the implementation of critiquing systems in other domains. PUBLICATIONS 1. Miller, P.L. (Ed.): Selected Topics in Medical Artificial Intelligence. New York: Springer-Verlag (in press). 2. Rennels, G.D., Miller, P.L.: Artificial intelligence research in anesthesia and intensive care. Journal of Clinical Monitoring (in press). 3. Miller, P.L., Rennels, G.D.: Prose generation from expert systems: An applied computational linguistics approach. Al Magazine (in press). 4. Rennels, G.D., Shortliffe, E.H., Stockdale, F.E., Miller, P.L.: A computational model of reasoning from the clinical literature. Computer Methods and Programs in Biomedicine 24:139-149, 1987. 5. Rennels, G.D., Shortliffe, E.H., Stockdale, F.E., Miller, P.L.: A structured representation of the clinical literature and its use in a medical management advice system. Bulletin du Cancer 74:215-220, 1987. E. H. Shortliffe 278 5P41 -RR00785- 15 Scientific Subproject Abstracts 6. Miller, P.L., Barwick, K.W., Morrow, J.S., Powsner, S.M., Riely, C.A.: Semantic relationships and medical bibliographic retrieval: A preliminary assessment. Computers and Biomedical Research 21:64-77, 1988. 7. Miller, P.L., Morrow, J.S., Powsner, SM., Riely, C.A.: Semantically assisted medical bibliographic retrieval: An experimental computer system. Bulletin of the Medical Library Association 76:13I-136, 1988. 8. Miller, P.L.: Exploring the critiquing approach: Clinical practice-based feedback by computer. Biomedical Measurement, lnformatics and Control (in press). 9. Rennels, G.D., Shortliffe, E.H., Stockdale, F.E., Miller, P.L.: A computational model of reasoning from the clinical literature. The Al Magazine (accepted pending revision). 10. Miller, P.L., Fisher, P.R.: Causal models in medical artificial intelligence. Proceedings of the Eleventh Symposium on Computer Applications in Medical Care, Washington, D.C., November 1987, pp. 17-22. II. Powsner, SM., Barwick, K.W., Morrow, J.S., Riely, C.A., Miller, P.L.: Coding semantic relationships for medical bibliographic retrieval: A preliminary study. Proceedings of the Eleventh Symposium on Computer Applications in Medical Care, Washington, D.C., November 1987, pp. 108-l 12. 279 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR007851 5 Stanford Project: BBICU -- BLACKBOARD APPLICATIONS IN THE INTENS!VE CARE UNIT Principal Investigator: Adam Seiver, M.D. Department of Surgery Palo Alto Veterans Medical Center Palo Alto, Calif Lawrer-e Fagan, M.D., Ph.D. Departnient of Medicine Stanford University Barbara Hayes-Roth, Ph.D. Department of Computer Science Stanford University We are designing a data-interpretation and therapy-planning system for the intensive care unit (ICU). Fundamental research issues in temporal reasoning are associated with the ICU application area including assimilation of incoming data, representation of time-oriented intervals, and description of ongoing physiological processes [Fagan 841. In addition, in ICUs of the 199Os, many more physiological measurements will need to be collected at frequent intervals, and increased access to the current medical record in coded format will be possible. Processing of incoming data will have to be opportunistic, selecting from a number of models that have different computational requirements and accuracy. We will use a blackboard architecture, known as BBl, that has evolved from earlier work on protein-structure elucidation and construction layout [Hayes-Roth 851. BBI is particularly well suited for the ICU project because it maintains separate blackboards for domain and control knowledge. Although we have selected the blackboard structure as the organizing principle, many knowledge representation issues remain. First, we must represent the structure and function (anatomy and physiology) of the respiratory system. By characterizing the pathophysiology in terms of generic faults, we will create a more flexible means to diagnose problems in unusual situations --- in contrast to the phenomenological rules used in earlier systems. Second, we must coalesce quantitative and qualitative models. The physiology of the respiratory and cardiac systems have been modeled in detail, but it is impractical to base the entire reasoning process on complex mathematical equations. Instead, we are developing methods to transform quantitative models into simpler formulations. We must make explicit the simplifying assumptions and associate them with their corresponding models, so that we can select an applicable model for situations of varying complexity. Using the approach of our project for planning treatments for cancer patients [Langlotz 871, we will use strategic knowledge to create patient-specific specializations of standard treatment plans. We will use decision analytic methods to evaluate and explain the various treatment options available at any point in time. The long-term goal of this project is to embed the decision-making components within the data management tasks of the ICU. REFERENCES I. Fagan, L., Kunz, J., Feigenbaum, E, and Osborn, J. Adapting a rule-based system for a monitoring task, in Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project, B. Buchanan and E. Shortliffe (eds.). Reading, MA: Addison-Wesley Publishing Co., 1984. E. H. Shortliffe 280 5P41 -RR00785-15 Scientific Subproject Abstracts 2. Hayes-Roth, B. A Blackboard architecture for control. (Artificial Intelligence) 26:251-321, 1985. 3. Langlotz, C., Fagan, L., Tu, S., Sikic, B., and Shortliffe, E. A therapy planning architecture that combines decision theory and artificial intelligence techniques. Computers and Biomedical Research 20:279-303, 1987. 281 E. H. Shortliffe Scientific Subproject Abstracts 5P41-RR0078515 National AIM Project: DECISION SUPPORT FOR TIME-VARYING CLINICAL PROBLEMS Principal Investigator: Lawrence Widman, M.D., Ph.D. Division of Cardiology University of Texas Health Science Center San Antonio, Texas 78284 Time-varying systems, which include many areas of medicine, science, economics, and business, can be described mathematically by differential equations. They are distinct from the pattern-matching and logic-based domains dealt with so successfully by existing expert system methods, because they can include feedback relationships. It is generally felt that they are best approached by enhancement of existing methods for deep model-based reasoning. The goal of this project is to develop Al methods for capturing and using knowledge about time-varying systems. The strategy is to address general problems in model- based knowledge representation and reasoning. The intermediate objective is to develop methods which are powerful enough to work in selected realistic situations yet are general enough to be transportable to other, unrelated knowledge domains. The tactical approach is to work on well-defined yet complex and interesting problems in the medical domain. We have, therefore, selected the human cardiovascular system as our prototype of a time-varying system, and are developing methods for representing and reasoning about its mechanical and electrical activities in the norma! and diseased states. REFERENCES 1. Widman, L.E. Knowledge-Based Fault Identification and "What If" Simulation in Symbolic Dynamic Systems Models. Proceedings of the 1988 Society for Computer Simulation Multiconference on Al and Simulation, pp. 89-94, 1988. 2. Widman, L.E., Lee, Y.-B., and Y.-H. Pao. Diagnosis of Causal Medical Models by Semi-Quantitative Reasoning. In: Miller, P.L. (ed.). Topics in Medical Artificial Intelligence, Springer-Verlag (in press) 3. Widman, L.E. Semi-Quantitative "Close Enough" Systems Dynamics Models: An Alternative to Qualitative Simulation. In: Widman, L.E., Loparo, K.A., and N.R. Nielsen (eds.). Artificial Intelligence, Simulation, and Modeling. John Wiley & Sons, New York (in press). 4. Widman, L.E., Loparo, K.A., and N.R. Nielsen (eds.). Artificial Intelligence, Simulation, and Modeling. John Wiley 8 Sons, New York (in press). 5. Widman, L.E. and G.L. Freeman. A-to-D Conversion from Paper Records with a Desktop Scanner and a Microcomputer. (submitted to the American Journal of Physiology). E. H. Shortliffe 282 5P41 -RR00785-15 Scientific Subproject Abstracts Stanford Project: GUIDON/NEOMYCIN -- KNOWLEDGE ENGINEERING FOR TEACHING MEDICAL DIAGNOSIS Principal Investigators: William J. Clancey, Ph.D. 701 Welch Road Department of Computer Science Stanford University Palo Alto, California 94304 (415) 723-l 997 (CLANCEY@SUMEX-AIM) Bruce G. Buchanan, Ph.D. Computer Science Department Stanford University The GUIDON/NEOMYCIN Project is a research program devoted to the development of a knowledge-based tutoring system for application to medicine. The key issue for the GUIDON/NEOMYCIN project is to develop a program that can provide advice similar in quality to that given by human experts, modeling how they structure their knowledge as well as their problem-solving procedures. The consultation program using this knowledge is called NEOMYCIN. NEOMYCIN's knowledge base, designed for use in a teaching application, is the subject material used by a family of instructional programs referred to collectively as GUIDON2. The problem-solving procedures are developed by running test cases through NEOMYCIN and comparing them to expert behavior. Also, we use NEOMYCIN as a test bed for the explanation capabilities incorporated in our instructional programs. The goal of current work is to construct a knowledge-based tutoring system that teaches diagnostic strategies explicitly. By strategy, we mean plans for establishing a set of possible diagnoses, focusing on and confirming individual diagnoses, gathering data, and processing new data. The tutorial program has capabilities to recognize these plans, as well as to articulate strategies in explanations about how to do diagnosis. The strategies represented in the program, modeling techniques, and explanation techniques are wholly separate from the knowledge base, so that they can be used with many medical (and non-medical) domains, That is, the target program will be able to be tested with other knowledge bases, using system-building tools that we provide. SOFTWARE AVAILABLE ON SUMEX GUIDON--A system developed for intelligent computer-aided instruction. Although it was developed in the context of MYCIN's infectious disease knowledge base, the tutorial rules will operate upon any EMYCIN knowledge base. NEOMYCIN--A consultation system derived from MYCIN, with the knowledge base greatly extended and reconfigured for use in teaching. In contrast with MYCIN, diagnostic procedures, common sense facts, and disease hierarchies are factored out of the basic finding/disease associations. The diagnostic procedures are abstract (not specific to any problem domain) and model human reasoning, unlike the exhaustive, top-down approach implicit in MYCIN's medical rules. This knowledge base is used in the GUIDON2 family of instructional programs, being developed on D-machines. 283 E. H. Shortliffe Scientific Subproject Abstracts 5P4 1 -RR00785- 15 REFERENCES 1. Clancey, W.J.: Knowledge-Based Tutoring: The GUIDON Program, Cambridge: The MIT Press, 1987. 2. Clancey, W.J.: Methodology for building an intelligent tutoring system. In Kintsch, Polson, and Miller, (Eds.), METHODS AND TACTICS IN COGNITIVE SCIENCE. L. Erlbaum Assoc., Hillsdale, NJ. 1984, (Also STAN-CS-81-894, HPP 81-18) 3. Clancey, W.J.: Acquiring, representing, and evaluating a competence model of diagnosis. In Chi, Glaser, and Farr (Eds.), THE NATURE OF EXPERTISE. 1988. E. H. Shortliffe 284 5P41 -RR00785-15 Scientific Subproject Abstracts National AIM Project: INTERNIST-I/QMR (Quick Medical Reference) Principal Investigators: INTERNIST-QMR PROJECT: Randolph A. Miller, M.D. (RMILLERGSUMEX-AIM) Jack D. Myers, M.D. (MYERS@SUMEX-AIM) Fred E. Masarie, Jr., M.D. (MASARIE@SUMEX-AIM) University of Pittsburgh Pittsburgh, Pennsylvania 15261 (412) 648-3190 The major goal of the INTERNIST-I/QMR Project is to produce a reliable and adequately complete diagnostic consultative program in the field of internal medicine. Although this program is intended primarily to aid skilled internists in complicated medical problems, the program may have spin-offs as a diagnostic and triage aid to physicians' assistants, rural health clinics, military medicine and space travel. In the design of INTERNIST-I and QMR, we have attempted to model the creative, problem- formulation aspect of the clinical reasoning process, and to design program components that can help physicians at key points in this process. The program can compose differential diagnoses, dynamically, on the basis of clinical evidence. During the course of an INTERNIST-I consultation, it is not uncommon for a number of such conjectured problem foci to be proposed and investigated. With the QMR program, the physician is given more control over the direction of the problem- solving activities than was possible with INTERNIST-I. QMR is broader in scope than INTERNIST-I in that it provides quick and efficient access to the INTERNIST-I/QMR knowledge base to provide low and intermediate levei informational support for physicians' decision-making, in addition to providing consultative advice. SOFTWARE AVAILABLE ON SUMEX Versions of the INTERNIST-I KB are available for experimental use, with permission from the developers. In the past year, copies have been shared with Drs. Gregory Cooper and Homer Chin of Dr. Shortliffe's group at Stanford. The INTERNIST-I and QMR knowledge bases are Copyright 1986-88 by the University of Pittsburgh. Quick Medical Reference and QMR are registered trademarks of the University of Pittsburgh. The project continues to be oriented primarily towards research and development; hence, a stable production version of the system is not yet available for general use. QMR has been shared on a restricted basis with a limited number of academic colleagues, who have agreed to give the QMR development team feedback on the program's strengths and weaknesses. REFERENCES 1. Myers JD. The Background of INTERNIST-I and QMR. In: Proceedings of the History of Medical lnformatics Conference. National Library of Medicine. pp. 195-197, November 1987. 2. Masarie, F.E., Miller, R.A. Medical Subject Headings and Medical Terminology: An analysis of terminology used in hospital charts. Bulletin of the Medical Library Association, 1987; 75:89-94. 3. Masarie FE, Miller RA. INTERNIST-I to Quick Medical Reference (QMR): Transition from a mainframe to a microcomputer. Proceedings Ninth Annual IEEE/Engineering in Biology and Biology Society. IEEE Press. pp. 1521- 1522, November 1987. 4. Parker, RC, Miller RA. Using causal knowledge to create simulated 285 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785-15 patient cases: The CPCS project as an extension of INTERNIST-I. Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care. pp. 473-480, November 1987. 5. Bankowitz RA, Blumenfeld BH, Miller RA, et al. User variability in abstracting and entering printed case histories with Quick Medical Reference (QMR). Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care. pp. 68-73, November 1987. 6. Masarie FE, Miller RA. Quick Medical Reference (QMR): An information management tool for clinical diagnosis. IN: Critical Reviews in Medical Informatics. Boca Raton, Florida, CRC Press, 1988. 7. Parker RC, Miller RA. Using causal knowledge to created simulated patient cases: The CPCS project as an extension of INTERNIST-I. IN: Miller PL (ed) Topics in Medical Artificial Intelligence. Computers and Medicine Series, Springer-Verlag, New York, 1988. 8. Challinor SM, McNeil MA, Bankowitz RA, et al. Evaluation of a Computer-Assisted General Medicine Diagnostic Consultation Service. Abstract presented at the 1 lth Annual SGIM Meeting, Washington, D.C., April 27-29, 1988. 9. Miller RA. From Automated Medical Records to Expert System Knowledge Bases: Common Problems in Representing and Processing Patient Data. Topics in Health Record Management. 75(3):23-36, 1987 10. Miller RA. Computer-based Diagnostic Decisionmaking. Medical Care. 25(12):Sl48-S152, 1987. E. H. Shortliffe 286 5P41 -RR00785- 15 Scientific Subproject Abstracts National AIM Project: MENTOR -- MEDICAL EVALUATION OF THERAPEUTIC ORDERS Principal Investigators: Stuart Speedie, Ph.D. (SPEEDIE@SUMEX-AIM) School of Pharmacy University of Maryland 20 N. Pine Street Baltimore, Maryland 21201 (301) 528-7650 Terrence F. Blaschke, M.D. (BLASCHKE@SUMEX-AIM) Department of Medicine Division of Clinical Pharmacology Stanford University Medical Center Stanford, California 94305 The goal of the MENTOR project is to implement and begin evaluation of a computer-based methodology for reducing therapeutic misadventures. The project uses an on-line expert system to continuously monitor the drug therapy of individual patients and generate specific warnings of potential and/or actual unintended effects of therapy. The appropriate patient information is automatically acquired through interfaces to a hospital information system. This data is monitored by a system that is capable of employing complex chains of reasoning to evaluate therapeutic decisions and arrive at valid conclusions in the context of all information available on the patient. The results reached by the system are fed back to the responsible physicians to assist future decision making. Specific objectives of this project include: ;. Implement a prototype computer-based expert system to continuously monitor in-patient drug therapy that uses a modular medical knowledge base and a separate inference engine to apply the knowledge to specific situations. 2. Select a small number of important and frequently occurring drug therapy problems that can lead to therapeutic misadventures and construct a comprehensive knowledge base necessary to detect these situations. 3. Evaluate the prototype MENTOR system with respect to its impact on the on the physicians' therapeutic decision making as well as its effects on the patient in terms of specific mortality and morbidity measures. The work in this project builds on the extensive previous work in drug monitoring done by these investigators in the Division of Clinical Pharmacology at Stanford and the University of Maryland School of Pharmacy. 287 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785-15 Stanford Project: MOLGEN -- AN EXPERIMENT PLANNING SYSTEM FOR MOLECULAR GENETICS Principal Investigators: Edward A. Feigenbaum, Ph.D. Department of Computer Science Stanford University Charles Yanofsky, Ph.D. (YANOFSKY@SUMEX-AIM) Department of Biology Stanford University Stanford, California 94305 (415) 725-3815 The MOLGEN project has focused on research into the applications of symbolic computation and inference to the field of molecular biology. This has taken the specific form of systems which provide assistance to the experimental scientist in various tasks, the most important of which have been the design of complex experiment plans and the analysis of nucleic acid sequences. Our current research concentrates on scientific discovery within the subdomain of regulatory genetics. We desire to explore the methodologies scientists use to modify, extend, and test theories of genetic regulation, and then emulate that process within a computational system. Theory or model formation is a fundamental part of scientific research. Scientists both use and form such models dynamically. They are used to predict results (and therefore to suggest experiments tc test the model) and also to explain experimental results. Models are extended and revised both as a result of logical conclusions from existing premises and as a result of new experimental evidence. Theory formation is a difficult cognitive task, and one in which there is substantia! scope for intelligent computational assistance. Our research is toward buildi??g a system which can form theories to explain experimental evidence, can interact with a scientist to help to suggest experiments to discriminate among competing hypotheses, and can then revise and extend the growing model based upon the results of the experiments. The MOLGEN project has continuing computer science goals of exploring issues of knowledge representation, problem-solving, discovery, and planning within a real and complex domain. The project operates in a framework of collaboration between the Heuristic Programming Project (HPP) in the Computer Science Department and various domain experts in the departments of Biochemistry, Medicine, and Biology. It draws from the experience of several other projects in the HPP which deal with applications of artificial intelligence to medicine, organic chemistry, and engineering. SOFTWARE AVAILABLE ON SUMEX SPEX system for experiment design. UNITS system for knowledge representation and acquisition. SEQ system for nucleotide sequence analysis. REFERENCES 1. Friedland, P.E.: Know/edge-based experiment design in molecular genetics, (Ph.D. thesis). Stanford Computer Science Report, STAN- cs-79-771. E. H. Shortliffe 288 5P41 -RR00785- 15 Scientific Subproject Abstracts 2. Friedland, P.E. and Iwasaki, Y.: The concept and implementation of skeletal plans, Journal of Automated Reasoning, 1(2):161-208, 1985. 3. Friedland, P.E. and Kedes, L.: Discovering the secrets of DNA, Communications of the ACM, 28( 11): 1164-l 186, November, 1985. 4. Stefik, M.J.: An examination of a frame-structured representation system, Proc. Sixth IJCAI, Tokyo, August, 1979, pp. 845-852. 5. Stefik, M.J.: Planning with constraints, (Ph.D. thesis). Stanford Computer Science Report, STAN-CS-80-784, March, 1980. 6. Karp, P., and D. Wilkins: An Analysis of the Deep/Shallow Distinction for Expert Systems. Stanford University Knowledge Systems Laboratory Report KSL-86-32, 1986. 7. Karp, P., and P. Friedland: Coordinating the Use of Qualitative and Quantitative Knowledge in Declarative Device Modeling. Stanford University Knowledge Systems Laboratory Report KSL-87-09, 1987. 8. Round, A.: QSOPS: A Workbench Environment for the Qualitative Simulation of Physical Processes. Stanford University Knowledge Systems Laboratory Report KSL-87-37, 1987. 9. Karp, P.: A Process-Oriented Model of Bacterial Gene Regulation, Stanford University Knowledge Systems Laboratory Technical Report KSL-88- 18. 289 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785-15 Stanford Project: ONCOCIN -- KNOWLEDGE ENGINEERING FOR ONCOLOGY CHEMOTHERAPY CONSULTATION Principal Investigator: Edward H. Shortliffe, M.D., Ph.D. Departments of Medicine and Computer Science Stanford University Medical Center Medical School Office Building Stanford, California 94305 (415) 723-6979 (SHORTLlFFE@SUMEX-AIM) Project Director: Dr. Lawrence M. Fagan (FAGANGSUMEX-AIM) The ONCOCIN Project is overseen by a collaborative group of physicians and computer scientists who are developing an intelligent system that uses the techniques of knowledge engineering to advise oncologists in the management of patients receiving cancer chemotherapy. The general research foci of the group members include knowledge acquisition, inexact reasoning, explanation, and the representation of time and of expert thinking patterns. Much of the work developed from research in the 1970's on the MYCIN and EMYCIN programs, early efforts that helped define the group's research directions for the coming decade. MYCIN and EMYCIN are still available on SUMEX for demonstration purposes. The prototype ONCOCIN system is in limited experimental use by oncologists in the Stanford Oncology Clinic. Thus, much of the emphasis of this research has been on human engineering so that the physicians will accept the program as a useful adjunct to their patient care activities. ONCOCIN has generally been weI!-accepted since its introduction, and we are now testing a version of the program which runs on professional workstations (rather than the central SUMEX computer) so that it can be implemented and evaluated at sites away from the University. Significant extensions to the basic ONCOCIN project have included the OPAL oncology knowledge acquisition system, the PROTEGE system for description of knowledge acquisition tools for clinical trial application areas, and the ONYX system for strategic therapy planning. SOFTWARE AVAILABLE ON SUMEX MYCIN-- A consultation system designed to assist physicians with the selection of antimicrobial therapy for severe infections. It has achieved expert level performance in formal evaluations of its ability to select therapy for bacteremia and meningitis. Although MYCIN is no longer the subject of an active research program, the system continues to be available on SUMEX for demonstration purposes and as a testing environment for other research projects. EMYCIN-- The "essential MYCIN" system is a generalization of the MYCIN knowledge representation and control structure. It is designed to facilitate the development of new expert consultation systems for both clinical and non-medical domains. ONCOCIN-- This system is in clinical use but requires Lisp machines to be run. Much of the knowledge in the domain of cancer chemotherapy is already well-specified in protocol documents, but expert judgments also need to be understood and modeled. E. H. Shortliffe 290 5P41 -RR00785- 15 Scientific Subproject Abstracts REFERENCES 1. Shortliffe, E.H., Scott, A.C., Bischoff, M.B., Campbell, A.B., van Melle, W. and Jacobs, C.D.: ONCOCIN: An expert system for oncology protocol management. Proc. Seventh IJCAI, pp. 876-881, Vancouver, B.C., August, 1981. 2. Duda, R.O. and Shortliffe, E.H.: Expert systems research. Science 220:261-268, 1983. 3. Langlotz, C.P. and Shortliffe, E.H.: Adapting a consultation system to critique user plans. Int. J. Man-Machine Studies 19:479-496, 1983. 4. Bischoff, M.B., Shortiiffe, E.H., Scott, A.C., Carlson, R.W. and Jacobs, C.D.: Integration of a computer-based consultant into the clinical setting. Proceedings 7th Annual Symposium on Computer Applications in Medical Care, pp. 149-152, Baltimore, Maryland, October 1983. 5. Hickam, D.H., Shortliffe, E.H., Bischoff, M.B., Scott, A.C., Jacobs, CD.: A study of the treatment advice of a computer-based cancer chemotherapy protocol advisor (Memo KSL-85-21). Annals of Internal Medicine 103(6 pt 1):928-936 (1985). 6. Kent, D.L., Shortliffe, E.H., Carlson, R.W., Bischoff, M.B., Jacobs, C.D.: Improvements in data collection through physician use of a computer- based chemotherapy treatment consultant (Memo KSL-85-22). Journal of Clinical Oncology 3: 1409- 1417, 1985. 7. Tsuji, S. and Shortliffe, E.H.: Graphical access to a medical expert system: I. Design of a knowledge engineer's interface (Memo KSL-85-11). Meth. Inf. Med., 25:62-70, 1986. -- 8. Lane, C.D., Differding, J.C., Shortliffe, E.H.: Graphical access to a medical expert system: II. Design of an interface for physicians (Memo KSL-85-15). Meth. Inf. Med., 25:143-150, 1986. -- 9. Shortliffe, E.H. Medical expert systems: Knowledge tools for physicians. Memo KSL-86-52. Special issue on Medical Informatics, West. J. Med. 145:830-839, 1986. 10. Walton, J.D., Musen, M.A., Combs, D., Lane, CD., Shortliffe, E.H., and Fagan, L.M. Graphical access to medical expert systems: Ill. Design of a knowledge-acquisition environment. Memo KSL-85-30. Methods of - Information e Medicine, 26:78-88, 1987. 11. Langlotz, C.P., Fagan, L.M., Tu, S.W., Sikic, B.I., and Shortliffe, E.H. A therapy planning architecture that combines decision theory and artificial intelligence techniques. KSL-85-55. Computers h Biomedical Research, 20:279-303, 1987. 12. Musen, M.A., Fagan, L.M., Combs, D.M., and Shortliffs, E.H. Use of a domain model to drive an interactive knowledge-editing tool (Memo KSL-86-24). International Journal of Man-Machine Studies - 26(1):105-121, 1987. 291 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785-15 National AIM Project: Computer-Aided Diagnosis of Lymph Node Pathology (PATHFINDER) Principal Investigator: Bharat Nathwani, M.D. Department of Pathology HMR 204 2025 Zonal Avenue University of Southern California School of Medicine Los Angeles, California 90033 (213) 226-7064 (NATHWANI@SUMEX-AIM) The Pathfinder Project is centered on the construction of an expert system for assisting pathologists with the diagnosis of tissue pathology. Pathfinder research is focused on the domain of lymph node pathology. The project is based at the University of Southern California in collaboration with the Stanford University Medical Computer Science Group. Ongoing AIM research has been addressing fundamental problems of knowledge representation, reasoning strategies, user modeling, explanation, and user acceptance. The central focus of research has been on developing tractable knowledge acquisition and reasoning methods for probabilistic reasoning. In addition to the ongoing theoretical research, the team have implemented an expert system program and a tool for knowledge acquisition. The Pathfinder expert system provides diagnostic advice on eighty common benign and malignant diseases of the lymph nodes based on 150 histologic features. Currently a more sophisticated knowledge base that represents probabilistic dependencies is being developed. A probabilistic knowledge acquisition tool has been implemented on the Macintosh that enables a user to assess probabilistic relationships by graphically noting similarities and differences between diseases hypotheses. SOFTWARE AVAILABLE ON SUMEX Pathfinder -- A pilot version of the Pathfinder program is available for experimentation on the DEC 2060 computer. This version is an early version of the program that has not been completely tested. REFERENCES 1. Horvitz, E.J., Heckerman, D.E., Nathwani, B.N. and Fagan, L.M.: Diagnostic Strategies in the Hypothesis-directed Pathfinder System, Node Pathology. HPP Memo 84-l 3. Proceedings of the First Conference on Artificial Intelligence Applications, Denver, Colorado, Dec., 1984. 2. Heckerman, D. E., and Horvitz, E. J., "The Myth of Modularity in Rule- based Systems," in Uncertainty in Artificial Intelligence, Vol. 2, J. Lemmer, L. Kanal, ed., North Holland, New York, 1987. 3. Horvitz, E.J., Heckerman, D.E., Nathwani, B.N. and Fagan, L.M.: The Use of a Heuristic Problem-solving Hierarchy to Facilitate the Explanation of Hypothesis-directed Reasoning. KSL Memo 86-2. Proceedings of Medlnfo, Washington D.C., October, 1986. 4. Horvitz, E. J., "Toward a Science of Expert Systems," Invited Paper, Computer Science and Statistics: Proceedings of the 18th Symposium on the Interface, American Statistical Association, March, 1986, pgs. 45-52. 5. Heckerman, D.E., "An Axiomatic Framework for Belief Updates," in E. H. Shortliffe 292 5P41 -RR00785-15 Scientific Subproject Abstracts Uncertainty in Artificial Intelligence, Vol. 2, J. Lemmer, L. Kanal, ed., North Holland; New York, 1987. 6. * Heckerman, D. E., and Horvitz, E. J., "The Myth of Modularity in Rule- based Systems," in Uncertainty in Artificial Intelligence, Vol. 2, J. Lemmer, L. Kanal, ed., North Holland, New York, 1987. 7. o Heckerman, D.E., and Horvitz, E.J., "On the expressiveness of rule- based systems for reasoning under uncertainty," Proceedings of the National Conference on Artificial Intelligence, Seattle, Washington, July, 1987. 8. * Horvitz, E. J., Heckerman, D. E., Langlotz, C. P., "A framework for comparing alternative formalisms for plausible reasoning," Proceedings of the AAAI," August, 1986, Morgan Kaufman, Los Altos, CA, 1986, 9. * Horvitz, E.J., Breese, J.S., Henrion, M., Decision Theory in Expert Systems and Artificial Intelligence, International Journal of Approximate Reasoning, Elsevier, N.Y. July, 1988. 10. o Heckerman, D.E., An Evaluation of Three Scoring Schemes, Proceedings of the 4th AAAI Workshop on Uncertainty in Artificial Intelligence, Minneapolis, MN., (to appear August 1988). 11. * Horvitz, E.J., A Multiattribute Utility Approach to Inference Understandability and Explanation, Tech. Report, KSL-28-87, Knowledge Systems Laboratory, Stanford, California, March, 1987. 293 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785- 15 Stanford Project: Principal Investigators: PROTEAN Project Oleg Jardetzky (JARDETZKYGSUMEX-AIM.STANFORD.EDU) Nuclear Magnetic Resonance Lab, School of Medicine Stanford University Medical Center Stanford, California 94305 Bruce G. Buchanan, Ph.D. (BUCHANAN@SUMEX-AlM.STANFORD.EDU) Computer Science Department Stanford University Stanford, California 94305 The goals of this project are related both to biochemistry and artificial intelligence: (a) use existing Al methods to aid in the determination of the 3-dimensional structure of proteins in solution (not from x-ray crystallography proteins), and (b) use protein structure determination as a test problem for experiments with the Al problem-solving structure known as the Blackboard Model. Empirical data from nuclear magnetic resonance (NMR) and other sources may provide enough constraints on structural descriptions to allow protein chemists to bypass the laborious methods of crystallizing a protein and using X-ray crystallography to determine its structure. This problem exhibits considerable complexity, yet there is reason to believe that Al programs can be written that reason much as experts do to resolve these difficulties. A prototype knowledge-based system assembles major secondary structures of a protein into families of structures compatible with a given set of distance constraints under the control of an explicit assembly strategy. Structures can also be refined at the atomic level of detail using constraints within secondary structures and between amino acid side chains to further restrict the 3-dimensional structure found. By generalizing this approach to the assembly of arrangements of objects subject to constraints, we have developed a language for specifying actions and control for problem solving in similar problem domains. REFERENCES 1. Altman, R. and Jardetzky, 0.: New strategies for the determination of macromolecular structures in solution. Journal of Biochemistry (Tokyo), Vol. 100, No. 6, p. 1403-1423, 1986. 2. Altman, R. and Buchanan, B.G.: Partial Compilation of Control Know/edge. To appear in: Proceedings of the AAAI 1987. 3. Brinkley, J., Cornelius, C., Altman, R., Hayes-Roth, B. Lichtarge, O., Duncan, B., Buchanan, B.G., Jardetzky, 0.: Application of Constraint Satisfaction Techniques to the Determination of Protein Tertiary Structure. Report KSL-86-28, Department of Computer Science, 1986. 4. Brinkley, James F., Buchanan, Bruce G., Altman, Russ B., Duncan, Bruce S., Cornelius, Craig W.: A Heuristic Refinement Method for Spatial Constraint Satisfaction Problems. Report KSL 87-05, Department of Computer Science. 5. Buchanan, B.G., Hayes-Roth, B., Lichtarge, O., Altman, A., Brinkley, J., Hewett, M., Cornelius, C., Duncan, B., Jardetzky, O.:The Heuristic Refinement Method for Deriving Solution Structures of Proteins. Report KSL-85-41. October 1985. E. H. Shortliffe 294 5P41 -RR00785-15 Scientific Subproject Abstracts 6. Garvey, Alan, Cornelius, Craig, and Hayes-Roth, Barbara: Computational Costs versus Benefits of Control Reasoning. Report KSL 87-l 1, Department of Computer Science. 7. Hayes-Roth, B.: The Blackboard Architecture: A Genera/ Framework for Problem Solving? Report HPP-83-30, Department of Computer Science, Stanford University, 1983. 8. Hayes-Roth, B.: BB7: An Environment for Building Blackboard Systems that Control, Explain, and Learn about their own Behavior. Report HPP-84-16, Department of Computer Science, Stanford University, 1984. 9. Hayes-Roth, B.: A Blackboard Architecture for Control. Artificial Intelligence 26:251-321, 1985. 10. Hayes-Roth, B. and Hewett, M.: Learning Control Heuristics in BBI. Report HPP-85-2, Department of Computer Science, 1985. 11. Hayes-Roth, B., Buchanan, B.G., Lichtarge, O., Hewett, M., Altman, R., Brinkley, J., Cornelius, C., Duncan, B., and Jardetzky, 0.: PROTEAN: Deriving protein structure from constraints. Proceedings of the AAAI, 1986, p. 904-909. 12. Jardetzky, 0.: A Method for the Definition of the Solution Structure of Proteins from NM/? and Other Physical Measurements: The LAC- Repressor Headpiece. Proceedings of the International Conference on the Frontiers of Biochemistry and Molecular Biology, Alma Alta, June 17-24, 1984, October, 1984. 13. Lichtarge, Olivier: Structure determination of proteins in solution by NMR. Ph.D. Thesis, Stanford University, November, 1986. 14. Lichtarge, Olivier, Cornelius, Craig W., Buchanan, Bruce G., Jardetzky, Oleg: Validation of the First Step of the Heuristic Refinement Method for the Derivation of Solution Structures of Proteins from NMR Data., April 1987. Submitted to Proteins: Structure, Function, and Genetics. 295 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785-15 National AIM Project: KNOWLEDGE ENGINEERING FOR RADIATION THERAPY Principal Investigator: Ira J. Kalet, Ph.D. School of Medicine University of Washington at Seattle Seattle, Washington 98195 (206) 548-4107 We are developing an expert system for planning of radiation therapy for head and neck cancers. The project will ultimately combine knowledge-based planning with numerical simulation of the radiation treatments. The numerical simulation is needed in order to determine if the proposed treatment will conform to the goals of the plan (required tumor dose, limiting dose to critical organs). The space of possible radiation treatments is numerically very large, making traditional search techniques impractical. Yet, with modern radiation therapy equipment, the design of treatment plans might be significantly aided by automatically generating plans that meet the treatment constraints. The project will result in systematization of knowledge about radiation treatment design, and will also provide an example of how to represent and solve design problems with a knowledge based system. This project has some relevance to computer science as well, in that our approach, if successful, may contribute to a better understanding of design problem solving with knowledge-based systems. REFERENCES 1. I. Kalet and W. Paluszynski: A Production Expert System for Radiation Therapy Planning. Proceedings of the AAMSI Congress 1985, May 20-22, 1985, San Francisco, California. Edited by Allan H. Levy and Ben T. Williams. American Association for Medical Systems and Informatics, Washington, D.C., 1985. 2. W. Paluszynski and I. Kalet: Radiation Therapy Planning: A Design Oriented Expert System. WESTEX-87 (Western Conference on Expert Systems), Anaheim, California, June 2-4, 1987. 3. I. Kalet and J. Jacky: Knowledge-based Computer Simulation for Radiation Therapy Planning. Proceedings of the Ninth International Conference on the use of Computers in Radiotherapy, Scheveningen, the Netherlands, June 1987. North Holland, 1987. 4. I. Kalet and W. Paluszynski: Knowledge-based Computer Systems for Radiotherapy Planning. American Journal of Clinical Oncology. In press. E. H. Shortliffe 296 5P41 -RR00785 15 Scientific Subproject Abstracts Stanford Project: RADIX -- DERIVING KNOWLEDGE FROM TIME-ORIENTED CLINICAL DATABASES PENGUIN -- Applying Database and Knowledge base Technology to Medical Instrumentation Principal Investigators: Gio C.M. Wiederhold, Ph.D. Departments of Computer Science and Medicine Stanford University Stanford, California 94305 (415) 497-0685 (WlEDERHOLD@SUMEX-AIM) Thierry Barsalou, M.D. Medical Computer Science Stanford University (BARSALOU@SUMEX-AIM) Robert L. Blum, M.D., Ph.D. Stanford University (BLUM@SUMEX-AIM) The RADIX research has been phased out during this year. RADIX had two main goals: (1) to explore the usefulness of knowledge-based techniques to derive medical knowledge from clinical database (DB) systems containing non-randomized, non-protocol patient observations and (2) to develop a program and set of techniques for automated summarization of patient records. The process of reliably deriving causal relationships has proven to be quite difficult because of the complexity of disease states and time relationships, strong sources of bias, and problems of missing and outlying data. However, the experience gained in the RADIX project and its predecessor project, RX, continues to influence basic research directions. The problem of automated knowledge acquisition remains an important area of research, and we expect that the foundations laid here will influence work of others as well. PENGUIN is a project aimed at developing methods for bringing together expert system and database technologies in an integrated advice system. Databases and expert systems share a common goal -- generating useful information for action -- but accomplish their tasks separately, using different principles. It is clear, however, that future information systems will require both the problem-solving capabilities of expert systems (ESs) and the data-handling capabilities of database management systems (DBMSs). Indeed, combining database and expert system technologies into expert database systems (EDSs) is an emerging research area. One can define an EDS as "a system for developing applications requiring knowledge-directed processing of shared information". From a perspective of developing advanced biomedical information systems, this definition conveys two goals: (1) enhancing DBMSs with structuring and manipulation tools that take more semantics into account: (2) allowing ESs to access and to handle efficiently information stored in database(s). In this research project, we investigate the hypothesis that the object-oriented approach to knowledge and information representation can serve as a unifying scheme for developing EDSs. We explore this hypothesis in a practical biomedical environment, Fluorescence Activated Cell Sorting (FACS). FACS is emerging as a major source of information for biomedical research and clinical practice. Currently, achieving good FACS performance requires analyzing and integrating various complex data and knowledge sources in order to fulfill the information needs of FACS investigators, This work 297 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785-15 involves a close collaboration between researchers in the Medical Information Sciences Program and the Departments of Computer Science and Genetics. REFERENCES PENGUIN PUBLICATIONS 1. Barsalou, Thierry and Gio Wiederhold: "Automating a Cell Counter"; to be published in The International Journal of Artificial Intelligence in Engineering, Computational Mechanics Publ., UK, 1988. 2. Barsalou, Thierry and Gio Wiederhold: "Applying a Semantic Model to an Immunology Database"; in W.W. Stead (editor), Proceedings of the Eleventh Symposium on Computer Applications in Medical Care, pages 871-877, IEEE Computer Society Press, Washington, D.C., November 1987. 3. Barsalou, Thierry, W.A. Moore, L.A. Herzenberg, and G. Wiederhold: "A Database System to Facilitate the Design of FACS Experiment Protocols" (abstract); Cytometry, Vol.97, August 1987. 4. Barsalou, T. and G. Wiederhold. Knowledge-based mapping of relations into objects. To appear in the International Journal of Al in Engineering, 1988. 5. Barsalou, T. An object-based architecture for biomedical expert database systems. Submitted to the IEEE Twelfth Symposium on Computer Applications in Medical Care, 1988. RADIX PUBLICATIONS Monograph 1. Blum, R.L.: Discovery and representation of causal relationships from a large time-oriented clinical database: The RX project. in D.A.B. Lindberg and P.L. Reichertz (Eds.), LECTURE NOTES IN MEDICAL INFORMATICS, Vol. 19, Springer-Verlag, New York, 1982. Journal Articles 1. Blum, R.L.: Computer-Assisted Design of Studies using Routine Clinical Data: Analyzing the Association of Prednisone and Cholesterol. Annals of Internal Medicine 104(6):858-868, June, 1986. 2. Blum, R.L.: Discovery, confirmation, and incorporation of causal relationships from a large time-oriented clinical database: The RX Project. Computers and Biomedical Research 15(2):164-l 87, April, 1982. 3. Walker, M.G. How Feasible is Automated Discovery? IEEE Expert 2( 1):70-82, Spring, 1987. Conference Proceedings 1. DeZegher-Geets, I.M., Freeman, A.G., Walker, M.G., Blum, R.L., and Wiederhold, G.C.M. Computer-aided Summarization of a Time-oriented Medical Data Base. In Proceedings of the Third Annual Conference on Computerization of Medical Records. Institute for Medical Record Economics, 1987. E. H. Shortliffe 298 5P41 -RR00785-15 Scientific Subproject Abstracts 2. Downs, S., Walker, M.G., and Blum, R.L. Automated Summarization of On-line Medical Records. In Proceedings of the Fifth World Congress on Medical lnformatics (Medinfo), pages 800-804. Elsevier Science Publishers, 1986. 3. Walker, M.G., and Blum, R.L. Towards Automated Discovery from Clinical Databases: the RADIX Project. In Proceedings of the Fifth World Congress on Medical lnformatics (Medinfo), pages 32-36. Elsevier Science Publishers, 1986. 299 E. H. Shortliffe Scientific Subproject Abstracts 5P41 -RR00785-15 Stanford Project: REFEREE Project Principal Investigators: Bruce G. Buchanan, Principal Investigator Computer Science Department Stanford University Stanford, California 94305 Byron W. Brown, Co-Principal Investigator Department of Medicine Stanford University Medical Center Daniel E. Feldman, Associate Investigator Department of Medicine Stanford University Medical Center The goals of this project are related both to medical science and artificial intelligence: (a) use Al methods to allow the informed but non-expert reader of the medical literature to evaluate a randomized clinical trial, and (b) use the interpretation of the medical literature as a test problem for studies of knowledge acquisition and fusion of information from disparate sources. REFEREE and REVIEWER, a planned extension, will be used to evaluate the medical literature of clinical trials to determine the quality of a clinical trial, make judgments on the efficacy of the treatment proposed, and synthesize rules of clinical practice. The research is an initial step toward a more general goal - building computer systems to help the clinician and medical scientist read the medical literature more critically and more rapidly for use in making clinical decisions. REFERENCES 1. Haggerty, J.: REFEREE and RULECRITIC: Two prototypes for assessing the quality of a medical paper. REPORT KSL-84-49. Master's Thesis, Stanford University, May 1984. 2. *Chavez, R. Martin and Cooper, G. F.: KNET: Integration Hypermedia and Normative Bayesian Modeling. REPORT KSL. Stanford University, March 1988. 3. `Lehmann, H. Knowledge Acquisition for Probabilistic Expert Systems. Submitted to Symposium on Computer Applications in Medical Care, 1988. E. H. Shortliffe 300 5P41 -RR00785- 15 References 1. Coulter, C. L. "Research Instrument Sharing." Science 201, 4354 (1978). 2. Lederberg, J. "Digital Communications and the Conduct of Science: The New Literacy." Proceedings of the IEEE 66, 11 (1978). 3. NLM Planning Panel 4. NLM Long Range Plan: Medical Informatics. Shortliffe, E.H., Panel 4 Chairman , National Library of Medicine, January, 1987. 4. Scheifler, R.W. and Gettys, J. The X Window System. Draft October 1986. To appear in a special issue of the ACM Transactions on Graphics -- user interface software 5. Steele, Guy L., Jr.. Common Lisp - The Language. Digital Press, Burlington, MA, 1984. 301 E. H. Shortliffe