MEDLINE to PubMed and Beyond

Presented by:: Janet Zipser; MEDLARS Management Section, NLM; December 1998

{Select this image to begin the slide presentation without the accompanying narrative.}

picture of the NLM MEDLINE to PubMed and Beyond

Late in the Civil War, while serving as a field surgeon for the Union at the Battle of Gettysburg, John Shaw Billings was wounded and re-assigned to the Office of the Medical Director of the Army of the Potomac in Washington, DC. As an assistant in the Surgeon General's office, one of his duties was to care for its library. The Library had started in 1836 and grown to 1800 volumes. It, along with the Army Medical Museum, had grown large enough to move into Ford's Theatre, which the government bought after Lincoln's assassination. By 1874, under Billing's leadership, the size of the Library grew to 50,000 items. Having succeeded in this goal, Billings next set out to prepare a catalog and index. Unfortunately Congress refused to fund the indexing project. So each night baskets of journals were delivered to Billing's Georgetown home, where he marked the articles he wanted indexed. Former soldiers who served as clerks copied down the titles on index cards. Billings and his assistant Robert Fletcher, would write down one or two appropriate subject headings.

This monumental contribution was best summed up in an editorial in the publication The Nation in 1881. The editorial, entitled A Great Catalogue said: "What would induce a man to spend years of labor in getting together and printing tens of thousands of titles of articles concerning the treatment of fevers, consumption, wounds, and abscesses. Among the chief motives, no doubt, is the desire to be of public service, a feeling very strong in all Americans, except perhaps politicans."

Billings, of course, was a renaissance man. He went on to design the Johns Hopkins Hospital and the New York City Public Library. While there were some excellent directors after Billings left the Library in the 1850s, none achieved greatness until Dr. Frank Bradway Rogers who led the library from 1949 to 1963.

By the 1950s the growth of biomedical literature strained the capabilities of NLM to produce timely printed indexes. In addition, the amount of information on a single subject was sufficient to make access by one aspect less than useful. Rogers set in motion a plan to mechanize the production of Index Medicus in the late 1950s. True computerization was not achieved until 1964, with $3 million in funding from what was then the National Heart Institute. The ability of computers to store thousands of citations, organize them by subject and author, produce camera-ready pages was unprecedented. Immediately, there was an increase in the depth of indexing from 2 headings per article to 10. Also, the number of articles indexed doubled.

During the early period in which the Library was using data processing equipment to prepare Index Medicus, staff learned a great deal about an alternative means of storing and retrieving information, namely the first generation of commercial electronic computers. Colonel Rogers, he was also in the military, envisioned and then wrote the specifications for what became the MEDLARS system. The heart of MEDLARS was to be a digital computer. Information representing the indexing or the same data appearing in Index Medicus was to be fed into the system, converted to magnetic tape, and manipulated in the computer.

The introduction of MEDLINE was not only a notable achievement itself but as NLM's current Deputy Director Kent Smith is fond of pointing out, we scaled the heights of bureaucratic nomenclature by creating an acronym of an acronym: For MEDLARS is an acronym for Medical Literature Analysis and Retrieval System. And MEDLINE stands for MEDLARS Online!

The MEDLARS system (MEDLINE was the only database) was developed by General Electric using a Honeywell computer. An integral part of the system was the development of photocomposition equipment known affectionately as the GRACE machine (Graphic Arts Composing Equipment). GRACE was scheduled for delivery in 1963 so as to be tested for the operational date of January 1964. Unfortunately GRACE delivery slipped and the Library had to face a major decision: delay the inauguration of MEDLARS or find another way to compose Index Medicus. The decision was made to keep to the original date by using the standard computer printer. GRACE arrived in 1964 and was used until 1975 when it was retired to the Smithsonian. GRACE performed the printing tasks in 18 hours that previously took weeks to perform.

A "demand search" service allowed librarians on behalf of health professionals to request a search. A multiple page form was completed, mailed to Bethesda, and worked on by a trained search analyst. In about 3 weeks a print-out was received by the analyst and mailed to the requestor.

With the success of MEDLARS came the need for more staff than NLM could hire. Accordingly, in 1967 the Library began to plan for a replacement system that would provide online automated support to create databases and to allow online formulation of searches (not online searching). In 1968, NLM began to ask many librarians about their requirements for an improved retrieval system. This led to the design of the AIM-TWX demonstration project. TWX was a network then owned by AT&T. Most libraries already used the TWX equipment for interlibrary loan. AIM-TWX demonstrated the value of online retrieval. Citations for about 100 journals (Abridged Index Medicus) were stored on the SDC or Systems Development Corporation computer in Santa Monica, California.

In 1968, SUNY at Syracuse had developed an online service based on MEDLARS and the SUNY Network also provided valuable experience and demonstrated the practicability of online bibliographic retrieval. AIM-TWX extended the concept to the entire nation. It was demonstrated in 1970 at MLA and well received. This slide shows AIM-TWX being demonstrated at AMA. It was clear that NLM had to revise its thinking to develop a system not just for online search formulation but online searching.

In October 1971 the old system for database creation and maintenance was coupled with SDC's retrieval system (later ELHILL) to mount the first online service.

What was an interactive, online retrieval system like in 1971? The only visible manifestation was the typewriter-like device, a terminal, so called because it was at one "end" of the service, connected to the user's telephone. The telephone connected the terminal to the computer that provided the service. This combination could be direct dial or other communication method.

Inexpensive, reliable telecommunications was essential for the success of MEDLARS, just as it is today for Internet access. Nationwide access through TWX machines was not a success. TWX rates were $42/hour compared to $27/hour for direct dial. As it happened, there was a fortunate conjunction of interests that resolved the problem. TYMSHARE, Inc. was interested in providing a data communications service under a new FCC directive and wanted a client with a public service to test the market; NLM needed a nationwide network but had neither money nor desire to build its own network.

The NLM computer was connected to Tymshare in February 1972, inaugurating the first commercial "value added network". By the end of 1972 about 150 libraries had access to MEDLINE^® all at medical schools and research facilities. For users in 40 cities, a local phone call could reach the network which would allow connection of the local terminal to the NLM computer. The rate was $6/hour, a 4-fold reduction over direct dial. The highest speed available was 30 characters/second. Most people had 10 characters/second Texas Instrument Silent 700s. We were the first customer of Tymshare technology that allowed a user to make a local call from their city to Bethesda and get back instant results. In other words, MEDLINE was the first remote access, real-time database in existence. In its own time, maybe as exciting as the killer aps of today! But in its early days, MEDLINE searching wasn't for every medical librarian.

In 1978 a meeting was held at NLM with 11 librarians representing each of the RML regions at that time. This group came to be known as the Bethesda 11. Some of you may know that Judith Topper a librarian from suburban New York, represented this area at the Bethesda 11 meeting. This group wanted a further expansion of MEDLINE for hospital libraries. After about 7 years only about 70 hospitals nationwide had access. NLM acknowledged its priority for MEDLINE access was RML, resource, and major medical schools. Remember there was an application process and there was a 3-week training requirement. Quite an inconvenience if you were running a one person hospital library. Also, many libraries had to buy a portable terminal for about $2,000. The hospital librarians wanted some more relevant information for them which lead to the creation of the Health Planning and Administration database.

The MEDLARS service continued to grow. In December of 1982 searching hours were extended to 3 AM - 5 PM weekdays and 8:30-5 on Saturdays. At the same time the MEDLINE backfiles were made available to search online. The famous NLM online pricing algorithm was used for the first time beginning in October 1983. We became the first system to move away from connect hour charges. The first class of the Basics of Searching MEDLINE for the health professional was offered in 1984 and in April of 1985 searching hours were extended to 11 at night.

Although librarians and information specialists were still being trained and using the "command language" mode of searching MEDLINE, in 1986 many health professionals began to use the Grateful Med software. Grateful Med took advantage of the expanding personal computer and modem market. The software included telecommunication dialing scripts that allowed end users to bypass installing additional communications software and writing the necessary scripts to dial up the National Library of Medicine's computer. The software also allowed the user to select terms from the controlled vocabulary and to automatically download retrieved references to their computer. Grateful Med was designed to simplify the search process for novice searchers. At its height, more than 100,000 copies of the disks were sold.

In July 1992, NLM announced that command language searching of MEDLARS was available via the Internet using the Telnet capability. And in April of 1996 as the use of Internet continued to grow, the Library announced a new addition to the Grateful Med family of user-friendly search software -- Internet Grateful Med, also known as IGM. IGM was the first World Wide Web search application available through NLM.

The very next year, in June 1997, NLM held a bi-partisan press event on Capitol Hill to announce that searching NLM databases via the World Wide Web would be free. The ceremonial first free MEDLINE search was conducted by Vice President Gore. Also present at the event and shown in the slide are Dr. Harold Varmus, NIH Director; Dr. Donald Lindberg, NLM Director; and Dr. David Lipman, Director, National Center for Biotechnology Information ?the developers of PubMed. At the June press event, Gore forecast the benefits of making the world's largest medical database available to consumers, health professionals, medical librarians, and scientists around the globe. The Vice President noted "This development, by itself, may do more to reform and improve the quality of health care in the United States than anything else we have done in a long time. "

At the same time PubMed was publicly discussed as another World Wide Web interface to the MEDLINE database in addition to Internet Grateful Med. The response to free Web searching by users of all kinds, health care professionals, researchers or the general public, has been overwhelming. The usage has grown in over a year from an annualized level of 7 million searches per year to 120 million. From a snapshot taken of PubMed use and dividing users into 3 general categories, the General Public approaches 30% of users. Others are Researchers 36%; Health Professionals 34%. Librarians are grouped with Healthcare Professionals in this survey.

Here are a few other staggering statistics to consider: approximately 10 million records are available in MEDLINE back to 1966; 120 million searches are conducted just on the NLM computer, and 400,000 new citations are added to MEDLINE each year. To keep up with the 400,000 articles, a physician could read 2 articles each day, every day of the year and by the end of the year fall 550 years behind.

This Capitol Hill event clearly was the public beginning of the newest chapter in NLM's history -- reaching out to the general public, the consumer. In 1990 the law that established NLM was amended to read "NLM shall publicize its products and services". No explicit language directs NLM to provide access to the general public but the times we live in are demanding this change. We no longer live in the society that would agree with this quote from the mid-19th century physician, Oliver Wendell Homes: "Your patient has no more right to all the truth than he has to all the medicine in your saddlebags." We now live in a society that genuinely seems to want access to medical information as evidenced by these actual headlines from major newspapers across the nation.

So, what is NLM doing and planning on doing to reach this diverse audience? We first began with the Public Library Pilot project and MedlinePlus, the consumer health Web page. The Library announced that 39 public library organizations with more than 200 locations in nine states - (Alabama, Georgia, Maryland, New York, Pennsylvania, South Carolina, Tennessee, Texas and Virginia) and the District of Columbia would take part in a pilot project designed to increase public awareness of and access to health information via the Internet. This project, Medical Questions, MEDLINE Has Answers, will evaluate the degree to which public libraries and the Internet can help meet the health information needs of the public. NLM's partners in the Medical Questions campaign are the National Network of Libraries of Medicine, the Kellogg Foundation, the Medical Library Association, the Public Library Association (a part of the American Library Association), and the Friends of the National Library of Medicine.

In October of this year, Dr. Lindberg announced the debut of NLM's consumer health page, MedlinePlus. He noted: "Recognizing that informed consumers are better patients but have varied health information needs, NLM has developed MedlinePlus, an easy-to-understand resource for the public which includes MEDLINE - the world's largest database of peer-reviewed information - as well as links to self-help groups, access to the NIH consumer health information, clearinghouses, health-related organizations, and clinical trials." Here's a slide of the MedlinePlus home page with the buttons to the various resources available.

In addition to general health topics, MedlinePlus provides access to extensive information for about 20 specific diseases and conditions ("Health Topics"). The site will be continuously expanded and it is expected to include comprehensive and cutting-edge information on hundreds of diseases and conditions in the coming months.

This year we also began indexing 12 consumer health publications which you see listed here . As alternative medicine has become more mainstream culminating it seems in the recent issue of JAMA featuring over 80 alternative medicine studies. NLM plans in 1999 to begin indexing 7 alternative medicine titles you see on this slide .

So, what's in the future? Easier interfaces to the information available at the NLM Web site that will accommodate our diverse customer base. We will strive to help the general public/consumer be able to easily navigate our site to find the information that will answer their questions and, at the same time, we will strive to continue supporting our customer base that needs to be able to manipulate the data/information in a more sophisticated manner, i.e., the librarian/information specialist.

Sometime early next year you will begin to see the first transition of unique journal citations to PubMed from other ELHILL databases. The first batch will include citations from HealthSTAR, AIDSLINE, and OLDMEDLINE. We will be moving away from "one-stop shopping by subject matter" to the new mindset of "one stop shopping by item type". So, for a while during a transition period, you will have to go to one interface to find journal citations, another to find monographs and audiovisuals, another to locate clinical trial information, and perhaps another to retrieve meeting abstracts. Hmm, that's not so friendly for the general public nor for the busy librarian. For that reason, NLM is beginning to plan and design for an overall "knowledge sources" protocol to collect information from various NLM applications transparently so it appears to the outsider that everything is actually in one database or databank. Staff in MEDLARS Management are affectionately calling the proposal "son of Grateful Med".

As I was preparing these remarks and consulting with Sheldon Kotzin who was originally scheduled to give this talk, Sheldon shared with me his interesting observation of a trend in NLM's online history. In the beginning of MEDLARS, NLM search analysts were the only individuals allowed to formulate and perform search strategies. NLM was finally convinced to train and allow academic medical school librarians to search. However, hospital librarians were not allowed this privilege. Some time later, times changed and hospital librarians were allowed to be part of the network. And then in the early to mid-1980s assisted by the release of the Grateful Med software, NLM began to extend their service to health professionals. And as you all know, this event was not welcomed by all. And now we are witnesses to an outreach to the general public. It seems that over time many of us have either been shut out or had the opinion that some group should be shut out or both! However, today's technology as I stated earlier is demanding that no group be disenfranchised.

Barbara Frey, Library Manager at New Britain General Hospital in New Britain, Connecticut said it best in a recent Gratefully Yours article entitled "Perpetual Transition": "Do I feel threatened by users doing their own searches, using their desktop, laptop, and palm-held computers, from their offices, homes, planes, trains, and automobiles? No. Water fluoridation did not eliminate the need for dentists, either. Systems keep changing. Some users keep up; others rely on the professional resource: the librarian. " She goes on to say: "It is important for the librarian to remain visible to the audience. Don't leave the stage!"

So, please consider and remember throughout these chaotic times: Since the late 1800s, we have been, are presently and will remain in "Perpetual Transition" but throughout NLM's history as well as MLA's history, the medical librarian has continued to remain one of the most useful and important information resources available.

BSD Home Page

Last updated: 13 February 2006
First published: 01 December 1998
Metadata| Permanence level: Permanence Not Guaranteed