Literature at the Human-Computer Seam

Michael D. Levi

Presented at the Modern Language Association 2000 Annual Convention

"Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation...A graphic representation of data abstracted from the banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data."

William Gibson, Neuromancer

Over the last fifty years computers and computing have irrevocably altered the literary process in ways ranging from the trivial to the profound. As information technology permeates the contemporary world its effects are felt not only in business, high-tech communications, and the sciences, but also by authors, book vendors, readers, and literary critics.

Some of the technology-generated changes may not have a measurable effect on the end literary product: despite the lamentations of many writers, it is not clear whether the physical act of writing on a word processor truly changes the written document. In the same vein, though it is astonishing that a book can be written in the United States, edited in Switzerland, typeset in India, and proofread in Scotland, all without once manifesting itself on paper, it is not clear how transportation over wires is qualitatively different from transportation via tramp steamer or cargo airplane, and whether it changes the substance of the final work.

On the other hand, the fact that literary works can now be made available to a huge audience without ever going through a critical review by editors or publishers, that such works can take advantage of group interactivity for both creation and modification, and that communications between authors and readers can be both immediate and unfiltered, may indeed change the very nature of literary creation. Similarly, the potential for non-linear narrative facilitated by hypertext and hypermedia, and the possibilities of fully immersive virtual reality raise the stakes for literary endeavors.

Before launching into ever more hyperbolic flights of speculation, however, it may be useful to recapitulate what is currently known about the perceptual and cognitive differences between the written word captured in ink on paper pages, and the same words captured in binary units displayed on a cathode ray tube, liquid crystal display unit, or comparable computer presentation device.

To start out, it is crucial to note that books are a very mature and stable technology. Admittedly, the mechanisms for producing relatively large volumes of printed publications have changed substantively from hand-copying in medieval scriptoria, to movable text physically laying a layer of ink on a page, to the current use of photocomposition technology. Far fewer significant changes in the physical manifestation of the end product have taken place, however. An eighteenth century reader of Swift or Defoe would have no trouble recognizing the latest John Grisham thriller for what it is. And, language differences aside, that same eighteenth century reader would have no trouble picking up The Brethren, orienting it right side up, locating the first page, and beginning to read.

Contrast this with the rate of change characteristic of today's computing devices:

  • Through the 1970's teletypes served as the primary interface between mainframe or mini computers and end users. Essentially printers with a keyboard, teletypes captured all human-computer communication sequentially on a roll of paper, often using only upper case letters. Specialty terminals did exist, but were rare and expensive, usually reserved for plotting coordinates and other low-resolution graphics tasks.
  • In the early 1980's personal computers came with video display terminals. These were also text-only devices, but allowed non-sequential positioning and updating of information.
  • By the late-1980's the windowed interface pioneered at Xerox PARC (Macintosh, Microsoft Windows, X-Windows, etc.) began to gain popularity. By the mid-1990's virtually all microcomputers employed a windowed interface.
  • Since then the most significant systems changes have been in smaller, more portable computing devices, often with wireless connectivity to larger networks. These include personal digital assistants, cellular phones, pagers, and "electronic books."

A 1970's computer scientist faced with an iMac or Pentium III would most likely have no idea where to begin. This hypothetical user, though computer-literate by the standards of the time, would probably be able to turn on the device, but might well never figure out how to operate a mouse, pull down the appropriate menu, and begin to operate his or her application of choice.

The point is that computers are still an emerging technology. The remainder of this essay will attempt to capture the current state of the art, knowing that the details will surely be dated almost as soon as they are written.

Inherent differences between books and computer screens can be divided into three types: perceptual, ontological, and residual.

Perceptual Differences

Perceptual differences relate to the sensory experience of the user, that which can be seen, touched, heard, or even smelled. The user may or may not be consciously aware of these factors.

The perceptual characteristics of computer screens have changed a great deal as the technology evolves, though for the most part these have been changes in degree rather than essence.

Book: Very high resolution. Computer Screen: Low to medium resolution.

The technology for placing ink on paper is very highly developed. Illustrations, especially fine-grained photographs, have a resolution of 3000 dots per inch (dpi) or better. Standard home printers have a resolution of 300 — 600 dpi. The best computer monitors, however, have a resolution only in the 200 - 300 dots per inch range, and typical monitors range from 70-120 dpi. A prestigious "Display Product of the Year" award was won in 1999 by a Silicon Graphics flat panel monitor with a resolution of only 110 dpi.

In addition to the obvious consequences (a high-quality art book, for example, can show detail impossible on a computer screen) there are other considerations as well. Resolution influences character shape and size, edge sharpness, and stroke width; inter-character and inter-line spacing; and all the other fine detail that characterizes typography.

Book: Direct physical interaction. Computer Screen: Indirect logical interaction.

A reader interacts with a book by manipulating it directly, touching and moving the cover and pages. A user almost always interacts with a computer screen at a remove, by manipulating a peripheral input device. Even when using a touch screen the user is logically manipulating an image on that screen, not the screen itself.

Other

Other purely perceptual differences include:

Book

Computer Screen

Stable image

Flickering image

Reflected light

Emitted light

Flat image

Parallax distortion

Matt surface (usually)

Reflective surface

Variable orientation (vertical vs. horizontal)

Fixed orientation

Ergonomic considerations (quite different for desktop displays and the various classes of handheld and portable devices) include the distance between the reading material and the reader, the angle of the reading material relative to the reader's head and body, the angle of the reading material on the reader's retina, the curvature of the computer screen, image distortion in the screen's corners, and the posture of the reader.

Ontological Differences

Ontological differences relate to the fundamental nature of paper and computer screens, and distinguish between the inherent characteristics of the two media.

Book: Permanent. Computer Screen: Transient.

Once ink is laid down on paper it stays there. A book is a relatively permanent artifact; a page once read will be identical on the next reading. The page continues to exist, unchanged, even between closed covers. This is not true for a computer screen. The glowing phosphors may illuminate for only an instant. A screen once read may easily change or even disappear before the next reading. After exiting a book-reading application or turning off the computer, the electronic page no longer exists..

Book: Static. Computer Screen: Dynamic.

Once ink is laid down on paper it does not move. Books are static artifacts. The content of any given page, along with all formatting such as typeface, font, size, spacing, margins, chapter identifiers or page markings, is fully determined during printing and remains unchanged throughout the life of the book. A computer screen, by contrast, can change. The reader can often adjust formatting. More radical and dramatic effects are also possible, such as scrolling text, animation, video, and other moving images.

Book: High affordance. Computer Screen: Low affordance.

Don Norman defines "affordance" as "the perceived and actual properties of a thing, primarily those fundamental properties that determine just how the thing could possibly be used." [Norman 1988] A book is a relatively simple artifact and can be used effectively with a minimum of training. Knowledge of how to use a book transfers readily from one volume to the next; once learned, no further lessons are necessary. This is emphatically not the case for a computer screen. Even once the standard set of input devices (keyboard, mouse, trackball, touch pad, touch screen, joystick) has been learned there is no guarantee that pressing a particular key will have the same effect in Microsoft Word or Living Books, or that clicking on visually similar icons will have similar results (not to speak of clicking once or twice, the right, middle, left, or only mouse button.) Even if tighter standards for menu selection, icon design, shortcut definitions, etc. were adopted and universally implemented, the affordance would still be by convention only, social rather than natural.

Book: High portability. Computer Screen: Mixed portability.

Most books (with the exception of oversized or exceptionally heavy volumes) can be moved with relative ease, even while reading. Though computer display devices have become far more portable in recent years, many are still confined to a desktop.

Book: Ease of annotation. Computer Screen: Possibility of annotation.

Making marks on a book — whether highlighting passages, jotting comments in the margin, dog-earing a page, or even tearing out a section — is simple and unexceptional. Annotating a computer screen is far more complex, and may or may not be possible depending on the software being used at the time.

Book: Reader in control. Computer Screen: Variable locus of control.

When reading a book the reader has full control over the interaction. He or she determines when to turn the page, how often to reread a particular sentence or paragraph, when to check the footnotes or return to the table of contents. Any user action is easy to reverse.

When reading from a computer screen this is not always the case. Typically control is a shared activity, where a user action produces a computer response, then the computer pauses for the next user action, etc. Poorly designed software often has too many or too few pauses for user confirmation, or performs unexpected actions. Reversing previous actions may be difficult or even impossible.

Book: Physical pages. Computer Screen: Simulated pages, if any.

A book clearly segments the text into discrete units. This segmentation is entirely artificial, in the sense that the division is based on text length rather than the logical structure of the content, and is a function of pages size, character size, spacing, etc. — layout rather than substance. Physical pages do have two incidental virtues. They provide visual/positional memory cues ("I remember exactly where on the page that passage can be found") and facing pages provide a context for the fairly abrupt visual shift from one page to the next. There is no comparable context information for the even more abrupt visual shift that accompanies turning a page, but backing up and refreshing short-term memory by looking at the last line of the preceding page is relatively straightforward.

Current computer screens typically hold less text than a printed page, so software that enforces a strict page-by-page metaphor exacerbates the difficulties of context switching without compensating with facing pages or an obvious and simple backtracking option.

A computer screen, however, can do away with the concept of a page altogether. Line by line vertical scrolling is common. Less common text advancement techniques have also been prototyped and tested, sometimes with positive results. "Rapid Serial Visual Presentation" (RSVP) flashes text at a fixed location on the screen, one word or short phrase at a time. Some research indicates that readers can perform approximately as efficiently with RSVP as with more common page-format reading. "Times Square" horizontal scrolling, like that found on many electronic billboards, may also work well when the scrolling is smooth, pixel by pixel, rather than jerky, character by character. [Muter 1996]

Book: Just a book. Computer screen: Full processing power of the computer.

The most profound ontological difference between books and computer screens as media, of course, is that the book is a fairly simple, single-purpose artifact, optimized for a narrow range of actions. The computer screen, by contrast, is a window into a general-purpose processing device that may not be optimized for anything in particular but can be retasked in a seemingly unlimited variety of ways. Even a relatively focused computing device such as an e-book or other information appliance can have all the power of a programmable chip behind the display.

Residual Differences

For the purpose of this discussion, a residual difference is defined as the effect that reading from a particular medium has on the reader, and describes what remains after the act of reading is complete. These characteristics are in many ways the most interesting, but the hardest to define and study unambiguously. The scientific studies that have been done are not conclusive, and strong disagreements remain among researchers.

The commonly studied residual effects are:

Reader Fatigue

Early studies showed that reading from a computer screen tired the reader more than reading from reasonable quality printed material. Visual fatigue was usually ascribed to awkward character rendering, poor choice of line length and other typographical elements, screen flicker and slow refresh rates. Physical fatigue is usually attributed to awkward posture and repetitive hand or finger movements.

More recent studies have noted a reduction in user fatigue, usually attributed to better quality displays. Most researchers still agree, however, that reading a computer screen is more physically taxing than reading a book. [Dillon 1992, Shneiderman 1998]

Reading Speed

Early studies showed a degradation of fifteen to thirty percent when comparing reading speed between a book and a computer screen. Though this is widely accepted in the literature, some studies do dispute these results [Dillon 1992, Muter 1996, O'Hara & Sellen 1997, Shneiderman 1998].

Reading Accuracy

Accuracy of reading is difficult to test empirically. Most studies use a proofreading task to measure reading accuracy, where they substitute misspelled, grammatically incorrect, or logically inappropriate words in a textual passage, and measure the reader's success rate in recognizing these errors.

Several studies show modest but statistically significant lower accuracy when reading from computer screens, others show little difference. [Gujar et al 1998, Shneiderman 1998]

Reading Comprehension

A number of studies show that user recollection and understanding of written materials is reduced when the material is read from a computer screen. One article suggests that the ease of annotating paper documents significantly enhances real-world comprehension and retention tasks [O'Hare 1998]. Another quite recent study shows a noticeable advantage for paper over computer screens in measures of author credibility, article understandability, article interestingness, and argument persuasiveness. [Murphy 2000]

Other studies show little or no difference [Dillon 1992].

User Preference

In almost all the studies where subjective user satisfaction measures were reported, the users expressed a preference for paper over a computer screen. This preference was stated even when the empirical data showed no advantage for the paper product [Gujar et al 1998]. One study [Sellen & Harper 1997] conducted an ethnography to determine long-term use of different media in a document-intensive organization. It found that the subjects of the study spent 97% of their work time writing, reviewing, and editing documents, sometimes alone and often in collaboration with colleagues. These were discretionary users who, in almost all cases, had a free choice between using printed materials or on-line materials. 51% of the time only paper listings were used; 35% of the time the same document was used in both paper and on-line forms, and only 14% of the time was the entire writing and rewriting process conducted on computers.

Conclusions

In looking over the various characteristics listed above, it appears that paper has an edge in the perceptual area; paper may have an edge in the residual area (it is instructive to note that among all the conflicting and ambiguous results reported in the research, not a single study showed a clear advantage for computer users); and the results are mixed in the ontological area, with paper having certain advantages but computer screens suggesting intriguing possibilities.

This makes sense, especially when comparing the adoption of computing technology to other historical introductions of new media types. When still photography was first introduced the exposure times required for a picture were so long that only still images (Matthew Brady's Civil War post-battle fields) or carefully staged portraits were possible. In essence photography used the techniques of the painter. After the technology had matured sufficiently, stilted poses gave way to natural settings and movements, and even high-speed sports photography became feasible. A similar evolution took place with early movies — many of the first motion pictures simply captured stage productions. As cinematographers learned their craft and the tools improved, movie making became an art form in its own right. Even to this day, however, painting and theater remain viable, vibrant forms of artistic expression. The newer media moved to occupy a different, previously non-existent niche.

One can expected a winnowing process similar to that of other media introductions to take places with literary expressions hosted on computing devices.

The perceptual advantages of paper products are unlikely to disappear. Given the remarkable stability of human sensory and cognitive abilities, it would be foolish to assume that technological change will eliminate all the drawbacks of computer screens identified to date. Paper will remain physiologically easier for people to read. Thus books will always remain the medium of choice for the presentation of lengthy streams of text intended for thorough and largely sequential reading. Novels are, quite simply, extremely well adapted to book form. Computer screens will never replace these bound volumes.

What is likely to change dramatically is the mechanism that produces books and delivers them to consumers. No longer will fixed-size production runs in centralized printing plants generate output that is disseminated through bookstores and (sometimes on-line) catalogs. Instead, the new production model will be volumes printed on demand by a distributed network of printing and binding machines able to access virtually any source text on-line and produce a book tailored to individual customer desires. This will include specifying content (which short stories, poems, or commentary to assemble in a collection) as well as format (type size, paper quality, binding, suite of illustrations, etc.)

Such a computer-enabled manufacturing process will, however, still produce books, books not much different than those read by Swift and Defoe.

More interesting to imagine is what artistic developments might be introduced that require computer screens for their presentation. At the present time computing technology is better suited for reference material than for narrative. It is uncertain how long this will last. Given the remarkable rate of change in all aspects of computing, it would be foolish to make any grand pronouncements or predictions based only on the characteristics of today's machines.

Literary computing will need to find its own creative niche where the unique characteristics of computing technology can be exploited to their full advantage. In order for this to take place, one or more conceptual artistic breakthroughs will be required. Radical creative paradigm shifts are even less predictable than technological ones. It is certain that we can anticipate a rich future. We just can't anticipate the form it will take.

References

Dillon, Andrew (1992) Reading from Paper versus Screens: A Critical Review of the Empirical Literature. Ergonomics: 3rd Special Issue on Cognitive Ergonomic, 35 (10) 1297-1326.

Dillon, Andrew (1996) Myths, Misconceptions and an Alternative Perspective on Information Usage and the Electronic Medium. Draft version of chapter in Rouet, J.F, Levonen, JJ, Dillon, A.P. & Spiro, R.J. (Eds.) Hypertext and Cognition. Hillsdale, NJ: Lawrence Erlbaum Asssociates. http://www.slis.indiana.edu/adillon/adillon-myths.html

Gujar, Anuj U. & Beverly L. Harrison, Kenneth P. Fishkin (1998) A Comparative Evaluation of Display Technologies for Reading. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 527-531.

O'Hara, Kenton & Abigail Sellen (1997) A Comparison of Reading Paper and On-Line Documents. Proceedings of CHI '97: Human Factors in Computing Systems, 335-342.

Murphy, P. Karen, Joyce Long, Theresa A. Holleran, Elizabeth Esterly (2000) Persuasion Online or on Paper: A New Take on an Old Issue. In C. S. Sperl (Chair), Changing Knowledge and changing beliefs: An examination of academic development, epistemology, and persuasion. Symposium to be presented at the annual meeting of the American Psychological Association, Washington, DC. <http://www.coe.ohio-state.edu/pkmurphy/Research/Presentations/Traditional versus computerized texts.doc>

Muter, Paul (1996) Interface Design and Optimization of Reading of Continuous Text. In van Oostendorp, H & de Mul, S. (Eds.) Cognitive Aspects of Electronic Text Processing. Norwood, NJ: Ablex Publishing Corp.

Norman, Donald A. (1988) The Design of Everyday Things. New York: Doubleday / Bantam Doubleday Dell Publishing Group, Inc.

Sellen, Abigail, Richard Harper (1997) Paper as an Analytic Resource for the Design of New Technologies. Proceedings of CHI '97: Human Factors in Computing Systems, 319-326.

Shneiderman, Ben (1998) Designing the User Interface, 3rd Edition. Reading, MA: Addison Wesley Longman, Inc.

 

Last Modified Date: July 19, 2008