Journal of Microcomputer Applications (1992) 15, 13-I 9

Adapting a programming editor (EMACS) for persons who are blind: issues, initiatives and problems

Walter Maner

Department of Computer Science, Bowling Green State University, Bouding Green, OH 43403 I USA.

This paper describes current research in the ‘programming by ear’ project begun by B. York and recently continued by the author. This research uses an extensively modified EMACS text editor to address general problems which constrain the effective design of human-computer interfaces for blind programmers. These include (1) an information conversion problem, (2) an information reduction problem, and (3) an information compression problem. Recent work has focused on the problem of providing an auditory equivalent for rapid visual browsing of program source code.

1. Introduction

This paper describes research initiatives made in response to three important issues which constrain the design of human-computer interfaces for blind programmers. We adopt the simplifying assumption that programming is a text-editing activity and ignore, for the purposes at hand, many valid issues regarding access to graphical computing environments. We further limit ourselves to the EMACS [l] family of text editors used by many professionals who program daily in the UNTXTM environment. Within this simpler text-only environment, one might assume that the mere addition of ‘screen reader’ technology to the standard programming environment would solve the access problem for blind programmers. This is not the case, however.

For the sake of argument, we will concede that blind programmers may be equally or better prepared to deal with data and procedural abstractions than their sighted counterparts. Everything else being equal, this could justify redoubled efforts to train visually handicapped persons for the computing professions. Perhaps this belief has motivated the work of the Association of Rehabilitation Programs in Data Process- ing [2]. Certainly we do not wish to deny the impressive successes of blind programmers such as Paul Bourgoin [3], Barbara Wegreich [4], and David McKenzie [5]. Despite their singular achievements, it does not appear to us that the ceteris paribus assumption is valid. Everything is not equal, not yet.

2. Three issues

We have sought to equalize opportunities for blind programmers by dealing with three interrelated issues involving information conversion, reduction and compression.

13 0745-7138/92/010013 +07 003.00/O 0 1992 Academic Press Limited

14 W. Maner

2. I Information conversion

Kokjer reports a human visual bandwidth of l,OOO,OOO bits per second. By comparison.

the human auditory bandwidth is only 10,000 bits per second, and the tactile bandwidth is a paltry 100 bits per second [6]. York and Karshmer state the issue crisply: ‘How can we design a programming support environment for individuals whose input bandwidths are two to four orders of magnitude less than their sighted counterparts in such a way as to permit comparable productivity levels?’ [7]. We agree emphatically with Millar that the question is not what the blind ‘can’ or ‘cannot’ do: ‘The question is rather what aspects of information depend on the sensory source by which the information is

conveyed, and how these can best be substituted.’ [8] We may call this the information conversion issue.

2.2 Information reduction

There is a comparable information reduction issue. We have seen successful sighted programmers scanning long listings of program source code at a prodigious rate, perhaps in excess of 600-1000 ‘words’ per minute. Presumably, they achieve such rates

by selectively attending to the major information-bearing tokens and selectively ignoring minor syntactic details. In some programming languages, readers can use text inden- tation (‘whitespace’) to regulate the depth of skimming. What is the auditoqv equivalent of such visual scanning? Clearly, piping unfiltered screen-fulls of text onto a high-speed

voice synthesizer is not a complete solution. Of course, it is important that synthesizer hardware be capable of operating in a ‘fast forward’ mode, but this capability alone does

not duplicate the structural and contextual information sighted programmers glean from skimming code.

2.3 Inform&on compression

In addition to the problems of information conversion and reduction, there exists an information compression issue. The sighted programmer is able to make good use of the visual display as an extension of working memory. The persistence of information on the screen and its random accessibility reduce the cognitive effort required to refresh short- term memory. This allows sighted programmers to cope effectively with the well-known

limit on items which can be held in working memory: seven, plus or minus two [9]. Since this persistent and accessible screen display is not available to blind individuals, they will need to optimize their use of scarce short-term memory through other means, chiefly through information compression.

3. The ‘programming by ear’ project

Elkes [lo] is right in wanting to address these issues on many fronts, from the re-design of the basic interface to the re-making of tools needed to engineer software within that environment.

The remainder of this paper describes such an environment. Essentially, it consists of an interface to EMACS which we have called Programming By Ear (PBE). It reflects research conducted by Bryant York at Boston University over a 4-year period, 1986 1989 [1 I]. Although York has moved on to other projects, his work has been continued by the author at Bowling Green State University. York’s efforts were inspired by Arthur

Adapting EMACS for the blind IS

Karshmer of New Mexico State University, creator of the ‘talking tactile terminal’ [ 121. The main goal of PBE is to ease the task of programming for blind individuals and, in the process, to open the field of computer programming to them. Sighted programmers may also benefit, but this has not been the focus.

4. Research initiatives

In this section, we limit ourselves to discussion of those PBE research initiatives which most directly address the issues of information conversion, reduction and compression.

4.1 The EMACS environment

PBE is a multi-sensory specialization of the programming environment normally provided with EMACS style text editors. The EMACS editor divides the screen into separate windows. Each window provides a view, often only a partial view, of a stream of characters stored in a scrollable buffer. Buffers can import from and export to files, or they can receive their content from keyboard input. Most of the machinery created for PBE either modifies or creates text for EMACS buffers. Users can silently browse the contents of these buffers using regular EMACS navigation commands or they can pipe all or part of the contents through a speech synthesizer. Text created only for immediate voice output is written to a virtual (invisible) buffer, which dumps it directly into the input stream for the voice chip. The windowing environment is currently programmed almost entirely in Lisp, the macro language which underlies EMACS. We make extensive use of EMACS Lisp primitives, such as those which hide portions of buffers from view, hence from being voiced.

4.2 Our basic thrust

A programming environment for the visually handicapped must aim to increase the rate, volume and utility of information intake. We improve rate by increasing information density. We improve volume by increasing available bandwidth. We improve utility by reducing extraneous noise. Sighted programmers may ease the strain of a high band- width medium (vision) by choosing to deal with lower information densities. Blind programmers, on the other hand, my ease the strain of a low bandwidth medium (touch or hearing) by choosing to deal with higher information densities. Both will want a low- noise channel.

4.3 Spec$c design problems

Various combinations of these methods may be needed to solve specific interface design problems. For illustration, we identify and discuss several prominent difficulties.

4.3.1 The visual structure problem. Consider the varying quantities of vertical and horizontal whitespace (blank lines and blanks) used by programmers to reveal the structure inherent in their code. It does not seem that a simple mapping of visual whitespace to auditory pause time will convey the meaning this whitespace has for sighted programmers. Braillers will have a better clue, but forming an impression of overall structure will still be difficult.

Our recent efforts to solve this structure-browsing problem have focused on design of

16 W. Maner

a ‘stubbing’ filter to allow hiding of detail. The approach was inspired by programmers’ use of empty procedure bodies or ‘stubs’ during early stages of testing. Our automatic Stubber condenses program text based on shallow analysis of language-specific entry and exit syntax, such as ‘BEGIN’ and ‘END’, or open and close punctuation. The Stubber can instantly shrink program text (‘zoom out’) to a skeleton by hiding any code lodged between paired entry and exit elements. Ellipses dots replace the hidden code. Stubbed sections can be unpacked one level at a time (‘zooming in’), or an entire buffer can be unpacked. This use of the Stubber assumes that entry and exit syntax is correct, including comment syntax, but the program may contain any other kind of syntax error. In an alternate mode, the Stubber can hide code based on the amount of whitespace used for indenting. The result is otherwise similar to the original mode, since changes in the amount of whitespace function effectively as entry and exit syntax. If a shrunken buffer is voiced, the blind programmer will hear only what their sighted counterparts would have seen. In its current form, the Stubber is a modest extension of the EMACS ‘outline’ mode.

4.3.2 The abstraction problem. Sometimes programmers need, not just local strut tural information, but a structural abstract of an entire program or module. The PBE project addressed this problem in accordance with Brooks’ theory of program compre- hension [13] by creating a Synopsizer similar in functionality to the C Abstractor [14]. The work has been described by Sinha [15]. If given a syntactically correct C program, the Synopsizer is capable of producing a synopsis at the module, function and block level. At the module level, for example, the Synopsizer repeats header comments, lists included files, distinguishes various kinds of identifiers, and names the functions defined in the module.

4.3.3 The search problem. Programmers may wish to search through code listings, not for literal strings, but for objects of a certain type (e.g. the next ‘case’ statement). York and Karshmer have described the mechanism they have used to broaden EMACS’ search and navigation resources [16]. An Earmarker adds invisible annotations to code in order to facilitate ‘search by construct’. It reads the parse tree and adds non-printing earmarks to the source code at points corresponding to nodes in the parse tree. There is a different type of mark for each different syntactic construct in the language. The search mechanisms normally present in EMACS have been extended to allow search of these hidden annotations. For example, a programmer may search earmarks to find any or all assignment statements containing a particular variable. Users may add earmarks of their own, in which case they become the auditory equivalent of Post-itTM notes.

4.3.4 The abbreviation problem. Considerable research has been devoted to the problem of determining when to provide diagnostic information during computer- assisted tutorials [17]. Too much information dispensed too frequently will break the continuity of the lesson. A similar but larger problem appears to exist for blind programmers. Relatively modest amounts of diagnostic output may disturb a program- mer’s train of thought is that programmer is forced to wait for the output to be voiced by a synthesizer. Some voice chips provide a means to cancel speech output, which is helpful but insufficient.

Adapting EMACS for the blind 17

The PBE project addressed this problem by designing a filter which can produce three

levels of abbreviation on output, three levels of expansion on input. During the creation of diagnostic output, the Abbreviator can map long phrases to pre-defined or user- defined short forms. During input, the Abbreviator can be made to work in reverse, mapping user-supplied short forms (‘macros’ or ‘aliases’) to their full-phrase equiva- lents. Users may adjust the level of abbreviation according to experience and familiarity.

Our expand-on-input mode merely extends the abbreviation facility already present in EMACS; the reduce-on-output mode represents an addition to the environment.

4.35 The voicing problem. Many of the more popular programming languages (e.g. C) offer poor speakability compared to natural languages. The problem is mostly due to punctuation, which voice synthesizers can voice or skip as the user requires. If

punctuation is voiced, piping lines of source code through the synthesizer produces an effect close to gibberish. On the other hand, if punctuation is silent, one may hear only a

barrage of run-together phrases. Our solution is to translate operators and other non-English tokens, including

punctuation, into canonical English phrases. Thus, ‘X : = 4;’ is voiced as ‘X takes four’ followed by a beep. Users may reconfigure the translation table to suit their preferences.

While this approach may seem straightforward, we have found that deeply parenthe- sized expressions can be difficult to understand no matter how they are voiced. We view Lisp as the worst case, so we rarely try to voice un-stubbed Lisp code.

Ultimately, for precise editing, we will want all punctuation to be voiced, so the paraphraser must be silenced at some point. A final difficulty is due to an oversight in the

process by which languages are standardized. National bodies normally standardize

national character sets but they do not standardize how various tokens of a program- ming language are to be pronounced. It is particularly difficult in the United States to reach any consensus about how to pronounce the various operators in the C language. This is as much a problem for those who read to the blind as it is for our research.

4.3.6 The yast forward’ problem. We have noted that sighted programmers often

want to scroll rapidly through source code, as rapidly as hardware permits. In a minute’s time, several thousand lines may scroll on and off the screen. They use the scroll key or some other repeating key as a ‘fast forward’ mechanism to position the visible window over the part of the buffer which requires their attention. Although some voice

synthesizers can generate speech at 700-1000 words per minute, even this accelerated rate is more than an order of magnitude slower than visual scrolling.

Currently, we are attempting to address this issue by providing a Hyperscanner. Invoking the Hyperscanner will cause only information-laden words to be sent to the synthesizer. We skip the frequently-occurring ‘noise’ words (e.g. articles and preposi- tions) which account for as much as 85% of English text. Obviously, this technique works better for comment text and documentation than for source code. For source code, the Hyperscanner treats language-specific reserved words, procedure names. and end-punctuation as information-laden. Depending on the language, this permits up to a 75% reduction in the volume of voiced output, which translates into an effective listening rate of 2000 words per minute. Once we are satisfied with the design of the Hyper- scanner, we plan to investigate its effect on program comprehension.

18 W. Maner

5. Project status

The various components of the PBE ‘system’ were created separately, using whatever hardware and software seemed appropriate for prototyping a particular component. Current efforts focus on integrating the various software components into a single programming environment. Within 2 years, we hope to make these components available to the Free Software Foundation for distribution with the rest of GNU EMACS. Before this can be considered, we must convert large amounts of VAX Lisp to equivalent EMACS Lisp (Elisp).

Acknowledgements

The author wishes to thank Bryant York and Arthur Karshmer for their co-operation and encouragement, and to acknowledge significant contributions made by others, including Shubba Chakravarthy, Patty DeCastro, Kenneth East, Yee Kwong Ngeow, Elizabeth Russel, Himaanshu Sinha, and Gale Haglund.

References 1.

2.

3. 4. 5. 6.

I.

8.

9.

10. 11.

12.

13.

14.

15.

16.

R. Stallman 1981. EMACS: the extensible, customizable, self-documenting display editor A 1 Memo 519a, Massachusetts Institute of Technology, Artificial Intelligence Laboratory, Cambridge, Massachusetts. S. M. Raucher 1981. A mind is too valuable a resource to waste. AEDS Monitor, 19/10-12, 8-9, 21. K. Mayo 1986. Promises to keep. Business Computer Systems 517, 24. B. Wegriech 1989. The PC is my lifeline. PC Computing 2/7, 87. Anonymous 1984. Blind programmer updating COBOL programs. Computerworld 18/17,28. K. J. Kokjer 1987. The information capacity of the human fingertip IEEE Transactions on Systems, Man, and Cybernetics SMC-17, 10&102. B. W. York & A. I. Karshmer 1989. An overview of T3-PBE: programming support for the visually impaired. SIGCAPH Newsletter 41, 17-20. S. Millar 1989. Perceptual and conceptual skills under visual handicap: perceptual and task factors in understanding braille and tactual graphs. Planning workshop on access to computers by blind individuals, October 47, Madison, Wisconsin. G. A. Miller 1956. The magical number seven, plus or minus two: some limits on our capacity for information processing. Psychological Review 6312, 81-97. J. G. Elkes 1982. Designing software for blind programmers. SZGCAPH Newsletter 30, 15-I 7. B. W. York 1987. PBE programming by ear, a programming environment for the visuahy handicapped. Boston University Computer Science Technical Report #87-009. A. I. Karshmer, H. R. Myler & R. Davis 1987. The architecture of an inexpensive talking-tactile terminal to aid the visually handicapped. Computer Standards and Interfaces 5 207-220. R. Brooks 1978. Using a behavioral theory of program comprehension in software engineering. Proceedings of the 3rd International Conference on Software Engineering, Atlanta, Georgia. Y. Chen & C. V. Ramamurthy 1986. The C information abstractor. Proceedings ufthe IEEE Computer Society’s Tenth Annual International Computer Software and Applications Conference (COMPSAC ‘86). H. S. Sinha 1988. PBE synopsizer: a synopsizer for the PBE environment. Boston University Computer Science Technical Report 88-006. B. W. York & A. I. Karshmer 1989. Tools to support blind programmers. Proceedings of the Seventeenth Annual ACM Computer Science Conference, February 21-23. Louisville. Kentucky.

Adapting EMACS for the blind 19

7. R. R. Burton & J. S. Brown 1982. An investigation of computer coaching for informal learning activities. Intelligent Tutoring Systems, 79-98.

Walter Maner is Associate Professor of Computer Science at Bowling Green State University in Bowling Green, Ohio, where he directs the artificial intelligence project. Prior to joining the faculty of BGSU in 1984, he held a joint appointment in computer science and philosophy at Old Dominion University in Norfolk. Virginia. Before that, he was a Fulbright Scholar at the University of Strasbourg and a Woodrow Wilson Fellow at Boston University. Maner co-chairs the National Conference on Computing and Values and is a board member of the Research Center on Computing.