Computer Networks and ISDN Systems 21 (1991) 315-319 315 North-Holland

Multimedia and high speed networking in MultiG

Bj~rn Pehrson and Stephen Pink Swedish Institute of Computer Science, Box 1263, S-164 28 Stockholm, Sweden

Local reviewer: Bengt Ahlgren, Swedish Institute of Computer Science.

Abstract

Pehrson, B. and S. Pink, Multimedia and high speed networking in MultiG, Computer Networks and ISDN Systems 21 (1991) 315-319.

This paper broadly describes some of the research activities in MultiG, a research program in multimedia and high-speed networking in Sweden, including those on multimedia applications using digital audio and video mixed with traditional applications, efficient implementation of network and transport protocols for gigabit networks (i.e., 6pm, the SICS Protocol Machine) and the MAC-level protocol, PTM, which is being designed and implemented for use on a fiber optic network.

Keywords. Gigabit networking, multimedia, efficient protocol implementation, media synchronisation.

1. I n t r o d u c t i o n : M u l t i G a p p l i c a t i o n s

MultiG is an ambitious research program in the area of multimedia applications and high-speed networking in Sweden in broad cooperation be- tween academia and industry. The goals are to strengthen the academic infrastructure and in- dustrial competitiveness, to integrate the major research sites in the Stockholm area, and to dem- onstrate operating prototypes of novel applica- tions and network concepts.

We focus on distributed workstation environ- ments supporting multiple users in concurrent co- operative editing of multimedia documents. The applications involve: - multimedia communication between multiple users. In the first step, a simple store-and-forward mail service and a real-time interactive talk service will be combined and extended to a - P ic tu rePhoneTa lk /mul t imed ia -mai l service (pptalk) based on general multimedia documents containing text, graphics, audio and video. In the next step, group communication and video con- ferencing (round table) will be supported. - distributed editors for multiple users including Commitment Control and Recovery (OSI/CCR),

featuring a replicated wyteboard. We will focus on a combination of traditional text editors and lan- guage oriented editors for graph-oriented graphic languages, in particular languages for specification of distributed systems [2,11]. - mult imedia m a n - m a c h i n e communicat ion which can extend the usability of new design tools (voice, handwriting, gestures, etc.).

The pptalk-service includes directory searching, addressing, QOS-negotiation and various repre- sentations of multimedia documents. A user will be able to access (or be reached via) the network anywhere from any kind of terminal equipment without having to remotely log in to a specific host, since it may be down at the moment.

If user A calls user B and B is logged in, the most advanced type of connection that both terminals (ranging from workstations of various capability to standard telephones) can cope with should be established. If B is not logged in or reachable via phone, A should be offered a mail editor.

If we combine the replicated wyteboard for concurrent editing and the pp ta lk / round table service, we have to synchronise the audio/video streams with the wyteboard replication and its

0169-7552/91/$03.50 © 1991 - Elsevier Science Publishers B.V. (North-Holland)

316 B. Pehrson, S. Pink / Multimedia and high speed networking in MultiG

edit operations. People can edit concurrently while talking to each other and perhaps seeing each other. This is an example of directly integrating an isochronous application with one that demands strict replication.

While using the distributed syntax-driven edi- tor, multiple users can work concurrently without interaction until they get into a conflict, e.g., in- consistent changes of mutual interfaces. When a conflict of a kind that should be resolved interac- tively occurs, a pptalk-connection could automati- cally be established. This involves directory check- ing, QOS-negotiation and a choice between inter- active and store-and-forward communication. It also involves synchronisation between the video- channel, the voice-channel, and the editing oper- ations, e.g., " I want to change this variable," (where the semantics of this is defined by the position of the mouse).

lnterprocess communication mechanisms

It is the operating system, traditionally, that has offered network services to the application. An application makes a request to the operating system kernel to open a connection to another host or to send a message to a network peer. But

as the properties of network traffic change and applications request support for complex multi- media services, the operating system should offer richer abstractions than connections and message service. Thus we define the notion of an "activity" in which an application participates, supported by the operating system and its network resources. An activity is an abstract description of a scene, the players in which are the applications that either request its creation or request to join. The description of the activity ultimately boils down to QOS requests that are satisfied or not depending on resource availability determined by the distrib- uted operating system. One such activity could be described as a round table wyteboard work ses- sion. This activity requires both one-way isochro- nous streams and reliable two-way connections. It demands deadline scheduling and synchronisation between its isochronous and reliable aspects. It is reasonable to see the notion of an activity as the abstraction for many kinds of event sequences present in computer supported cooperative work applications. This sort of activity presents run-time and network designers with a novel problem: het- erogeneous synchronisation, i.e., synchronising network traffic whose components have totally different transfer characteristics.

A number of attempts have been made to de-

Bjrrn Pehrson received his BSc. from the University of Stockholm in 1966. After some years with IBM Nordic Laboratories he received his MSc. in 1969 and his Ph.D. degree in automatic control in 1975 from the University of Uppsala. During the period 1969-1985 he worked as a lecturer, researcher and professor at Uppsala University in the area of methods and tools for design of distributed systems. Since 1985, when the Swedish Institute of Computer Science (SICS) was formed, he has been managing the Distributed Systems Laboratory at SICS, and is also an adjunct professor in computer networks at the Royal Institute of Technology in Stockholm.

Stephen Pink has been a researcher in the Distributed Systems Laboratory of the Swedish Institute of Computer Science since 1988. After getting a Bachelor's degree from the University of Minnesota (1970) and a Master's degree from CorneU University (1973) he was a faculty member at the University of Rochester (1974-1979) and has worked as a software engineer at Cadmus Computer Systems and Symbolics, both in Massachussetts. His main research interests are in high speed networking, efficient protocol implementation and operating systems.

B. Pehrson, S. Pink / Multimedia and high speed networking in MultiG 317

sign synchronisation mechanisms for isochronous streams [12,13], the focus of attention being the "lip-synch" necessary to support an integrated digital audio and video service. Here, the streams are viewed as separate simplex channels whose synchronisation points are manipulated mostly at the sink ends. Separation of communication chan- nels is necessary because of the different QOS parameters belonging to each stream, and the con- sequent degradation of either the audio or video in having to "catch up" with the other [1].

Mechanisms th~it deal with the synchronisation of simplex isochronous streams might not, how- ever, manage their synchronisation with the relia- ble two-way byte-streams or multicast reliable datagram connections that are used to support the concurrent editing application described above. Many of the present solutions for synchronisation assume point-to-point or unicast traffic, where round tables are by nature group oriented. Such solutions synchronise at the sink end of the stream, where in a round table mixed traffic scenario it would be more efficient to synchronise at one or more of the source ends. The initiator of the traffic may be the only one to know when a request /reply operation is completed, so the ini- tiator should be held responsible for blocking the isochronous streams when synchronisation is nec- essary.

The various transfer characteristics of multi- media applications imply that there are a number of appropriate models for interprocess communi- cation: streaming mechanisms for isochronous traffic, reliable byte streams for bulk transfers, causal broadcast (as used in the Isis toolkit from Cornell University [3]) for control of wyteboard replication, shared memory and remote procedure calling. As in all parts of the MultiG architecture, we will carefully measure the overhead created by the use of these mechanisms, trading functionality for performance only when the situation demands it. For example, traditional remote procedure cal- ling sets a lower limit on network latency. The synchronous request /response nature of the mechanism may itself prove to be a bottleneck in a low latency network, when transfer speeds begin approaching propagation delay boundaries. Thus, either a tradeoff is made between performance and convenient programming paradigm, or a rede- sign of the basic mechanism should be considered. It has been suggested that programs instead of

procedures (i.e., larger chunks of computations) should be sent and waited for by a client in a request/response communication [14].

High speed protocol processing and the 6pm

To a large extent, the design of the transport protocol must match the network service below. Our medium speed networks have so far been stateless, in the sense of offering only datagram service. It is tempting to continue using the data- gram model as the speed of the network increases. But there have been powerful arguments advanced that a stateless datagram network (such as T C P / I P provides) would be helpless in the face of conges- tion created within the round-trip time of a high speed network [18]. Others have argued, however, that the design of present datagram networks can be subtly altered to provide the congestion control needed [19]. We consider this to be an experimen- tal issue, and as far as possible, we will be par- ticipating in the experiment. We will start building our network with T C P / I P , but new transport protocols such as TP + + from Bellcore [7], desig- ned for connection-oriented networks, as well as VMTP [5] for connectionless networks, will be considered.

Current research [6], including our own [9,15,17], has shown that implementation tech- niques for middle layer protocols play a large part in their performance. We have therefore initiated a project within MultiG called 6pm (SICS Proto- col Machine) to realize the middle layers of our protocol architecture. The 6pro will, among other things, provide the network interface to the high speed network below, opening up that traditional bottleneck. Using dedicated memory and its own set of processors, a multi-threaded implementa- tion of the protocol stack from transport down to medium access, and real time scheduling, the 6pm will provide host applications with a high per- centage of the available bandwidth of the network.

The first implementation of the 6pm will be a four processor (Motorola 88000-family) node of the Data Diffusion Machine (DDM) [10]. The DDM is a general purpose multiprocessor, being built at SICS, with a hierarchical structure and local memory caches providing a virtual global memory through an efficient cache coherency pro- tocol. The software architecture of the 6pm will be

318 B. Pehrson, S. Pink /Multimedia and high speed networking in MultiG

integrated with the DDM by having the 6pm run the same operating system kernel as the DDM, in the first instance the Mach kernel [16], and by having the middle layer protocols modules execut- ing as dedicated Mach threads [4] scheduled spe- cially for the needs of protocol execution.

We believe that integrating the software and hardware of our protocol machine with that of the host computer will eliminate much of the latency introduced by the interrupts and network/host copies seen in many outboard protocol processors. In particular, the 6pm should limit the number of copies from network to application to one. In many protocol processing systems there are two sources of extra data copying.

First, there must be a copy of the data from network interface to system buffers (or vice versa) to insure that they are aligned on buffer boundaries. This copy is necessary because the (transport) protocol headers encapsulated within the medium access layer frame can be of varying length. This occurs, in turn, because most protocol processors attempt to be generic in terms of the transport protocols which multiplex upon them, and no provision is made to group data packets in memory by protocol headers, making it necessary for the protocol processing software to copy align the data. The 6pm will avoid the cost of this extra data copy by reading and writing all packets be- longing to a particular connection to and from a contiguous section of memory. Since it is guaran- teed that there will be only one transport protocol per connection, the protocol server (and hence the user process) will know at connection setup time the size of the protocol header and will be able, in the case of receiving, to map its buffers onto the user data that are arriving at the network inter- face, and in the case of transmitting, to align its user buffers so that they can be sent directly onto the network with no extra copy.

Second, there is a copy from system buffers to user buffers. (This is true in many UNIX and UNIX-like systems.) This copy can be eliminated using the copy-on-write sharing available in the Mach operating system. The user application can share data with the 6pm protocol server, which will also run as a user program, simply by map- ping its buffers to those of the server. If the user application writes to the shared data, a copy is made only of the particular page of virtual mem- ory that is altered. The size of the (software) page

is determined at boot time, and can be adjusted to be a power of two of the hardware page size for that processor. (In the case of the Motorola 88200, the hardware page size is 4096 bytes.) In most cases, the amount of data copied after a copy-on- write fault is much less than the amount that ends up being copied when all data from a network session is copied from system to user buffers.

The optimizations described above allow the data on a network connection to flow directly into the waiting buffers of an application, thereby avoiding the latency of all copies but the one necessary to make the network transfer. This should help satisfy the low-latency and low-jitter requirements of applications which use digital audio and video. It also leaves in the hands of the application, which after all knows best, what the error recovery strategy will be for a particular connection. We believe that this is the most flexi- ble approach for host protocol processing in a high speed, mixed multimedia environment.

L o w e r l a y e r s

In order to have appropriate and stable operat- ing testbeds for application development in the short run, the plants are to move on from the LAN/FDDI- techno logy to ATM-technology pro- vided by our industrial partners. The research activities in the lower layers, however, will be focused on new technology operating in the multi-gigabit range.

A new T D M technique, PTM, Programmable Transfer Mode [8], is being designed and imple- mented. PTM is a hybrid between STM and ATM with more dynamic bandwidth allocation than STM and without the performance degrading need to process a header in each cell as is necessary in ATM. Some basic ideas are: - preallocate bandwidth (time-slots) to all net- work nodes to avoid competition about a common resource when it is most harmful, and provide a mechanism for fast background reallocation. - use circuits to concentrate all processing in a connection establishment phase and make the data transfer phase extremely simple. Circuit connec- tions allow resource reservation to be made in advance, and the ordered delivery of data in the network preserves the synchronisation mecha- nisms achieved in the higher layers.

B. Pehrson, S. Pink / Multimedia and high speed networking in MultiG 319

- connect each node to more than one fiber and provide switching. - use wavelength multiplexing to increase the use of each node.

PTM provides a synchronous network without internal buffering. Connections could be regarded as DMA-channels between memory spaces ad- dressable by the communicating hosts.

In our first PTM network prototype, each node can connect to three dual fiber buses. The address- ing scheme allows 1D, 2D or 3D (cubic) mesh networks. PTM does not, in contrast to, e.g., DQDB, require optical termination in each node. This means that, in combination with wavelength multiplexing and switching, the fiber media can carry traffic in the Terabi t / s range, but nodes can still operate in the Gigabit /s range.

C o n c l u s i o n

We have attempted to describe some of the main research topics in MultiG. The application we have focused on is PicturePhoneTalk (pptalk) with concurrent editing. As underlying support for this application (and others) we are building a high speed network, capable of multi-gigabit per second operation, and an efficient protocol ma- chine to widen the bottlenecks in traditional pro- tocol processing and host /network interfaces. Thus the MultiG project is working toward a broad solution to the problems of future multi- media applications and high speed networks.

R e f e r e n c e s

[1] D.P. Anderson, R. Wahbe, The DASH network communi- cation architecture, Report No. UCB/CSD 88/462, November 8, 1988, Computer Science Division (EECS), University of California, Berkeley.

[2] B. Backlund, O. Hagsand and B. Pehrson, Generation of graphic language-oriented design environments, SICS Re- search Report, R89011, ISSN 0283-3638.

[3] K. Birman and T. Joseph, Exploiting virtual synchrony in distributed systems, in: Proc. ACM SIGOPS SOSPll (1987) 123-138.

[4] D.L. Black, Scheduling support for concurrency and parallelism in the Mach operating system university, IEEE Comput., May (1990) 35-43.

[5] D.R. Cheriton, VMTP: A transport protocol for the next generation of communications systems, in: Proc. ACM SIGCOMM, "88, Stanford, CA (August 1988).

[6] D. Clark, V. Jacobson, J. Romkey and H. Salwen, An analysis of TCP processing overhead, IEEE Comm. Mag. June (1989).

[7] David C. Feldmeier, Multiplexing issues in communica- tion system design, in: Proe. ACM SIGCOMM 90, Philadelphia, PA (September 1990).

[8] L. Gauffin, L. Hakansson and B. Pehrson, Multi-Gigabit networking based on PTM, A TDM, medium access tech- nique with dynamic bandwidth allocation, in: Proc. 1st MultiG Workshop, Stockholm (1990).

[9] P. Gunningberg, M. Bjorkman, E. Nordmark, S. Pink, P. Sjodin and J. Stromquist, Application protocols and per- formance benchmarks, IEEE Comm. Mag. June (1989).

[10] S. Haridi and E. Hagersten, The data diffusion machine, in: Proc. Parallel Architectures and languages Europe (1989).

[11] G.E. Kaiser, S.M. Kaplan and J. Micallef, Multiuser, distributed language-based environments, IEEE Software November (1987).

[12] C. Nicolau, An architecture for real-time multimedia com- munication systems, IEEE J. Selected Areas Comm. 8 (3) (1990).

[13] R. Steinmetz, Synchronization Properties in Multimedia Systems, IEEE J. Selected Areas Comm. 8 (3) (1990).

[14] Sun Microsystems, Inc., Network Extensible File System Protocol Specification, Mountain View, CA. Feb. 12, 1990.

[15] E. Nordmark and P. Gunningberg, SPIMS: a tool for protocol implementation performance measurements, in: Proc. 13th Conference on Local Area Networks, Minneapo- lis, MN, (October 1988).

[16] R. Rashid, A. Tevanian, M. Young, D. Golub, R. Baron, D. Black, W.J. Bolosky and J. Chew, Machine-indepen- dent virtual memory management for paged uniprocessor and multiprocessor systems, IEEE Trans. Comput. August (1988).

[17] P. Sjodin, P. Gunningberg, E. Nordmark and S. Pink, Towards protocol benchmarks, in: H. Rudin and R. Wil- liamson, eds, Protocols for High-Speed Networks (selected papers from IFIP WG 6.1/WG 6.4, Zurich, May 1989) (North-Holland, 1989).

[18] L. Zhang, A new architecture for packet switching net- work protocols, Ph.D. Thesis, Electrical Engineering and Computer Science Dept., Massachussets Institute of Tech- nology, Cambridge, MA, July 1989.

[19] Van Jacobson, presentation made at the Internet Research Steering Group Workshop, Cambridge, MA, Jan 1990. Available as lnternet Request for Comments (RFC) 1152, SRI Network Information Center, Menlo, Park, CA.