Darpa Internet

8/8/2019 Darpa Internet

1/9

THE DESIGN PHILOSOPHY OF THE DARPA INTERNETPROTOCOL~SDavid D. Clark

Massachusetts nstitute of TechnologyLaboratory for Computer ScienceCambridge, Ma. 02139

AbstractThe Internet protocol suite, TCP/IP, was first proposedfifteen years ago. It was developed by the DefenseAdvanced Research Projects Agency (DARPA), and hasbeen used widely in military and commercial systems.While there have been papers and specifications thatdescribe how the protocols work, it is sometimes difficultto deduce from these why the protocol is as it is. Forexample, the Internet protocol is based on aconnectionless or datagram mode of service. Themotivation for this has been greatly misunderstood. Thispaper attempts to capture some of the early reasoningwhich shaped he Internet protocols.

I. Int reductionFor the last 15 years , the Advanced Research ProjectsAgency of the U.S. Department of Defense has beendeveloping a suite of protocols for packet switchednetworking. These protocols, which include the InternetProtocol (IP), and the Transmission Control Protocol(TCP), are now U.S. Department of Defense standardsfor intemetworking, and are in wide use in thecommercial networking environment. The ideasdeveloped in this effort have also influenced otherprotocol suites, most importantly the connectionlessconfiguration of the IS0 protocols, 3*4.

This work was support4n part y the Defense Advanced ResearchProjects Agency (DARPA) under Contract No. NOO OIJ-83-K-0125.Permission 10 copy without fee all or part of this material is granted providedthat the copies are not made or distributed for direct commercial advantage.the ACM copyright notice and the title of the publication and its date appear.and notice is given that copying is by permission of the Association forComputing Machinery. To copy otherwise. or to republish. requires a fee and/or specific permission.

o 1988 ACM 0-89791-279-9/88/008/0106 $1.50

While specific information on the DOD protocols is fairlygenerally available, 6. , it is sometimes difficult todetermine the motivation and reasoning which led to thedesign.In fact, the design philosophy has evolved considerablyfrom the first proposal to the current standards. Forexample, the idea of the datagram, or connectionlessservice, does not receive particular emphasis in the firs tpaper, but has come to be the defining characteristic ofthe protocol. Another example is the layering of thearchitecture into the IP and TCP layers. This seems basicto the design, but was also not a part of the originalproposal. These changes in the Internet design arosethrough the repeated pattern of implementation andtesting that occurred before the standards were set.The Internet architecture is still evolving. Sometimes anew extension challenges one of the design principles,but in any case an understanding of the history of thedesign provides a necessary context for current designextensions. The connectionless configuration of IS0protocols has also been colored by the history of theInternet suite, so an understanding of the Internet designphilosophy may be helpful to those working with ISO.This paper catalogs one view of the original objectives ofthe Internet architecture, and discusses the relationbetween these goals and the important features of thepnXocols.

2. Fundamental GoalThe top level goal for the DARPA Internet Architecturewas to develop an effective technique for multiplexedutilization of existing interconnected networks. Someelaboration is appropriate to make clear the meaning ofthat goal.The components of the Internet were networks, whichwere to be interconnected to provide some larger service.The original goal was to connect together the ori BnalARPANET with the ARPA packet radio network. , inorder to give users on the packet radio network access othe large service machines on the ARPANET. At thetime it was assumed that there would be other sorts of

106


2/9

networks to interconnect, although the local area networkhad not yet emerged.An alternative to interconnecting existing networkswould have been to design a unified system whichincorporated a variety of different transmission media, amulti-media network. While this might have permitted ahigher degree of integration, and thus better performance,it was felt that it was necessary to incorporate the thenexisting network architectures if Internet was to be usefulin a practical sense. Further, networks representadministrative boundaries of control, and it was anambition of this project to come to grips with the problemof integrating a number of separately adminis tratedentities into a comm on utility.The technique selected for multiplexing was packe tswitching. Au alternative such as circuit switching couldhave been considered, but the applications beingsupported, such as remote login, were naturally served bythe packe t switching paradigm, and the networks whichwere to be integrated together in this project were packetswitching networks. So packet switching was acceptedas a fundamental component of the Internet architecture.The fmal aspect of this fundamental goal was theassumption of the particular technique for interconnectingthese networks. Since the technique of store and forwardpacket switching, as demonstrated in the previousDARPA project, the ARPANET, was well understood,the top level assumption was that networks would beinterconnected by a layer of Internet packet switches,which were called gateways.From these assumptions comes the fundamental structureof the Internet: a packet switched communicationsfacility in which a number of distinguishable networksam connected together using packet communicationsprocessors called gateways which implement a store aridforward packet forwarding algorithm.

3. Second Level GoalsThe top level goal sta ted in the previous section containsthe word effective, without offering any definition ofwhat an effective interconnection must achieve. Thefollowing list summarizes a more detailed se t of goalswhich were established for the Internet architecture.

1. Internet commuuication must continuedespite loss of networks or gateways.

2. The Internet m ust support multiple types ofcommunications service.3. The Internet architecture mustaccommodate a variety of networks.4. The Internet architecture must permitdistributed management of its resources.

5. The Internet architecture must be costeffective.6. The Internet architecture must permit hostattachment with a low level of effort.7. The resources used in the iutemetarchitecture must be accountable.

This set of goals might seem to be nothing more than achecklist of all the desirable network features. It isimportant to understand that these goals are in order ofimportance, and an entirely d ifferent network architecturewould result if the order were changed. For example,since this network was designed to operate in a militarycontext, which implied the possibility of a hostileenvironment, survivability was put as a first goal, andaccountability as a last goal. During wartime. one is lessconcerned with detailed accounting of resources usedthan with mustering whatever resources are available andrapidly deploying them it-i an operational manner. Whilethe architects of the Internet were mindful ofaccountability, the problem received very little attentionduring the early stages of the design. aud is only nowbeing considered. An architecture primarily forcommercial deployment would clearly place these goalsat the opposite end of the list.Similarly, the goal that the architecture be cost effectiveis clearly on the list, but below certain other goals, suchas distributed management, or support of a wide varietyof networks. Other protocol suites, including some of themore popular commercial architectures, have beenoptimized to a particular kind of network, for example along haul store and forward network built o f mediumspeed telephone lines, and deliver a very cost effectivesolution in this context, in exchange for dealingsomewhat poorly with other kinds of nets, such as localarea nets.The reader should consider carefully the above list ofgoals, and recognize that this is not a motherhood list,but a set of priorities which strongly colored the designdecisions within the Internet architecture. The followingsections discuss the relationship between this list and thefeatures of the Internet.

4. Sur\i\rability in the Face of FailureThe most important goal on the list is that the Internetshould continue to supply communications service, eventhough networks and gateways are failing. In particular,this goal was interpreted to mean that if two entities arecommuuicating over the Internet. and some failure causesthe Internet to be temporarily disrupted and reconfiguredto reconstitute the service, then the entitiescommunicating should be able to continue withouthaving to reestablish or reset the high level s tate of theirconversation. More concretely, at the service interface ofthe transport layer, this architecture provides no facilityto communicate to the client of the transport service that

107


3/9

the synchronization between the sender and the receivermay have been lost. It was an assumption in thisarchitecture that synchronization would never be lostunless there was no physical path over which any sort ofcommunication could be achieved. In other words, at thetop of transport, there is only one failure, and it is totalpartition. The architecture was to mask completely anytransient failure.To achieve this goal, the state information whichdescribes the on-going conversation must be protected.Specific examples of state information would be thenumber of packets transmitted, the number of packetsacknowledged, or the number of outstanding flow controlpermissions. If the lower layers of the architecture losethis information, they will not be able to tell if data hasbeen lost, and the application layer will have to cope withthe loss of synchrony. This architecture insisted that thisdisruption not occur, which meant that the stateinformation must be protected from loss.In some network architectures, this state is stored in theintermediate packet switching nodes of the network. Inthis case, to protect the information from loss, it mustreplicated. Because of the distributed nature of thereplication, algorithms to ensure robust replication arethemselves difficult to build, and few networks withdistributed state information provide any sort ofprotection against failure. The alternative, which thisarchitecture chose, is to take this information and gatherit a t the endpoint of the net, at the entity which is utilizingthe service of the network.reliability I call this approach tofate-sharing. The fate-sharing modelsuggests hat it is acceptable to lose the state informationassociated with an entity if, at the same time, the entityitself is lost. Specifically, information about transportlevel synchronization is stored in the host which isattached to the net and using its communication service.There are two important advantages to fate-sharing overreplication. First, fate-sharing protects against anynumber of intermediate failures, whereas replication canonly protect against a certain number (less than thenumber of replicated copies). Second, fate-sharing ismuch easier to engineer than replication.There are two consequences o the fate-sharing approachto survivability. First. the intermediate packet switchingnodes, or gateways, must not have any essential stateinformation about on-going connections. Instead, theyare stateless packet switches, a class of network designsometimes called a datagram network. Secondly, rathermore trust is placed in the host machine than in anarchitecture where the network ensures the reliabledelivery of data. If the host resident algorithms thatensure the sequencing and acknowledgment of data fail,applications on that machine are prevented fromoperation.Despite the the fact that survivability is the first goal inthe list, it is still second to the top level goal ofinterconnection of existing networks. A more survivable

technology might have resulted from a single multi-media network design. For example, the Internet makesvery weak assumptions about the ability of a network toreport that it has failed. Internet is thus forced to detectnetwork failures using Internet level mechanisms, withthe potential for a slower and less specific error detection.

5. Types of ServiceThe second goal of the Internet architecture is that itshould support, at the transport service level, a variety oftypes of service. Different types of service aredistinguished by differing requirements for such things asspeed, latency and reliability. The traditional type ofservice is the bidirectional reliable delivery of data. Thisservice, which is sometimes called a virtual circuitservice, is appropriate for such applications as remotelogin or tile transfer. It was the first serv ice provided inthe Internet architecture, using the Transmission ControlProtocol (TCP). It was early recognized that even thisservice had multiple variants, because remote loginrequired a service with low delay in delivery, but lowrequirements for bandwidth, while file transfer was lessconcerned with delay, but very concerned with highthroughput. TCP attempted to provide both these typesof service.The initial concept of TCP was that it could be generalenough to support any needed type of service. However,as the full range of needed services became clear, itseemed too difficult to build support for all of them intoone protocol.The first example of a service outside the range of TCPwas support for XNET ,2 the cross-Internet debugger.TCP did not seem a suitable transport for XNET forseveral reasons. First, a debugger protocol should not bereliable. This conclusion may seem odd, but underconditions of stress or failure (which may be exactlywhen a debugger is needed) asking for reliablecommunications may prevent any communications at all.It is much better to build a service which can deal withwhatever gets through, rather than insisting that everybyte sent be delivered in order. Second, if TCP is generalenough to deal with a broad range of clients, it ispresumably somewhat complex. Again, it seemed wrongto expect support for this complexity in a debuggingenvironment, which may lack even basic servicesexpected in an operating system (e.g. support for timers.)So XNET was designed to run directly on top of thedatagram service provided by Internet.Another service which did not fu TCP was real timedelivery of digitized speech, which was needed to supportthe teleconferencing aspect of command and controlapplications. III real time digital speech, the primaryrequirement is not a reliable service, but a service whichminimizes and smooths the delay in the delivery ofpackets. The application layer is digitizing the analogspeech, packetizing the resulting bits, and sending themout across the network on a regular basis. They must

108


4/9

arrive at the receiver at a regular basis in order to beconverted back to the analog signal. If packets do notarrive when expected, it is impossible to reassemble thesignal in real time. A surprising observation about thecontrol of variation in delay is that the most serioussource of delay in networks is the mechanism to providereliable delivery. A typical reliable transport protocolresponds to a missing packet by requesting aretransmission and delaying the delivery of anysubsequent packets until the lost packet has beenretransmitted. It then delivers that packet and allremaining ones in sequence. The delay while this occurscan be many times the round trip delivery time of the net,and may completely disrupt the speech reassemblyalgorithm. In contrast, it is very easy to cope with anoccasional missing packet. The missing speech cansimply be replaced by a short period of silence, which inmost cases does not impair the intelligibility of thespeech o the listening human. If it does, high level errorcorrection can occur, and the listener can ask the speakerto repeat the damagedphrase.It was thus decided, fairly early in the development of theInternet architecture, that more than one transport servicewould be required, and the architecture must be preparedto tolerate simultaneously transports which wish toconstrain reliability, delay, or bandwidth. at a minimum.This goal caused TCP and IP, which originally had beena single protocol in the architecture, to be separated ntotwo layers. TCP provided one particular type of service,the reliable sequenceddata stream, while IP attempted toprovide a basic building block out of which a variety oftypes of service could be built. This building block wasthe datagram, which had also been adopted to supportsurvivability. Since the reliability associated with thedelivery of a datagram was not guaranteed, but besteffort, it was possible to build out of the datagram aservice that was reliable (by acknowledging andretransmitting at a higher level), or a service which tradedreliability for the primitive delay characteristics of theunderlying network substrate. The User DatagramProtocol (UDP)13 was created to provide a application-level interface to the basic datagram service of Internet.The architecture did not wish to assume that theunderlying networks themselves support multiple types ofservices, because this would violate the goal of usingexisting networks. Instead, the hope was that multipletypes of service could be constructed out of the basicdatagram building block using algorithms within the hostand the gateway. For example, (although this is not donein most current implementations) it is possible to takedatapams which are associated with a controlled delaybut unreliable service and place them at the head of thetransmission queues unless their lifetime has expired, inwhich case they would be discarded; while packetsassociated with reliable streams would be placed at theback of the queues, but never discarded, no matter howlong they had been n the net.

It proved more difficult than first hoped to providemultiple types of service without explicit support fromthe underlying networks. The most serious problem wasthat networks designed with one particular type of servicein mind were not flexible enough to support otherservices. Most commonly, a network will have beendesigned under the assumption that it should deliverreliable service, and will inject delays as a part o fproducing reliable service, whether or not this reliabilityis desired. The interface behavior defined by X.25, forexample, implies reliable delivery, and there is no way toturn this feature off. Therefore, although Internetoperates successfully over X.25 networks it cannotdeliver the desired variability of type service in thatcontext. Other networks which have an intrinsicdatagram service are much more flexible in the type ofservice they will permit. but these networks are much lesscommon, especially in the long-haul context.

6. Varieties of NetworksIt was very important for the success of the Internetarchitecture that it be able to incorporate and utilize awide variety of network technologies, including militaryand commercial facilities. The Internet architecture hasbeen very successful in meeting this goal: it is operatedover a wide variety of networks, including long haul nets(the ARPANET itself and various X.25 networks), localarea nets (Ethernet, ringnet, etc.), broadcast satellite nets(the DARPA Atlantic Satellite Network, I5 operating at64 kilobits per second and the DARPA ExperimentalWideband Satellite Net,16 operating within the UnitedStates at 3 megabits per second), packet radio networks(the DARPA packet radio network, as well as anexperimental British packet radio net and a networkdeveloped by amateur radio operators), a variety of seriallinks, ranging from 1200 bit per second asynchronousconnections to TI links, and a variety of other ad hocfacilities, including intercomputer busses and thetransport service provided by the higher layers of othernetwork suites, such as IBMs HASP.The Internet architecture achieves this flexibility bymaking a minimum set of assumptions about the functionwhich the net will provide. The basic assumption is thatnetwork can transport a packet or datagram. The packetmust be of reasonable size, perhaps 100 bytes minimum,and should be delivered with reasonable but not perfectreliability. The network must have some suitable form ofaddressing if it is more than a point to point link.There are a number of services which are explicitly notassumed from the network. These include reliable orsequenceddelivery, network level broadcast or multicast,priority ranking of transmitted packet,multiple types of serv ice, and mtemal knyJ&te z;failures, speeds, or delays. If these services had beenrequired, then in order to accommodate a network withinthe Internet, it would be necessaryeither that the networksupport these services directly, or that the networkinterface software provide enhancements to simulate

109


5/9

these services at the endpoint of the network. It was feltthat this was an undesirable approach, because theseservices would have to be re-engineered andreimplemented for every single network and every singlehost interface to every network. By engineering theseservices at the transport, for example reliable delivery viaTCP, the engineering must be done only once, and theimplementation must be done only once for each host.After that, the implementation of interface software for anew network is usually very simple.

7. Other GoalsThe three goals discussed so far were those which had themost profound impact on the design on the architecture.The remaining goals, because they were lower inimportance, were perhaps less effectively met, or not socompletely engineered. The goal of permittingdistributed management of the Internet has certainly beenmet in certain respects. For example, not all of thegateways in the Internet are implemented and managedby the same agency. There are several differentmanagement centers within the deployed Internet, eachoperating a subset of the gateways, and there is a two-tiered routing algorithm which permits gateways fromdilferent administrations to exchange routing tables, eventhough they do not completely trust each other, and avariety of private routing algorithms used among thegateways in a single administration. Similarly, thevarious organizations which manage the gateways are notnecessarily the same organizations that manage thenetworks to which the gateways are attached.On the other hand, some of the most significant problemswith the Internet today relate to lack of sufficient tools fordistributed management,especially in the area of routing.In the large intemet being currently operated, routingdecisions need to be constrained by policies for resourceusage. Today this can be done only in a very limitedway, which requires manual setting of tables. This iserror-prone and at the same time not sufficientlypowerful. The most important change in the Internetarchitecture over the next few years will probably be thedevelopment of a new generation of tools formanagement of resources in the context of multipleadministrations.It is clear that in certain circumstances, the Internetarchitecture does not produce as cost effective autilization of expensive communication resources as amore tailored architecture would. The headers of Internetpackets am fairly long (a typical header is 40 bytes), andif short packets are sent, this overhead is apparent. Theworse case, of course, s the single character remote loginpackets, which carry 40 bytes of header and one byte ofdata. Actually, it is very difficult for any protocol su ite toclaim that these sorts of interchanges are carried out withreasonable efficiency. At the other extreme, largepackets for file transfer, with perhaps 1,000 bytes of data,have an overhead for the header of only four percent.

Another possible source of inefficiency is retransmissionof lost packets. Since Internet does not insist that lostpackets be recovered at the network level, it may benecessary to retransmit a lost packet from one end of theInternet to the other. This means that the retransmittedpacket may cross several intervening nets a second time,whereas recovery at the network level would not generatethis repeat traffic. This is an example of the tradeoffresulting from the decision, discussed above, of providingservices from the end-points. The network interface codeis much simpler, but the overall efficiency is potentiallyless. However, if the retransmission rate is low enough(for example, 1%) then the incremental cost is tolerable.As a rough rule of thumb for networks incorporated intothe architecture, a loss of one packet in a hundred is quitereasonable, but a loss of one packet in ten suggests thatreliability enhancements be added to the network if thattype of service is required.The cost of attaching a host to the Internet is perhapssomewhat higher than in other architectures, because allof the mechanisms to provide the desired types of service,such as acknowledgments and retransmission strategies,must be implemented in the host rather than in thenetwork. Initially, to programmers who were not familiarwith protocol implementation, the effort of doing thisseemed somewhat daunting. Implementors tried suchthings as moving the transport protocols to a front endprocessor, with the idea that the protocols would beimplemented only once, rather than again for every typeof host. However, this required the invention of a host tofront end protocol which some thought almost ascomplicated to implement as the original transportprotocol. As experience with protocols increases, theanxieties associated with implementing a protocol suitewithin the host seem to be decreasing, andimplementations are now available for a wide variety ofmachines, including personal computers and othermachines with very limited computing resources.A related problem arising from the use of host-residentmechanisms is that poor implementation of themechanism may hurt the network as well as the host. Thisproblem was tolerated, because the initial experimentsinvolved a limited number of host implementations whichcould be controlled. However, as the use of Internet hasgrown, this problem has occasionally surfaced in aserious way. In this respect, the goal o f robustness, whichled to the method of fate-sharing, which led to host-resident algorithms, contributes to a loss of robusmess ifthe host misbehaves.The last goal was accountability. In fact , accounting wasdiscussed in the first paper by Cerf and Kahn as animportant function of the protocols and gateways.However, at the present time, the Internet architecturecontains few tools for accounting for packet flows. Thisproblem is only now being studied, as the scope of thearchitecture is being expanded to include non-militaryconsumers who are seriously concerned withunderstanding and monitoring the usage of the resourceswithin the intemet.

110


6/9

8. Architecture and ImplementationThe previous discussion clearly suggests that one of thegoals of the Internet architecture was to provide wideflexibility in the service offered. Different transportprotocols could be used to provide different types ofservice, and different networks could be incorporated.Put another way, the architecture tried very hard not toconstrain the range of service which the Internet could beengineered to provide. This, in turn, means that tounderstand the service which can be offered by aparticular implementation of an Internet, one must looknot to the architecture, but to the actual engineering of thesoftware within the particular hosts and gateways, and tothe particular networks which have been incorporated. Iwill use the term realization to describe a particular setof networks, gateways and hosts which have beenconnected together in the context of the Internetarchitecture. Realizations can differ by orders ofmagnitude in the service which they offer. Realizationshave been built out of 1900 bit per second phone lines,and out of networks only with speeds greater than 1megabit per second. Clearly, the throughput expectationswhich one can have of these realizations differ by ordersof magnitude. Similarly, some Internet realizations havedelays measured in tens of milliseconds, where othershave delays measured in seconds. Certain applicationssuch as real time speech work fundamentally differentlyacross these two realizations. Some Intemets have beenengineered so that there is great redundancy in thegateways and paths. These Internets are survivable,because resources exist which can be reconfigured afterfailure. Other Internet realizations, to reduce cost, havesingle points of connectivity through the realization, sothat a failure may partition the Internet into two halves.The Internet architecture tolerates this variety ofrealization by design. However, it leaves the designer ofa particular realization with a great deal of engineering todo. One of the major struggles of this architecturaldevelopment was to understand how to give guidance tothe designer of a realization, guidance which would relatethe engineering of the realization to the types of servicewhich would result. For example, the designer mustanswer the following sort of question. What sort ofbandwidths must he in the underlying networks, if theoverall service is to deliver a throughput of a certain rate?Given a certain model of possible failures within thisrealization, what sorts of redundancy ought to beengineered nto the realization?Most of the known network design aids did not seemhelpful in answering these sorts of questions. Protocolverifiers, for example, assist n confirming that protocolsmeet specifications. However, these tools almost neverdeal with performance issues, which are essential to theidea of the type of service. Instead, they deal with themuch more restricted idea of logical correctness of theprotocol with respect to specification. While tools toverify logical correctness are useful, both at thespecification and implementation stage. they do not helpwith the severe problems that often arise related to

performance. A typical implementation experience isthat even after logical correctness has been demonstrated,design faults are discovered that may cause aperformance degradation of an order of magnitude.Exploration of this problem has led to the conclusion thatthe difficulty usually arises, not in the protocol itself, butin the operating system on which the protocol runs. Thisbeing the case, it is difficult to address the problemwithin the context of the architectural specification.However, we still strongly feel the need to give theimplementor guidance. We continue to struggle with thisproblem today.The other class of design aid is the simulator, which takesa particular realization and explores the service which itcan deliver under a variety of loadings. No one has yetattempted to construct a simulator which take intoaccount the wide variability of the gatewayimplementation, the host implementation, and thenetwork performance which one sees within possibleInternet realizations. It is thus the case that the analysisof most Internet realizations is done on the back of anenvelope. It is a comment on the goal structure of theInternet architecture that a back of the envelope analysis,if done by a sufficiently knowledgeable person, is usuallysufficient. The designer of a particular Internetrealization is usually less concerned with obtaining thelast five percent possible in line utilization than knowingwhether the desired type of service can be achieved at allgiven the resources at hand at the moment.The relationship between architecture and performance isan extremely challenging one. The designers of theInternet architecture felt very strongly that it was aserious mistake to attend only to logical correctness andignore the issue of performance. However, theyexperienced great difficulty in formalizing any aspect ofperformance constraint within the architecture. Thesedifficulties arose both because he goal of the architecturewas not to constrain performance, but to permitvariability, and secondly (and perhaps morefundamentally), because there seemed to be no usefulformal tools for describing performance.This problem was particularly aggravating because thegoal of the Internet project was to produce specificationdocuments which were to become military standards. Itis a well known problem with government contractingthat one cannot expect a contractor to meet any criteriawhich is not a part of the procurement standard. If theInternet is concerned about performance, therefore, it wasmandatory that performance requirements be put into theprocurement specification. It was trivial to inventspecifications which constrained the performance, forexample to specify that the implementation must becapable of passing 1.000 packets a second. However, thissort of constraint could not be part of the architecture,and it was therefore up to the individual performing theprocurement to recognize that these performanceconstraints must be added to the specification, and tospecify them properly to achieve a realization whichprovides the required types of service. We do not have a

111


7/9

good idea how to offer guidance in the architecture for complete review of the history of TCP itself wouldthe person performing this task. require another paper of this length.

9. DatagramsThe fundamental architectural feature of the Internet isthe use of datagrams as the entity which is transportedacross the underlying networks. As this paper hassuggested, there are several reasons why datagrams areimportant within the architecture. First, they eliminatethe need for connection state within the intermediateswitching nodes, which means that the Internet can bereconstituted after a failure without concern about state.Secondly, the datagram provides a basic building blockout of which a variety of types of service can beimplemented. In contrast to the virtual circuit, whichusually implies a fixed type of service, the datagramprovides a more elemental service which the endpointscan combine as appropriate to build the type of serviceneeded. Third, the datagram represents the minimumnetwork service assumption, which has permitted a widevariety of networks to be incorporated into variousInternet realizations. The decision to use the datagramwas an extremely successful one, which allowed theInternet to meet its m ost important goals verysuccessfully.There is a mistaken assumption often associated withdatagrams, which is that the motivation for datagrams isthe support of a higher level service which is essentiallyequivalent to the datagram. In other words, it hassometimes been suggested hat the datagram is providedbecause the transport service which the applicationrequires is a datagram service. In fact, this is seldom thecase. While some applications in the Internet, such assimple queries of date servers or name servers, use anaccess method based on an unreliable datagram, mostservices within the Internet would like a moresophisticated transport model than simple datagram.Some services would like the reliability enhanced, somewould like the delay smoothed and buffered, but almostall have some expectation more complex than adatagram. It is important to understand that the role ofthe datagram n this respect s as a building block, and notas a service in itself.

IO. TCPThere were several interesting and controversial designdecisions in the development of TCP, and TCP itselfwent through several major versions before it became areasonably stable standard. Some of these designdecisions, such as window management and the nature ofthe port address structure, are discussed in a series ofimplementation notes ublished as part of the TCPprotocol handbook.7 P But again the motivation for thedecision is sometimes acking. ln this section, I attempt tocapture some of the early reasoning that went into partsof TCP. This section is of necessity incomplete; a

The originaI ARPANET host-to host protocol providedflow control based on both bytes and packets. Thisseemed overly complex, and the designers of TCP feltthat only one form of regulation would he sufficient. Thechoice was to regulate the delivery of bytes, rather thanpackets. Flow control and acknowledgment in TCP isthus based on byte number rather than packet number.Indeed, in TCP there is no significance to thepacketization of the data.This decision was motivated by several considerations,some of which became irrelevant and others of whichwere more important that anticipated. One reason toacknowledge bytes was to permit the insertion of controlinformation into the sequence space of the bytes, so thatcontrol as well as data could be acknowledged. That useof the sequence space was dropped, in favor of ad hoctechniques for dealing with each control message. Whilethe original idea has appealing generality, it causedcomplexity in practice.A second reason for the byte stream was to permit theTCP packet to be broken up into smaller packets ifnecessary n order to fit through a net with a small packetsize. But this function was moved to the IP layer when IPwas split from TCP, and IP was forced to invent adifferent method of fragmentation.A third reason for acknowledging bytes rather thanpackets was to permit a number of small packets to begathered together into one larger packet in the sendinghost if retransmission of the data was necessary. It wasnot clear if this advantage would be important; it turnedout to be critical. Systems such as UNIX which have ainternal communication model based on single characterinteractions often send many packets with one byte ofdata in them. (One might argue from a networkperspective that this behavior is silly, but it was a reality,and a necessity for interactive remote login.) It was oftenobserved that such a host could produce a flood ofpackets with one byte of data, which would arrive muchfaster than a slow host could process them. The result islost packets and retransmission.If the retransmission was of the original packets, the sameproblem would repeat on every retransmission, with aperformance impact so intolerable as to preventoperation. But since the bytes were gathered into onepacket for retransmission, the retransmission occurred ina much more effective way which permitted practicaloperation.On the other hand, the acknowledgment of bytes could beseen as creating this problem in the first place. If the basisof flow control had been packets rather than bytes, thenthis flood might never have occurred. Control at thepacket level has the effec t, however, of providing asevere limit on the throughput if small packets are sent. Ifthe receiving host specifies a number of packets to

112


8/9

receive, without any knowledge of the number of bytes ineach, the actual amount of data received could vary by afactor of 1000, depending on whether the sending hostputs one or one thousand bytes in each packet.In retrospect, the correct design decision may have beenthat if TCP is to provide effective support of a variety ofservices, both packets and bytes must be regulated, aswas done in the original ARPANET protocols.Another design decision related to the byte stream wasthe End-Of-Letter flag, or EOL. This has now vanishedfrom the protocol, replaced by the push flag, or PSH. Theoriginal idea of EOL was to break the byte stream intorecords. It was implemented by putting data fromseparate records into separate packets, which was notcompatible with the idea of combining packets onretransmission. So the semantics of EOL was changed toa weaker form, meaning only that the data up to this pointin the stream was one or more complete application-levelelements, which should occasion a flush of any internalbuffering in TCP or the network. By saying one ormore rather than exactly one, it became possible tocombine several together and preserve the goal ofcompacting data in reassembly. But the weaker semanticsmeant that various applications had to invent an ad hocmechanism for delimiting records on top of the datastream.In this evolution of EOL semantics, there was a littleknown intermediate form, which generated great debate.Depending on the buffering strategy of the host, the bytestream model of TCP can cause great problems in oneimprobable case. Consider a host in which the incomingdata is put in a sequenceof fixed size buffers. A buffer isreturned to the user either when it is full, or an EOL isreceived. Now consider the case of the arrival of an out-of-order packet which is so far out of order to he beyondthe current buffer. Now further consider that afterreceiving this out-of-order packet, a packet with an EOLcauses he current buffer to be returned to the user onlypartially full. This particular sequence of actions has theeffect of causing the out of order data in the next buffer tobe in the wrong place, becauseof the empty bytes in thebuffer returned to the user. Coping with this generatedbook-keeping problems in the host which seemedunnecessary.To cope with this it was proposed that the EOL shoulduse up all the sequence space up to the next valuewhich was zero mod the buffer size. In other words, itwas proposed that EOL should be a tool for mapping thebyte stream to the buffer management of the host. Thisidea was not well received at the time, as it seemed muchtoo ad hoc, and only one host seemed to have thisproblem. In retrospect, t may have been the correct idea

This use of EOL was properly called Rubber EOL but itsdetractors quickly called it rubber baby buffer bumpers in an attemptto ridicule tbc idea. &edit mus t go to the creator of the ide+ BillPlummcr, for sticking to his guns in the face of detractors saying theabove to him ten times fast.

to incorporate into TCP some means of relating thesequence space and the bu ffer management algorithm ofthe host. At the time, the designers simply lacked theinsight to see how that might be done in a sufficientlygeneral manner.

Il. ConclusionIn the context of its priorities, the Internet architecturehas been very successful. The protocols are widely usedin the commercial and military environment, and havespawned a number of similar architectures. At the sametime, its success has made clear that in certain situations,the priorities of the designers do not match the needs ofthe actual users. More attention to such things asaccounting, resource management and operation ofregions with separateadministrations are needed.While the datagram has served veIy well in solving themost important goals of the Internet, it has not served sowell when we attempt to address some of the goals whichwere further down the priority list. For example, thegoals of resource management and accountability haveproved difficult to achieve in the context of datagrams.As the previous section discussed, most datagrams are apart of some sequence of packets from source todestination, rather than isolated units at the applicationlevel. However, the gateway cannot directly see theexistence of this sequence, because it is forced to dealwith each packet in isolation. Therefore, resourcemanagement decisions or accounting must be done oneach packet separately. Imposing the datagram model onthe intemet layer has deprived that layer of an importantsource of information which it could use in achievingthese goals.This suggests that there may be a better building blockthan the datagram for the next generation of architecture.The general characteristic of this building block is that itwould identify a sequence of packets traveling from thesource to the destination, without assuming any particulartype of service with that service. I have used the wordflow to characterize this building block. It would benecessary or the gateways to have flow state in order toremember the nature of the flows which are passingthrough them, but the state information would not becritical in maintaining the desired type of serviceassociated with the flow. Instead, that type of servicewould be enforced by the end points, which wouldperiodically send messages o ensure that the proper typeof service was being associated with the flow. In thisway, the state information associated with the flow couldbe lost in a crash without permanent disruption of theservice features being used. I call this concept softstate, and it may very well permit us to achieve ourprimary goals of survivability and flexibility, while at thesame time doing a better job of dealing with the issue ofresource management and accountability. Exploration ofalternative building blocks constitute one of the currentdirections for research within the DARPA InternetPrOgrtUtl.

113


9/9

12. Acknowledgments -- A HistoricalPerspectiveIt would be impossible to acknowledge all thecontributors to the Internet project; there have literallybeen hundreds over the 15 years of development:designers, implementors, writers and critics. Indeed, animportant topic, which probably deserves a paper initself, is the process by which this project was managed.The participants came from universities, researchlaboratories and corporations, and they united (to someextent) to achieve this common goal.The original vision for TCP came from Robert Kahn andVinton Cerf, who saw very clearly, back in 1973, how aprotocol with suitable features might be the glue thatwould pull together the various emerging networktechnologies. From their position at DARPA, they guidedthe project in its early days to the point where TCP and IPbecame standards or the DOD.The author of this paper oined the project in the mid-70s,and took over architectural responsibility for TCP/IP in1981. He would like to thank all those who have workedwith him, and particularly those who took the time toreconstruct some of the lost history in this paper.

References1.

2.

3.

4.

5.

6.

7.

V. Cerf, and R. Kahn, A Protocol for PacketNetwork intercommunication, IEEETransactions Communications, Vol.Corn-22, No. 5, May1974 pp. 637-648.ISO, Transport Protocol Specification, Tech.report IS-8073, International Organization forStandardization, September 1984.ISO, Protocol for Providing the Connectionless-Mode Network Service, Tech. report DIS8473,International Organization for Standardization,1986.R. Callon, Internetwork Protocol, Proceedingsofthe IEEE, Vol. 71, No. 12, December 1983, pp.1388-1392.Jonathan B. Pastel, Intemetwork ProtocolApproaches, IEEE TransactionsCommunications, Vol. Corn-28, Nd:4, April 1980, pp. 605-611.Jonathan B. Postel, Carl A. Sunshine, DannyCohen, The ARPA Internet Protocol,Computer Networks 5, Vol. 5, No. 4, July 1981,pp. 261-27 1.Alan Shehzer, Robert Hinden, and Mike Brescia,Connecting Different Types of Networks withGateways, Data Communications, August 1982.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

J. McQuillan and D. Walden, The ARPANetwork Design Decisions , ComputerNetworks, Vol. 1, No. 5, August 1977, pp.243-289.R.E. Kahn, S.A. Gronemeyer, J. Burdifiel, E.V.Hoversten, Advances in Packet RadioTechnology, Proceedings of the IEEE, Vol.66, No. 11, November 1978, pp. 1408-1496.B.M. Leiner, D.L. Nelson, F.A. Tobagi, Issuesin Packet Radio Design, Proceedings of theIEEE, Vol. 75, No. 1, January 1987, pp. 6-20.

Transmission Control Protocol RFC-793,&DN Protocol Handbook, Vol.2, September 1981, pp, 2.179-2.198.Jack Haverty, XNET Formats for InternetProtocol Version 4 IEN 158, DDN ProtocolHandbook, Vol. 2, October 1980, pp. 2-345 to2-348.Jonathan Postel, User Datagram Protocol NIC-RFC-768, DDN Protocol Handbook, Vol.2. August 1980, pp. 2.175-2.177.I. Jacobs. R. Binder, and E. Hoversten, GeneralPurpose Packet Satellite Networks, Proceedingsof the IEEE, Vol. 66, No. 11, November 1978, pp1448-1467.C. Topolcic and J. Kaiser, The SATNETMonitoring System, Proceedings of the IEEE-MILCOM Boston, MA, October 1985, PP.26.1.1-26.1.9.W.Edmond, S.Blumenthal, A.Echenique,S.Storch, T.Calderwood, and T.Rees, TheButterfly Satellite IMP for the Wideband PacketSatellite Network , Proceedings of the ACMSIGCOMM 86, ACM, Stowe, Vt., August 1986,pp. 194-203.David D. Clark, Window and AcknowledgmentStrategy in TCP NlC-RFC-813, DDN ProtocolHandbook, Vol. 3, July 1982, pp. 3-5 to 3-26.David D. Clark, Name, Addresses, Ports, andRoutes NIC-RFC-814, DDN ProtocolHandbook, Vol. 3, July 1982, pp. 3-27 to 3-40.

114

Documents

Darpa Internet