Application Layer

CT 542

Content• DNS – Domain Name Server

– DNS Name Space– DNS Records– Name Servers

• E- Mail– Architecture and Services– Message format– Message transfer

• SMTP, SSMTP– Message final delivery

• POP3, IMAP, Web mail• WWW

– Architecture– Static web documents

• HTML overview– Dynamic web documents– HTTP

DNS• IP addresses are difficult to remember

– JohnDoe@140.203.8.22 would be difficult to remember– If JohnDoe’s mail server moves to another machine, then his e-mail address

would not be valid anymore– Something like JohnDoe@wuzwuz.ucg.ie would be appropriate

• Need some mechanisms to translate wuzwuz.ucg.ie to the IP address– To solve this problem DNS was invented

• It is a hierarchical, domain-based naming scheme and a distributed database system for implementing this naming scheme

• Usage:– Map name onto an IP address, an application program calls an library

procedure, called resolver, passing it the name as a parameter (i.e. gethostbyname() is an resolver)

– The resolver sends UDP packet to a local DNS server which looks up the name and returns the IP address to the reslover

– The resolver returns the IP address to the application, which can establish an TCP/IP connection with the destination (or send UDP packets)

DNS Name Space

• Internet is divided into over 200 top level domains– Each domain is divided into sub-domains, which are further partitioned, etc..– All domains can be represented by a tree

• The leaves of the tree represent domains that have no sub-domains (but contain machines)

• A leaf domain may contain a single host or represent a company and contain thousands of hosts

• Top level domains could be generic and country domains

Domain Names• Can be either absolute (ends with a period i.e. eng.sun.com.) or

relative (it doesn’t end with a dot)– Relative ones have to be interpreted in a context to find the true meaning– Both of them refers to a specific node in the tree and all the nodes under it

• Are case insensitive (edu, Edu, EDU are same thing)• Components names can be up to 63 characters and full names should

not exceed 255 characters• There is no rule against registering under two top level domains

(sony.com and sony.nl)• Each domain controls how it allocates the domains under it

– i.e. Japan makes a domains ac.jp and co.jp that mimic edu and com– Netherlands doesn’t make this distinction

• To create a new domain, permission is required from the domain that will include it; once created, it can create sub-domains without having to ask permission from the higher up domains.

Resource Records• Every domain, either single host or a top level domain,

should have a set of resource records• When a resolver gives a domain name to DNS, what it gets

back are the resource records associated with that name; thus the primary function of DNS is to map domain names onto resource records

• A resource record has five parts:– Domain name– Time to Live– Class– Type– Value

Resource Records• The Domain_name tells the domain to which this record

applies– Normally many records exist for each domain and each copy of the

database holds info about multiple domains– This field is the primary search key used to satisfy queries– The order of the records in the database is not important

• Time_to_live field gives an indication of how stable the record is– Info that is highly stable, is assigned a large value, such as 86400

(number of seconds in a day)– Info that is highly volatile is assigned a small value, such as 60 (1

minute)

• Class field is always IN for Internet information

Resource Records

• Type field tells what kind of record this is

An SOA record provides the name of the primary source of information about the name server’s zone, the e-mail address of its administrator, a unique serial number and various flags and timeouts

Address record is the most important record. It holds a 32 bit IP address for some host; Every Internet host must have at least one IP address; some hosts have two or more IP addresses (being connected to multiple networks, having one type A resource record per network connection); DNS can be made to cycle through those (for first request to return first record, for second request to return the second A type record, etc…)

MX record specifies the name of the host prepared to accept e-mail for the specified domain; it is used because not every machine is prepared to accept e-mail. If someone wants to send e-mail to bill@microsoft.com , the sending host needs to find a mail server at microsoft.com that is willing to accept e-mail. MX record can provide this information

The NS record specify name servers. i.e., every DNS database normally has an NS record for each of the top-level domainsCNAME records allow aliases to be created. In example:

cs.mit.edu 86400 IN CNAME lcs.mit.edu

Creates an alias for lcs.mit.edu (the real domain name)

PTR record points to another name (like CNAME). Unlike CNAME (which is an macro definition), PTR is a DNS data type. It is used to associate a name with an IP address to allow lookups of the IP address and return the name of the corresponding machine. This is called reverse lookup.

HINFO records allow people to find out what kind of machine and OS system a domain corresponds to. TXT records allow domains to identify themselves in arbitrary ways.

Resource Records Example

Name servers

• One DNS server could service all requests– In practice it will be overloaded

• To solve this, DNS name space is divided in non overlapping zones– Each zone contains some part of the tree and name servers holding zone info

• A zone would have a primary DNS (gets info from disk)• One or more secondary DNS (get info from the primary DNS)

Name Servers

• In the example, a resolver on flits.cs.vu.nl is looking for IP address of linda.cs.yale.edu (using recursive query; some servers don’t implement recursive query and return the address of the next server to try)– The resolver sends a query containing the domain name sought, the type (A)

and class (IN)– The query is forwarded by the local name server to the name server for

domain edu, that is found in its database….etc• Once the records get back to cs.vu.nl name server, they will be

entered in a local cache, in case they are needed later; this info is not authoritative, since changes at cs.yale.edu will not propagate to all the caches in the world; this is the reason why time_to_live should not be too big

E-Mail

• Architecture and services

• User agent

• Message formats

• Message transfer agents– SMTP

• Final delivery

E-Mail Architecture

• E-mail system consists of two parts– User agents, which allow people to read and send email

• Local programs that provide a command based or graphical method for interacting with e-mail system

– Message transfer agents, which move the messages from source to destination

• Are typically system daemons or processes that run in background, having the job to move messages

E-Mail functions

• E-mail system functions– Composition – refers to the process of creating messages

and answers; although any text editor can be used for the text of the message, the system itself can provide assistance with addressing and numerous header fields attached to each message

– Transfer – refers to moving messages from the originator to the recipient; this requires establishing a connection to the destination or some intermediate machine, outputting the message and releasing the connection

E-mail functions

• E-mail system functions– Reporting – has to do with telling the originator what

happened to the message; Was it delivered? Was it rejected? Was it lost?

– Displaying – showing the incoming message is important, so the people can read their e-mail; sometime conversion or a special viewer is required (i.e. if the message is a PS message or an audio file)

– Disposition – what the recipient does after the message has been received; possibilities include throwing it away before reading it, throwing it away after reading it, saving it and so on.

E-mail architecture and functions• Distinction between

envelope and its contents;– envelope

encapsulates the message and contains info needed for transporting the message, such as destination address, priority and security level

– Message has two parts: headers (interpreted by the user agent) and body (info for the human recipient)

E-mail user agent

• Sending e-mail– User must provide the message and the destination address

(user@dns-address)– User agents may support mailing lists

• Receiving e-mail– When an user agent is started, it looks at user’s mailbox before

displaying anything– Then it may announce the number of messages in the mailbox

E-mail message format• Basic ASCII e-mail message using RFC 822

– Messages consists of a primitive envelope (described in RFC821), some number of header fields, a blank line and then the message body

– Each header field (logically) consists of a single line of ASCII text, a colon and, for most fields, a value (see next slide)

– RFC822 was designed long ago and doesn’t clearly distinguish between the envelope fields and the header fields

• This was revised in RFC 2822, however, wasn’t possible to completely redo it due to the widespread usage

E-mail message format

RFC 822 header fields related to message transport.

Some fields used in the RFC 822 message header.

E-mail message format

• MIME – the Multipurpose Internet Mail Extensions– Solves problems related to send and receive of messages in non

Latin alphabets, languages with no alphabets (Chinese or Japanese), messages not containing text at all (video or audio)

– RFC1341 and updated in RFCs 2045-2049

• The basic idea of MIME is to continue to use RFC 822 format, but to add structure to the message body and define encoding rules for non-ASCII messages– By not deviating the RFC822, MIME messages can be sent using

the existing mail programs and protocols

– All it has to be changed is the receiving and sending programs, which each user can do it

MIME message format

RFC 822 headers added by MIME.

Content-Description: header is an ASCII string telling what is the message; this header is needed so the recipient will know whether it is worth decoding and reading the message

Content-Id: header identifies the content

Content-Transfer-Encoding: header tells how the body is wrapped for transmission through a network that may object to most characters other than letters, numbers and punctuation marks. Five schemas are provided (the simplest is ASCII text. ASCII characters use 7 bits (one bit is not used) and can be carried directly by the mail protocol, provided that no line exceeds 1000 characters.

See next slide for clarification.

Content-Type: header specifies the nature of the message body. Severn types are defined by the RFC 2045, each of which has one or more subtypes. The type and subtype is separated by a slash, as in:

Content-Type: video/mpeg

The subtype must be given explicitly in the header; no defaults are provided. See the following slides for clarification.

MIME – Content Transfer Encoding• Content-Transfer-Encoding: header specifies how the body

is wrapped for transmission– the simplest is ASCII text. ASCII characters use 7 bits (one bit is

not used) and can be carried directly by the mail protocol, provided that no line exceeds 1000 characters

– The next simplest scheme is the same thing, but using 8 bit characters, all values from 0 to 255. This encoding scheme violates the original Internet e-mail protocol, but it is used by some parts of the Internet that implement some extensions to the original protocol. While declaring the encoding doesn’t make it legal, at least explain things when something goes wrong. Still this schema is respecting the 1000 chars per line rule

– Even worst are messages that don’t respect the 1000 characters per line rule; Executable programs fall in this category. No guarantee is given that messages in binary will arrive correctly, but some people try

MIME – Content Transfer Encoding– The correct way to encode binary messages is to use base64

encoding, sometimes called ASCII armor. In this scheme, groups of 24 bits are broken up into four 6 bits units, with each unit being sent as a valid ASCII character. The coding is ‘A’ for 0, ‘B’ for 1 and so on, followed by the 26 lower-case letters, the ten digits, and finally ‘+’ and ‘–’ for 62 and 63; The ‘==‘ and ‘=‘ sequences are used to indicate that the last group contained only 8 respectively 16 bits. Carriage returns and line feeds are ignored and can be inserted freely in the encoded data, to keep the lines short enough. Arbitrary binary data can be sent safely using this scheme.

– For messages that are almost entirely ASCII but with a few non ASCII characters, the base64 encoding is somewhat inefficient; instead an encoding known as quoted-printable encoding is used. This is just 7 bits ASCII, with all the characters above 127 encoded as an equal sign followed by the character’s value as two hexadecimal digits.

MIME message format • MIME Content-Type: header types and subtypes

Text type is for ASCII text; Text/Plain is for allowing ordinarily messages to be transported in MIME with a few extra headers; Text/Enriched allows for a simple markup language to be included in the text; this language provide system independent way to express boldface, italics, etc…(i.e. the <bold>time</bold> has come is displayed like the time has come ); RFC 2854 adds another subtype: text/html and RFC3023 defines text/xml

Image – used to transmit still pictures; JPEG and GIF support were built in nearly all existing browsers, therefore, they are in the list from the beginning. Other types are added to the list

Audio Video: types are for sound and movies; video contains only visual information, not the sound track; RFC 3003 is adding a new audio subtype: audio/mpeg for supporting e-mail with MP3 files

Application type is a catchall for formats that require external processing not covered by any of the other types. An octet-stream is just a sequence of un-interpreted bytes. To interpret Postscript, the user agent can just call an external PostScript interpreter

Message type allows one message to be fully encapsulated inside another (useful for message forwarding); When a complete RFC822 message is encapsulated in the outer message, then rfc822 type should be used; partial subtype can be used to break messages into pieces; external-body subtype is used to provide FTP (url) address for the body

Multipart type allows messages to contain more than just one part, with the beginning and the end of each part very well defined.

MIME example

• A birthday greeting is transmitted as both text and as a song; the receiver user agent will fetch the sound file (birthday.snd)

Email message transfer• Message transfer mechanism is concerned with relaying

messages from the originator to the destination– This can be done by establishing an transport level connection

between the source and the destination and then just transfer the message

• SMPT – Simple Mail Transfer Protocol– Source machine establishes a TCP connection on port 25 on

destination machine, where SMPT daemon listens. This daemon accepts the incoming connections and copies messages from them into the appropriate mailboxes

– If a message can’t be delivered, an error report containing the first part of the undeliverable message is returned to the sender

– It is a simple ASCII protocol

SMTP Protocol• Connection establishment (on port 25)• Data exchange

– the client machine (operating as a client) waits for the destination machine (operating as a server) to talk first;

– the server begins by sending a line of text giving its identity and telling whether is prepared to receive mail;

• if it is not, then the client releases the connection and tries again latter

– If the server is willing to accept mail, then the client announces whom the e-mail is coming from and whom it is going to

– If such recipient exists at the server end, then the client get the go-ahead to send the message

– The client sends the message, server acknowledges it

• Connection is released

• Stages of SMTP protocol – C – client– S – server

SMTP typical problems• Some old implementations can’t handle more than 64KB

message length• If the server and client have different timeouts, one of them

may give up while the other is still busy, unexpectedly terminating the connection

• In some situations infinite mail storms can be triggered– If host 1 is holding mailing list A and host 2 holds mailing list B

and each list contains an entry for other one, then a message sent to either list could generate a never ending amount of e-mail traffic unless it is checked

• RFC281 defines ESMTP (Extended SMTP)– Clients wanting to use it should start initially with EHLO instead

HELO; if this is rejected then the server is regular SMTP server

Final delivery• Assuming that all machines can send and receive mail all

the time, the e-mail model so far works• This model breaks for people accessing Internet over a

dialup connection– What happens when Elinor wants to send Carolyn e-mail and

Carolyn is not currently online?• One solution is to have a message transfer agent on ISP

machine; since this transfer agent can be online all the time, e-mail can be sent 24 hours a day– This solution creates another problem: how does the user gets e-

mail from ISP’s message transfer agent• Solution to create another protocol that allows user transfer agents (on client

PCs) to contact the message transfer agent (on ISP’s machine) and allow e-mail to be copied from ISP to the user

• One such protocol is POP3 (Post Office Protocol Version 3), RFC 1939

Final delivery

(a) Sending and reading mail when the receiver has a permanent Internet connection and the user agent runs on the same machine as the message transfer agent.

(b) Reading e-mail when the receiver has a dial-up connection to an ISP.

POP3

• Starts when the user starts the mail reader

• Mail reader calls up the ISP (if there is no connection) and establishes a TCP connection with the message transfer agent on port 110;– Authorization

• Having user logged in by sending its username and password

– Transactions• User collecting the e-mails and marking them for deletion

– Update• Causes the e-mails to be deleted

POP3

• Sequence to receive three messages

IMAP

• POP3 works fine for users with one e-mail account with one ISP, accessed from one PC– If mail was accessed from different locations, user may loose e-

mails, security issues may appear, etc

• An alternative final delivery protocol, IMAP (Internet Message Access Protocol), defined in RFC2060– Instead assuming that all messages will be downloaded and work

offline after that (like POP3), IMAP assumes that all e-mail will remain on the server indefinite in multiple mailboxes

– Provides extensive mechanisms to read messages or parts of messages, mechanisms to create, destroy and manipulate multiple mailboxes.

POP3 vs. IMAP

Web Mail

• Various companies (i.e. Hotmail and Yahoo) provide e-mail service using Web mail.

• Normal message transfer agents are listening on port 25 for incoming SMTP connections

• Messages are delivered using special web pages; when the user goes to the e-mail Web page, a form is presented in which the user is asked for a login name and password.

WWW

• Architectural Overview

• Static Web Documents

• Dynamic Web Documents

• HTTP – The HyperText Transfer Protocol

• Performance Enhancements

• The Wireless Web

Architecture Overview

Web is a collection of web pages

Each page contains links to other pages

Hypertext – idea of having one page point to another

Browser – program to view pages

Hyperlinks – strings of text that are links to other pages

Example:

• Typical web page

• The page reached by clicking on

Department of Animal Psychology.

Architectural Overview

• Browser displays a page on the client machine– Click on a link, the browser sends a message to the abcd.com web server asking it for the

page– When page arrives, it is displayed; if it contains a hyperlink on a page on xyz.com, that is

clicked, then the browser will send a message to xyz.com server and the process continues

Client side• Pages are named using URL (Uniform Resource Locators) (i.e.

http://www.abcd.com/products.html)– Name of protocol (http)– DNS name of the machine where the page is located (www.abcd.com)– The name of the file containing the page (products.html)

• When the link is selected:– Browser detects the URL (by reading the input)– Browser asks DNS server for IP address of www.abcd.com– DNS replies with IP address: 156.106.192.32– Browser makes a TCP connection on port 80 to 156.106.192.32– It sends a request asking for file /products.html– Web server www.abcd.com sends file /products.html – TCP connection is released– Browser displays all the text in /products.html

• Web pages are written in standard language called HTML– A page may consist of a formatted document in PDF format, an icon in GIF

format, a video in MPEF format, a song in MP3 format, or any other format

Client side

• The browser may have problems interpreting all of these formats … rather than making the browsers larger and larger, a more general solution is adopted.

• When a server returns a page, it usually returns some information about the page– MIME type of the page

• Pages of type text/html are just displayed directly • If MIME type is not of a built in type, then the browser consults an internal table with

associations between MIME types and viewers• Two possibilities

– Plug-ins – special modules that the browser loads in its memory space– Helper applications – separate process that takes as parameter the name of the file to

display

Server side

• Typical web server operations:– Accept TCP connection– Get the name of the file requested– Get the file (from disk) – this can be a

lengthy operation, since every disk access takes in average 5ms (access time) + time to read the file (up the file length);

– Return the file to the client– Release the TCP connection

• Improvements– Maintain a cache with last “n”

most accessed files– Multithreaded server

Server side

• If too many requests come into one second, the CPU will not be able to manage the load, no matter how many disks are used in parallel– The solution is to add more nodes (computers), possible with

replicated disks (server farms)– A front end still accepts incoming requests and “sprays” them over

multiple CPUs rather than multiple threads– Individual machines may be multithreaded and pipelined as before

Server side

• Problems with server farms:– Cache shared between threads is lost (because each process has its own

memory)• One solution is to have the front-end smart enough to keep track where it sends each

request and send subsequent requests to the same page to the same node

– Clients TCP connection terminates at the front-end, so the reply must go through the front-end (a)

• One solution is to perform a trick known as TCP handoff, where the TCP endpoint is passed to the processing node, so it can reply directly to the client (b)

URL – Uniform Resource Locator

• Some common URLs

Stateless and cookies

• Web is stateless, no concept of login session; the browser sends a request to the server, and gets back a file; server forgets it seen that particular client– A quick solution would be to observe client’s IP addresses (not good since they could be

NAT-ed)• Cookies (Netscape) solve this problem, by having the server supply additional

information when a client requests a page; this info may include a cookie, that is a small (4KB) file or string– Contains up to five fields– When the browser sends a request to a page, it first checks to see if it has an associated

cookie with the domain the request is going to. If yes, then it appends this cookie to the request; the server gets it and interpret it any way it wants

HTML – HyperText Markup Language

• (a) HTML source code – Markup language containing explicit commands for formatting

• (b) Formatted page

Typical HTML tags

HTML versions

• HTML 1.0 was one way• Large demand for two-way traffic

– HTML 2.0 included forms; contain boxes or buttons that allow users to fill in information or make choices and send the information back to the server

Forms

• (a) HTML source code (containing forms)• (b) Formatted page• Possible response from the user (request from the browser to the server, with filled in values)

Dynamic Web documents

• So far, the model we have used is a static one– A client requests a file, the server sends it back

• Content has become more and more dynamic (generated on demand)– Content generation can take place at both server side

and client side.

Server side dynamic web page

• Steps in processing a HTML form

PHP form processing example

(a) A Web page containing a form.

(b) A PHP script for handling the output of the form.

(c) Output from the PHP script when the inputs are "Barbara" and 24 respectively.

Client-Side Web Page Generation• CGI, PHP, JSP and ASP solve the problem of handling

forms and interaction from forms– They can accept incoming information from forms, look up info in

one or more databases, generate HTML pages with the results– They can’t respond to mouse movements or interact directly with

users• Client side web page generation

– It is required to have scripts embedded into HTML pages, scripts that will be executed on client’s machine; starting with HTML 4.0, such scripts are permitted starting with tag <script>

• JavaScript is one popular client side scripting language– Use of applets (small java programs compiled for JVM)

• They can be embedded into the HTML code between <applet> … </applet> tags and can be interpreted by JVM capable browsers

– Use of ActiveX controls (compiled for the native processor, very fast)

JavaScript used for processing a form

• Same output as the PHP example

Dynamic web page generation

(a) Server-side scripting with PHP.

(b) Client-side scripting with JavaScript.

HTTP• HyperText Transfer Protocol is the transfer protocol used

throughout the World Wide Web• It specifies what messages clients may send to servers and

what responses they get back in return • Each interaction consists of one ASCII request followed by

one RFC 822 MIME-like response– All clients and all servers should obey this protocol– It is defined in RFC 2616

• Connections– A browser contacts a server by making a TCP connection on port

80• In HTTP 1.0 after a connection was setup, one request was sent over, one

response was sent back; then TCP connection was released• In HTTP 1.1 persistent connections are supported

HTTP Methods

The GET method requests the server to send the page . The page is suitably encoded in MIME. The vast majority of the requests on the web are GETs.

GET filename HTTP/1.1

Where filename is the resource name to be fetched and 1.1 is the protocol version being used

The HEAD method just asks for the message header, without the actual page. This method can be used to get a page’s time of last modification, to collect information for indexing purposes, or just to test an URL for validity

The PUT method is the reverse of GET. Instead of reading a page, it writes a page. This method makes it possible to build a collection of web pages on a remote web server. The body of the request contains the page. It may be encoded using MIME, in which case, the lines following the PUT might include Content-Type and authentication headers to prove that the caller indeed has permission to perform the operation

POST is similar to PUT. It too, bears an URL, but instead of replacing the existing data, it appends it to any existing data. Posting a message to a newsgroup or adding a file to a bulletin board are example of the usage.

DELETE does what you expect: deletes a page from a remote web server; authentication and permissions play a major role here. There is no guarantee that a DELETE operation is successful since even if the remote web server is willing to delete a page, the file permissions may be set in such a way that the server process doesn’t have rights to do it.

TRACE method is used for debugging. It instructs the server to send back the request. This is useful when requests are not processed correctly and the client wants to know it sent the proper request.

CONNECT – is not used at the moment

OPTIONS – provides a way for the client to query the server about its properties or those of a specified file

HTTP methods

• Every request gets back an response consisting of– A status line

• A three digits status code telling whether the request was satisfied and if not, why not

– Possible additional information

References

• Andrew S. Tanenbaum – Computer Networks, ISBN: 0-13066102-3