Web Technologieskti.tugraz.at/staff/vsabol/courses/wt/wt_slides_bwp.pdf · Basic Web Programming Web Technologies VU (706.704) Vedran Sabol ISDS, TU Graz Oct 14, 2019 Vedran Sabol

Web TechnologiesVU (706.704)

Vedran Sabol

ISDS, TU Graz

Oct 12, 2020

Vedran Sabol (ISDS, TU Graz) Web Technologies Oct 12, 2020 1 / 67

Outline

1 Internet and the Web

2 Web Applications

3 Asynchronous Web Applications

4 Ajax Discussion

5 HTTP

6 Server-side Functionality

Vedran Sabol (ISDS, TU Graz) Web Technologies Oct 12, 2020 2 / 67

Basic Web ProgrammingWeb Technologies VU (706.704)

Vedran Sabol

ISDS, TU Graz

Oct 12, 2020

Vedran Sabol (ISDS, TU Graz) Basic Web Programming Oct 12, 2020 3 / 67

Internet and the Web

The Web

Three essential components

System of unique resource identifiers: the URL

Language for publishing in the Web: HTML

Data transfer protocol: HTTP



Internet - Technical Background

Internet is the net of the nets

TCP/IP is the communication protocol for the Internet

A computer communication protocol is a description of the rulescomputers must follow to communicate with each other

TCP/IP stands for Transmission Control Protocol / Internet Protocolthat work together



Technical Background

Inside the TCP/IP standard there are a number of protocols forhandling data communication

IP (Internet Protocol) communication between computers (routing,IP addresses): connectionless, packet-oriented

TCP (Transmission Control Protocol) communication betweenapplications: reliable, connection-oriented

UDP (User Datagram Protocol) simple, fast communication betweenapplications: unreliable, connectionless

ICMP (Internet Control Message Protocol) for errors and statistics,used by routers

DHCP (Dynamic Host Configuration Protocol) for dynamic IPaddressing and config. (IP address, subnet mask, default gateway)




IP addressing:

IPv4: 32 bit long (4 bytes) / dot notation

e.g. 2166031126 = 10000001000110110000001100010110→ 10000001.00011011.00000011.00010110→ 129.27.3.22

IPv6: 128 bit (2128 different addresses, 7 ∗ 1023 pro m2)

draft standard in Dec 1998, Internet standard since Jul 2017

as of 2019 majority of traffic still over IPv4




hierarchical naming service (e.g. news.tu-graz.ac.at → 129.27.3.22)

decentralised infrastructure: DNS name servers translate of names toIP addresses

int com edu gov mil org net jp�

us nl . . .

acm ieeesun yale ac co oce vu�

jack�

cs engeng jill�

keio nec

flitscs cslai linda fluit

cs

pc24robot

Generic Countries

Figure: Domain Name Space




Higher-layer applications are referred to by a port identifier inTCP/UDP messages

The port identifier and IP address together form a socket

The end-to-end communication between two hosts is uniquelyidentified on the Internet by the four-tuple

(source port, source address, destination port, destination address)

TCP/UDP and specific ports are the basis for the Internet services



Services in the Internet

Telnet: virtual terminal providing bidirectional text communication

Email: exchange of digital messages (non real-time)

Usenet (news): discussion system - post/receive messages on topics

FTP: file transfer protocol

WWW: information space consisting of addressable, interlinkedhypertext documents



Protocols on top of TCP/IP

Specific for a service

RFCs: Request for Comments

publications by Internet Engineering Task Force (IETF,http://www.ietf.org/rfc.html)

describe methods and behaviours applicable to the functioning of theInternet

HTTP1.0: RFC 1945

HTTP/1.1 (current since 1999): RFC 2616

FTP: RFC 959


http://www.ietf.org/rfc.html


Protocols on top of TCP/IP

Figure: Protocols



Uniform Resource Locator (URL)

Unique reference to a resource: addressability as core Web concept

The protocol, followed by ://

host address as a domain name or IP address

optional - :port number

path of the ressource on the host

http://kti.tugraz.at:

80/staff/vsabol/courses/wt/wt_slides_intro.pdf

descriptiveness, readability, memorizable

use readable words, ”-” to separate words, avoid timestamps orcounters


http://kti.tugraz.at:80/staff/vsabol/courses/wt/wt_slides_intro.pdf

http://kti.tugraz.at:80/staff/vsabol/courses/wt/wt_slides_intro.pdf

Web Applications

Web application models

Static Web model

The Web started as a static information system

This model was very efficient and scalable

No processing on the server side

Caching is easily possible


Web Applications


Figure: Static Web Model


Web Applications


Template Web model

Early dynamic web applications

Web-gateways to existing (database) applications

Templates dynamically filled with information

Still very efficient and scalable

Still possible to cache queries on data (if data did not change)


Web Applications


Figure: Template Web Model


Web Applications


Next generation of Web applications dealt with personalized content

Provide different content for each user

Depending on user preferencesDepending on the state of applications

E.g. online shops


Web Applications


Need to track users and their sessions

Cookies store web site data

HttpSession at the level of servlets, JSPs

Manage session information, bind user objects to session

Keep user-specific objects in the server’s memory

Make user-specific objects persistent and load them as the need arises


Web Applications


Figure: Personalized Web Model


Web Applications


User-specific objects tend to grow in size very fast

Keeping them in memory might compromise performance andscalability on the server side

Early solutions: make server clusters

You need to share session-related objects between cluster nodes

Complexity arises and again performance and scalability suffer


Web Applications


Also problems with caching

Every response is generated according to the request and the currentstate of session on the server

Solution: calculate all possible combinations of request/session andcache them on the server

However, server processing is still needed to decide what to pull outof the cache


Asynchronous Web Applications

Asynchronous application model

Instead: move application state and session data to the client

Execute application logic on the client

Update the page as the need arises by fetching the new data from theserver

Still problems because old Web applications have been page-based

Usability leaves much to be desired




With the introduction of XMLHttpRequest object in browser it ispossible to make an HTTP request from a browser in the background

The users do not notice this at all

With JavaScript the response can be processed asynchronously

The parts of a page can be updated dynamically

Usability tremendously improved




Figure: Client Web Model



XMLHttpRequest Object

W3C publishes specifications for XMLHttpRequest

Methods:

open(method, url, async)get/setRequestHeadersend(data)

Specifies a callback function for processing of request (asynchrony)

Listener called in UI thread: onreadystatechange




XMLHttpRequest.readyState property changes accordingly

0 - unsent, 1 - opened, 2 - headers received, 3 - loading4 - done

XMLHttpRequest.status property: HTTP status codes

responseType property (”text”, ”document”, ”json”...)

Response: responseText, responseXML




var req;

function loadXMLDoc(url)

{

if (window.XMLHttpRequest) {

req = new XMLHttpRequest();

req.onreadystatechange = processReqChange;

req.open("GET", url, true);

req.send(null);

}

}




function processReqChange()

{

// only if req shows "complete"

if (req.readyState == 4) {

// only if "OK"

if (req.status == 200) {

// ...processing statements go here...

} else {

alert("There was a problem retrieving

the XML data:\n" + req.statusText);

}

}

}



AJAX

Ajax = Asynchronous JavaScript and XML

Components of the Ajax mix

Standards based presentation using (X)HTML and CSSDynamic display and interaction using DOMData interchange and manipulation using XML and JSONAsynchronous data retrieval using XMLHttpRequestJavaScript that binds all together



AJAX Web Model



AJAX Web Model



AJAX

The intent is to make web pages feel more responsive

By exchanging small amounts of data with the server behind thescenes

The entire web page does not have to be reloaded each time the usermakes a change

Meant to increase the web page’s interactivity, speed, and usability



AJAX: Discussion

If used properly on both the client side and the server side exhibitsgreat advantages

Properly on the client side: conform with (X)HTML, CSS andJavaScript standards

Properly on the server: no state on the server



AJAX: XML

Data retrieved from the server is in XML format: need to process theXML on the client

XML data is parsed by the browser in a DOM object

Tree structure that is traversed to retrieve the informationProblems: different browsers have different DOM implementations,XML verbosity and complexity

Alternative: JSON (JavaScript Object Notation) gaining popularity



AJAX: JSON

Readability

Smaller data footprint (= less network overhead)

Usually faster to parse then XML

Straightforward JS object serialisation (eval(), JSON.parse(), JSON.stringify())

Platform independent



AJAX: JSON

{"class":

[{"student":{"id":1},"name":"Linda Jones",

"legacySkill":"Access, VB 5.0"},

{"student":{"id":2},"name":"Adam Davidson",

"legacySkill":"Cobol, MainFrame"},

{"student":{"id":3},"name":"Charles Boyer",

"legacySkill":"HTML, XML"}]}


Ajax Discussion

Ajax architecture

Ajax architectural style involves a browser-side language (JS) makingasynchronous HTTP requests

An Ajax application is a web client that runs inside a web browser

A user makes a request for the main URL of the application

The server serves a page that contains an embedded script


Ajax Discussion

Ajax architecture

The browser renders the page

Behind the scene HTTP requests are made as response to user events

The script parses the response and uses the data to modify the GUI

The same architecture as a rich client GUI applications


Ajax Discussion

Ajax architecture

The downside: every application state has the same URL

Addressability of the Web is destroyed!

The Ajax application talks to Web services and uses addressability

But the Ajax application is not fully addressable


Ajax Discussion

Solving Ajax problems

Users see a HTML page (that changes) and one URL

User can not bookmark application states

Cannot send a link to a state per email

Cannot use the back button

Easy for an Ajax application to use all the Web benefits but leavenone for the end-user

Long-time issue with Gmail and Google Maps - now (at leastpartially) addressed


Ajax Discussion


Ajax applications disconnect the end user from the HTTPrequest-response cycle

When you visit the URL of an Ajax application you leave the Web

From that point you are using a GUI application that makes HTTPrequests for you

But an Ajax application can give its users the benefits of the Web


Ajax Discussion


Solution: implement some of the browser features in the application

Some extra DOM work keeps the URL up to date as you navigate

Manage navigation history: insert back and forward links usingHTML5 APIs (next 2 lectures)


HTTP

HTTP

Original intention to retrieve and publish hypertext documents

HTTP is a request/response protocol between a client and a server

Connectionless protocol

Connection is closed by the server after the response is sent

A server serves a number of Web resources, e.g. HTML pages, images

Each resource is uniquely addressable by its URL


HTTP

HTTP Request Message

Request line, e.g. GET /lectures/wt/ HTTP/1.1

Headers such as: Host, Accept-Language, ...

An empty line

Optionally message body

Figure: HTTP Request Message Example (Source)


http://www3.ntu.edu.sg/home/ehchua/programming/webprogramming/http_basics.html

HTTP

HTTP Request Methods

As of HTTP 1.1: 8 request methods

GET: request for a representation of a resource

Representation in a particular media format: text/html, image/png, ...

Should not have side-effects: only retrieves the data

HEAD: asks for response identical to GET but without message body

To retrieve meta-information on the resource

E.g. last-modified date: check against a local cache copy


HTTP


POST: submits data to the server for processing by a specifiedresource (e.g. HTML forms)

Data included and encoded in the message body

PUT: uploads a representation of a specific resource

Data in the message body


HTTP


DELETE: deletes a specified resource

TRACE: echoes the request to see changes by the intermediaries

OPTIONS: Retrieves methods supported by the server for the URL

CONNECT: Converts the connection to a tunnel typically to facilitateSSL communication (https)


HTTP

HTTP Safe Methods

HEAD, GET, OPTIONS, and TRACE are defined as safe

These methods should not have side-effects

They should not change the state of resources on a server

But: GET often misused to e.g. delete a database record on a server

POST, PUT, DELETE change the resource state


HTTP

HTTP Idempotent Methods

Multiple indentical requests must have the same result

GET, HEAD, PUT, DELETE, OPTIONS and TRACE are idempotent

Misusing GET for changing the state violates idempotence definition


HTTP

HTTP Response

The first line of response is the status code

Headers (for handling response, e.g. encoding, language, length,last-modified...)

Message body (representation of a resource)


HTTP

HTTP Response

Figure: HTTP Status Codes


HTTP

HTTP Response: Example

HTTP/1.1 200 OK

ETag: W/"413-1160316312000"

Last-Modified: Sun, 08 Oct 2006 14:05:12 GMT

Content-Type: text/html

Content-Length: 413

Date: Mon, 12 Nov 2007 11:08:09 GMT

Server: Apache-Coyote/1.1

<html>

...


HTTP

HTTP/2

Next version of the HTTP (since HTTP1.1 update in 1999)

IETF standard since May 2015 (RFC 7540)

Triggered by Google’s SPDY protocol

Does not change how Web applications work

Mainly same methods, status codes, header fields

New features improving performance and decreasing latency

Changes in framing and transporting dataCompression, parallel loading, server-push etc.


Server-side Functionality

Server-side

In the beginnnig Web servers served documents, images, etc.

Static information stored on the server side (file system)

HTML forms: dynamic server response

Server responds differently depending on submitted values



Server-side

Need to extend the functionality of Web servers

Don’t add the new functionality into Web servers directly

Just allow Web servers to communicate with external programs

External programs generate dynamic content depending on valuessubmitted by HTML form

Dynamic content forwarded to Web server

Web server responds with dynamic content



Communication between Web server and external programs

How should Web server communicate with external programs?

Passing parameters, getting response, etc.

Standardized communication mechanism

Standard created by Web consortium



Common Gateway Interface (CGI)

CGI is a specification of communication between Web server andexternal programs

Current version CGI 1.1: http://www.w3.org/CGI/

Very general approach, can be applied for different applications

Not only HTML form evaluation

Web server must implement CGI specification

All major Web servers do! (e.g. Apache, IIS, etc.)


http://www.w3.org/CGI/


Common Gateway Interface (CGI)

A Web server is configured to interpret a URL as a CGI script

E.g. using a filename extension

CGI scripts: external programs written in a scripting language

Read some input, process it and output results

Results incorporated into the Web page served by the Web-Server



CGI Specification

Environment variables

System specific variables set by the Web server

External program reads environment variables and obtains data aboutclient request

REQUEST_METHOD, PATH_INFO, QUERY_STRING,

CONTENT_LENGTH, CONTENT_TYPE, REMOTE_ADDR, REMOTE_HOST, etc.

Alternative input: standard input parsed by the script

Script output: standard output



CGI Specification

HTTP method used by the client: GET or POST

GET method: external program reads environment variables

QUERY_STRING special environment variable containing datasubmitted by user

POST method: external program reads from standard input

External program needs to parse the input

UNIX shell scripts, Perl scripts, PHP scripts, C programs, C++programs, JavaScript...



CGI Example

#!/bin/sh

# send http-header and a newline afterwards:

echo "Content-Type: text/html"

echo ""



CGI Example

# send html content:

echo "<HTML>"

echo " <HEAD>"

echo " <TITLE>Hello World CGI</TITLE>"

echo " </HEAD>"

echo " <BODY>"

echo " Hello World ("

date "+%T, %d.%m.%Y"

echo ")"

echo " </BODY>"

echo "</HTML>"



PHP, Java, ...

CGI performance issue: external program started each time a script isexecuted

Extensions for all major programming languages: PHP, Python, ...

E.g. Apache module architecture: embedding scripting in the server

Runs much faster than CGI

Java: Apache Tomcat

Java servlets and Java server pages



What to take?

Depeneds on personal preferences and requirements

Performance, e.g. interpreted vs. compiled language

Dynamically vs. statically typed languages (fast development vs. errorchecking)

Scalability and failover capability of the framework

Requirements, e.g. embed an existing Java application

Popular on the Web: PHP, Python, Ruby, ...

In business environments mostly Java



Security

Check parameters carefully!!!

if($email =~ /[^a-zA-Z0-9_\-\.@]/){

$_ = "The email address should be of

the form <i>user\@server</i>!";

}else{

$_ = qx($finger $email);

}



Security

Suppose this e-mail address: something ; mail [email protected] >/etc/passwd

Basically you let other people start programs on the server

Check what they want to do on your server!!!

Not only CGI! (PHP, Java Servlets, etc.)


Documents

Web Technologieskti.tugraz.at/staff/vsabol/courses/wt/wt_slides_bwp.pdf · Basic Web Programming Web Technologies VU (706.704) Vedran Sabol ISDS, TU Graz Oct 14, 2019 Vedran Sabol