SYSTEM ADMINISTRATIONChapter 13

Security Protocols

Internet Protocol Security (IPSec)

• IPSec is an IETF standard designed to provide secure communications across both public and private networks.

• IPSec can deter several types of threats, including denial-of-service, identity spoofing, and packet sniffing.

How IPSec Works• IPSec relies on key management functions through the

use of Internet Key Exchange (IKE). • IKE provides the exchange of the required key types

between the source and destination machines that will allow identification and authentication.

• The key types supported by IPSec are:o Pre-shared Keys – same key installed on source and

destination devices.o Public Key Cryptography – also known as PKI,

requires a certificate to generate a key pair (public key and private key).

(continued)

How IPSec Works(continued)

o Digital Signatures –allows a sending device to add digital code to a transmission, thus “sealing” the transmission.

• Two types of headers are used with IPSec:

o Authentication header (AH) – provides data integrity.

o Encapsulating security payload (ESP) – provides data integrity and confidentiality.

IPSec Modes of Operation• Transport Mode

– IPSec in transport mode encrypts the payload of the packet only.

– Original IP headers remain intact with correct information. Intervening devices know the real addresses of the source and destination.

• Tunnel Model– Tunnel mode allows the entire datagram to be encrypted.

– The real source and destination addresses are hidden, replaced by the source and destination addresses of the routers that handle the process.

– End-systems do not need any configuration when deploying IPSec in tunnel mode.

Virtual Private Networks (VPNs)

• The VPN is a transmission between two systems that makes use of the public infrastructure as the medium for transmission, extending the boundary of the private network.

• VPNs rely on tunneling to create a safe transmission.• The tunneling protocol “wraps’ the packet (often just

the header), creating a virtual tunnel through which the data can be transmitted.

• The encapsulation provides the needed routing information.

(continued)

Virtual Private Networks (VPNs)(continued)

• VPN transmissions usually contain an encrypted payload.

• The advantages of VPNs include:o Safety of transmissiono Flexibility in the business environmento Lower transmission costso Lower administrative overhead

Point-to-Point Tunneling Protocol

• PPTP is built on PPP used for remote access connections.

• Transmissions are subject to setup negotiation, authentication, and error-checking.

• PPTP supports a multiprotocol environment, using IP as the transport protocol, but allowing other protocols (IPX, NetBEUI) to be used for communication on the remote network.

• PPTP uses MPPE as its encryption protocol on Microsoft networks.

• PPTP supports 40-bit, 56-bit, and 128-bit encryption schemes.

Layer 2 Tunneling Protocol (L2TP)

• L2TP is a relatively new tunneling protocol, built by combining Microsoft’s PPTP and Cisco’s L2F technology.

• L2TP uses a five-step process for encapsulation.

Deploying L2TP and IPSec

• L2TP and IPSec are used together on Microsoft networks to provide secure communications over the Internet or intranet.

• When combined, L2TP provides the tunnel and IPSec provides the payload encryption necessary for security.

• To communicate using L2TP/IPSec, both the source and destination devices must understand the mechanisms and be configured to use them.

Secure Sockets Layer (SSL)• Secure Sockets Layer (SSL) is a protocol that has

been designed to provide a secure connection over an insecure network, such as the Internet.

• SSL runs above the TCP/IP protocol and below some of the higher-level protocols such as Hypertext Transfer Protocol (HTTP) and File Transfer Protocol (FTP).

• SSL uses a series of keys, public and private, to encrypt the data that is transported across the secure connection.

(continued)

Secure Sockets Layer (SSL)(continued)

• The RSA algorithm, or cipher, is a commonly used encryption and authentication algorithm that includes the use of a digital certificate.

• The public key is made available to whomever needs it, while the private key is stored in a central location and never made public.

• Data that is encrypted with the public key can be decrypted only with the private key.

SSL Server Authentication

• SSL server authentication allows a client computer to identify the server that it is talking with.

• A client using SSL-enabled software uses a public key to verify that the server’s certificate and public ID are correct and valid and that they have been issued by a certificate authority (CA) that is listed on the client’s list of trusted CAs.

SSL Client Authentication

• SSL client authentication is used to verify the client’s identity.

• SSL-enabled server software checks the client’s certificate and public ID to ensure they are correct and valid and that they have been issued by a CA listed on the server’s list of trusted CAs.

Encrypted SSL Connection

• The encrypted SSL connection ensures that all of the information transferred between the SSL-enabled client and SSL-enabled server are encrypted and decrypted during transmission.

• Also, all of the data transmitted across the connection contains a mechanism to detect tampering, so the data can be checked to see if it was altered during the transfer process.

SSL Subprotocols• The SSL Handshake Protocol

– An SSL session begins with the SSL handshake process. – The handshake process is an exchange of messages that

the server uses to authenticate itself to the client using a public key.

– The client and the server cooperate to create symmetric keys that will be used for the encryption, decryption, and tamper-detection processes that occur during data transmission.

– If necessary, the handshake process will also allow the client to authenticate itself to the server.

• The SSL Record Protocol– The SSL Record protocol is used to define the message

format that is used to transmit encrypted data. – The record protocol uses a series of algorithms that are

generated by the handshaking process to encrypt the transmitted data.

Man-in-the-Middle Attack

• The Man in the Middle is a rogue program that intercepts all communication between the client and a server during an SSL session.

Kerberos• Kerberos is a secure system, using strong encryption

processes that are designed to provide authentication for users and services that need to communicate and be validated on a network.

• Kerberos provides a way to prove identity in order to gain access to other network resources.

• Kerberos works through the use of encrypted tickets and server processes that run on one or more third-party trusted servers.

• The principals and the Kerberos server all share a secret password.

• This secret password is used to verify that messages are authentic.

 

Understanding the Kerberos Process

• Begin the process by requesting authentication from the third-party, trusted Kerberos server.

• This authentication server (AS) will create a session key, or “ticket-granting ticket” (TGT).• TGT goes to a ticket-granting server (TGS).

• TGS verifies the ticket time stamps it and returns it to the principal that submitted it.

• Ticket can be sent to accessible service.

• Service can accept or reject the ticket.

• Since the ticket was time stamped by the TGS, it is valid for more than one session.

• Kerberos is the default encryption and security system used with Microsoft Windows 2000 operating systems.