Design of a Scalable Clearing House Architecture

Lakshminarayanan Subramanian

Chen-Nee Chuah

Ramakrishna Gummadi

ICEBERG Design Review Jan 12, 2000

Basic Questions in Mind!!!

What is a Clearing House? What are the basic requirements of the

Clearing House? What are the services it supports? What are the goals of our design? What are the assumptions we make in our

design?

Clearing House

Coordinates interactions between the various ISPs in the network.

What kind of interactions? Performs path discovery and resource

reservation. Services wide-area call requests. Provide QoS guarantees. Secure billing services.

Support for multicast and mobility.

Present Scenario

H1

H3

ISP1

ISP3

ISP2

H2

ISP4

Goals of our design

Scalability- throughput, state maintained Optimize network utilization Dynamic call-routing Continuous path monitoring for QoS Reduce response time for call requests Support multicast, mobility and secure billing Recovery from link,node and packet failures Security and Privacy

How do we achieve it!!!

Build a logical hierarchy in the network Distribute state and resources among the

nodes in the hierarchy and create a distributed database

Aggregate requests and bound queue size Hierarchical and dynamic routing of call

requests Continuous monitoring of resources Separate resource reservation and call-setup

Assumptions

Edge routers can collect traffic statistics and estimate bandwidth requirements

Control and data paths are separated Clearing House and ISP trust each other Routers can measure queueing delay

statistics Possible to introduce a monitoring system

into existing ISP architecture

Clearing House Structure

ISP1

ISP3

ISP2

ISP4

ICH

ICH

ICHICH

ECH

Clearing House Infrastructure

External Clearing House(ECH) as third party agent to coordinate inter-ISP traffic.

Internal Clearing House(ICH) services intra-ISP traffic and acts as a monitoring agent for external traffic.

ECH organized as a hierarchy of nodes. ECH stores inter-ISP network state and ICH

stores intra-ISP network state in a distributed manner.

Hierarchical Structure

Divide network into non-intersecting basic domains(e.g.. Cluster area codes)

Recursively join physically adjacent domains to form larger logical domains.

Generate a hierarchical tree of domains in the network.

Associate a distributed ECH with every domain in the tree.

Hierarchical Clearing House

ISP1

ISP3

ISP2

ISP4

ICH

ICH

ICH

ECHICH

ECH

ECHDomain

External Clearing House

Performs hierarchical routing and computes near optimal path for call requests.

Aggregates call requests. Collects statistics on resource reservations and

delay statistics from ISPs. Performs extra resource reservations for call

requests if necessary. Monitors performance of traffic. Stores billing prices of ISPs within its domain

Internal Clearing House

Every ISP has an ICH. Routes intra-ISP calls. Monitors and predicts incoming and

outgoing traffic in edge routers Performs advanced reservations for

predicted traffic and updates ECH. Determines link reservations in ISP and

updates traffic routing table of routers.

Hierarchical Routing

1a 1b 1c

Inter-domain and Intra-domain paths

Domain 1

Clearing house state

Billing information is present in CH of basic domain.

Each CH maintains aggregated state of its domain. Calls between two sub-domains of its domain. Aggregated connectivity graph between

domains. Reservation and delay status along links and

nodes in the graph. Pricing information between domains.

Other Enhancements

Caching Cache computed inter-domain paths

RxW scheduling Maximize throughput without affecting response time.

Measuring QoS parameters Multicast support Dynamic path routing to support mobility Secure billing architecture Fault tolerance

Support for Multicast and Broadcast Trees

• Nodes up in the hierarchy find inter-domain multicast tree.• Local nodes find intra-domain optimal tree.

Edge router

Level 0

L1

L1

Moving Object

DomainStructure

Scalable Infrastructure for supporting Mobility

Strengths

State of network distributed among various CH nodes.

Aggregation of call requests. Response time depends on locality. Bounded queue size. Path discovery is distributed. Localized billing – makes it real-time. Core routers do not maintain much state info. Caching, scheduling improve performance.

Clearing House Design:Resource Reservation Strategies

Chen-Nee Chuah

ICEBERG Design Review

Jan 12, 2000

ISP 1ISP 2

ISP 3

Example Scenario

Quality of Service?

Resource Reservation

H2

H3H1

ISP 1ISP 2

ISP 3

Example Scenario

H2

H3H1SLA

SLA

SLA: Agreements that describe the volume of traffic exchanged, bandwidth reserved and price

Challenges

How is the SLA between two ISPs established? How do SLAs reflect dynamic traffic patterns? What happens when it involves more than 2 ISPs?

=> Clearing House provides a scalable approach to address these questions

Hierarchical Clearing House

source

ISP n

destination

Edge Router

CH1

ICH

CH1

CH2

ISP2

Distributed database & bandwidth brokering agent • reservation status, % link utilization, traffic predictor• establish advanced reservation (based on traffic predictor)

UpdatesUpdates

ISP1

Adapt Reservation

ICH ICH

Resource Reservation Infrastructure

H1

ISP1

H2

ICH

Edge Router

Edge Router

ICH

Assume the Edge Router collects traffic statistics

e.g. average aggregate incoming and outgoing traffic volume

estimates dynamic change of bandwidth requirements statistical techniques (Kalman

filter)

sends regular updates to LCH aggregates reservation

requestsISP2

Static & Dynamic Reservations

H1

ISP1

H2

ICH

Edge Router

Edge Router

Internal Clearing House Maintain intra-ISP reservation

status Establish static reservations

based on mean aggregate traffic for different time of the day.

Adapt reservations on a smaller time-scale based on existing reservation and bandwidth predictor.

Send regular update to GCHStatic Reservation

Dynamic Reservation

CH

Properties

Aggregation of Signaling Resource reservation requests are aggregated at various

levels (ER -> ICH, ICH-> CH1 etc.)

De-couple notifications & reservation requests notifications: updates on reservation status, % link

utilization, traffic predictor reservation requests: initiation of SLA renegotiations

Hierarchical Approach Static and Dynamic Reservations

reduce reservation setup time compensate for the coarse granularity of the notifications

Clearing House Hierarchical Tree

Notifications (every u s)- Reservation status- Link utilization- Bandwidth predictor

CH1

ICH

CH2

CH1

ICH ICH ICH

Adapt Reservations- Advance reservations- Process reservation requests

Aggregate reservation requests (Ta)

LCHLCH

Int-Serv Approach

End-to-end notifications & reservation requests ISP 2 notifies next-hop ISPs and negotiate new SLA

with them. When all downstream ISPs accept the SLAs, an ISP notifies upstream ISPs and set up new SLAs.

When original SLA is accepted, all SLAs from source to sink are updated.

source

ISP1

ISP n

destination

BB BB

BB

ISP2

Diff-Serv Approach

Limited or no notifications Trade-off end-to-end QoS for scalability

source

ISP1

ISP n

destination

Edge Router

BB BB BB BB

ISP2

Evaluation

Overall Performance Metrics Trade-offs between scalability, QoS and signaling

complexity

• Effect of aggregation on QoS – e.g. % blocking/dropping

• Choice of signaling between CHs Link efficiency

Bandwidth Estimator How well do the predictors track the traffic fluctuation?

• Window of measurement?

Clearing House Design:Billing, Security and Privacy

Ramakrishna Gummadi

ICEBERG Design Review

Jan 12, 2000

Basic Goals

Support Scalable, Secure and Correct Billing

Support Trust Management for Traffic Monitoring

Support Privacy Management

Tasks while supporting Secure and Scalable Billing

Must scale to millions of calls per day Must perform authentication (in both

directions), authorization, and correct billing

Approaches to support Secure and Scalable Billing

Use a level of indirection through authorization and billing tickets

Generate these tickets offline Perform offline settlements with the user and

various ISPs Use aggregation for storing and verifying tickets

to reduce storage space Use X.509 certificates, passwords or Public-key

challenge/response for mutual authentication

Notes on Authorization and Billing Tickets

Both used as level of indirection, for achieving scalability, while maintaining high security and requiring little trust

Both like Kerberos, a scalable security service, using tickets for authentication and secrecy

Both acquired by the user once at the beginning, and used as needed

Notes on Authorization and Billing Tickets (contd..)

Authorization tickets used to establish that call corresponds to resources reserved

Billing tickets used to charge the user for time spent on the call

Billing tickets can be returned by the user at end of call, or more can be acquired during duration of call, as needed, to maintain correct billing records

Performance Optimizations

Can use shared-key techniques in using authorization tickets

A lot depends on the degree of trust between the CH and the ISPs (though the ISPs themselves don’t need to trust each other)

If trust possible, we can use shared-key cryptography for billing (no non-repudiation support)

Lots of performance and storage improvement through aggregation

Trust Management

CH can incorporate a Trust Management module to: Provide a standard, general-purpose mechanism for

specifying application security and credentials Directly authorize security-critical actions, like network

monitoring Bind keys directly to authorization records to perform

specific tasks Describe delegation of trust and subsume the role of

public key certificates

Privacy Management

Privacy management very difficult in the current Internet, more so in ICEBERG (because of billing)

Privacy of users and participating ISPs needed

User privacy with respect to participating ISPs achieved by anonymization in the form of indirect authorization and billing tickets