Unique Opportunities in Experimental Computer Systems Research

- the Berkeley Testbeds

David Culler

http://www.cs.berkeley.edu/~culler

U.C. Berkeley

Grad Student Presentations

8/27/1999

8/26/99 Grad Presentation 2

Emerging Problems of Scale

Scalable, AvailableInternet Services

Info. appliances

ClientServer

Clusters

Massive Cluster

Gigabit Ethernet

8/26/99 Grad Presentation 3

Convergence at the Extremes

• Powerful Services on “Small” Devices– massive computing and storage in the infrastructure

– active adaptation of form and content “along the way”

• Extremes more alike that either is to the middle– More specialized in function

– Communication centric design

» wide range of networking options

– Federated System of Many Many Systems

– Hands-off operation, mgmt, development

– High Reliability, Availability

– Scalability

– Power and space limited

– simplicity

• They have to “work or die!”

8/26/99 Grad Presentation 4

100 node Ultra/Myrinet NOW

•GLUnix

•Active Messages

•xFS

•Fast sockets,

•MPI, and SVM

•Titanium and Split-C

•ScaLapack

8/26/99 Grad Presentation 5

Novel Systems Design

• Virtual networks– integrate communication events into virtual memory system

• Implicit Co-scheduling– cause local schedulers to co-schedule parallel computations

using a two-phase spin-block and observing round-trip

• Co-operative caching– access remote caches, rather than local disk, and enlarge global

cache coverage by simple cooperation

• Reactive Scalable I/O

• Network virtual memory, fast sockets

• ISAAC “active” security

• Internet Server Architecture

• TACC Proxy architecture

8/26/99 Grad Presentation 6

The Millennium Vision

• To work, think, and study in a computationally rich environment with deep information stores and powerful services

– test ideas through simulation

– explore and investigate data and information

– share, manipulate, and interact through natural actions

• Organized in a manner consistent with the University setting

– clusters of clusters

– Computational Economy

• Novel modes of interacting with large amounts of data

8/26/99 Grad Presentation 7

The Millennium Community

School of Info. Mgmt and Sys.

Computer Science

Electrical Eng.

Mechanical Eng.

BMRC

Nuclear Eng.

IEORCivilEng.

MSMEInst. OfTransport

Business

Chemistry

Astro

Physics

Biology

EconomyMath

8/26/99 Grad Presentation 8

NT Workstations for Sci. & Eng.

SIMS

C.S.

E.E.

M.E.

BMRC

N.E.

IEORC. E. MSME

Transport

Business

Chemistry

Astro

Physics

Biology

EconomyMath

8/26/99 Grad Presentation 9

SMP => storage, small-scale parallelism

SIMS

C.S.

E.E.

M.E.

BMRC

N.E.

IEORC. E. MSME

Transport

Business

Chemistry

Astro

Physics

Biology

EconomyMath

8/26/99 Grad Presentation 10

Group Cluster of SMPs => Parallelism

SIMS

C.S.

E.E.

M.E.

BMRC

N.E.

IEORC. E. MSME

NERSC

Transport

Business

Chemistry

Astro

Physics

Biology

EconomyMath

8/26/99 Grad Presentation 11

Campus Cluster => large-scale Parallelism

SIMS

C.S.

E.E.

M.E.

BMRC

N.E.

IEORC. E. MSME

NERSC

Transport

Business

Chemistry

Astro

Physics

Biology

EconomyMath

8/26/99 Grad Presentation 12

Gigabit Ethernet Connectivity

Gigabit Ethernet

SIMS

C.S.

E.E.

M.E.

BMRC

N.E.

IEORC. E. MSME

NERSC

Transport

Business

Chemistry

Astro

Physics

Biology

EconomyMath

8/26/99 Grad Presentation 13

FIAT LUX: crossing areas

• Combines– Image Based Modeling and Rendering, – Image Based Lighting, – Dynamics Simulation and – Global Illumination in a completely novel fashion to achieve

unprecedented levels of scientific accuracy and realism

• Computing Requirements– 15 Days of worth of time for development.– 5 Days for rendering Final piece.– 4 Days for rendering in HDTV resolution on 140 Processors

• Storage– 72,000 Frames, 108 Gigabytes of storage– 7.2 Gigs after motion blur– 500 MB JPEG

• premiere at the SIGGRAPH 99 Electronic Theater– http://fiatlux.berkeley.edu/

8/26/99 Grad Presentation 14

An upcoming Comp. Econ Experiment

• Two identical 32 proc Millennium Clusters

• One open shop

• One with usage based on bid-based proportional share scheduling

8/26/99 Grad Presentation 15

Ninja: Push Services into an Active Infrastr.

Servers

Clients

ClientsClients

ClientsClients

Clients

Servers

Servers

Infrastructure Services

Open

=> enable Distributed Innovation of Scalable, Avail. Services

8/26/99 Grad Presentation 16

Universal Computing Lab (464 Soda)

• Computing in the infra, in the walls, on the desk, in your hand, ...

• Not just a new project, a new computing culture

8/26/99 Grad Presentation 17

universal

Function: adjective

1 : including or covering all or a whole collectively or distributively without limit or exception

2 a : present or occurring everywhere b : existent or operative everywhere or under all conditions <universal cultural patterns>

3 a : embracing a major part or the greatest portion (as of mankind) <a universal state> <universal practices> b : comprehensively broad and versatile <a universal genius>

4 a : affirming or denying something of all members of a class or of all values of a variable b : denoting every member of a class <a universal term>

5 : adapted or adjustable to meet varied requirements (as of use, shape, or size)