GSLIS ProseminarGSLIS ProseminarFebruary 24, 2003February 24, 2003

The Evolution of the Net:The Evolution of the Net:Predicting Network InfrastructurePredicting Network Infrastructure

Bruce R. SchatzGraduate School of Library and Information ScienceUniversity of Illinois at Urbana-Champaign (USA)

Graduate School of InformaticsKyoto University (August 2001 – July 2002)

schatz@uiuc.edu, www.canis.uiuc.edu

The Future of InfrastructureThe Future of InfrastructureThe Future of InfrastructureThe Future of Infrastructure

Niels Bohr on Quantum Theory“Prediction is very Difficult, especially about the Future”

What is Infrastructure?What is Infrastructure? Infra means Inside Structure means Support

Infrastructure is: Essential Services for Everyday Life Universal Protocols to Help Users

Infrastructure Evolves to Deeper Levels New Technologies become Generic Enough

Outline of TalkOutline of TalkThe Evolution of Infrastructure

Amateurs become Experts via Technology

The Waves of the Net Data Information Knowledge

The Laws of the Net The Basic Units of Evolution

Evolution of Infrastructure Evolution of Infrastructure Evolution of Infrastructure Evolution of Infrastructure

Everyone becomesAn Expert with

Interactive Support

The Structures of Everyday LifeThe Structures of Everyday Life Bodies (individuals)

Food and Clothes Buildings (groups)

Houses and Cities

Transportation (physical interactions) Rails (trains) and Roads (cars)

Communication (logical interactions) Phones (talking) and Computers (retrieving)

How Infrastructure ScalesHow Infrastructure Scales Experts are Effective but Expensive

Initially: All things done by Experts Eventually: Most things done by Amateurs

Volume forces Provider Pyramid A few experts at the top A lot of amateurs at the bottom

Infrastructure supports Ordinary People Everyone becomes a little bit of an Expert Most interactions done by amateurs

Example: TalkingExample: Talking Transmission of Messages

Generated by a person and Sent to another Network connects Operators to Operators

The Telegraph of the 19th Century Station to Station, trained professionals Operator transmits coded message

The Telephone of the 20th Century Person to Person, untrained amateurs Person speaks in ordinary words Infrastructure transmits coded message

Example: RetrievingExample: Retrieving Search of Archives

Generated by many and Retrieved by many Network connects Archives to Users

Bibliographic Search of the 20th Century Station to Station, trained professionals Librarian queries central archives

Web Search of the 21st Century Person to Person, untrained amateurs Person types in ordinary words Infrastructure queries central archives

Provider Pyramids EmergeProvider Pyramids Emerge Telephone Network was Station-Station

Person talked to Telephone Operator E.g. “get me a doctor!” Operator did manual switching to Person

Volume forced Network to Person-Person In 1900, everyone must become an operator In 2000, everyone is an operator! Amateurs use directories then dial numbers

Infrastructure supports Everyday Activity Automatic switching scales to volume

The Evolution of InfrastructureThe Evolution of Infrastructure Experts handle Everyday Structures

Provide services that become essential Increased Volume forces Provider Pyramid

Amateurs handle most Interactions Infrastructure supports Amateurs Free Labor at Acceptable Quality

Evolution proceeds by Copying Experts New Services approximate Experts Infrastructure adopts new Protocols Generic Technologies that always work

The Waves of the NetThe Waves of the NetThe Waves of the NetThe Waves of the Net

Network InfrastructureEvolves from Data to

Information to Knowledge

What is the Net?What is the Net? The Net connects Everyone to Everything

Global Network Infrastructure Universal Protocols to support Interactions

The Functionality of the Net Evolves Data to Information to Knowledge Deeper Levels with Stronger Technologies

At Any Level, the User only sees Outside Infrastructure is Inside, thus Invisible The Web is just the Present not the Future Someday, the Net will be true Cyberspace

Waves of the Net - LevelsWaves of the Net - Levels Data – The Past

Transmission across Networks (Physical) Telephone Operators & Switching Gateways

Information – The Present Retrieval across Spaces (Logical) Reference Librarians & Federated Repositories

Knowledge – The Future Analysis across Models (Reality) Doctors & Categories

Wisdom – The Goal

Waves of the Net - ServicesWaves of the Net - Services Data – The Past

File Transfer Basic Unit is Bits within Packets

Information – The Present Concept Navigation Basic Unit is Words within Documents

Knowledge – The Future Structure Correlation Basic Unit is Features within Vectors

Waves of the Net -- ProtocolsWaves of the Net -- Protocols Data – The Internet

Packet Subwave: TCP/IP (Transmission) File Subwave: FTP (File Transfer Protocol)

Information – The Interspace Document Subwave: HTTP/HTML (Search) Concept Subwave: CSP (Concept Switching)

Knowledge – The Intermodel Vector Subwave: VTP/VCPL (Cluster Pattern) Structure Subwave: SMP (Structure Matching)

Transmitting DataTransmitting Data Telephone Network of the 19th Century

Wave 0, Multiplexed Telegraph Digital Bits basic Syntax for Transmission

Packets are the paradigm Packets are grouped Bits Address and Routing for Packets Bits interpreted only by local computers

The InternetThe Internet First Universal Physical Wave

Wave 1, Protocols across Networks Universal across packet networks (IP) Files from computer to computer

Distributed File Systems Files cluster Bits within Computers Assemble Files from Bits using Packets (TCP) Transmit Files across Computers (FTP)

Searching InformationSearching Information WorldWide Web of the 20th Century

Wave 2, Hyperlink Transmission (HTTP) Digital Words basic Semantics for Search

Documents are the paradigm Documents are grouped Words (HTML) Classification and Indexing for Documents Words interpreted only by local repositories

The InterspaceThe Interspace First Universal Logical Wave

Wave 3, Protocols across Spaces Universal across document spaces Concepts from repository to repository

Distributed Concept Systems Concepts cluster Words within Repositories Assemble Concepts from Words using Documents Search Concepts across Repositories

Universal Information UnitsUniversal Information Units Basic Concepts of Natural Language

Fundamental Grouping needed for Semantics Specialized to specific Community Collections

Essential Technologies all Converging Document Representation (XML, RDF) Language Parsing (multiword noun phrases) Statistical Indexing (contextual frequencies) Peer-Peer Networking (local computations) Concept Switching (vocabulary translations)

Same Process, Deeper LevelSame Process, Deeper Level The Interspace, using Libraries

Subject thesaurus, yields Concept space Subject classification, yields Concept map From words to concepts From networks to spaces

The Intermodel, using Lifestyles Reality problem, yields Structure model Reality solution, yields Structure cluster From features to structures From spaces to models

Analyzing KnowledgeAnalyzing Knowledge Lifestyle Grid of the 21st Century

Wave 4, Full-spectrum Questionnaires Digital Features basic Pragmatics for Analysis

Vectors are the paradigm Vectors are grouped Features Comparison and Clustering for Vectors Features interpreted only by local situations

Healthcare InfrastructureHealthcare Infrastructure Monitors for each Individual

Capture everyday lifestyle status Customized questionnaires for healthcare Build detailed database of health vectors

Managers for each Individual Daily Feature Record for each Individual Detailed Databases for whole Population Analyze Clusters of Similar Individuals

The IntermodelThe Intermodel First Universal Representational Wave

Wave 5, Protocols across Models Universal across vector models Features from individual to individual

Distributed Structure Systems Structures cluster Features within Individuals Assemble Structures from Features using Vectors Analyze Structures across Individuals

Universal Knowledge UnitsUniversal Knowledge Units Basic Structures of Everyday Life

World Categories needed for Pragmatics

Similar Structures from Many Subjects Medicine, categories of health status Computing, TCOL versus UNCOL Anthropology, cross-cultural universals Psychology, basic natural categories Architecture, design patterns for comfort life History, structures of everyday life

Realizing WisdomRealizing Wisdom Zen Buddhism

The Ocean including all Waves Direct Interaction with Reality Beyond Words and Concepts

Zen Koans Curriculum process of realization exercises Become “one with the koan”, achieve Satori Richi -- Reality Challenge, e.g. “Mu” Kikan – Reality Interaction, e.g. “Oak Tree”

The IntermindThe Intermind First Universal Spiritual Wave

Wave N, Protocols across Minds Universal across Mind Realities Hive Mind on the Net, There is no Self!

Religions to model Lifestyles richi (koan) for monitor (health) kikan (koan) for manager (health) Mimic basic structures not details Interactive navigation through automated

clusters not automated decision-making

Metaphors across WavesMetaphors across Waves

Capturing Operators yields Internet

Capturing Librarians yields Interspace

Capturing Doctors yields Intermodel

Capturing Roshi-s yields Intermind

The Laws of the NetThe Laws of the NetThe Laws of the NetThe Laws of the Net

Predicting Units of Network Infrastructure

for Deeper Evolution

Philosophy of InformaticsPhilosophy of Informatics Physics in the 20th Century

Newtonian Mechanics is Western philosophy Deterministic Laws predict all motion Quantum Mechanics is Eastern philosophy Probabilistic Computations are uncertain

Informatics in the 21st Century Database Retrieval is Western philosophy Deterministic Rules match correct results Information Retrieval is Eastern philosophy Statistical Clusters suggest useful results

Laws of MotionLaws of MotionIsaac Newton, 1687, Principia Mathematica

 I. Every body continues in its state of rest, or of uniform motion in a right [straight] line, unless it is compelled to change that state by forces impressed upon it.II. The change of motion is proportional to the motive force impressed; and is made in the direction of the right line in which that force is impressed.III. To every action there is always opposed an equal reaction; or, the mutual actions of two bodies upon each other are always equal, and directed to contrary parts.

Laws of InfrastructureLaws of InfrastructureBruce Schatz, 2004, The Evolution of the Net 

I. Every infrastructure undergoes a characteristic evolution, from station-station to person-person, from trained experts to untrained amateurs, to handle the volume of a universal service.II. Most of the volume of a universal service is handled by the users themselves, with infrastructure support provided by modeling an appropriate expert.III. Each new basic structure models a new expert in terms of previous structures, and each successful evolution brings the structures closer to approximating reality.

Levels of AbstractionLevels of Abstraction Deeper Levels drive Later Waves

Each Wave has a fundamental level Deeper composed from Shallower

Each Level built upon the Previous Level units level N-2 ordered by (ob) units level N-1 Context of N-2 within N-1 yields N

Laws predict Future Units Information Space, Knowledge Model

Units of AbstractionUnits of Abstraction Metaphors

structures ob situations Situations

categories ob structures Structures

concepts ob categories Categories

documents ob concepts Concepts

terms ob documents Documents

words ob terms

The Science of InformaticsThe Science of Informatics New Units from Old Units

Predict Future Units from Current Units

New Waves from Old Waves Predict Future Protocols from Current

Same Processes, New Technologies Context as Meaning, Statistics as Truth Deeper Levels are Closer to Reality