Upload
doanlien
View
227
Download
0
Embed Size (px)
Citation preview
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
Informatics luis rocha 2017
I501introduction
to informatics
introduction to informaticslecture 3
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Readings until now
This week Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters: 1 and 2 Gleick, J. [2011]. The Information: A History, a Theory, a Flood. Random
House. Chapter 8. Optional
McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5:115-133.
Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters 11, and 12. Presentations
Freeth, Tony. 2014. “Eclipse Prediction on the Ancient Greek Astronomical Calculating Machine Known as the AntikytheraMechanism.” PloS One 9 (7): e103275. Plank, Nicholas
Wasserman, M., X.H.T. Zeng, and L.A.N. Amaral [2015]. “Cross-evaluation of metrics to estimate the significance of creative works”. PNAS 112 (5) 1281-1286. Boothby, Clara
Lecture Notes The Nature of Information
Available http://canvas.iu.edu and listed at
http://informatics.indiana.edu/rocha/academics/i501 Also check out
Links and notes at http://sciber.blogspot.com/
introduction to informatics
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017 The external tape as a general principle (system) of universal computing
Charles Babbage (1791 – 1871) and Ada Lovelace (1815-1852)
Analytical engine Separated memory store from a
central processing unit (or ‘mill’) Cogs not just numbers
variables Programmable
instructions on punched cards Inspired by the Jacquard Loom
Ada Lovelace: the science of operations Set of (recursive) rules for producing
Bernoulli numbers (a program) Separation of variable and operational
(data) cards would punch out cards for later use
“the Engine eating its own tail.” (Babbage)
distinction between numbers that mean thingsand numbers that do things.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Alan Turing (1912-1954)
“The chemical basis of morphogenesis” Turing, A. M. Phil. Trans. R. Soc. Lond.
B 237, 37–72 (1952). Reaction-diffusion systems
“Computing machinery and intelligence” Turing, A. M. Mind 49, 433–460 (1950).
The “Turing Test” “On computable numbers with an
application to the Entscheidungsproblem” Turing, A. M. Proc. Lond. Math. Soc. s2–42, 230–265 (1936–37). Turing machine, universal
computation, decision problem
key contributions (most relevant to biocomplexity)
Brenner, Sydney. [2012]. “Life’s code script.” Nature 482 (7386): 461-461.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
computation as number crunching
Computing with numbers It is always possible to encode
any symbol string into a sequence of integers
A sequence of integers can be mapped into a single natural number Gödelization (prime factorization)
Computer scientists can concentrate on functions that take a single number as input and output Computation as a mapping of
numbers to other numbers Models of Computation
How to construct number mappings Turing machine, general
recursive functions, λ-calculus
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Turing’s tape
“On computable numbers with an application to the Entscheidungsproblem” Turing, A. M. Proc. Lond. Math. Soc. s2–42, 230–265 (1936–37).
Turing machine, universal computation, decision problem Machine’s state is controlled by a program, while data
for program is on limitless external tape every machine can be described as a number that can be
stored on the tape for another machine Including a Universal machine
distinction between numbers that mean things (data) and numbers that do things (program)
A fundamental principle of computation
“The fundamental, indivisible unit of information is the bit. The fundamental, indivisible unit of digital computation is the transformation of a bit between its two possible forms of existence: as [memory] or as [code]. George Dyson, 2012.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
At every discrete time instance the machine is in a single state
Program is a state transition table
state Read symbol
Next state
Write symbol
Tape move
0 4 1 - left
1 - 0 1 right
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
At every discrete time instance the machine is in a single state
Program is a state transition table
state Read symbol
Next state
Write symbol
Tape move
0 4 1 - left
1 - 0 1 right
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
computation
Process of rewriting strings in a formal system according to a program of rules Operations and states are syntactic Symbols follow syntactical rules Rate of computation is irrelevant
Program determines result, not speed of machine Physical implementation is irrelevant for result
Computer Physical device that can reliably
execute/approximate a formal computation Errors always exist Design aims to make rate and dynamics irrelevant
some facts
“[…] essential elements in the machine are of a binary […] nature. Those whose state is determined by their history and are time-stable are memory elements. Elements of which the state is determined essentially by the existing amplitude of a voltage or signal are called ‘gates’”. Bigelow et al, 1947
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
John Von Neumann (1903-1957)
realizing the power of Turing’s tape physical (electronic) computers emphasized the importance of the stored-
program concept (the external tape) EDVAC
allows machine to modify its own program von Neumann architecture: The functional
separation of storage from the processing unit. programs can exist as data (two roles)
Converts tape to fixed-address memory (random-access memory)
Turing machines beyond the decision problem
“Let the whole outside world consist of a long paper tape”. —John von Neumann, 1948
“ ‘Words’ coding the orders are handled in the memory just like numbers” --- distinction between numbers that mean things and numbers that do things.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
John Von Neumann (1903-1957)
realizing the power of Turing’s tape physical (electronic) computers emphasized the importance of the stored-
program concept (the external tape) EDVAC
allows machine to modify its own program von Neumann architecture: The functional
separation of storage from the processing unit. programs can exist as data (two roles)
Converts tape to fixed-address memory (random-access memory)
Turing machines beyond the decision problem
“Let the whole outside world consist of a long paper tape”. —John von Neumann, 1948
“ ‘Words’ coding the orders are handled in the memory just like numbers” --- distinction between numbers that mean things and numbers that do things.
“Since Babbage’s machine was not electrical, and since all digital computers are in a sense equivalent, we see that this use of electricity cannot be of theoretical importance.… The feature of using electricity is thus seen to be only a very superficial similarity.” (Alan Turing)
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
Informatics luis rocha 2017
I501introduction
to informatics
design principles of computationBabbage/Lovelace, Turing’s tape, and roles of information
distinction between numbers that mean things and numbers that do things.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Nature.com; ANDY POTTS; TURING FAMILY
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
other pre-cybernetics developments
“The Cerebral Inhibition Meeting” New York City, May 1942
Organized by Frank Freemont-Smith of the Josiah Macy Jr. Foundation
From the human sciences Lawrence Frank, Margaret Mead and Gregory Bateson
From the sciences Warren McCulloch and Arturo Rosenblueth
Result Huge excitement about Rosenblueth’s presentation of concepts
from Norbert Wiener and Julien Bigelow Homeostasis, purposeful action (goal-direction), aiming A new paradigm of interdisciplinary research?
Goal-directed actions Controversial: explaining actions in terms of future events, violating
cause and effect Teleological mechanisms
Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: State-
determined systems The mathematics were accessible
1942 meeting
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
McCulloch & Pitts
McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5:115-133. A Turing machine program could be implemented in a finite
network of binary neuron/switches Neurons as basic computing unit of the brain Circularity is essential for memory (closed loops to sustain memory)
Brain (mental?) function as computing Others at Macy Meeting emphasized other aspects of brain
activity Chemical concentrations and field effects (not digital)
Ralph Gerard and Fredrik Bremmer
Memory can be maintained in circular networks of binary switches
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Cybernetics was born
The Feedback Mechanisms and Circular Causal Systems in Biology and the Social Sciences March 1946 (10 meetings between 1946
and 1953) Interdisciplinary
Since a large class of ordinary phenomena exhibit circular causality, and mathematics is accessible, let’s look at them with a war-time team culture
Participants John Von Neumann, Leonard Savage,
Norbert Wiener, Arturo Rosenblueth, Walter Pitts, Margaret Mead, Heinz von Foerster, Warren McCulloch, Gregory Bateson, Claude Shannon, Ross Ashby, etc.
Key concepts Homeostasis, Circular causality
requiring negative feedback (postulated to be very common)
Present state becomes input for action at next moment: State-determined systems
The mathematics were finally accessible
post-war science: the Josiah Macy Jr. Foundation Meetings
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Cybernetics was born
The Feedback Mechanisms and Circular Causal Systems in Biology and the Social Sciences March 1946 (10 meetings between 1946
and 1953) Interdisciplinary
Since a large class of ordinary phenomena exhibit circular causality, and mathematics is accessible, let’s look at them with a war-time team culture
Participants John Von Neumann, Leonard Savage,
Norbert Wiener, Arturo Rosenblueth, Walter Pitts, Margaret Mead, Heinz von Foerster, Warren McCulloch, Gregory Bateson, Claude Shannon, Ross Ashby, etc.
Key concepts Homeostasis, Circular causality
requiring negative feedback (postulated to be very common)
Present state becomes input for action at next moment: State-determined systems
The mathematics were finally accessible
post-war science: the Josiah Macy Jr. Foundation Meetings
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
cybernetics
Synthetic approach Engineering-inspired Supremacy of mechanism
Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology
All can be axiomatized and computed Mculloch&Pitts’ work was major influence
“A logical calculus of the ideas immanent in nervous activity”. Bulletin of Mathematical Biophysics 5:115-133 (1943).
A Turing machine (any function) could be implemented with a networkof simple binary switches (if circularity/feedback is present)
post-war science
Macy Conferences: 1946-53
Warren S. McCulloch
Claude Shannon
Margaret Mead
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
other key concepts
Norbert Wiener and Arturo Rosenblueth Goal-directed behavior and negative feedback (control) Homeostasis and circular causality
In machines and biology Automata Theory (Von Neumann) Communication and Information
The fundamental idea is the message, even though the message may not be sent by man and the fundamental element of the message is the decision” (Norbert Wiener)
Shannon’s Information and Wiener’s Communication Theory Natural semiotics (McCulloch and others later get into Peircean Semiotics)
“functional equivalence” of systems (general systems) Bio-inspired mathematics and engineering and computing/mechanism-inspired
biology and social science
at the Macy meetings
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
other key concepts
Gregory Bateson and Margaret Mead Homeostasis and circular causality in society
Transvestite ceremony to diffuse aggressive action in Iatmul culture
Learning and evolution Can a computer learn to learn?
A new organizing principle for the social sciences (control and communication) As much as evolution was for Biology
Lawrence Frank The new interdisciplinary concepts needed a
new kind of language Higher generality than what is used in single
topic disciplines A call for a science of systems
Yehoshua Bar-Hillel Optimism of a new (cybernetics and
information) age “A new synthesis […] was destined to open
new vistas on everything human to help solve many of the disturbing open problems concerning man and humanity”.
at the Macy meetings
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
British Cybernetics
The Ratio Club (starting in1949) British cybernetics meetings
William Ross Ashby, W. Grey Walter, Alan Turing. etc “computation or the faculty of mind which calculates, plans
and reasons” Also following Wiener’s use of “Machina ratiocinatrix” in
Cybernetics (1948), following Leibniz’ “calculus ratiocinator”
Turing as cybernetician
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
British Cybernetics
The Ratio Club (starting in1949) British cybernetics meetings
William Ross Ashby, W. Grey Walter, Alan Turing. etc “computation or the faculty of mind which calculates, plans
and reasons” Also following Wiener’s use of “Machina ratiocinatrix” in
Cybernetics (1948), following Leibniz’ “calculus ratiocinator”
Turing as cybernetician
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Shannon’s mouse
trial and error algorithm information as reduction of uncertainty in the
presence of alternatives (combinatorics) lifelike behavior
trial and error to learn path from many alternatives adapts to new situations
how is learning achieved? Correct choices, information gained from reduced
uncertainty, must be stored in memory memory of information as a design principle
of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays
Brain as (switching) machine
controlling information to achieve life-like behavior
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
next class
Week 4 Lecture
Prokopenko, Mikhail, Fabio Boschetti, and Alex J. Ryan. [2009]. "An information-theoretic primer on complexity, self-organization, and emergence. “ Complexity 15(1): 11-28
Presentations & Discussion Barabasi and Albert (1999) Emergence of Scaling in
Random Networks, Science 296 (5439). Erkol, Sirag
Hofman, Jake M., Amit Sharma, and Duncan J. Watts. "Prediction and explanation in social systems." Science 355.6324 (2017): 486-488. Kaminski, Patrick
Lecture Notes The Nature of Information
Optional Aleksander, I. [2002]. “Understanding Information Bit by
Bit”. In: It must be beautiful : great equations of modern science. G. Farmelo (Ed.), Granta, London.
Readings (available in Canvas)