View
216
Download
0
Embed Size (px)
Citation preview
February 1, 2011 1
February 1, 2011 2http://www.2wheelbikes.com/sitebuilder/images/cable-lock-comb-bike-accessories-469x345.jpg
How many bit strings of length n?
Adding one bit doubles the number
February 1, 2011 3
length Bit strings of that length Count
1 0, 1 2 = 21
2 00, 0110, 11
4 = 22
3 000, 001, 010, 011,100, 101, 110, 111
8 = 23
n 0 followed by strings of length n-1,1 followed by strings of length n-1
2n-1 + 2n-1 = 2 x 2n-1=2n
Text8 bits per character
“A” = 01000001
“(” = 00101000
How many combinations of 8 bits?
2· 2· 2· 2· 2· 2· 2· 2 = 28 = 256
February 1, 2011 4
Hexadecimal Digits
February 1, 2011 5
0000 0001 0010 0011 0100 0101 0110 0111
0 1 2 3 4 5 6 7
1000 1001 1010 1011 1100 1101 1110 1111
8 9 A B C D E F
ASCIIAmerican Standard Code for Information Interchange
Character represented by Hex xy, e.g. 4B is “K”
February 1, 2011 6
xy
0 1 2 3 4 5 6 7 8 9 A B C D E F
0
1
2 sp ! " # $ % & ' ( ) * + , - . /
3 0 1 2 3 4 5 6 7 8 9 : ; < = > ?
4 @ A B C D E F G H I J K L M N O
5 P Q R S T U V W X Y Z [ \ ] ^ _
6 ` a b c d e f g h i j k l m n o
7 p q r s t u v w x y z { | } ~ del
ASCII UnderneathEmails
Web pages
February 1, 2011 7
February 1, 2011 8http://farm4.static.flickr.com/3021/2494096946_2bf86f8571.jpg?v=0
What if you need more than 256 characters?
Unicode
32 bits per character
February 1, 2011 9
February 1, 2011 10
February 1, 2011 11
February 1, 2011 12
February 1, 2011 13
How many Unicode characters?
32 bits each, so there are in all
February 1, 2011 14
†
2324,294,967,296
February 1, 2011 15
Binary counting1+1=10, or 1+1=0 and carry 1
February 1, 2011 16
+1 +1 +1
0 1 1 0 0 1 1 1
+ 1
0 1 1 0 1 0 0 0
Positive and Negative Numbers
Signed and unsigned numbersUnsigned: 28=256 bit patterns represent 0 … 255Signed: 28 bit patterns represent -128 … +127Leftmost bit = sign bit: 0 => 0 or pos, 1 => negLargest 8-bit positive number = 01111111 = 1270 = 00000000Most negative negative number =
10000000 = -128
February 1, 2011 17
Negative numbers
February 1, 2011 18
+1 +1 +1 +1 +1 +1 +1
1 1 1 1 1 1 1 1
+ 1
0 0 0 0 0 0 0 0
-1 = 11111111so addition works the same for positive and negative numbers
Biggest NumbersBiggest positive number = 01111111 (like
999999 on a car odometer)
Most negative negative number = 10000000
February 1, 2011 19
Biggest Positive Number + 1 “=”
Most Negative Negative Number
February 1, 2011 20
+1 +1 +1 +1 +1 +1 +1
0 1 1 1 1 1 1 1
+ 1
1 0 0 0 0 0 0 0
OVERFLOW!
The Comair Christmas “Glitch”
16 bits for monthly count of crew changes
Biggest positive 16-bit number =32,767
December was a bad month, lots of snowstorms, lots of flights rescheduled
As Christmas approached the count went from 32,767 to -32,768 by adding 1
February 1, 2011 21
The Y2010 “Glitch”Binary representation of decimal 10 =
00001010
Binary Coded Decimal = write decimal 10 as sequence of 4-bit binary codes for digits Decimal 10 = BCD 0001 0000
What if you write decimal 10 in BCD but some other program reads it as decimal? Binary 0001 0000 = decimal 16
February 1, 2011 22
Bytes 1 byte = 8 bits = 2 hex digits = 1 character
210 =1024 bytes = 1 kilobyte = 1KB
220 =1,048,576 bytes = 1 megabyte = 1MB
230 bytes = 1 gigabyte = 1GB = “a billion”
240 bytes = 1 terabyte = 1TB = “a trillion”
250 bytes = 1 petabyte = 1PB = “a quadrillion”
260 bytes = 1 exabyte = 1EB = a quintillion bytes
270 = zetta
280 = yotta
February 1, 2011 23
KAll this terminology based on the accident that
Which is 1K?
There are new standard names: 1 kibibyte = 1000 bytes vs. 1 kilobyte = 1024 bytes
But almost no one uses “kibi-”, “mebi-”, etc.
February 1, 2011 24
†
10242101031000
February 1, 2011 25
February 1, 2011 26
Moore’s Law (1965)
The number of transistors on a silicon chip doubles every 18 [or 12, or 24] months
1965: 64 = 26
2008 = 2 billion ~ 231
25 doublings in 43 years = one doubling every 20+ months
February 1, 2011 27
Example of linear increase
February 1, 2011 28
Example of exponential increase
Now for the y axis use instead lg(y) = the exponent e such that 2e=y
February 1, 2011 29
Same plot, using lg(y) instead of y
February 1, 2011 30
One of the Greatest Engineering Achievements
An increase by a factor of 225 is about 30 millionfold
If human speed had increased that much over the past 43 years, we would now be traveling faster than the speed of light
February 1, 2011 31
ProbabilitiesFair coin: P(heads) = 1/2
Fair die: P(rolling 3) = 1/6
Fair card deck: P(hearts) = 1/4
P(ace) = 1/13
February 1, 2011 32
Probabilities of Independent Events Multiply
P(heads and then heads) = 1/2 · 1/2 = 1/4
P(3 and then 4) = 1/6 · 1/6 = 1/36
P(ace and ace) = 1/13·1/13 = 1/169 ≈ .0059 but only if the first card drawn is replaced and the deck is completely reshuffled, otherwise the events are not independent
P(ace and ace without reshuffling) = 1/13 · 3/51 ≈ .0045
February 1, 2011 33
Unlikely Events
How likely are 100 heads in a row?
(1/2)100 ≈ 10-32 =
.00000000000000000000000000000001
February 1, 2011 34
How Small is 2-100 ≈ 10-
30?Age of universe ≈ 1018 sec = 1027
nanoseconds (1 nanosecond = 1 ns = 1 billionth of a second = 10-9 sec)
If you do all 100 coin flips in a billionth of a second, you will get the 100-heads event about once every thousand lifetimes of the universe
1030 = 103 ·1027
This is “never” for all practical purposes
February 1, 2011 35
Morse’s telegraph1844 1848
February 1, 2011 36
Morse Code (1838)A B C D E F G H I J K L M
.- -… -.-. -.. . ..-. --. …. .. .--- -.- .-.. --
N O P Q R S T U V W X Y Z
-. --- .--. --.- .-. … - ..- …- .-- -..- -.-- --..
February 1, 2011 37
Morse Code (1838)A
.08B
.01C
.03D
.04E
.12F
.02G
.02H
.06I
.07J
.00K
.01L
.04M
.02
.- -… -.-. -.. . ..-. --. …. .. .--- -.- .-.. --
N.07
O.08
P.02
Q.00
R.06
S.06
T.09
U.03
V.01
W.02
X.00
Y.02
Z.00
-. --- .--. --.- .-. … - ..- …- .-- -..- -.-- --..
February 1, 2011 38
How Long are Morse Codes on Average?
Not the average of the lengths of the letters: (2+4+4+3+…)/26 = 82/26 ≈ 3.2
We want the average a to be such that in a typical real sequence of say 1,000,000 letters, the number of dots and dashes should be about a·1,000,000
The weighted average:
(freq of A)·(length of code for A)
+ (freq of B)·(length of code for B)
+ …
= .08·2 + .01·4 + .03·4 + .04·3+… ≈ 2.4February 1, 2011 39
Data vs. InformationMessage sequence:
“yea,” “nay,” “yea,” “yea,” “nay,” “nay” …
In ASCII, 3·8 = 24 bits of data per message
But if the only possible answers are “yea” and “nay,” there is only 1 bit of information per message
Entropy is a measure of the information content of a message, as opposed to its size
Entropy of this message sequence = 1 bit/msg
February 1, 2011 40
Squeezing out the “Air”Suppose you want to ship pillows in boxes
and are charged by the size of the box
Lossless data compression
Entropy = lower limit of compressibility
February 1, 2011 41
Claude Shannon (1916-2001)
A Mathematical Theory of Communication (1948)
February 1, 2011 42
Communication over a Channel
February 1, 2011 43
Source Coded Bits Received Bits Decoded Message
S X Y T Channel symbols bits bits symbolsEncode bits before putting them in the channelDecode bits when they come out of the channel
E.g. the transformation from S into X changes“yea” --> 1 “nay” --> 0
Changing Y into T does the reverseFor now, assume no noise in the channel, i.e. X=Y