TM TT BI GING L THUYT THNG TIN
CHNG 1:
TIN TC
1.1H THNG TRUYN TIN (HT3)Ngun tin
Si(t)
Ngun tin
Ngun tin
S0(t)
Ngun tin:
Nhiu
+ L tp hp cc tin HT3 dng lp cc bn tin khc nhau trong s truyn. +
Ngun tin c m hnh ho ton hc bng bn qu trnh sau: Qu trnh ngu nhin lin
tc. Qu trnh ngu nhin ri rc. Dy ngu nhin lin tc. Dy ngu nhin ri rc.
S0(t) = Nm Si(t) + Na(t) + Si(t): Tn hiu vo & S0(t): tn hiu ra
ca knh tin + Nm (t), Na(t) : c trng cho nhiu nhn, nhiu cng.
Knh tin: l ni din ra s truyn lan ca tn hiu mang tin v chu tc ng
ca nhiu.
Nhn tin: l u cui ca HT3 lm nhim v khi phc tin tc ban u.Nguon tin
M ha ngun M ha knh B iu ch Pht cao tn Knh tin Nhan tin Gii m ngun
Gii m knh Gii iu ch
Thu cao tn
H thng truyn tin s (ri rc) Hai vn c bn ca h thng truyn tin: + Vn
hiu sut, ni cch khc l tc truyn tin ca h thng. + Vn chnh xc, ni cch
khc l kh nng chng nhiu ca h thng. 1.2 S O THNG TINLy thuyet thong
tin trang: 1
a. Lng o tin tc: Ngun A c m tn hiu ng xc xut, mt tin do ngun A
hnh thnh lmt dy n k hiu ai bt k (ai A).
Lng tin cha trong mt ai bt k: I(ai)=logm I(x) = n.log m (1) (2)
Lng tin cha trong mt dy x gm n k hiu:
n v lng o thng tin thng c chn l c s 2. - Khi m k hiu ca ngun tin
c xc xut khc nhau v khng c lp thng k vi nhau th I(xi) = log
(1/p(ai)) Lng tr ring: I(xi) = -log p(xi) (4) L lng tin ban u c xc
nh bng xc xut tin nghim. (3)
Lng tin cn li ca xi sau khi nhn c yj c xc nh bng xc xut hu
nghim.I ( xi / y i ) = log p ( xi ) yj
(5)xi ) yj
Lng tin tng h:p( I ( xi / y i ) = I ( xi ) I ( xi / y i ) =
log
(6)
p ( xi )
c tnh ca lng tin: + I(xi) I(xi ; yi) + I(xi) 0 (8) (9) (7)
+ I(xi.yi) = I(xi) + I(yi) - I(xi; yi)
Khi cp xi, yj c lp thng k vi nhau th I(xi; yi) = 0 Ta c: I(xi;
yi) = I(xi) + I(yi) (10) Lng tin trung bnh: l lng tin tc trung bnh
cha trong m k hiu bt k ca ngun cho.I ( x) = p ( x ) log p ( x)X
(11)
Lng tin tng h trung bnh:I ( X , Y ) = p ( x, y ) logXY
p( x / y) p( x)
(12)
Lng tin ring trung bnh c iu kin:I (Y / X ) = p ( x, y ) log( y /
x )XY
(13)
b. Entrpi ngun ri rc: l mt thng s thng k c bn ca ngun. V ngha vt
l bt ng v lng thng tin tri ngc nhau, nhng v s o chng bng nhau:H ( X
) = I ( X ) = p ( x) log p ( x)
(1)
Ly thuyet thong tin
trang: 2
c tnh ca Entrpi H(X): + H(X) 0 + H(X) = 0 khi ngun tin ch c mt k
hiu + H(X)max khi xc sut xut hin cc k hiu ca ngun bng nhau.
Entrpi ng thi: l bt nh trung bnh ca mt cp (x,y) bt k trong tch
XY.H ( XY ) = p ( x, y ) log p ( x, y )XY
(2)
Entrpi c iu kin:H ( X / Y ) = p ( x, y ) log p ( x / y ) XY
(3)
1.3 THNG LNG CA KNH THNG TIN: Tc thit lp tin ca ngun: R= n0.H(X)
(bps) (1) + H(X); entrpi ca ngun. + n0 : s k hiu c lp trong mt n v
thi gian Thng lng ca knh C l lng thng tin ti a knh cho qua i trong
mt n v thi gian m khng gy sai nhm. C(bps) Thng thng R < C, R tin
ti gn C ta dng php m ho thng k ti u tng Entrpi. C = Rmax = n0.
H(X)max d ca ngun:r =1 H (X ) H ( X ) max
a. Thng lng knh ri rc khng nhiu: (bps) (2)
(3)
Dng phng php m ha ti u gim d ca ngun n khng hoc s dng d ca ngun
xy dng m hiu chng nhiu. b. Thng lng knh ri rc c nhiu: R = noI(X;Y)
= n0[H(X)-H(X/Y)] (bps) Tc lp tin cc i trong knh c nhiu: C = Rmax =
n0[H(X)-H(X/Y)]max (bps) (5) (4)
Ly thuyet thong tin
trang: 3
CHNG 2: M HA NGUN TIN2-1. M HIU 2-1-1 M hiu v cc thng s c bn ca
m hiu: C s ca m (m) l s cc k hiu khc nhau trong bng ch ca m. i vi m
nh phn m= 2. di ca m n l s k hiu trong mt t m. Nn di cc t m nh nhau
ta gi l m u, ngc li l m khng u. di trung bnh ca b m:n = p ( x i )n
ii =1
(1)
+ p(xi): xc sut xut hin tin xi ca ngun X c m ha. + ni : di t m
tng ng vi tin xi. + N: Tng s t m tng ng vi tng s cc tin ca xi
Tng hp cc t hp m c th c c: N0=2n., nu:+ NN0 ta gi l m y 2-1-2 iu
kin thit lp m hiu: iu kin chung cho cc loi m l quy lut m bo s phn
tch cc t hp m. iu kin ring cho cc loi m:
+ i vi m thng k ti u: di trung bnh ti thiu ca m. + i vi m sa
sai: kh nng pht hin v sa sai cao. 2-1-3. PHNG PHP BIU DIN M. aCc
bng m: Tin T m Mt to m: bi = K 2 K 1K =1 n
a1 00
a2 01
a3 100
a4 1010
a5 1011
(1)
K
=0 hay 1;
K: s th t ca k hiu trong t m
c. hnh m:Cy m
Ly thuyet thong tin
trang: 4
0 1 0 1 0
0v1
0
1 2 34
03
0V1
1
a (00) a2(01) 1
1
0 0
1 1
0
2
a3(100)
a4(1010) a5(1011)
o hnh ket cau
Hm cu trc ca m: 2 Khi ni = 2 G(ni) = 1 Khi ni= 3 2 Khi ni = 4
2-1-4 iu kin m phn tch c : M c tnh Prphic Bt k dy cc t m no ca b m
cng khng c trng vi mt dy t m khc ca cng b m. M c tnh prphic nu bt k
t hp m no cng khng phi l prphic ca mt t hp no khc cng b m. iu kin m
c tnh prphic:
a-
2j =1
n
j
G( j) 1
M h thng c tnh phphic c xy dng t mt m prphic no bng cch ly mt s
t hp ca m prphic gc lm t hp s ng v cc t hp cn li lm t hp cui. Ghp
cc t hp s ng vi nhau v ni mt trong cc t hp cui vo thnh t hp m mi gi
l m h thng c tnh prphic. V d: Ly b m prphic 1,00,010,011 Cc t hp s
ng: 1,00,010 Mt t hp cui: 011 Gi : n1, n2,, ni l di cc t hp s ng 1,
2,, k l di cc t hp cui
-
- S c th c c cc dy ghp bng cc t hp s ng c di nj bng :g(nj) =
g(nj-n1) + g(nj-n2) + + g(nj-ni) Trong : nj 1; g(0) = 1 ; g(nj <
0) = 0 (1)
Nu ch dng mt t hp cui , hm cu trc m s l:G(nj) = g(nj- ) (2) (3)
+ T (1) v (2) ta c cng thc truy chng tnh G(nj) G(nj) = G(nj-n1) +
G(nj-n2) + + G(nj-ni) + T (1) ta c: n1=1, n2=2, n3=3 v =3 g(nj) =
g(nj-1) + g(nj-2) + g(nj-3)Ly thuyet thong tin trang: 5
Trong : nj +1; G(nj = ) = 1; G(nj < ) = 0
g(nj=1) = g(0) + g(-1) + g(-2) = 1 c 1 dy 1 g(nj=2) = g(1) +
g(0) + g(-1) = 2 c 2 dy: 00 v 11 g(nj=3) = g(2) + g(1) + g(0) = 4 c
4 dy: 111, 100, 001, 010 + T (3) ta c: G(nj) = G(nj-1) + G(nj-2)
+G(nj-3) Trong : nj= +1=4 ; G(nj=3) = 1 ; G(nj
