General InformationEEL 6532 - Information Theory and Coding. Spring 2010.

Lecture time: M, Wd 10:30 - 11:45 AM,Lecture hall: HEC 0111

Instructor: Dan C. MarinescuOffice Hours: M, Wd 12 - 1:00 PM. Office Location: HEC 439B.Final Exam:

Course Objectives: (a) Introduce to the students to concepts essential for communication -a critical element of the information revolution. (b) Provide basic understanding of basicconcepts regarding information such as entropy, mutual entropy, and relative entropy. (c)Stress the physical nature of information and touch on reversibility. (d) Stress the role ofcoding for data compression, encryption, and fault-tolerance.

Textbook: The students are not required to buy any textbook for this class. Infoma-tion Theory concepts are covered in Section 4.1 - 4.7 of the textbook “From Classical toQuantum Information Theory” by Dan C. Marinescu and Gabriela M. Marinescu, to bepublished in Summer 2010 by Morgan Kaufmann, a division of Elsevier. Coding is cov-ered in Chapter 5, Sections 5.1 to 5.20 of the same book. The text is avalable online athttp://www.cs.ucf.edu/∼dcm/QCV2.pdf.

Additional references:1. T. M. Cover and J. A. Thomas ”Elements of Information Theory,” Willey, 1991.2. S. A. Vanstone and P.C. van Oorschot ”An Introduction to the Error Correcting Codes

with Applications” Kluwer, 1989.

Assignments: There are several homework assignments.

Class attendance and grading: The students are strongly encouraged to attend every classand be active. Our policy is simple, based on professional standards: on exams you shouldnot collaborate. On all other assignments you are welcome to work with anyone else on ideasand understanding, but your writing should be your own and you should carefully acknowledgeall contributions of ideas by others, whether from classmates or from papers you have read.Acts of academic dishonesty will not be tolerated; when detected they lead to unconditionalfailure of the class. The grading scheme is :

• Homeworks: 35%

• Class participation: 15%

• Midterm: 20%

• Final: 30%

Syllabus: The schedule is tentative and will be adjusted to the needs of the class; when atopic seems more difficult we shall spend more time on it.

1

• Mathematical Foundations (2 weeks) a. Elements of probability theory (continuous anddiscrete random variables, moments, joint and conditional probabilities, central-limittheorem. Linear algebra (vector spaces, linear transformations, matrices) c. Algebraicstructures (groups, fields, rings).

• Physical Nature of Information. Landauer’s Theorem. Thermodynamic, von Neumann, and Shannon Entropy. Joint, Conditional, and Relative Entropy (2 weeks)

• General Architecture of Communication Systems. Encoding and Decoding, Source andChannel Models. Mutual Information and Channel Capacity (2 weeks)

• Fano Inequality; Data Processing Inequality Shannon’s Coding Theorems (1 week)

• Error Detection and Error Correction. Block Codes. Decoding Policies (1 week)

• Linear codes. Cosets, Syndrome and Standard Array Decoding of Linear Codes (1 week)

• Hamming and Reed-Muller Codes (1 week)

• Cyclic Codes. Generator and Parity-Check Matrix of a Cyclic Code (1 week)

• Hamming, Singleton, Gilbert-Varshamov, and Plotkin Bounds (1 week)

• Reed-Solomon Codes, Concatenated Codes, Decoding Complexity, Turbo Codes (1week)

• Midterm and review (1 week)

2