ELN 4403 Hata Düzeltme Kodlarına Giriş Course Outline

Transkript

2011-12 Güz
ELN 4403
Hata Düzeltme Kodlarına Giriş
2011-12 Güz
Tuncay ERTAŞ
Course Outline
z
z
z
Introduction
Fundamentals of Information Theory
Error-Control Coding
©
2
1-1
2011-12 Güz
Introduction
z
The broadly based mathematical discipline of
information theory has made fundamental
contributions to

z
Communications
Computer Science
Statistical Physics
Statistical Inference
Probability and Statistics
We will approach the information theory in the context
of communications
©
3
Fundamentals of Information Theory
z
This part addresses the issues

Entropy as the basic measure of information

Source coding theorem and data compaction

Mutual information and channel capacity

Channel coding theorem

Rate-distortion theory for source coding
©
4
1-2
2011-12 Güz
Error Control Coding
z
As the different ways of implementing Shannon’s
channel coding theorem, this part includes:

z
Linear block codes
Convolutional Codes
Trellis Codes
Turbo Codes
Each coding technique involves the use of

Channel encoder in the Transmitter
Decoding algorithm in the Receiver
©
5
Block Diagram of a Communication System
©
5
1-3
2011-12 Güz
©
5
Değerlendirme
z
Dersten geçme notu
Â
Â
Â
z
Proje %25
Dönem içi Sınavı %25
Dönem sonu Sınavı %50
Ev ödevleri belirtilen tarihte sınıfta teslim edilir. Zamanında
verilmeyen ödevler kesinlikle kabul edilmez.
z
Derse devam esastır
z
Derslere %70 devam alamayanlar, doğrudan başarısız sayılırlar
z
Derste bulunmayanların, derslerde yapılan duyurulardan doğan
yükümlülüklerinin sorumlukları kendilerine aittir
Dersin Sayfası: http://home.uludag.edu.tr/~tertas
©
6
1-4
2011-12 Güz
Karşılıklı Güven Politikası
z
z
z
z
Projeleri öncelikle kendiniz yapmalısınız
Takıldığınız yerleri bir sınıf arkadaşınızla veya
benimle tartışmanız tavsiye edilir
Diğer bir kişinin çözümüne bakmak, kısmen veya
tamamen kopya etmek kabul edilmez
Dersi alan herkesin bu işbirliği ve karşılıklı güven
politikasını kabul ve uymayı taahhüt ettiği farz
edilir
©
7
Father of Information Theory
Claude Elwood Shannon (1916-2001)
©
8
1-5
2011-12 Güz
Bir Çin Atasözü
İşitirsem unuturum
Görürsem hatırlarım
Yaparsam öğrenirim
©
9
History of Information Theory
z
In general there are three types of Information
• Syntactic information
Related to the symbols from which the messages are built-up and
to their interrelations
• Semantic information
Related to the meaning of messages
• Pragmatic information
Related to the usage and effect of messages
©
10
1-6
2011-12 Güz
Syntactic information mainly considers the form of information
Semantic and Pragmatic information are related to the
information content
Consider the following sentences
1. Ali was brought to the railway station by Taxi
2. The taxi brought Ali to the railway station
3. There is a traffic jam on E5, between Istanbul and Edirne
4. There is a traffic jam on E5 in Turkey
©
11
Sentences 1 and 2 are syntactically different but semantically
identical. Equally informative.
Sentences 3 and 4 are different not only with respect to their syntax
but also in their semantics.
Sentence 3 gives more precise information than that of sentence 4.
The semantic and the pragmatic aspects of information are studied
in British tradition of information theory. So, it is more related to
philosophy and psychology.
American tradition deals with the syntactic aspects of IT. In this
approach, there is full abstraction from the meaning aspects of
information. A rather technical approach to IT remains.
©
12
1-7
2011-12 Güz
There, basic questions are the measurement of syntactic
information
1. The fundamental limits on the amount of information that can
be transmitted
2. The fundamental limits on the compression of information
which can be achieved, and how to build information
processing systems to approach these limits
In American tradition,
Information Theory is sometimes referred to as Communication
Theory, Mathematical Information Theory, or shortly
Information Theory
©
13
Advance of Information Theory
In 1924, H. Nyquist published an article rising the matter of how
characters could be sent over a telegraph channel with max.
Possible speed without distortion.
In 1928, R.V.L. Hartley first tried to define the measure of
information.
In 1948, Claude Elwood Shannon published his article
“A Mathematical Theory of Communication”, and generally known
to be the founder of the American tradition in IT.
He extended the theories of Nyquist and Hartley and laid the
foundation of present-day information theory by associating
information with uncertainty using the concept of probability.
©
14
1-8
2011-12 Güz
Shannon’s Theory of Information
Information Theory provides answers to two
fundamental questions
What is the irreducible complexity below which the signal can not
be compressed?
Entropy of a source
What is the ultimate transmission rate for reliable communication
over a noisy channel?
Capacity of a channel
©
15
1-9

ELN 4403 Hata Düzeltme Kodlarına Giriş Course Outline

Transkript

Benzer belgeler

Z. Caner Taşkın - Department of Industrial Engineering