Information Theory and Coding (330:542)

World Wide Web Homepage

INSTRUCTOR:
Christopher Rose
can be reached at *crose@ece.rutgers.edu*
Office hours are 1 hour prior to class. Email questions preferable if possible.
**Signup here for mailing list/newsgroup**

MISCELLANEOUS: Information Sheet, Syllabus , E-Mail Archive

- Problem Set 1
- Problem Set 2
- Quiz 1 1996
- TAKEHOME EXAM
- Problem Set 3
- Problem Set 4
- Quiz 2 1996
- Quiz 2
- Problem Set 5
- Problem Set 6
- Final 1996
- Final

- Problem Set 1 Solutions
- Problem Set 2 Solutions
- Problem Set 3 Solutions
- Takehome Solutions
- Problem Set 4 Solutions (not proofed yet)
- Problem Set 5 Solutions (not proofed)
- Quiz 2 Solutions
- Problem Set 6 Solutions (not proofed)

(from Dorin Comaniciu) This message is related to the optimization procedure of the Fly-By-Night corporation (ECE542 Quiz2 Spr.96 Q3) I explain below what the four figures contain:

Figure 1 : The mutual info I(X;Y)=H(Y)-H(Y|X) of a Binary Symmetric Channel (with a crossover probability of e=0.05) is plotted as a function of the variables 0<=px0<=1 and 0<=px1<=1. The variables px0 and px1 are considered independent.

Figure 2 : The same as Fig.1 but rotated with 90 deg. As one can see, the maximum of the surface I(X,Y) does not belong to the plane px0+px1=1. Therefore, an unconstrained optimization procedure (such that of Fly-By-Night) will obtain a capacity greater than the true capacity. The true capacity is TC=0.7136 bits (for e=0.05) and corresponds to the maximum value of the surface I(X;Y), subject to the constraint px0+px1=1.

Figure 3 : We present here the plots of px0(N) and px1(N) given by the optimization procedure of Fly-by-Night (BSC with e=0.05). The initial values are px0(0)=0.25 and px1(0)=0.75. As the optimization advances, the two variables px0 and px1 tend to a value around 0.36, corresponding to the maximum value of the surface I(X,Y) (see Fig.2).

Figure 4 : The plot of I(X,Y) as a function of the iteration N. The Fly-by-Night procedure performs quite well and it finds the absolute maximum of the surface I(X,Y) (see Fig.2), which is higher (~ 0.78) than the true capacity (TC=0.7136). However we are NOT interested in finding the absolute maximum of the surface I(X,Y). The channel capacity is defined as the maximum of I(X,Y) subject to the constrained px0+px1=1.

If you have questions, comments or other suggestions please
email your comments to *crose@ece.rutgers.edu*