Back to ECE home pageINSTRUCTOR: Christopher Rose can be reached at crose@ece.rutgers.edu Office hours are 1 hour prior to class. Email questions preferable if possible. Signup here for mailing list/newsgroup
MISCELLANEOUS: Information Sheet, Syllabus , E-Mail Archive
(from Dorin Comaniciu) This message is related to the optimization procedure of the Fly-By-Night corporation (ECE542 Quiz2 Spr.96 Q3) I explain below what the four figures contain:
Figure 1 : The mutual info I(X;Y)=H(Y)-H(Y|X) of a Binary Symmetric Channel (with a crossover probability of e=0.05) is plotted as a function of the variables 0<=px0<=1 and 0<=px1<=1. The variables px0 and px1 are considered independent.
Figure 2 : The same as Fig.1 but rotated with 90 deg. As one can see, the maximum of the surface I(X,Y) does not belong to the plane px0+px1=1. Therefore, an unconstrained optimization procedure (such that of Fly-By-Night) will obtain a capacity greater than the true capacity. The true capacity is TC=0.7136 bits (for e=0.05) and corresponds to the maximum value of the surface I(X;Y), subject to the constraint px0+px1=1.
Figure 3 : We present here the plots of px0(N) and px1(N) given by the optimization procedure of Fly-by-Night (BSC with e=0.05). The initial values are px0(0)=0.25 and px1(0)=0.75. As the optimization advances, the two variables px0 and px1 tend to a value around 0.36, corresponding to the maximum value of the surface I(X,Y) (see Fig.2).
Figure 4 : The plot of I(X,Y) as a function of the iteration N. The Fly-by-Night procedure performs quite well and it finds the absolute maximum of the surface I(X,Y) (see Fig.2), which is higher (~ 0.78) than the true capacity (TC=0.7136). However we are NOT interested in finding the absolute maximum of the surface I(X,Y). The channel capacity is defined as the maximum of I(X,Y) subject to the constrained px0+px1=1.
Back to ECE home page
If you have questions, comments or other suggestions please email your comments to crose@ece.rutgers.edu