I know about Wikipedia and MacKay's Information Theory, Inference, and Learning Algorithms (is it appropriate as textbook?). A textbook starting with Shannon's entropy and going through Conditional entropy and Mutual information is sought... Any idea? If you are following such a course at your university, which textbook is used?
Thanks.