Information Theory, Inference and Learning Algorithms Review

Average Reviews:

(More customer reviews)Uniting information theory and inference in an interactive and entertaining way, this book has been a constant source of inspiration, intuition and insight for me. It is packed full of stuff - its contents appear to grow the more I look - but the layering of the material means the abundance of topics does not confuse.
This is _not_ just a book for the experts. However, you will need to think and interact when reading it. That is, after all, how you learn, and the book helps and guides you in this with many puzzles and problems.

Click Here to see more reviews about: Information Theory, Inference and Learning Algorithms

Information theory and inference, often taught separately, are here united in one entertaining textbook. These topics lie at the heart of many exciting areas of contemporary science and engineering - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics, and cryptography.This textbook introduces theory in tandem with applications. Information theory is taught alongside practical communication systems, such as arithmetic coding for data compression and sparse-graph codes for error-correction. A toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, are developed alongside applications of these tools to clustering, convolutional codes, independent component analysis, and neural networks.The final part of the book describes the state of the art in error-correcting codes, including low-density parity-check codes, turbo codes, and digital fountain codes -- the twenty-first century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, David MacKay's groundbreaking book is ideal for self-learning and for undergraduate or graduate courses. Interludes on crosswords, evolution, and sex provide entertainment along the way.In sum, this is a textbook on information, communication, and coding for a new generation of students, and an unparalleled entry point into these subjects for professionals in areas as diverse as computational biology, financial engineering, and machine learning.

Get 21% OFF

Click here for more information about Information Theory, Inference and Learning Algorithms

Read More...

Probabilistic Graphical Models: Principles and Techniques (Adaptive Computation and Machine Learning series) Review

Average Reviews:

(More customer reviews)Stanford professor, Daphne Koller, and her co-author, Professor Nir Friedman, employed graphical models to motivate thoroughgoing explorations of representation, inference and learning in both Bayesian networks and Markov networks. They do their own bidding at the book's web page, [...], by giving readers a panoramic view of the book in an introductory chapter and a Table of Contents. On the same page, there is a link to an extensive Errata file which lists all the known errors and corrections made in subsequent printings of the book - all the corrections had been incorporated into the copy I have. The authors painstakingly provided necessary background materials from both probability theory and graph theory in the second chapter. Furthermore, in an Appendix, more tutorials are offered on information theory, algorithms and combinatorial optimization. This book is an authoritative extension of Professor Judea Pearl's seminal work on developing the Bayesian Networks framework for causal reasoning and decision making under uncertainty. Before this book was published, I sent an e-mail to Professor Koller requesting some clarification of her paper on object-oriented Bayesian networks; she was most generous in writing an elaborate reply with deliberate speed.

Click Here to see more reviews about: Probabilistic Graphical Models: Principles and Techniques (Adaptive Computation and Machine Learning series)

A general framework for constructing and using probabilistic models ofcomplex systems that would enable a computer to use available information for makingdecisions.

Get 21% OFF

Click here for more information about Probabilistic Graphical Models: Principles and Techniques (Adaptive Computation and Machine Learning series)

Read More...

Analogy-Making as Perception: A Computer Model (Neural Network Modeling and Connectionism) Review

Average Reviews:

(More customer reviews)Melanie Mitchell's analogy-making as perception is a remarkably original book. It documents an artificial intelligence project known as copycat, which was implemented as the author's PhD project under Douglas Hofstadter.
Copycat is unlike anything in artificial intelligence. It is not a symbolic system, neither a connectionist one. The major goal of the project is to study the nature of concepts. Concepts, as we all know, are flexible, context-sensitive creatures. For instance, DNA has nothing to do with a computer program, but there is a sense on which we can see DNA as a computer program that guides embrionary development. DNA can also be seen as a zipper, as it "zips down" in two parts (for cell reproduction). Still another view would be DNA as a will, for it carries valuable hereditary "property". Now, DNA is in truth just a molecule, and nothing else. The question is, how can we see the same thing (such as DNA) as so many different things? Moreover, how can these fluid context-sensitive concepts be implemented in rigid, rule-obeying computers?
To which the answer is: what we view is the abstract roles that DNA plays in embrionary development, cell division, and in individual reproduction. And this is the very idea of "Analogy-making as perception".
Well, not so fast. The copycat project is not designed to grasp such extremely complex subjects as DNA, but, on the other hand, it presents a computational architecture that suggests what the nature of concepts is like, and how flexible concepts may emerge from inflexible mechanisms.
Copycat can solve analogy problems such as abc->abd:ijk-> ?. But it is not restricted to trivial ones. Consider the following analogy: abc ->abd:xyz->?. How would you solve it? How do you think that copycat solves it?
Obviously, this project doesn't fit in very easily in classical artificial intelligence, as it attacks some of the most pervasive ideas of the field, such as the separation of perception and cognition. In fact, I think this book redefines the major questions of artificial intelligence (and although Mitchell does not state it, I think the copycat model does not fall prey to either the frame problem or to the symbol grounding problem).
It is very unfortunate that this is not one of the best-selling books in AI. But I believe that it will ultimately make its mark on the History of the field, if for no other reason than it simply is the right approach to genuine intelligence and authentic understanding.
Should one day Amazon.com let me give a 6-star to a book, but charge me a dollar for giving it, this is one that would definitely deserve to be such a 6-star.
============================================
PS. I would also recommend Hofstadter's Fluid Concepts and Creative Analogies; and Robert French's Subtlety of Sameness.

Click Here to see more reviews about: Analogy-Making as Perception: A Computer Model (Neural Network Modeling and Connectionism)

The psychologist William James observed that "a native talent forperceiving analogies is... the leading fact in genius of every order." Thecentrality and the ubiquity of analogy in creative thought have been noted again andagain by scientists, artists, and writers, and understanding and modeling analogicalthought have emerged as two of the most important challenges for cognitivescience.Analogy-Making as Perception is based on the premise that analogy-making isfundamentally a high-level perceptual process in which the interaction of perceptionand concepts gives rise to "conceptual slippages" which allow analogies to be made.It describes Copycat - a computer model of analogymaking, developed by the authorwith Douglas Hofstadter, that models the complex, subconscious interaction betweenperception and concepts that underlies the creation of analogies.In Copycat, bothconcepts and high-level perception are emergent phenomena, arising from largenumbers of low-level, parallel, non-deterministic activities. In the spectrum ofcognitive modeling approaches, Copycat occupies a unique intermediate positionbetween symbolic systems and connectionist systems a position that is at present themost useful one for understanding the fluidity of concepts and high-levelperception.On one level the work described here is about analogy-making, but onanother level it is about cognition in general. It explores such issues as thenature of concepts and perception and the emergence of highly flexible concepts froma lower-level "subcognitive" substrate.Melanie Mitchell, Assistant Professor in theDepartment of Electrical Engineering and Computer Science at the University ofMichigan, is a Fellow of the Michigan Society of Fellows. She is also Director ofthe Adaptive Computation Program at the Santa Fe Institute.

Click here for more information about Analogy-Making as Perception: A Computer Model (Neural Network Modeling and Connectionism)

Read More...

Cognitive Modeling (Bradford Books) Review

Average Reviews:

(More customer reviews)This book could be considered to be a collection of articles on the `computational theory of mind.' Although the articles are somewhat out of date, due to the advances in neuroscience and cognitive science that have occurred since the time of publication of the book, it does serve as a good motivation for the understanding of more recent developments. I did not read all of the articles in the book, and so my review will be confined to the ones that I did.
The article on ACT in chapter 2 is basically a theory of cognition that is based on recursion. Referring to ACT as a "simple theory of complex cognition", John Anderson, the author of the article, wants to simulate the manner in which humans develop recursive programs. The machine that is to simulate this makes use of `production rules,' in its knowledge base, which the author claims is exhaustive enough to produce complex cognition. To produce true machine intelligence, all one has to do is to tune these production rules and make use of them as needed. As the author describes it, the original ACT theory was based on human associative memory, but the one described in this article is called ACT-R, and can simulate adaptive behavior in the presence of a noisy environment. The author describes various simulations using ACT-R, and concludes that it is sensitive to prior information and to information about what is appropriate response to the situation it finds itself in. The author stresses more than once the simplicity of the ACT-R system: it is able to encode data from the environment as declarative knowledge, encode the changes in the environment as procedural knowledge, and encode the statistics of this knowledge use in the environment.
Another highly interesting article is the one by Alan Prince and Paul Smolensky on the application of optimization theory to linguistics. Called `optimality theory' by the authors in their extensive research on the topic, in the article they discuss the relations between optimality in grammar and optimization in neural networks. The authors discuss with great clarity the role that constraints play in the construction of linguistic structures, and the fact that these constraints typically conflict with each other. This conflict between grammatical constraints must thus be managed by a successful grammatical architecture. Optimality theory asserts that these constraints are universal in the sense that they are present in every language. The connection of optimality theory with neural networks arises when one is interested in finding out if the properties of optimality theory can be explained in terms of fundamental principles of cognition. The computational theory of neural networks the authors believe holds some clues on these properties. In order to make the connection with grammatical issues, as abstract as they are, and because neural networks are highly nonlinear dynamical systems, one must find a way of encapsulating the complicated behavior of neural networks. The authors accomplish this by the use of Lyapunov functions, which for reasons of consistency of terminology they call `harmony functions.' For those neural networks admitting a harmony function, the initial activation pattern flows through the network to construct a pattern of activity that maximizes "harmony." Most interestingly, the harmony function for a neural network performs the same function as does the mechanisms needed for well-formed grammar. The patterns of activation are thus a mathematical analog of the structure of linguistic representations. However, the authors are careful to note that not every weighting scheme for the neural network will give a possible human language. It is here where the constraints play an essential role in limiting the possible linguistic patterns and relations.
The article by Keith Holyoak and Paul Thagard discusses the construction of a correspondence between a source analog and of a target. This is the so-called analogical mapping, which is constructed using a collection of structural, semantic, and pragmatic constraints. In the view of the authors, the concept of analogy can be broken down into four components, namely the selection of a source analog, the actual mapping, an analogical inference (transfer), and the actual learning that takes place. The authors omit discussion of the last component in this article. The finding of the correspondences between the two analogs can result in a combinatorial explosion, and so use is made of appropriate constraints. These constraints consist of those that exemplify structural consistency, those of semantic similarity, and lastly of pragmatic centrality. The theory of analogical mapping that the authors propose is governed by these constraints. They discuss the ACME (Analogical Constraint Mapping Engine) algorithm as one that constructs a network of units representing mapping hypotheses and eventually converges to a state that represents the best mapping. They list several applications of ACME, such as radiation problems, attribute mappings, chemical analogies, and the classical `farmer's dilemma' problem. ACME was also able to simulate a number of empirical results related to human analogical reasoning. The analogical mapping they discuss is most powerful in a specific domain however. This domain-specificity is a typical restriction for most of the efforts in learning theory and artificial intelligence.

Click Here to see more reviews about: Cognitive Modeling (Bradford Books)

Computational modeling plays a central role in cognitive science. Thisbook provides a comprehensive introduction to computational models of humancognition. It covers major approaches and architectures, both neural network andsymbolic; major theoretical issues; and specific computational models of a varietyof cognitive processes, ranging from low-level (e.g., attention and memory) tohigher-level (e.g., language and reasoning). The articles included in the bookprovide original descriptions of developments in the field. The emphasis is onimplemented computational models rather than on mathematical or nonformalapproaches, and on modeling empirical data from human subjects.

Get 19% OFF

Click here for more information about Cognitive Modeling (Bradford Books)

Read More...

Modeling and Reasoning with Bayesian Networks Review

Average Reviews:

(More customer reviews)While pursuing my PhD at UCLA, I took Professor Darwiche's classes and had the privilege of using the pre-release version of this book. Before taking professor Darwiche's class, I had spent a good deal of time while working on my masters degree working on Bayesian networks. I found that much of the literature on Bayesian networks was inaccessible to someone new to the field. There simply was not a comprehensive resource that would explain Bayesian networks from the beginning in a through and clear manner. I say with confidence that this has now changed.
The book begins with the fundamentals of logic. It continues on to describe the properties of the Bayesian network graph such as independence relationships and d-separation as well as how the parameters of a Bayesian network work.
There are then in depth discussions of the various queries we are able to perform on Bayesian networks and the algorithms for accomplishing them. These include queries such as probability of evidence, most probable explanation and probabilistic inference. Techniques such as summing out, pearl's polytree algorithm and belief propagation are described elogently and clearly.
The book also contains information on the current state of the art research going on in the field. This book is a valuable resource for anyone new to or ingrained in the use of Bayesian Networks. A book of this scope and target was sorely needed and I for one am glad it has arrived. I would and have recommended this to any of my peers in the field.

Click Here to see more reviews about: Modeling and Reasoning with Bayesian Networks

This book provides a thorough introduction to the formal foundations and practical applications of Bayesian networks. It provides an extensive discussion of techniques for building Bayesian networks that model real-world situations, including techniques for synthesizing models from design, learning models from data, and debugging models using sensitivity analysis. It also treats exact and approximate inference algorithms at both theoretical and practical levels. The author assumes very little background on the covered subjects, supplying in-depth discussions for theoretically inclined readers and enough practical details to provide an algorithmic cookbook for the system developer.

Get 17% OFF

Click here for more information about Modeling and Reasoning with Bayesian Networks

Read More...