Uncertainty and Information: Foundations of Generalized Information Theory

Kybernetes

ISSN: 0368-492X

Article publication date: 1 August 2006

895

Citation

Klir, G.J. (2006), "Uncertainty and Information: Foundations of Generalized Information Theory", Kybernetes, Vol. 35 No. 7/8, pp. 1297-1299. https://doi.org/10.1108/03684920610675283

Publisher

:

Emerald Group Publishing Limited

Copyright © 2006, Emerald Group Publishing Limited


This presents a range of theories about uncertainty, all of them mathematical and allowing quantitative treatment. A definition of uncertainty is automatically associated with one of information, if it is accepted that information is that which reduces uncertainty. The book is about information‐based uncertainty and uncertainty‐based information.

That uncertainty can usefully be treated by means other than those of classical probability theory has been acknowledged in recent decades by the wide utilisation of fuzzy techniques, and these are treated here, but in company with a variety of others. The validity of the classical theory, and its usefulness where applicable, are not disputed, but it is argued that other theories can take over where the classical one fails. It is argued that the choice of method should be problem‐driven rather than theory‐driven.

The treatment is put into a very general context by representing the range of problems arising in science on a two‐dimensional plot with “complexity” as one axis and “randomness” the other. It is observed, with a quotation from Kelvin, that science was once thought to be concerned only with precise data, but later had to accept uncertainty and statistical methods, first with the gas laws and then in thermodynamics. Established methods of science (prior to the developments now treated) are applicable to problems represented either in a region near the origin of the plot, labelled “organised simplicity” or one remote from it in both dimensions labelled “disorganised complexity”. The former corresponds to the precise‐data methods of Kelvin's time, and the latter to thermodynamics and the like. The remaining space, external to these two regions, is labelled “organised complexity” and represents the range of problems amenable to solution using the theories in this book or expected developments from them. The new methods stem from two advances in mathematics, namely the generalisation of classical set theory to include fuzzy sets, and a generalisation of classical measure theory allowing the requirement of additivity to be replaced by the weaker monotonicity.

The two‐dimensional plot of problem types is taken from a paper by Warren Weaver on “science and complexity”. (It is not the one that forms the latter part of the famous book by Shannon and Weaver, although it dates from about the same time.) George Klir sees the breakthrough to treatment of “organised complexity” as providing a basis for scientific advance in the next half century.

Among the non‐classical theories treated is that of Hartley, which predates the Shannon version. It is insisted that Hartley's measure of information should not be regarded simply as the special case of Shannon's when the probabilities are equal, but as a measure based on possibility (and necessity) that is useful irrespective of probabilities. The generalisation of measure theory to allow non‐additivity leads to a generalisation of standard integration called the Choquet integral, and this is the basis of a range of theories that include the well‐known Dempster‐Schafer theory of evidence as a limiting case.

Other methods are applicable when the probability itself is imprecise, and fuzzy methods are fully treated. The relevance of the latter to natural‐language processing is mentioned, with acknowledgement that more needs to be done in this area, and that even scientific communication has to accept the imprecision of natural language.

The treatment is extremely comprehensive, with discussion of the fuzzification of uncertainty theories (that were not intrinsically fuzzy) and of methodological issues including the transfer of uncertainty indications between methods. Pointers are given for future research. The presentation is clear, though highly mathematical, and the text is supplemented by student exercises, glossaries of key concepts and of symbols, as well as subject and name indexes. There is an extensive bibliography of 29 pages, with some of the entries referring to books and reviews noted as having further extensive lists. The subject‐matter of the book comes mainly from sources quoted and some is original research by the author.

The book is clearly a major work that will be referred to for many years to come, probably well into the half century in which it is predicted these issues will come to the fore. There are just two criticisms that seem warranted, one of them relating to the notes on the back cover and one to a statement in the Preface.

The back‐cover notes refer to “examples and illustrations to clarify complex material and demonstrate practical applications”. The reference to practical applications is somewhat misleading since most of the examples are mathematical in nature, without direct reference to the “real world”. Arguing from the book alone it could be difficult to persuade a determined sceptic that its contents have practical value. That they definitely do is demonstrated elsewhere, notably in a paper () in which the author has been obliged to take a more polemic stance. Examples are given there of practical problems that resist solution by the classical approach but are readily analysed using Dempster‐Schafer theory. There is also a passing reference in the book, on page 186, to a connection with theory of economics, and other practical issues including consequences of imprecise measurement receive attention.

It is also possible to criticise the author's contention in his Preface that the book can be used for self‐study by someone familiar with fundamentals of set theory, probability theory and the calculus. I think it is fair to say that this is not quite true and is belied by a number of later recommendations for supplementary reading, including on Measure Theory. However, the rewards for making the effort are great since this is a timely and admirable unified treatment of material previously only to be found in scattered and difficult books and papers, supplemented by original research of the author. This will be a standard and much prized work for teaching and reference for many years to come, as the value of the new methods comes to be accepted by statisticians and theorists.

References

Halmos, P.R. (1950), Measure Theory, van Nostrand, New York, NY.

Klir, G.J. (1989), “Is there more to uncertainty than some probability theorists might have us believe?”, Int. J. General Systems, Vol. 15 No. 4, pp. 34778.

Related articles