Lost in Math by Sabine Hossenfelder

The book makes great points, especially regarding the faulty incentive structure of scientists and the charade of “research proposals” for funding. My only qualm is that the reader may come away thinking there are no objective measures of goodness for models, which is not true. Statistical learning theory, Bayesian Occam factors, algorithmic complexity etc. each grapple with this problem, yet not much of this is mentioned.
Full post... Related link

Happy birthday Raymond Smullyan

A mathematician friend of mine recently told me of a mathematician friend of his who everyday "takes a nap". Now, I never take naps. But I often fall asleep while reading -- which is very different from deliberately taking a nap! I am far more like my dogs Peekaboo, Peekatoo and Trixie than like my mathematician friend once removed. These dogs never take naps; they merely fall asleep. They fall asleep wherever and whenever they choose (which, incidentally is most of the time!). Thus these dogs are true sages.

I think this is all that Chinese philosophy is really about; the rest is mere elaboration!

Raymond Smullyan, The Tao is Silent (1977)

Full post...

Our Mathematical Universe by Max Tegmark

Consider a computer simulation of a mini universe, a universe complex enough to give rise to sentient beings. As Greg Egan points out in Permutation City, (and as is obvious to anybody who has played The Sims), the subjective flow of time in the simulated universe could be many times faster or slower than ours. In fact, once we have the simulation going, we can fast forward, rewind back, play the frames out of order, and it wouldn't matter one bit to the subjective experience of the simulated folk. What if we stopped the simulation? What if we saved the whole history on a DVD and put it on a shelf? Again, the subjective experience of the simulated, which arise from the relationships between their successive moments, would be uneffected. We would just be missing the opportunity to observe them in action, and interact with them.

That brings us to the next natural question: do they need the simulation, the DVD, do they need us at all? Don't they just "exist" independently whether or not we happen to be watching, recording, interacting or anybody has ever thought of them? Max Tegmark advances the Mathematical Universe Hypothesis, which says (1) our universe at the most basic level IS (not just described by) a mathematical structure (an electron or a photon can be defined exactly by a handful of numbers and there is nothing left over), and that (2) any mathematically consistent (computable?) structure must exist in the same sense as we do!

Needless to say this has led to quite a bit of spirited discussion around the web, I would recommend Scott Aaronson's blog post and the comments therein.

Full post... Related link

A Mathematician's Lament by Paul Lockhart

This is a wonderful and heartfelt book about the poetry of mathematics and how we fail to introduce it to students at schools and to society in general. I can finally stop feeling guilty about my obsession with popular math books and cool problems even though I know they probably have no practical value to me or to society whatsoever. After all it is not practical value that makes reading poetry a pleasure! I am thankful to all my wonderful math teachers who bequeathed me this guilty pleasure without my knowledge or consent in spite of the horrible curriculum, text books, and test system. And I highly recommend the author's second book "Measurement" to those who seek an alternative to the typical 10 pound high school textbooks full of repetitive and unimaginative problems.

Full post... Related link

Semantic Structures by Ray Jackendoff

Jackendoff divides the study of the language faculty into three components: phonological, syntactic, and conceptual. In speech recognition a model that tells us what word sequences are likely is essential to recognition accuracy, because the acoustic information is often ambiguous (e.g. "She kissed this guy." vs. "She kissed the sky."). Stated in Jackendoff's terms, a probabilistic model of the syntactic component is helpful in disambiguating what is going on in the phonological component. Similarly I think a model of what is likely in the conceptual component is essential to resolving ambiguities in the syntactic component. If "what is likely to be said" is essential in interpreting "what is heard", then "what is likely to be meant" is similarly essential in interpreting "what is said".

Unfortunately we do not have good conceptual models yet, so computational linguists still try to make do with error prone hand tagging and shallow machine learning to disambiguate senses, references, and relations.

On a side note, each component in Jackendoff's work is modeled after the generative paradigm which, for the syntactic component, is described as follows:

1. Speakers can understand and create an indefinite number of sentences they have never heard before.
   
2. Therefore the repertoire of syntactic structures cannot be characterized as a finite list of sentences.
   
3. Nor can it be characterized as an infinite list of possible sentences because we have finite brains.
   
4. Thus it MUST be mentally encoded in terms of a finite set of primitives and a finite set of principles of combination that collectively generate the class of possible sentences.

Am I the only one befuddled by this argument? Primitives plus means of combination is certainly one way to create infinity using finite means, but why assume it is the only way? Dynamic systems, random processes, who knows what else can lead to infinite possible outcomes from a finite initial endowment. Why just present two strawmen, finite and infinite lists, as the only alternatives to discrete primitives and combination? Why after a couple of paragraphs further narrow the description to "the argument from creativity to the NECESSITY for principles or rules in syntactic knowledge"? Discrete primitives with finite and definite constraints and rules of combination is one way to build a representational system, unlikely to be the correct way for all three components of language, and certainly not the only way.

See also: Plausibility vs. Inference.

Full post... Related link

From Eternity to Here by Sean Carroll

Why is the future different than the past? The fundamental rules of physics seem as symmetric in time as they are for different directions of space, yet you can't unscramble an egg and you can't remember the future. The answer seems to lie in the concept of entropy, which tends to increase over time in a closed system unlike energy and momentum which stay constant. Yet, entropy is one of these concepts that lead to quite a bit of confusion, sort of like (and sometimes for the same reasons as) probability. Sean Carrol's wonderful book tackles the mysteries of entropy and arrow of time. Here are some entropy puzzles and some further reading:

Entropy puzzles:

• If you find yourself in a universe with low entropy it is as likely to have come from a higher entropy state as it is to evolve into a higher entropy state. So how do you know you are not a Boltzmann brain?
  
• Take a box with a partition in it, with gas A on one side, gas B on the other side, and both gases are at the same temperature and pressure. Remove the partition. If gas A and B are different gases, there is an entropy that arises due to the mixing. If the gases are the same, no additional entropy is calculated. What if you thought they were the same gas and years later it was discovered that they happened to be two different isotopes? (See Gibbs paradox and E.T. Jaynes' paper). More generally this microstate / macrostate business seems completely user defined and arbitrary, so how can it have real physical effects?
  
• In a reversible system there must be just as many paths that decrease the entropy as that increase the entropy. Why don't we observe as many of the first type as the second?
  
• A gas squeezed in the corner of a room will tend to spread thereby increase its disorder and entropy. If we add an attractive force like gravity matter seems to clump together rather than spread out. How does clumping together increase entropy?
  
• A rotting plant turns into dust and gas which increases disorder and entropy. A seed turns a bunch of gas and dust into a full grown tree which seems to decrease entropy. This can only happen because the seed is not a closed system and is using the energy from the sun and ends up increasing the overall entropy of the universe at the end. When, how, and why does this type of thing happen?
  

Further reading:

• From Eternity to Here: Sean Carroll's book that inspired this post.
  
• Labyrinth of Time: Michael Lockwood's book on the arrow of time with a bit more philosophy and a bit less black hole physics.
  
• Time and Chance: David Z. Albert's book gives one of the clearest discussions of thermodynamics (both classical and statistical) and its relation to the arrow of time.
  
• Good and Real: Gary L. Drescher's book that tackles not only the arrow of time, but quantum indeterminacy, consciousness, free will, and ethics. This is one of my favorite books which inspired a few earlier posts.
  
• E. T. Jaynes has a number of articles that clarify some of the mysteries. See also these links.
  
• Permutation City: One of my favorite sci-fi novels by Greg Egan explores the nature of time, simulation and reality. Maybe instants are not ordered in time at all, the state of our short term memory seems to give this impression.
  
• Max Tegmark at MIT came up with the Mathematical universe hypothesis which is reminiscent of the Dust theory that underlies Permutation City. See also Hans Moravec's essay.
  

Full post... Related link

Adam's Tongue by Derek Bickerton

Bickerton presents his theory on the origin of language. Some notes:

- Human language is qualitatively distinct from animal communication systems (ACSs): most words are incomplete by themselves and need to be combined to express meaning whereas animal calls are self sufficient.

- A better model for origins of language may be pidgins (languages created by people that live together but do not share a common language).

- A protolanguage would have words as we know them that combine but no well defined rules of syntax or morphology.

- ACSs are all about the here and now, most words refer to things outside the current happenings.

- ACSs are mainly manipulative whereas language mainly informative.

- Communicative units come in three flavors: icons resemble the thing talked about, indices point to them, and symbols do neither. (function words which do not refer at all form a separate group). Displacement (referring to things that are not in the here/now) is only possible with symbols and icons. Iconic signs may have been the first displaced ones paving the way for symbols.

- "Since we usually regard language as no more than the means by which we express our thoughts, it seems natural to think that language should issue from intelligence, rather than vice versa. It seemed equally obvious, to naive observers, that the earth was the center of the universe, and the sun, moon, and planets all went around it." pp.58

- Categories are different from concepts. pp.87. Concepts you can think about or think with, whereas all you can do with categories is to tell whether something belongs in them. pp.205

- The ACSs of ants and bees may be closer to humans because they exhibit displacement (about food sources distant in time and space). ch.7

- Crucial as speciation is, it's still far from completely understood. pp.149

- The real breakthrough in language had to be displacement rather than arbitrariness (of signs). pp.160

- Bickerton's sequence:
1. Animals have concepts that won't merge.
2. Protohumans start talking.
3. Talking produces typically human concepts.
4. Merge appears and starts merging concepts.
5. The brain maybe gets rewired.
6. Capacities for complex thought planning etc. develop.

Chomsky's version has concepts appear first and talking last. pp.189.

- Thoughts like "roses are red" are offline as opposed to online thinking about here and now. (still makes the hearer do stuff: think of roses, think of red, merge...) pp.193

- Categories have to be only detailed enough to distinguish between appropriate reactions (affordances?). pp.206.

- I don't quite know what he's saying about memory pp.207 or recursion pp.244.

- Posted using BlogPress from my iPhone


Full post... Related link

Author Alerts

I have looked for a way to get alerted about new releases from my favorite authors for a long time. I think Amazon used to support this in the past but they no longer do. Barnes and Noble has writer alerts for only a small list of authors. This is such an obvious feature for a bibliophile that I do not understand why nobody supports it. I was about to write my own code but luckily ran into www.authoralerts.com first. Highly recommended.

Full post... Related link

The Trouble with Physics by Lee Smolin

Smolin's book made me think of a dilemma I face often. I find the current system of scientific funding disturbing. Chief among the "values" of a scientist is absolute honesty. Yet, the project proposals we need to fill periodically ask us to describe what we are going to do in detail for the next three years. I don't know what I am going to do during the next three weeks! It depends on what results I am going to get using my current approach during the next couple of days. Maybe I will have a brilliant idea that will change my whole approach to the problem. Maybe I will be taken over with another problem. Honestly I don't know. The only thing I can promise is that I will put all my working energy on making progress on the problem that I find most promising at the time. But apparently that is not enough to get funding, and we are forced to either (i) bend the truth, or (ii) tie ourselves to an approach that we will most likely find suboptimal in the near future.

To me the answer is simple: scientists should be funded not on promises about the future (which nobody can honestly make, let alone scientists whose job is to explore the unknown), but on past performance. That leaves the problem of young scientists who have no past. There should be a reasonable amount of seed funding for such people, just enough to make sure an adventurous spirit has enough time to risk his career tackling an important and deep problem.

Smolin's book should be required reading by all who manage scientists and scientific funding. If you are not interested in the debate on string theory, just read the last few chapters on how science works based on a shared ethic, and why we should take a bit more risk on "Seers" who tend to obsess about high risk problems and may take a long time (sometimes forever) producing anything valuable.

Chapter 17 proposes the shared ethic among scientists rather than some abstract "scientific method" as chiefly responsible for the success of science. Chapter 18 draws a distinction between two types of scientists "Seers" and "Craftspeople". In fact pretty much the whole book is an elaboration of how and why the scientific establishment does not provide enough room for "Seers" who by nature like to obsess about high risk problems and need much longer incubation times.

I find the shared ethic of science to be one of the most important creations of human culture. I had long held the view that science was about "what is" and not about "what ought to be", thus science and ethics had nothing to do each other. Recently I started to see the ethic of scientists as people, if not the result of their work, as being very relevant. Dennis Overbye describes it best:

"Not only does science not provide any values of its own, say its detractors, it also undermines the ones we already have, devaluing anything it can’t measure, reducing sunsets to wavelengths and romance to jiggly hormones. It destroys myths and robs the universe of its magic and mystery. So the story goes. But this is balderdash. Science is not a monument of received Truth but something that people do to look for truth. That endeavor, which has transformed the world in the last few centuries, does indeed teach values. Those values, among others, are honesty, doubt, respect for evidence, openness, accountability and tolerance and indeed hunger for opposing points of view."

Full post... Related link

Dennett in Istanbul

As part of Sabanci University's Darwin Year Celebration activities, Prof. Daniel C. Dennett is going to give a talk entitled "Darwin's strange inversion of reasoning" at the Sakip Sabanci Museum on April 10, 2009 at 16:00.

Dennett's 1995 book Darwin's Dangerous Idea argues that natural selection is a blind and algorithmic process sufficiently powerful to account for the generation and evolution of life, minds, and societies. I am looking forward to his talk, an earlier version of which I had seen at MIT when the book first came out.

These days he has taken on religious fundamentalism (see Breaking the Spell). One of his proposed solutions to fight ignorance and intolerance is to teach children about ALL of world's religions instead of brainwashing them with a single system of thought, or leaving them vulnerable by not teaching them about religion at all.

If you have not had the pleasure of listening to Dennett before, I recommend his many recorded talks available at the following websites: TED talks, Wikipedia, Reitstoen.com, and his homepage.

Dennett is quite popular in the Artificial Intelligence / Cognitive Science community due to his refreshingly rational explanations of perplexing issues like consciousness and free will. You may not agree with the specifics of his theories, but at least he makes a convincing case that there is no need for "magic dust" to explain these natural phenomena. Talking about interesting psychological results in Sweet Dreams he says:

I often discover skeptics who are quite confident that I am simply making these facts up! But we must learn to treat such difficulties as measures of our frail powers of imagination, not insights into impossibility.

Yet some of his adversaries take the failure of their imagination for a physical model of the mind as evidence of its impossibility, succumb to mysterianism, take comfort in the assumption that some questions will never be answered, or look for magic dust in the depths of quantum theory. Dennett thinks one day we will find a psychological explanation for their defect.

Many of Dennett's books that I have mentioned in this blog are on the mysteries of the mind. If you get sleepy when you read philosophy, then I especially recommend Mind's I which is one of my favorite collections of philosophical fiction. Here is a list of his books which I hope to turn into an annotated bibliography at some point:

Content and Consciousness (1969)
Brainstorms: Philosophical Essays on Mind and Psychology (1978)
Mind's I: Fantasies and Reflections on Self and Soul (1981)
Elbow Room: The Varieties of Free Will Worth Wanting (1984)
The Intentional Stance (1987)
Consciousness Explained (1991)
Darwin's Dangerous Idea: Evolution and the Meanings of Life (1995)
Kinds of Minds: Towards and Understanding of Consciousness (1996)
Brainchildren: Essays on Designing Minds (1998)
Freedom Evolves (2003)
Sweet Dreams: Philosophical Obstacles to a Science of Consciousness (2005)
Breaking the Spell: Religion as a Natural Phenomenon (2006)

Full post... Related link

Incandescence by Greg Egan

Greg Egan's latest novel Incandescence is a great thought experiment exploring how a primitive alien race living inside the tunnels of an asteroid orbiting around a neutron star can find evidence for Newtonian physics and even general relativity using simple experiments inside their closed world. The motion around the star causes a nontrivial field of forces inside the asteroid. There is no weight at the center. Away from the center, the amount and direction of the weight depends on the location relative to the center. Zak, the alien Newton, starts to explain this state of affairs with a simple assumption: "Weight is the difference between preferred and actual motion."

For example, when an object is placed north or south of the center, it is pulled towards the center. Here is what an object left to free fall from the north of the center looks like to an observer inside the asteroid:

However from a different perspective, one can see that the object is actually moving around its own slightly tilted orbit around the star. And if the object is supported by a floor and is prevented from moving toward the center, it will experience a gravitational pull toward the center -- thus an example of the maxim: "Weight is the difference between preferred and actual motion."

When two object are placed toward and away from the sun with respect to the center of the asteroid, they feel a pull away from the center:

Again, from a different perspective, this can be seen as the objects trying to follow their own natural orbits, and weight being the difference between preferred and actual motion:

As a final example, suppose an object at the center is given a radial push. Unlike the objects in the previous example, this object will not accelerate away from the center, but keep cycling in an elliptical course as seen from inside the asteroid:

The standard explanation is in terms of Coriolis force due to the object's motion balancing the tidal acceleration. From the orbital perspective, our radial push places the object in an elliptical orbit crossing the circular orbit of the planet:

Starting with these and many other interesting observations, Zak and his friends uncover the secrets of their universe before ever stepping outside their closed world and ever seeing their sun, their orbit, or fixed stars. The book is a treat for aficionados of gedankenexperiments and qualitative physics.

P.S. I'd like to thank Greg Egan for allowing the use of these animations from his page, which hosts a treasure of information about science and science fiction.

Full post... Related link

Reckoning with Risk by Gerd Gigerenzer

This is the last one of the new breed of "innumeracy" books I have read. Not one of the better ones. Still ok for generating examples for my probability course. Other examples of the genre include:

- Innumeracy by Paulos
- A mathematician reads the newspaper by Paulos
- How to lie with statistics by Huff

Not in the same genre but the following books on the history of probability and statistics may also be useful for stories and examples:

- Games Gods and Gambling by David
- The Probabilistic Revolution by Kruger et.al.
- The History of Statistics by Stigler.
- The Lady Tasting Tea by Salsburg.

The website
http://www.planetqhe.com/beta/information/home.htm contains nice problems and animations as well as some book recommendations and links.

- The Magical Maze by Ian Stewart
- Chance Rules by Brian Everitt
- Inevitable Illusions by Massimo Piattelli-Palmarini
- Taking Chances by John Haigh
- Fifty Challenging Problems in Probability by Frederick Mosteller
- Cartoon Guide to Statistics by Gonick and Smith


Full post...

Project Orion by George Dyson

"A universal library, whether of books, genotypes, or technologies, forms an expanding cloud of possibilities in a multidimensional space. The laws of nature form an outermost bound. A smaller cloud, condensed out of this atmosphere of possibilities, represents the organisms of technologies that can be assembled from available parts. Finally, a small central core - where we live - represents the books, organisms, or technologies that exist at the present time. Instead of building outward by small increments, Ted (Taylor) sought to develop Orion the other way around: start with the laws of nature; delineate the bounds first of possibility and then of practicality; finally, trace a path backward to existing technology so as to advance not by increments but by leaps and bounds." -- quote from pp. 98
Full post...

The Tao is Silent - Raymond Smullyan

At all costs, the Christian must convince the heathen and the
atheist that God exists, in order to save his soul. At all costs,
the atheist must convince the Christian that the belief in God is
but a childish and primitive superstition, doing enormous harm to
the cause of true social progress. And so they battle and storm and
bang away at each other. Meanwhile the Taoist Sage sits quietly by
the stream, perhaps with a book of poems, a cup of wine, and some
painting materials, enjoying the Tao to his hearts content, without
ever worrying whether or not the Tao exists. The Sage has no need to
affirm the Tao; he is far too busy enjoying it!

-– Raymond Smullyan, the Tao is Silent
Full post... Related link

Top ten books

I need to work on this list a bit more:

1. Tao is silent (Raymond Smullyan)
2. Mind's I (Hofstadter and Dennett)
3. Godel, Escher, Bach (Hofstadter)
4. Zen and the Art of Motorcycle Maintenance (Pirsig)
5. Good and Real (Gary L. Drescher)
6. Cosmos (Carl Sagan)
7. A History of Pi (Beckmann)
8. Skeptics and True Believers
9. The Social Brain (Gazzaniga)
10. On Intelligence (Jeff Hawkins)
11. A Different Universe (Laughlin)
12. The Naked Ape (Desmond Morris)
13. Structure and Interpretation of Computer Programs (Abelson and Sussman)
14. One two three ... infinity (George Gamow)
15. Notes from underground (Dostoyevsky)
16. Sparse distributed memory (Kanerva)
17. Society of mind (Minsky)
18. The limits of mathematics (Chaitin)
19. The age of intelligent machines (Kurzweil)
20. Skunk works (Rich and Janos)
21. The making of the atomic bomb (Rhodes)
22. The case for mars (Zubrin)
23. Night comes to the cretaceous (Powell)
24. Genome (Ridley)
25. Probability Theory (Jaynes)
    

Full post...

Good and Real by Gary Drescher

I finally received Drescher's new book. I had written about it before in this blog entry. Here is an excerpt on ethics:

Ethical relativism is an understandable reaction to centuries of opposite excess, in which the powerful have proclaimed tha their self-serving values - entitlement to conquer vulnerable peoples, and to the subservience of slaves, women, and the poor, for instance - are moral absolutes that others are obliged to yield to and could rightly be forced to yield to. Ironically, these putatively absolute moral values have been rationalized by appeal to two quintessentially relativist sources: faith and tradition. Faith, after all, is just a strong subjective feeling as to what's true. And tradition is just an appeal to ancient peer pressure (typically intimately interwoven with faith). Deriving moral foundations from tradition leads us to conform to the beliefs and behavior of people who - though they did their best with the knowledge then available - lived long before even a high-school education existed. But as every child learns (or should), the claim that "everyone else does it," or has always done it, is a poor justification for one's conduct.

P.S. Drescher does defend an objective approach to ethics.

P.P.S. The link below is a fun story I found on Jeff Bridges' website that this excerpt reminded me of.


http://www.jeffbridges.com/because.html



Full post...

Prediction, determinism, and free will

[Books_] Imagine standing around in a train station when you notice a TV monitor hanging on the wall. You recognize your own face looking back at you from the screen. You think no big deal, it must be one of those closed circuit security monitors with a camera hidden someplace. But then the you on the screen starts walking and gets on a train you know that is not due for another five minutes. The monitor is connected to a magic camera that shows what is going to happen five minutes in the future! Well, do you have the freedom not to get on that train now? Or will you somehow get compelled to get on the train against your will? If you do have the freedom to betray the monitor, does that mean a future showing monitor is impossible to build? Maybe quantum uncertainty or chaos make future prediction at that scale technically infeasible. Even for God? What if the universe was deterministic? Would that help things?

I thought about this problem on and off for a number of years until one day I ran into a paper by Gary L. Drescher. I had met Drescher once at the AI lab, enjoyed his book Made Up Minds like most fellow grad students, admired his explanation of quantum mechanics, and puzzled over his refusal to get any support from DARPA. And here he was again, with the solution to the puzzle.

To avoid the complications of the real universe Gary's trick is to think of made up universes with simple rules. This prevents confusion and eliminates prejudice. So going back to my train station, imagine a universe simulated on a computer that has completely deterministic rules. A bit like the Matrix movies. You program the simulation and let it run until sentient beings evolve and start building train stations. It is your program, and it is deterministic, so you know what is going to happen at any point in time. When in doubt, you can just load the program on a second computer and fast forward to see the future. Do YOU have the power to place a monitor that shows the future at the train station in your simulation? Well of course, you might say, you wrote the program, you own the computer, you can do anything. Not quite, it turns out...

You can certainly put a monitor in the train station and pump images of five minutes in the future to it. But trouble starts as soon as one of your simulated creatures actually looks at the monitor and decides what to do based on it. To decide what to put on the monitor at that moment, you need to fast forward. To fast forward you need to simulate what the creature is going to do. What the creature is going to do depends on what it sees on the monitor. Hmm, we have a circular impasse. Well, you might say, why not run a bunch of simulations putting various future movies on the monitor to see how the creature is going to react - one movie shows the creature getting on the train, another shows not getting on the train, for example. You fast forward in each case, then choose the movie that actually turns out to be correct. The trouble is none of the movies are going to turn out to be correct if you have a particularly stubborn creature that keeps doing the opposite of what it sees on the screen. And that brings us to the crux of the matter - the "creature" does not even have to be a sentient being. It can be a very simple automaton with the only talent of doing the opposite of whatever it observes on the monitor. And you are helpless.

So there is nothing in principle against an omniscient creator who knows the future. He just can't share the information with the inhabitants. Unless of course he forcibly compels them to obey his prophecies, but that would be lame, interfering with their free will. Determinism and (somebody else's) knowledge of the future are not inconsistent with your free will, being forced into a particular future is.

The real universe has the added problems of quantum uncertainty and chaotic behavior which may make future prediction physically impossible. But our story shows that knowledge of your own future is logically impossible - as far as impossibilities go, that one is hard to beat ;)

Any errors and misunderstandings in the discussion above belong to me, and I acknowledge Drescher for the rest. I am looking forward to his new book.

Full post... Related link

Mathematics of Generalization

[Learning_ Books_] Mathematics of Generalization was one of those books that had been waiting on my shelf for quite some time. Partly because I was intimidated by the possible difficulty of mathematics. It turns out the articles are very well written and not that intimidating at all. I will post small perls of wisdom I get from the book in the comments of this post.

Full post... Related link

Fooled by Randomness (Nassim Nicholas Taleb)

[Books_] I read this on Feyzu's recommendation, highly recommended. A wall street person's perspective on uncertainty and how people are misled by it. The probability concepts are elementary but certainly important and generally under-appreciated by professionals and the public. There is a lot of hostility towards economists, especially financial economists which adds spice. (I seem to like books whose authors have the self confidence to attack well respected fields, persons: The history of pi and A different universe being other examples).
Full post... Related link

Information Theory, Inference, and Learning Algorithms by David MacKay

[Books_] I just discovered this wonderful book by David MacKay. It has a very good interpretation of Occam's razor in Bayesian terms. Very nice examples of Bayesian inference and pitfalls of orthodox statistics. The book is at the intersection of communication, inference, and learning (although very little learning). I am still undecided about the pedagogic value of information theory for teaching inference - for the most part the information theory results can be stated in probability terms and vice versa. It inspired me to replan my machine learning course more on foundations and less on various algorithmic incarnations. The book is freely available on the net at the above address (which is another reason I appreciated it and bought a hardcopy at Amazon).
Full post... Related link