October 18, 2012

My comments on Norvig's comments on Chomsky's comments on Statistical Learning

Noam Chomsky made a few negative comments on statistical learning last year during the MIT150 Symposium on Brains, Minds, and Machines. Peter Norvig, who was also at the symposium, later published a provocative essay on Chomsky's comments.

I particularly enjoyed the statistical analysis of Chomsky's famous example: "Pereira (2001) showed that such a (statistical, finite-state) model, augmented with word categories and trained by expectation maximization on newspaper text, computes that (a) 'colorless green ideas sleep furiously' is 200,000 times more probable than (b) 'furiously sleep ideas green colorless'."

If interested in a colorful follow up to this discussion, see Straw men and Bee Science by Mark Liberman on the Language Log and the related comments.

Full post...

Writing a paper with a 1st year PhD student

For more see http://researchinprogress.tumblr.com.
Full post...