Big questions

While browsing some interesting talks on the web, I realized some of the "big questions" that cause much confusion fall into one of two categories.

1. Physics does not have X. Is X something real and missing in physics, or is X an illusion, a modeling tool, a result of our limited information, agent based perspective, or stupidity? Examples: consciousness, free will, causality and directionality of time.
2. Physics does have X which doesn't make sense. Is X something real, or a modeling tool? Examples: randomness, entropy, various sorts of quantum weirdness.

Here are some of the related links:

• I was trying to find out what fellow MIT AI Lab alum Gary Drescher is up to after writing Good and Real (which covers most of the big questions mentioned above). I found this video of a talk on causality and choice he gave at the Singularity Summit in 2009. He argues convincingly that determinism and choice are not at odds with each other.
• Speaking of causality, Michael Nielsen (author of Quantum Computation and Quantum Information, as well as a free online book on Neural Networks and Deep Learning) has a nice article summarizing the key points of Judea Pearl's Causality book. The epilogue of the book (well worth the read if you don't have the time or patience for the whole book) is a lecture Pearl gave some years ago where he covers the history of the confusion about causality, why physics (if we model the whole universe) does not require it, how economics and other social sciences (where lack of controlled experiments make causality detection difficult) still do not adequately model it, and how it is actually possible in some cases to derive causality from purely observational data without resorting to controlled experiments.
• Another regular at the Singularity Summit is Eliezer Yudkowsky. I first discovered his posts on Less Wrong, which has a lot of good material that identifies common sources of confusion when thinking about the big questions.

Full post...

Monument Valley

For all of you Escher fans out there...

Full post... Related link

Poof

Full post...

Bret Victor

Bret Victor - Inventing on Principle from CUSEC on Vimeo.

Bret Victor's inspirational talk with his views on (1) how to flourish fragile ideas, and (2) how to live your life. For more from Bret, check out his website.
Full post...

Vi Hart

For more from my 7 year old daughter's new favorite educator Vi Hart check out Khan Academy, YouTube, other videos, Wikipedia, or her blog.
Full post...

My comments on Norvig's comments on Chomsky's comments on Statistical Learning

Noam Chomsky made a few negative comments on statistical learning last year during the MIT150 Symposium on Brains, Minds, and Machines. Peter Norvig, who was also at the symposium, later published a provocative essay on Chomsky's comments.

I particularly enjoyed the statistical analysis of Chomsky's famous example: "Pereira (2001) showed that such a (statistical, finite-state) model, augmented with word categories and trained by expectation maximization on newspaper text, computes that (a) 'colorless green ideas sleep furiously' is 200,000 times more probable than (b) 'furiously sleep ideas green colorless'."

If interested in a colorful follow up to this discussion, see Straw men and Bee Science by Mark Liberman on the Language Log and the related comments.

Full post...

Writing a paper with a 1st year PhD student

For more see http://researchinprogress.tumblr.com.
Full post...

Probabilistic Programming

The probabilistic programming language Church brings together two of my favorite subjects: Scheme and Probability. I highly recommend this tutorial to graduate students interested in machine learning and statistical inference. The tutorial explains probabilistic inference through programming starting from simple generative models with biased coins and dice leading up to hierarchical, non-parametric, recursive and nested models. Even at the undergraduate level, I have long thought probability and statistics should be taught in an integrated manner instead of their current almost independent treatment. One roadblock is that even the simplest statistical inference (e.g. three tosses of a coin with an unknown (uniformly distributed) weight results in H, H, T; what is the fourth toss?) requires some calculus at the undergraduate level. Using a programming language like Church may allow an instructor to introduce basic concepts without students getting confused about the details of integration.
Full post...

Istanbul Marathon 2010

Some pictures from the Istanbul Marathon...

Here is our route from runkeeper.com:



Some useful links:
evolutionrunning.com: How to run without injury.
Galloway's Marathon book: Everybody can do it!
Born to run: To get inspired.

And some pictures:

























Full post... Related link

Author Alerts

I have looked for a way to get alerted about new releases from my favorite authors for a long time. I think Amazon used to support this in the past but they no longer do. Barnes and Noble has writer alerts for only a small list of authors. This is such an obvious feature for a bibliophile that I do not understand why nobody supports it. I was about to write my own code but luckily ran into www.authoralerts.com first. Highly recommended.

Full post... Related link

How to speak

******************
KOC UNIVERSITY
ECOE 590 SEMINAR
******************

Speaker : Patrick Winston (video presentation)
Title : How to speak
Date : 17 November 2009, Tuesday
Time : 17:00
Place : ENG B30
Refreshments will be served at 16:45

Abstract: In this skillful lecture, Professor Patrick Winston of the Massachusetts Institute of Technology offers tips on how to give an effective talk, cleverly illustrating his suggestions by using them himself. He emphasizes how to start a lecture, cycling in on the material, using verbal punctuation to indicate transitions, describing "near misses" that strengthen the intended concept, and asking questions. He also talks about using the blackboard, overhead projections, props, and "how to stop."

Video available at http://isites.harvard.edu/fs/html/icb.topic58703/winston1.html

Full post... Related link

Natural Language Processing summer course at Sabanci University

This summer Kemal Oflazer, Dilek Hakkani-Tur ve Gokhan Tur are offering a Statistical Natural Language Processing course at Sabanci University. A draft syllabus is included below.

STATISTICAL NLP CLASS:

Kemal Oflazer:

• Overview of NLP (2 hours)
• NLP Applications
• Processing pipeline: Basic steps and how they feed into each other and how they are used by applications

• Morphological Analysis (could be skipped or shortened) (2 hours)

• Introduction to Statistical Models, n-gram language modeling, (2hours)
• Applications to simple sequence problems (tagging English and/or deascifier)

• Morphological Disambiguation (applications to Turkish)

• HMMs (formal treatment (backward-forward + viterbi) + applications to tagging) (2-3 hours)

• CFGs and Probabilistic CFGs (3-4 hours)
• Inside-outside algorithm for training PCFGs
• Parsing with PCFGs
• Machine Translation (MT) (3-4 Hours)
• Brief overview Classical Symbolic MT
• Statistical Machine Translation
• Word-based Models
• Phrase-based Models
• Syntax-based models
• Dealing with Morphology in SMT

Dilek Hakkani-Tur:

Elements of Information Theory / Advanced Language Modeling and Applications

• Entropy/Perplexity/Mutual Information
• Noisy Channel Model
• Sequence classification / HMM
• Sample classification / Naive Bayes
• Smoothing
• Adaptation
Named Entity Extraction (NE)
• Using HMM for NE
• Using CRF for NE
• Using Boosting/MaxEnt/SVM for NE
• Spoken Language Understanding (SLU) as Template Filling
• HMM approaches (AT&T vs BBN)
• Hidden Vector State Models
• Latent Semantic Analysis
• Sample-classification based (Boosting/MaxEnt/Decision Trees)
• Summarization
• Greedy Algorithms, MMR
• TextRank/LexRank
• Classification based extractive summarization
• Global Models for Summarization: Linear Programming approaches
• Question Answering
• Spoken Dialog Systems and Dialog Management (DM)
• Dialog Systems
• DM
• Finite State Models
• Agent Models
• Reinforcement Learning

Gokhan Tur

• Topic Classification
• Discriminative classification: SVM/Boosting
• Generative classification: language model, document similarity, vector-space-model
• Feature selection/transformation (LDA)
• Latent semantic indexing
• SLU as Intent Determination
• Semantic Role Labeling
• Robustness to ASR
• Topic Clustering
• K-Means
• Top/Down vs. Bottom/Up
• Topic Segmentation
• HMM
• TextTiling
• Markov Chains
• Sentence Segmentation
• HMM
• CRF
• Hybrid
• Active Learning/Semi-Supervised Learning/Unsupervised Learning/Model Adaptation/Robustness

Full post...

Top science videos

I have not been this excited about the internet since 1994, when people at the AI Lab would run mosaic everyday and check to see if there were any new pages. My new addiction is videos! You can find quite a number of lectures, interviews etc. and it somehow makes a big difference to see the person talking, rather than just listening or reading. Here is a partial list of my favorites.

• http://www.ted.com/tedtalks
• http://www.poptech.com/popcasts
• http://www.communistrobot.com
  
• http://www.edge.org/edge_video.html
• http://mitworld.mit.edu/index.php
  
• http://videolectures.net
  
• http://www.singinst.org/media
  
• http://www.longnow.org/projects/seminars/
• http://www.transhumanism.org/tv
  
• http://meaningoflife.tv/
• http://www.bbc.co.uk/sn/tvradio/programmes/horizon/broadband/archive/feynman/
• http://googleresearch.blogspot.com/2006/12/google-research-picks-for-videos-of.html
  
• http://sapere.alice.it/extra/126/23_settembre.html
• http://www.thetech.org/genetics/pov_atkins/index.html
• http://www.thefutureofscience.org/23rd_evolMind.htm
  
• http://technetcast.ddj.com/tnc_play_stream.html?stream_id=256
• http://sss.stanford.edu/coverage/audioandvideo/
  
• http://www.aaai.org/AITopics/html/show.html
• http://web.mit.edu/webcast/csail/2006/
  
• http://www.quiprocone.org/Protected/Lecture1.htm
• http://swiss.csail.mit.edu/classes/6.001/abelson-sussman-lectures

Full post...

sodarace

This is how physics should be taught in high school!
Full post... Related link

STORYTRON - Interactive Storytelling

Wouldn't you like to be a character in one of your favorite novels and participate in directing the course of the story events? Chris Crawford, an ex computer game designer, has set out to give us the technology to do just that. Writers, instead of depicting a fixed course of events, will spend their time defining characters and circumstances. The reader, instead of just following a fixed course of events, will participate as one of the characters and watch as others in the story act and react to his actions. I am looking forward to their first demos, which should be available soon. I am also fascinated with the technology from an AI perspective: in particular the question of how much depth do the characters need to have in order for the story that unfolds from their interaction to be interesting.
Full post... Related link

Best visual illusion ever

This has got to be the strongest visual illusion I have seen. BTW I got to it from the web site of Jeff Bridges, which is very cool - www.jeffbridges.com. And i got to that using Stumble! - www.stumbleupon.com... I know I know I should be working right now...
Full post... Related link