August 21, 2012

PAC Bayes isn’t dead

Filed under: Uncategorized — heavytailed @ 2:28 am

Mea culpa. See this lecture. There are some nice points to it: first, a nudge in my direction that anyone who tries to be ultra rigorous is going to come in last place; second, a nice statement about the relationship between machine learning and computer science:

In real software applications that compute by optimizing, a fundamental issue is what is the objective function. … Machine learning is about getting the objective function that the software optimizes when it runs.

His approach to PAC-Bayes analysis (fix prior, posterior, etc) is pretty nice. Nevertheless, I don’t like it as much as the concept of Luckiness (despite the fact that luckiness seems to have faded in favor of Pac Bayes), as it doesn’t make as nice a statement about what a serendipitous compatibility between feature distribution and hypothesis class “looks like” (e.g. My tree is actually a decision list, or my SVM has a really large margin).

Anyway, back to implementing test classes.


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: