Is there a supervised learning algorithm that takes tags as input, and produces a probability as output?

Question

Let's say I want to determine the probability that I will upvote a question on SO, based only on which tags are present or absent.

Let's also imagine that I have plenty of data about past questions that I did or did not upvote.

Is there a machine learning algorithm that could take this historical data, train on it, and then be able to predict my upvote probability for future questions? Note that it must be the probability, not just some arbitrary score.

Let's assume that there will be up to 7 tags associated with any given question, these were drawn from a superset of tens of thousands.

I hope that it can make quite sophisticated connections between tags, rather than each tag simply contributing to the result is a "linear" way (much as words do in a Bayesian spam filter).

So for example, it might be that the word "java" increases my upvote probability, except when it is present with "database", however, "database" might increase my upvote probability when present with "ruby".

Oh, and it should be computationally reasonable (training within an hour or two on millions of questions).

What approaches should I research here?

Answer 1

Node's activation functions are often sigmoid curves, which has very little to do with probabilities. Given that there probably aren't many tags per message, you could just create"n-gram" tags and implement naive Bayes. Regression trees would also provide an empirical probability at the leaf nodes, using +1 for upvote and 0 for no upvote. See http://www.stat.cmu.edu/~cshalizi/350-2006/lecture-10.pdf for some readable lecture notes and http://sites.google.com/site/rtranking/ for an open-source implementation.

If you wish to learn about Supervised Learning then visit this Types of Machine Learning.