Machine Learning - Classification algorithm

Question

I want to find the following probability:

P(y=1/n=k; thetha)

Read as:

Probability, The prediction is class 1 given number of words = k, parametrized by thetha

A traditional classification doesn't have the conditional probability (right)

P(y = 1; thetha)

How do I solve this?

EDIT:

For example, lets say I want to predict whether an email is spam or not based on the number of attachments. Let y=1 indicate spam and y=0 be non-spam.

So,

P(y = 1/num_attachements=0; some attributes)
and so on!!

Is it making any sense?

Very unclear question. Specify what is the application of this task, what impact make number of word k and what are other parameters theta. Also there's no such thing as "traditional classification". Most probably you are interested in classifiers based on probability theory (such as Naive Bayes), which may be easily combined with other probability theory methods. — ffriend
– ffriend, Commented Feb 23, 2012 at 23:41
@ffriend: please take a look at the edits Thanks let me know incase its still unclear — frazman
– frazman, Commented Feb 23, 2012 at 23:45

ffriend · Accepted Answer · 2012-02-24 19:11:46Z

1

Normally number of attachments is just another attribute, so your probability is the same as

P(y = 1 | all attributes)

However, if you have some special treatment of attachment (say, other attributes are numeric and attachment is boolean) you can compute them separately and then combine as:

P(C|A, B) = P(C|A) * P(C|B) / P(C)

where C stands for event y = 1, A - for attachments and B for other attributes.

See this paper for description of several Nave Bayes classifiers.

edited Feb 24, 2012 at 19:11

answered Feb 24, 2012 at 0:07

ffriend

28.7k14 gold badges93 silver badges137 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Edouard Over a year ago

Your formula makes no sense to me. You have P(y = 1 | A, B) = 1/Z * P(y = 1, A, B) = 1/Z * P(y = 1) * P(A, B | y = 1), where Z = P(A, B). Then, naive bayes model assumes that A and B are independent knowing y = 1. So you get P(y = 1 | A, B) = 1/Z * P(y = 1) * P(A | y = 1) * P(B | y = 1).

ffriend Over a year ago

@Edouard: actually you are right - I made standard but unreasonable for this case assumption that P(y = 1) is equal to P(y = 0) (that is probabilities for all classes are equal). I updated my answer to take into account this assumption and still use probabilities P(C|A) and P(C|B) in case they are calculated separately. Your last formula is equivalent and is another way to calculate same thing - you can get my formula by applying Bayes rule to both P(A|C) and P(B|C) in yours.

Matt Alcock · Accepted Answer · 2012-02-23 23:46:03Z

1

Use a Naive Baisean classifier. You can code one yourself quite quickly or use/look at the nltk library.

answered Feb 23, 2012 at 23:46

Matt Alcock

13k15 gold badges49 silver badges61 bronze badges

Collectives™ on Stack Overflow

Machine Learning - Classification algorithm

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related