CHAPTJZR 6 BAYESIAN LEARNING in Software

Generator Denso QR Bar Code in Software CHAPTJZR 6 BAYESIAN LEARNING

One highly practical Bayesian learning method is the naive Bayes learner, often called the naive Bayes classijier In some domains its performance has been shown to be comparable to that of neural network and decision tree learning This section introduces the naive Bayes classifier; the next section applies it to the practical problem of learning to classify natural language text documents The naive Bayes classifier applies to learning tasks where each instance x is described by a conjunction of attribute values and where the target function f ( x ) can take on any value from some finite set V A set of training examples of the target function is provided, and a new instance is presented, described by the tuple of attribute values ( a l ,a 2 a,) The learner is asked to predict the target value, or classification, for this new instance The Bayesian approach to classifying the new instance is to assign the most probable target value, VMAP, given the attribute values ( a l ,a2 a,) that describe the instance

= argmax P(vjlal,a 2 a,)

Now we could attempt to estimate the two terms in Equation (619) based on the training data It is easy to estimate each of the P ( v j ) simply by counting the frequency with which each target value vj occurs in the training data However, estimating the different P(al, a 2 a,lvj) terms in this fashion is not feasible unless we have a very, very large set of training data The problem is that the number of these terms is equal to the number of possible instances times the number of possible target values Therefore, we need to see every instance in the instance space many times in order to obtain reliable estimates The naive Bayes classifier is based on the simplifying assumption that the attribute values are conditionally independent given the target value In other words, the assumption is that given the target value of the instance, the probability of observing the conjunction a l , a 2 a, is just the product of the probabilities for the individual attributes: P(a1, a2 a, 1 v j ) = P(ai lvj) Substituting this into Equation (619), we have the approach used by the naive Bayes classifier

Naive Bayes classifier:

= argmax P (vj)

P (ai1vj)

(620)

where V N B denotes the target value output by the naive Bayes classifier Notice that in a naive Bayes classifier the number of distinct P(ailvj) terms that must

be estimated from the training data is just the number of distinct attribute values times the number of distinct target values-a much smaller number than if we were to estimate the P(a1, a2 anlvj) terms as first contemplated To summarize, the naive Bayes learning method involves a learning step in which the various P(vj) and P(ai Jvj)terms are estimated, based on their frequencies over the training data The set of these estimates corresponds to the learned hypothesis This hypothesis is then used to classify each new instance by applying the rule in Equation (620) Whenever the naive Bayes assumption of conditional independence is satisfied, this naive Bayes classification VNB is identical to the MAP classification One interesting difference between the naive Bayes learning method and other learning methods we have considered is that there is no explicit search through the space of possible hypotheses (in this case, the space of possible hypotheses is the space of possible values that can be assigned to the various P(vj) and P(ailvj) terms) Instead, the hypothesis is formed without searching, simply by counting the frequency of various data combinations within the training examples