Naive Bayes (NB) is a classic supervised machine learning algorithm that is based upon the traditional statistical principal: Bayes’ Theorem. The technique is referred to as naive due to the naive assumption made that each of the input variables are independent from one another. In general NB models calculates probabilities. Given input data the algorithm determines the likelihood of the input feature belonging to a specific class. To perform NB the model must initially be taught on training data to then accurately interpret input data and make class predictions. NB is most commonly used when dealing with text data and /or categorical or count based response features.

More specifically the Multinomial NB algorithm is ideally designed towards working with discrete count data. In this version of NB the algorithm assumes that the features follow a multinomial distribution and takes into account how often an item appears. Multinomial NB is often used in text classification. For example, given the frequencies of certain words the algorithm can predict who the author of the text is likely to be. Alternatively Bernoulli NB is designed for binary or boolean features and predictions. For this instance the algorithm does not count frequencies rather determines the presence or lack of presence of a feature. This approach is commonly used in email spam detection, is the incoming email spam or not - binary. Another common NB approach is Gaussian NB which in turn assumes that features follow a Gaussian - or normal - distribution. This approach is most applicable to dealing with numeric continuous feature variables such as age or salaries. Finally, Categorical NB, which is a helpful approach when dealing with a response that has a fixed number of discrete categories. This approach is often applied when looking at population demographic attributes.

For all NB models (although often less necessary for Gaussian models dealing with continuous data) smoothing is required to adjust probability estimates and avoid zero values. Smoothing instead applies very small probabilities to unseen events, rather than zero. Avoiding zero is necessary because the presenece of a zero in the probability equation will result in the entire probability becoming 0 even if the potential class is otherwise an accurate fit.