2. Types of classifiers
Models of selected classifiers
Linear discriminant classifiers.
A linear classifier separates of two or more classes of objects by linear decision boundary.
Classifier name
Description
Is a simple classifier. NMC computes mean for every class and assigns sample to nearest mean (minimum distance to the mean). Results are presented as in linear classification boundaries. Prior probabilities are not used.
A classifier with a linear decision boundary, generated by fitting class conditional densities to the data and using Bayes’ rule. The model fits a Gaussian density to each class, assuming that all classes share the same covariance matrix. The fitted model can also be used to reduce the dimensionality of the input by projecting it to the most discriminative directions.
Quadratic discriminant classifiers.
A quadratic classifier separates two or more classes of objects by a quadric decision boundary.
Classifier name
Description
A classifier with a quadratic decision boundary, generated by fitting class conditional densities to the data and using Bayes’ rule. The model fits a Gaussian density to each class.
Other discriminant classifiers.
This type of classifier separates two or more classes of objects by decision boundary, which is not linear or quadratic.
Classifier name
Description
The Naive Bayes Classifier estimates parameters for every class and every feature separately. Total class densities are constructed by assuming independency of particular feature densities and their multiplication. The Bayesian classifier is based on statistical theories of learning – Bayes theorem.
Visualization of discriminant function
On the basis of module 1 and knowledge of the different shapes of discriminant function, we can now show the difference between the two groups of classifiers. The results, for generated dataset of five classes, are presented in figure below.
The left side shows dwo decision boundaries for linear classifier. A top right subplot shows quadratic classifier and bottom right — Bayes classifier. We can draw such figure using following script.
Controlling of training process
Some classifiers have as input parameters not only dataset. For example LDA classifier has a shrinkage parameter, with values between 0 and 1. For different values of this parameter we get different decision regions.
from sklearn import discriminant_analysis
classifiers = [
discriminant_analysis.LinearDiscriminantAnalysis(solver='lsqr',shrinkage=0),
discriminant_analysis.LinearDiscriminantAnalysis(solver='lsqr',shrinkage=.5),
discriminant_analysis.LinearDiscriminantAnalysis(solver='lsqr',shrinkage=1)
]
classifier_names = [0, .5, 1]More classifiers
Classifier name
Description
A decision tree is loosely defined as a hierarchical classification procedure determined by a sequence of questions. Once the first question has been asked, the choice of subsequent questions depends on the answer to the current question. This can be represented by a directed graph known as a tree.
Classifier name
Description
The perceptron is a type of artificial neural network invented in 1957 by Frank Rosenblatt. It can be seen as a linear classifier. The perceptron is a binary classifier which maps its input x (a real-valued vector) to an output value f(x) (a single binary value) across the matrix.
Classifier name
Description
Neural networks have emerged as an important tool for classification. The recent vast research activities in neural classification have established that neural networks are a promising alternative to various conventional classification methods. The advantage of neural networks lies in the following theoretical aspects. First, neural networks are data driven self-adaptive methods in that they can adjust themselves to the data without any explicit specification of functional or distributional form for the underlying model. Second, they are universal functional approximators in that neural networks can approximate any function with arbitrary accuracy.
Classifier name
Description
SVMs identify support vectors (SVs) H1 and H2 that will create a margin between the two classes, thus ensuring that the data is “more separable” than in the case of the conventional classifier. This problem is equivalent to searching for a higher dimension where the data are linearly separable, and then designing a linear classifier in that higher dimension.
Excercise
Generate two datasets, choose three classifiers and analyse one, chosen parameter from each of them in the context of decision space.
Last updated