Tag Archives: Machine Learning

Undocumented Machine Learning (VI): Logistic Regression

In this blog post, I will discuss the logistic regression model and show some thought of mine regarding the relation between logistic regression and naive Bayes. 1. Logistic Regression The binary classifier of logistic regression is to assume the label … Continue reading

Posted in Machine Learning | Tagged

Undocumented Machine Learning (V): Naive Bayes

In this blog post, I will show that the decision function of Naive Bayes is the same as logistic regression for observation from exponential family. 1. Naive Bayes with Gaussian The basic assumption of Naive Bayes is . The posterior … Continue reading

Posted in Machine Learning | Tagged | 1 Comment

Undocumented Machine Learning (IV): Exponential Family

1. Exponential Family The exponential family of distributions over , given the parameter , is defined to be the set of distributions of the form or equivalently Here is called the natural parameter of the distribution, and is some function … Continue reading

Posted in Machine Learning | Tagged | 1 Comment

Undocumented Machine Learning (III): From Probabilistic Regression to Gaussian Process

In this post, I will show how directly derive the Gaussian process formulation of nonlinear regression from probabilistic linear regression model. Doing this, some common misunderstanding of GP will be clear. First, for reference purpose, let’s write down the probabilistic … Continue reading

Posted in Machine Learning | Tagged | 1 Comment

Undocumented Machine Learning (II): Kernel Regression

Last time I talked about linear ridge regression. You might think the trick in last post is trivial. However, applying the trick for nonlinear regression won’t be so obvious any more. It is natural to extend the linear ridge regression … Continue reading

Posted in Machine Learning | Tagged

Undocumented Machine Learning (I): Linear Regression

In this post, I will discuss a seldom documented aspect or trick for one of the simplest model: linear regression. I will show how to make the solution of ridge regression translation invariant and what the meaning of the bias … Continue reading

Posted in Machine Learning | Tagged

Undocumented Machine Learning

Having been working on Machine Learning for a while, I realize that many models work only if you have an deep understanding of them and use them correctly, even for the simplest ones. In the most of time, the literatures … Continue reading

Posted in Machine Learning | Tagged