
Recent Posts
Recent Comments
Archives
Categories
Meta
Author Archives: statinfer
Undocumented Machine Learning (VI): Logistic Regression
In this blog post, I will discuss the logistic regression model and show some thought of mine regarding the relation between logistic regression and naive Bayes. 1. Logistic Regression The binary classifier of logistic regression is to assume the label … Continue reading
Posted in Machine Learning
Tagged Machine Learning
Undocumented Machine Learning (V): Naive Bayes
In this blog post, I will show that the decision function of Naive Bayes is the same as logistic regression for observation from exponential family. 1. Naive Bayes with Gaussian The basic assumption of Naive Bayes is . The posterior … Continue reading
Undocumented Machine Learning (IV): Exponential Family
1. Exponential Family The exponential family of distributions over , given the parameter , is defined to be the set of distributions of the form or equivalently Here is called the natural parameter of the distribution, and is some function … Continue reading
Undocumented Machine Learning (III): From Probabilistic Regression to Gaussian Process
In this post, I will show how directly derive the Gaussian process formulation of nonlinear regression from probabilistic linear regression model. Doing this, some common misunderstanding of GP will be clear. First, for reference purpose, let’s write down the probabilistic … Continue reading
Undocumented Machine Learning (II): Kernel Regression
Last time I talked about linear ridge regression. You might think the trick in last post is trivial. However, applying the trick for nonlinear regression won’t be so obvious any more. It is natural to extend the linear ridge regression … Continue reading
Posted in Machine Learning
Tagged Machine Learning
Undocumented Machine Learning (I): Linear Regression
In this post, I will discuss a seldom documented aspect or trick for one of the simplest model: linear regression. I will show how to make the solution of ridge regression translation invariant and what the meaning of the bias … Continue reading
Posted in Machine Learning
Tagged Machine Learning
Undocumented Machine Learning
Having been working on Machine Learning for a while, I realize that many models work only if you have an deep understanding of them and use them correctly, even for the simplest ones. In the most of time, the literatures … Continue reading
Posted in Machine Learning
Tagged Machine Learning