( See [6] for a good introduction to ML and Bayesian estimation. )
Maximum likelihood estimation (MLE) is a way to estimate model
parameters based on parameterized probability density functions.
\begin{enumerate}
\item Construct a conditional model $P(x|\theta)$ which gives the
probability density function (PDF) of observables $x$ in terms of
model parameters $\theta$.
\item Interpret the PDF as a function $L(\theta) = P(x_0 | \theta)$,
setting the observables to a particular observed outcome $x_0$, and
find the $\theta_0$ that maximizes this function.
\end{enumerate} This gives a $\theta_0$ that \emph{best explains} the
data, as the probability of observing $x_0$ is highest for the
parameter vector $\theta_0$.
A simple example of MLE is linear least squares estimation with
uniform noise assumption.
From [6] p. 91, the Kalman filter arrises in a Bayesian framework from
calculating updates of conditional probabilities, taking into account
the next observation step. In the case where PDFs of parameter priors
and noise sources are gaussian, there is a efficient update mechanism
to compute parameter representation of these PDFs (mean and covariance
matrix). In general the KF gives the best possible linear estimator
in a MSE sense.
In [6], p. 71-77 the differences between ML, MMSE and MAP estimators
is explained using the concept of risk[7] which combines cost with
probability. An estimator minimises risk. Different cost functions
give rise to different estimators.
In [8] a brief explanation is given about how one arrives at the
Kalman filter relations from a recursive Bayesian setting.
% [1] http://www.tina-vision.net/docs/memos/1996-002.pdf
% [2] ftp://ftp.esat.kuleuven.ac.be/pub/SISTA/nackaerts/other/alln.ps.gz
% [3] http://en.wikipedia.org/wiki/Maximum_likelihood
% [4] http://en.wikipedia.org/wiki/Likelihood
% [5] http://en.wikipedia.org/wiki/Ordinary_least_squares
% [6] http://www-sigproc.eng.cam.ac.uk/~sjg/book/digital_audio_restoration.zip
% [7] http://en.wikipedia.org/wiki/Risk_%28statistics%29
% [8] http://en.wikipedia.org/wiki/Kalman_filter#Relationship_to_recursive_Bayesian_estimation