( See [6] for a good introduction to ML and Bayesian estimation. ) Maximum likelihood estimation (MLE) is a way to estimate model parameters based on parameterized probability density functions. \begin{enumerate} \item Construct a conditional model $P(x|\theta)$ which gives the probability density function (PDF) of observables $x$ in terms of model parameters $\theta$. \item Interpret the PDF as a function $L(\theta) = P(x_0 | \theta)$, setting the observables to a particular observed outcome $x_0$, and find the $\theta_0$ that maximizes this function. \end{enumerate} This gives a $\theta_0$ that \emph{best explains} the data, as the probability of observing $x_0$ is highest for the parameter vector $\theta_0$. A simple example of MLE is linear least squares estimation with uniform noise assumption. From [6] p. 91, the Kalman filter arrises in a Bayesian framework from calculating updates of conditional probabilities, taking into account the next observation step. In the case where PDFs of parameter priors and noise sources are gaussian, there is a efficient update mechanism to compute parameter representation of these PDFs (mean and covariance matrix). In general the KF gives the best possible linear estimator in a MSE sense. In [6], p. 71-77 the differences between ML, MMSE and MAP estimators is explained using the concept of risk[7] which combines cost with probability. An estimator minimises risk. Different cost functions give rise to different estimators. In [8] a brief explanation is given about how one arrives at the Kalman filter relations from a recursive Bayesian setting. % [1] http://www.tina-vision.net/docs/memos/1996-002.pdf % [2] ftp://ftp.esat.kuleuven.ac.be/pub/SISTA/nackaerts/other/alln.ps.gz % [3] http://en.wikipedia.org/wiki/Maximum_likelihood % [4] http://en.wikipedia.org/wiki/Likelihood % [5] http://en.wikipedia.org/wiki/Ordinary_least_squares % [6] http://www-sigproc.eng.cam.ac.uk/~sjg/book/digital_audio_restoration.zip % [7] http://en.wikipedia.org/wiki/Risk_%28statistics%29 % [8] http://en.wikipedia.org/wiki/Kalman_filter#Relationship_to_recursive_Bayesian_estimation