Feel Statistics

Newton-Raphson method

Newton-Raphson method, also called the Newton’s method, is a root-finding algorithm that uses the Taylor series of a function $f(x)$ in the vicinity of a suspected root. Given an initial guess of the root $x_0$ , the Taylor series of $f(x)$ about the point $x=x_0+\varepsilon_0$ is given by

$f(x_0+\varepsilon)=f(x_0)+f\prime(x_0)\varepsilon_0+...$

If $x=x_0+\varepsilon_0$ is the root, then $f(x_0)=0$ . Thus we can get

$\varepsilon_0=-\frac{f(x_0)}{f\prime(x_0)}$ .

By letting $x_1=x_0+\varepsilon_0$ , we can calculate a new $\varepsilon_1$ , and so on. At the nth step, we can get

$x_n=x_{n-1}-\frac{f(x_{n-1})}{f\prime(x_{n-1})}$ .

Newton-Raphson can be used to obtain maximum likelihood estimation of a statistical model. For MLE, after we get the log-likelihood function, we take the first derivative and set it to 0. In this case, it likes to find the root of a function. Thus, Newton-Raphson method can be used directly.

This entry was posted on Tuesday, April 1st, 2008 at 7:20 pm and is filed under Research. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

You must be logged in to post a comment.

Newton-Raphson method

Leave a Reply