In numerical analysis, finite differences are widely used for approximating derivatives, and the term "finite difference" is often used as an abbreviation of "finite difference approximation of derivatives". Finite difference approximations are finite difference quotients in the terminology employed above.
Finite differences were introduced by Brook Taylor in 1715 and have also been studied as abstract self-standing mathematical objects in works by George Boole (1860), L. M. Milne-Thomson (1933), and Károly Jordan [de] (1939). Finite differences trace their origins back to one of Jost Bürgi's algorithms (c. 1592) and work by others including Isaac Newton. The formal calculus of finite differences can be viewed as an alternative to the calculus of infinitesimals.
Three basic types are commonly considered: forward, backward, and central finite differences.
A forward difference, denoted of a functionf is a function defined as
Depending on the application, the spacing h may be variable or constant. When omitted, h is taken to be 1; that is,
A backward difference uses the function values at x and x − h, instead of the values at x + h and x:
If h has a fixed (non-zero) value instead of approaching zero, then the right-hand side of the above equation would be written
Hence, the forward difference divided by h approximates the derivative when h is small. The error in this approximation can be derived from Taylor's theorem. Assuming that f is twice differentiable, we have
The same formula holds for the backward difference:
However, the central (also called centered) difference yields a more accurate approximation. If f is three times differentiable,
The main problem with the central difference method, however, is that oscillating functions can yield zero derivative. If f (nh) = 1 for n odd, and f (nh) = 2 for n even, then f ′(nh) = 0 if it is calculated with the central difference scheme. This is particularly troublesome if the domain of f is discrete. See also Symmetric derivative.
Authors for whom finite differences mean finite difference approximations define the forward/backward/central differences as the quotients given in this section (instead of employing the definitions given in the previous section).
In an analogous way, one can obtain finite difference approximations to higher order derivatives and differential operators. For example, by using the above central difference formula for f ′(x + h/2) and f ′(x − h/2) and applying a central difference formula for the derivative of f ′ at x, we obtain the central difference approximation of the second derivative of f:
Similarly we can apply other differencing formulas in a recursive manner.
Second order forward
Second order backward
More generally, the nth order forward, backward, and central differences are given by, respectively,
Note that the central difference will, for odd n, have h multiplied by non-integers. This is often a problem because it amounts to changing the interval of discretization. The problem may be remedied taking the average of δn[ f ](x − h/2) and δn[ f ](x + h/2).
Forward differences applied to a sequence are sometimes called the binomial transform of the sequence, and have a number of interesting combinatorial properties. Forward differences may be evaluated using the Nörlund–Rice integral. The integral representation for these types of series is interesting, because the integral can often be evaluated using asymptotic expansion or saddle-point techniques; by contrast, the forward difference series can be extremely hard to evaluate numerically, because the binomial coefficients grow rapidly for large n.
The relationship of these higher-order differences with the respective derivatives is straightforward,
Higher-order differences can also be used to construct better approximations. As mentioned above, the first-order difference approximates the first-order derivative up to a term of order h. However, the combination
approximates f ′(x) up to a term of order h2. This can be proven by expanding the above expression in Taylor series, or by using the calculus of finite differences, explained below.
If necessary, the finite difference can be centered about any point by mixing forward, backward, and central differences.
For a given polynomial of degree n ≥ 1, expressed in the function P(x), with real numbers a ≠ 0 and b and lower order terms (if any) marked as l.o.t.:
After n pairwise differences, the following result can be achieved, where h ≠ 0 is a real number marking the arithmetic difference:
Only the coefficient of the highest-order term remains. As this result is constant with respect to x, any further pairwise differences will have the value 0.
Let Q(x) be a polynomial of degree 1:
This proves it for the base case.
Let R(x) be a polynomial of degree m − 1 where m ≥ 2 and the coefficient of the highest-order term be a ≠ 0. Assuming the following holds true for all polynomials of degree m − 1:
Let S(x) be a polynomial of degree m. With one pairwise difference:
As ahm ≠ 0, this results in a polynomial T(x) of degree m − 1, with ahm as the coefficient of the highest-order term. Given the assumption above and m − 1 pairwise differences (resulting in a total of m pairwise differences for S(x)), it can be found that:
This completes the proof.
This identity can be used to find the lowest-degree polynomial that intercepts a number of points (x, y) where the difference on the x-axis from one point to the next is a constant h ≠ 0. For example, given the following points:
We can use a differences table, where for all cells to the right of the first y, the following relation to the cells in the column immediately to the left exists for a cell (a + 1, b + 1), with the top-leftmost cell being at coordinate (0, 0):
To find the first term, the following table can be used:
This arrives at a constant 648. The arithmetic difference is h=3, as established above. Given the number of pairwise differences needed to reach the constant, it can be surmised this is a polynomial of degree 3. Thus, using the identity above:
Solving for a, it can be found to have the value 4. Thus, the first term of the polynomial is 4x3.
Then, subtracting out the first term, which lowers the polynomial's degree, and finding the finite difference again:
4 − 4(1)3 = 4 − 4 = 0
109 − 4(4)3 = 109 − 256 = −147
772 − 4(7)3 = 772 − 1372 = −600
2641 − 4(10)3 = 2641 − 4000 = −1359
6364 − 4(13)3 = 6364 − 8788 = −2424
Here, the constant is achieved after only two pairwise differences, thus the following result:
Solving for a, which is −17, the polynomial's second term is −17x2.
Moving on to the next term, by subtracting out the second term:
0 − (−17(1)2) = 0 + 17 = 17
−147 − (−17(4)2) = −147 + 272 = 125
−600 − (−17(7)2) = −600 + 833 = 233
−1359 − (−17(10)2) = −1359 + 1700 = 341
−2424 − (−17(13)2) = −2424 + 2873 = 449
Thus the constant is achieved after only one pairwise difference:
It can be found that a = 36 and thus the third term of the polynomial is 36x. Subtracting out the third term:
17 − 36(1) = 17 − 36 = −19
125 − 36(4) = 125 − 144 = −19
233 − 36(7) = 233 − 252 = −19
341 − 36(10) = 341 − 360 = −19
449 − 36(13) = 449 − 468 = −19
Without any pairwise differences, it is found that the 4th and final term of the polynomial is the constant -19. Thus, the lowest-degree polynomial intercepting all the points in the first table is found:
Arbitrarily sized kernelsEdit
Using linear algebra one can construct finite difference approximations which utilize an arbitrary number of points to the left and a (possibly different) number of points to the right of the evaluation point, for any order derivative. This involves solving a linear system such that the Taylor expansion of the sum of those points around the evaluation point best approximates the Taylor expansion of the desired derivative. Such formulas can be represented graphically on a hexagonal or diamond-shaped grid.
This is useful for differentiating a function on a grid, where, as one approaches the edge of the grid, one must sample fewer and fewer points on one side.
The details are outlined in these notes.
The Finite Difference Coefficients Calculator constructs finite difference approximations for non-standard (and even non-integer) stencils given an arbitrary stencil and a desired derivative order.
which holds for any polynomial function f and for many (but not all) analytic functions. (It does not hold when f is exponential type. This is easily seen, as the sine function vanishes at integer multiples of ; the corresponding Newton series is identically zero, as all finite differences are zero in this case. Yet clearly, the sine function is not zero.) Here, the expression
is the "falling factorial" or "lower factorial", while the empty product(x)0 is defined to be 1. In this particular case, there is an assumption of unit steps for the changes in the values of x, h = 1 of the generalization below.
To illustrate how one may use Newton's formula in actual practice, consider the first few terms of doubling the Fibonacci sequencef = 2, 2, 4, ... One can find a polynomial that reproduces these values, by first computing a difference table, and then substituting the differences that correspond to x0 (underlined) into the formula as follows,
The finite difference of higher orders can be defined in recursive manner as Δn h ≡ Δh(Δn − 1 h). Another equivalent definition is Δn h = [Th − I]n.
The difference operator Δh is a linear operator, as such it satisfies Δh[αf + βg](x) = α Δh[ f ](x) + β Δh[g](x).
It also satisfies a special Leibniz rule indicated above,
Δh(f (x)g(x)) = (Δhf (x)) g(x+h) + f (x) (Δhg(x)). Similar statements hold for the backward and central differences.
Formally applying the Taylor series with respect to h, yields the formula
where D denotes the continuum derivative operator, mapping f to its derivative f ′. The expansion is valid when both sides act on analytic functions, for sufficiently small h. Thus, Th = ehD, and formally inverting the exponential yields
This formula holds in the sense that both operators give the same result when applied to a polynomial.
Even for analytic functions, the series on the right is not guaranteed to converge; it may be an asymptotic series. However, it can be used to obtain more accurate approximations for the derivative. For instance, retaining the first two terms of the series yields the second-order approximation to f ′(x) mentioned at the end of the section Higher-order differences.
The analogous formulas for the backward and central difference operators are
The calculus of finite differences is related to the umbral calculus of combinatorics. This remarkably systematic correspondence is due to the identity of the commutators of the umbral quantities to their continuum analogs (h → 0 limits),
A large number of formal differential relations of standard calculus involving
functions f (x) thus map systematically to umbral finite-difference analogs involving f (xT−1 h).
For instance, the umbral analog of a monomial xn is a generalization of the above falling factorial (Pochhammer k-symbol),
hence the above Newton interpolation formula (by matching coefficients in the expansion of an arbitrary function f (x) in such symbols), and so on.
For example, the umbral sine is
As in the continuum limit, the eigenfunction of Δh/h also happens to be an exponential,
and hence Fourier sums of continuum functions are readily mapped to umbral Fourier sums faithfully, i.e., involving the same Fourier coefficients multiplying these umbral basis exponentials. This umbral exponential thus amounts to the exponential generating function of the Pochhammer symbols.
A generalized finite difference is usually defined as
where μ = (μ0, …, μN) is its coefficient vector. An infinite difference is a further generalization, where the finite sum above is replaced by an infinite series. Another way of generalization is making coefficients μk depend on point x: μk = μk(x), thus considering weighted finite difference. Also one may make the step h depend on point x: h = h(x). Such generalizations are useful for constructing different modulus of continuity.
The generalized difference can be seen as the polynomial rings R[Th]. It leads to difference algebras.
As a convolution operator: Via the formalism of incidence algebras, difference operators and other Möbius inversion can be represented by convolution with a function on the poset, called the Möbius functionμ; for the difference operator, μ is the sequence (1, −1, 0, 0, 0, …).
Multivariate finite differencesEdit
Finite differences can be considered in more than one variable. They are analogous to partial derivatives in several variables.
Some partial derivative approximations are:
Alternatively, for applications in which the computation of f is the most costly step, and both first and second derivatives must be computed, a more efficient formula for the last case is
since the only values to compute that are not already needed for the previous four equations are f (x + h, y + k) and f (x − h, y − k).
^ abcPaul Wilmott; Sam Howison; Jeff Dewynne (1995). The Mathematics of Financial Derivatives: A Student Introduction. Cambridge University Press. p. 137. ISBN 978-0-521-49789-3.
^ abcPeter Olver (2013). Introduction to Partial Differential Equations. Springer Science & Business Media. p. 182. ISBN 978-3-319-02099-0.
^ abcM Hanif Chaudhry (2007). Open-Channel Flow. Springer. p. 369. ISBN 978-0-387-68648-6.
^Jordán, op. cit., p. 1 and Milne-Thomson, p. xxi.
Milne-Thomson, Louis Melville (2000): The Calculus of Finite Differences (Chelsea Pub Co, 2000) ISBN 978-0821821077
^"Finite differences of polynomials". February 13, 2018.
^Fraser, Duncan C. (January 1, 1909). "On the Graphic Delineation of Interpolation Formulæ". Journal of the Institute of Actuaries. 43 (2): 235–241. doi:10.1017/S002026810002494X. Retrieved April 17, 2017.
^Burkard Polster/Mathologer (2021). " Why don't they teach Newton's calculus of 'What comes next?' " on YouTube
^Newton, Isaac, (1687). Principia, Book III, Lemma V, Case 1
^Iaroslav V. Blagouchine (2018). "Three notes on Ser's and Hasse's representations for the zeta-functions" (PDF). Integers (Electronic Journal of Combinatorial Number Theory). 18A: 1–45. arXiv:1606.02044.
^König, Jürgen; Hucht, Fred (2021). "Newton series expansion of bosonic operator functions". SciPost Physics. 10 (1): 007. arXiv:2008.11139. Bibcode:2021ScPP...10....7K. doi:10.21468/SciPostPhys.10.1.007. S2CID 221293056.
^Richtmeyer, D. and Morton, K.W., (1967). Difference Methods for Initial Value Problems, 2nd ed., Wiley, New York.
^Boole, George, (1872). A Treatise On The Calculus of Finite Differences, 2nd ed., Macmillan and Company. On line. Also, [Dover edition 1960]
^Jordan, Charles, (1939/1965). "Calculus of Finite Differences", Chelsea Publishing. On-line: 
^Zachos, C. (2008). "Umbral Deformations on Discrete Space-Time". International Journal of Modern Physics A. 23 (13): 2005–2014. arXiv:0710.2306. Bibcode:2008IJMPA..23.2005Z. doi:10.1142/S0217751X08040548. S2CID 16797959.
^Curtright, T. L.; Zachos, C. K. (2013). "Umbral Vade Mecum". Frontiers in Physics. 1: 15. arXiv:1304.0429. Bibcode:2013FrP.....1...15C. doi:10.3389/fphy.2013.00015. S2CID 14106142.
Levy, H.; Lessman, F. (1992). Finite Difference Equations. Dover. ISBN 0-486-67260-3.
Ames, W.F. (1977). Numerical Methods for Partial Differential Equations. New York, NY: Academic Press. Section 1.6. ISBN 0-12-056760-1.