1 9
2
3 4
5
6
7
8
10
11
Figure 4.3: Triangulation of a two-dimensional region with 11 nodes. Contourlines for the basis function φ8(x, y) are also shown as dashed lines.
takes the value 1 at the node (xj, yj) and 0 at all other nodes. This function is continuous across the boundaries between triangles and nonzero only for the triangles that have Node j as a corner. For example, Figure 4.3 indicates contour lines for the basis functionφ8(x, y) as dashed lines.
Using (4.32) in (4.31) gives anN×N linear system of the formKc=F where Kij =−
Z Z
Ω∇φj· ∇φidx dy. (4.33)
These gradients are easy to compute and in fact are constant within each triangle since the basis function is linear there. Since ∇φi is identically zero in any triangle for which Node i is not a corner, we see that Kij = 0 unless Nodes i andj are two corners of a common triangle. For example, in Figure 4.3 the eighth row of the matrixK will have have nonzeros only in columns 6, 7, 8, 10, and 11.
Note also that K will be a symmetric matrix, since the expression (4.33) is symmetric in iand j.
It can also be shown to be positive definite.
For a typical triangulation on a much finer grid, we would have a large but very sparse matrix K.
The structure of the matrix, however, will not generally be as simple as what we would obtain with a finite difference method on a rectangular grid. The pattern of nonzeros will depend greatly on how we order the unknowns and equations. Direct methods for solving such systems efficiently often rely on sophisticated graph theoretical algorithms for ordering the unknowns to minimize the bandwidth; see for example [DER86]. In practice iterative methods are often used to solve the sparse linear system.
Chapter 5
Iterative Methods for Sparse Linear Systems
This chapter contains a brief overview of several iterative methods for solving the large sparse linear systems that arise from elliptic equations, either from finite-difference approximations (Chapter 3) or from finite element approximations (Section 4.3). Large sparse linear systems arise from many other practical problems too, of course, and the methods discussed here are important more generally.
The classical Jacobi and Gauss-Seidel methods will first be introduced for the 5-point Laplacian.
Next we will analyze the convergence properties of these methods based on viewing them in terms of matrix splittings.
The SOR and conjugate gradient methods will also be briefly introduced. The reader can find considerably more theoretical analysis of these methods in the literature. See for example [GO89], [GO92], [Gre97], [HY81], [She94], [TB97], [Var62], [You71].
5.1 Jacobi and Gauss-Seidel
Except when the matrix has very special structure and fast direct methods of the type discussed in Section 3.5 apply, iterative methods are usually the method of choice for large sparse linear systems.
In this section two classical iterative methods, Jacobi and Gauss-Seidel, are introduced to illustrate the main issues. It should be stressed at the beginning that these are poor methods in general which converge very slowly, but they have the virtue of being simple to explain. More efficient methods are discussed later in this chapter.
We again consider the Poisson problem where we have the system of equations (3.10). We can rewrite this equation as
uij =1
4(ui−1,j+ui+1,j+ui,j−1+ui,j+1)−h2
4 fij. (5.1)
In particular, note that for Laplace’s equation (where fij ≡0) this simply states that the value ofuat each grid point should be the average of its four neighbors. This is the discrete analog of the well-known fact that a harmonic function has the following property: The value at any point (x, y) is equal to the average value around a closed curve containing the point, in the limit as the curve shrinks to the point.
Physically this also makes sense if we think of the heat equation. Unless the temperature at this point is equal to the average of the temperature at neighboring points, there will be a net flow of heat towards or away from this point.
The equation (5.1) suggests the following iterative method to produce a new estimateu[k+1]from a current guessu[k]:
u[k+1]ij = 1
4(u[k]i−1,j+u[k]i+1,j+u[k]i,j−1+u[k]i,j+1)−h2
4 fij. (5.2)
61
This is theJacobiiteration for the Poisson problem, and it can be shown that for this particular problem it converges from any initial guessu[0] (though very slowly).
Here is a short section ofmatlabcode that implements the main part of this iteration:
for iter=0:maxiter for j=2:(m+1)
for i=2:(m+1)
unew(i,j) = 0.25*(u(i-1,j) + u(i+1,j) + u(i,j-1) + u(i,j+1) - h^2 * f(i,j));
end end u = unew;
end
where it is assumed that u initially contains the guess u[0] and that boundary data is stored into u(1,:), u(m+2,:), u(:,1), and u(:,m+2). The indexing is off by one from what one might expect sincematlabbegins arrays with index 1, not 0.
Note that one might be tempted to dispense with the variable unewand replace the above code by for iter=0:maxiter
for j=2:(m+1) for i=2:(m+1)
u(i,j) = 0.25*(u(i-1,j) + u(i+1,j) + u(i,j-1) + u(i,j+1) - h^2 * f(i,j));
end end end
This would not give the same results, however. In the correct code for Jacobi we compute new values ofubased entirely on old data from the previous iteration, as required from (5.2). In the second code we have already updatedu(i-1,j)andu(i,j-1)before we updateu(i,j), and these new values will be used instead of the old ones. The latter code thus corresponds to the method
u[k+1]ij =1
4(u[k+1]i−1,j+u[k]i+1,j+u[k+1]i,j−1+u[k]i,j+1)−h2
4 fij. (5.3)
This is in fact what is known as theGauss-Seidelmethod, and it would be a lucky coding error since this method generally converges about twice as fast as Jacobi’s method.
Convergence of these methods will be discussed in Section 5.2. First we note some important features of these iterative methods:
• The matrixAis never stored. In fact, for this simple constant coefficient problem, we don’t even store all the 5m2 nonzeros which all have the value 1/h2 or −4/h2. The values 0.25 and h2 in the code are the only values that are “stored”. (For a variable coefficient problem where the coefficients are different at each point, we would in general have to store them all.)
• Hence the storage is optimal — essentially only the m2 solution values are stored in the Gauss-Seidel method. The above code for Jacobi uses 2m2 since unew is stored as well as u, but one could eliminate most of this with more careful coding.
• Each iteration requiresO(m2) work. The total work required will depend on how many iterations are required to reach the desired level of accuracy. In Chapter 5 we will see that with these particular methods we requireO(m2logm) iterations to reach a level of accuracy consistent with the expected global error in the solution (ash→0 we should require more accuracy in the solution to the linear system). Combining this with the work per iteration gives a total operation count
ofO(m4logm). This looks worse than Gaussian elimination with a banded solver, though since logm grows so slowly with m it is not clear which is really more expensive for a realistic size matrix. (And the iterative method definitely saves on storage.)
Other iterative methods also typically require O(m2) work per iteration, but may converge much faster and hence result in less overall work. The ideal would be to converge in a number of iterations that is independent of hso that the total work is simplyO(m2). Multigrid methods can achieve this, not only for Poisson’s problem but also for many other elliptic equations.