Sparse matrix - Citizendia

A sparse matrix obtained when solving a finite element problem in two dimensions. The finite element method (FEM (sometimes referred to as finite element analysis) is a numerical technique for finding approximate solutions of Partial differential The non-zero elements are shown in black.

In the mathematical subfield of numerical analysis a sparse matrix is a matrix populated primarily with zeros. Mathematics is the body of Knowledge and Academic discipline that studies such concepts as Quantity, Structure, Space and Numerical analysis is the study of Algorithms for the problems of continuous mathematics (as distinguished from Discrete mathematics) In Mathematics, a matrix (plural matrices) is a rectangular table of elements (or entries) which may be Numbers or more generally

Conceptually, sparsity corresponds to systems which are loosely coupled. Consider a line of balls connected by springs from one to the next; this is a sparse system. By contrast, if the same line of balls had springs connecting every ball to every other ball, the system would be represented by a dense matrix. The concept of sparsity is useful in combinatorics and application areas such as network theory, of a low density of significant data or connections. Combinatorics is a branch of Pure mathematics concerning the study of discrete (and usually finite) objects Network theory is an area of Applied mathematics and part of Graph theory.

Huge sparse matrices often appear in science or engineering when solving partial differential equations. Science (from the Latin scientia, meaning " Knowledge " or "knowing" is the effort to discover, and increase human understanding Engineering is the Discipline and Profession of applying technical and scientific Knowledge and In Mathematics, partial differential equations ( PDE) are a type of Differential equation, i

When storing and manipulating sparse matrices on a computer, it is beneficial and often necessary to use specialized algorithms and data structures that take advantage of the sparse structure of the matrix. A computer is a Machine that manipulates data according to a list of instructions. In Mathematics, Computing, Linguistics and related subjects an algorithm is a sequence of finite instructions often used for Calculation A data structure in Computer science is a way of storing Data in a computer so that it can be used efficiently Operations using standard matrix structures and algorithms are slow and consume large amounts of memory when applied to large sparse matrices. Sparse data is by nature easily compressed, and this compression almost always results in significantly less memory usage. Computer data storage, often called storage or memory, refers to Computer components devices and recording media that retain digital Indeed, some very large sparse matrices are impossible to manipulate with the standard algorithms.

1 Storing a sparse matrix
- 1.1 Example
- 1.2 Diagonal matrices
2 Bandwidth
- 2.1 Reducing bandwidth
3 Reducing fill-in
4 Solving sparse matrix equations
5 See also
6 References
7 Further reading

Storing a sparse matrix

The naive data structure for a matrix is a two-dimensional array. A data structure in Computer science is a way of storing Data in a computer so that it can be used efficiently Each entry in the array represents an element a_i,j of the matrix and can be accessed by the two indices i and j. For a m×n matrix we need at least enough memory to store (m×n) entries to represent the matrix.

Many if not most entries of a sparse matrix are zeros. The basic idea when storing sparse matrices is to store only the non-zero entries as opposed to storing all entries. Depending on the number and distribution of the non-zero entries, different data structures can be used and yield huge savings in memory when compared to a naïve approach. A data structure in Computer science is a way of storing Data in a computer so that it can be used efficiently Computer data storage, often called storage or memory, refers to Computer components devices and recording media that retain digital

One example of such a sparse matrix format is the (old) Yale Sparse Matrix Format[1]. It stores an initial sparse m×n matrix, M, in row form using three one-dimensional arrays. Let NNZ denote the number of nonzero entries of M. The first array is A, which is of length NNZ, and holds all nonzero entries of M in left-to-right top-to-bottom order. The second array is IA, which is of length $m + 1$ (i. e. , one entry per row, plus one). IA(i) contains the index in A of the first nonzero element of row i. Row i of the original matrix extends from A(IA(i)) to A(IA(i+1)-1). The third array, JA, contains the column index of each element of A, so it also is of length NNZ.

For example, the matrix

[ 1 2 0 0 ]
[ 0 3 9 0 ]
[ 0 1 4 0 ]

is a three-by-four matrix with six nonzero elements, so

A  = [ 1 2 3 9 1 4 ]
IA = [ 1 3 5 7 ]
JA = [ 1 2 2 3 2 3 ]

Another possibility is to use quadtrees. A quadtree is a Tree data structure in which each internal node has up to four children

Example

A bitmap image having only 2 colors, with one of them dominant (say a file that stores a handwritten signature) can be encoded as a sparse matrix that contains only row and column numbers for pixels with the non-dominant color. In Computer graphics, a bitmap or pixmap is a type of memory organization or Image file format used to store Digital images The A signature (from Latin signare, " Sign " is a handwritten (and sometimes stylized depiction of someone's name nickname or even a simple In Digital imaging, a pixel ( pict ure el ement is the smallest piece of information in an image

Diagonal matrices

A very efficient structure for a diagonal matrix is to store just the entries in the main diagonal as a one-dimensional array, so a diagonal n×n matrix requires only n entries. In Linear algebra, a diagonal matrix is a Square matrix in which the entries outside the Main diagonal (↘ are all zero

Bandwidth

The lower bandwidth of a matrix A is the smallest number p such that the entry a_ij vanishes whenever i > j + p. Similarly, the upper bandwidth is the smallest p such that a_ij = 0 whenever i < j − p (Golub & Van Loan 1996, §1. 2. 1). For example, a tridiagonal matrix has lower bandwidth 1 and upper bandwidth 1. In Linear algebra, a tridiagonal matrix is a matrix that is "almost" a Diagonal matrix.

Matrices with small upper and lower bandwidth are known as band matrices and often lend themselves to simpler algorithms than general sparse matrices; one can sometimes apply dense matrix algorithms and simply loop over a reduced number of indices. In Mathematics, particularly Matrix theory, a band matrix is a Sparse matrix, whose non-zero entries are confined to a diagonal band, comprising

Reducing bandwidth

The Cuthill-McKee algorithm can be used to reduce the bandwidth of a sparse symmetric matrix. In the mathematical subfield of Matrix theory the Cuthill–McKee algorithm is an Algorithm to reduce the bandwidth of sparse In Linear algebra, a symmetric matrix is a Square matrix, A, that is equal to its Transpose A = A^{T} There are, however, matrices for which the Reverse Cuthill-McKee algorithm performs better. In the mathematical subfield of Matrix theory the Cuthill–McKee algorithm is an Algorithm to reduce the bandwidth of sparse

The U.S. National Geodetic Survey (NGS) uses Dr. The National Geodetic Survey and the Office of Coast Survey are the two successor agencies in the United States to the U Richard Snay's "Banker's" algorithm because on realistic sparse matrices used in Geodesy work it has better performance.

There are many other methods in use.

Reducing fill-in

"Fill-in" redirects here. For the puzzle, see Fill-In (puzzle). Fill-Ins (also known as Fill-It-Ins) are a type of Logic puzzle commonly printed in various Puzzle magazines.

The fill-in of a matrix are those entries which change from an initial zero to a non-zero value during the execution of an algorithm. To reduce the memory requirements and the number of arithmetic operations used during an algorithm it is useful to minimize the fill-in by switching rows and columns in the matrix. The symbolic Cholesky decomposition can be used to calculate the worst possible fill-in before doing the actual Cholesky decomposition. In the mathematical subfield of Numerical analysis the symbolic Cholesky decomposition is an Algorithm used to determine the non-zero pattern for the L In Mathematics, the Cholesky decomposition is named after André-Louis Cholesky, who found that a symmetric Positive-definite matrix can be

There are other methods than the Cholesky decomposition in use. In Mathematics, the Cholesky decomposition is named after André-Louis Cholesky, who found that a symmetric Positive-definite matrix can be Orthogonalization methods (such as QR factorization) are common, for example, when solving problems by least squares methods. While the theoretical fill-in is still the same, in practical terms the "false non-zeros" can be different for different methods. And symbolic versions of those algorithms can be used in the same manner as the symbolic Cholesky to compute worst case fill-in.

Solving sparse matrix equations

Both iterative and direct methods exist for sparse matrix solving. In Computational mathematics, an iterative method attempts to solve a problem (for example an equation or system of equations by finding successive Approximations One popular iterative method is the conjugate gradient method. In Mathematics, the conjugate gradient method is an Algorithm for the Numerical solution of particular systems of linear equations, namely those

References

Tewarson, Reginald P, Sparse Matrices (Part of the Mathematics in Science & Engineering series), Academic Press Inc. A matrix representation is a method used by a Computer language to store matrices of more than one dimension in memory. The Pareto principle (also known as the 80-20 rule, the law of the vital few and the principle of factor sparsity) states that for many events 80% An irregular matrix, or ragged matrix, can be described as a matrix that has a different number of elements in each row A skyline matrix, or a variable band matrix, is a form of a Sparse matrix storage format for a square, banded (and typically symmetric In Computer science, a sparse array is an Array in which most of the elements have the same value (known as the default value -- usually 0 or null) A Sparse graph code is a Code which is represented by a Sparse graph. In Computer science, a sparse file is a type of Computer file that attempts to use File system space more efficiently when blocks allocated to the file Professor Reginald P Tewarson (Reginald Prabhakar Tewarson is the author of the book Sparse Matrices and many articles on sparse matrices, the solution of differential equations , May 1973. (This book, by a professor at the State University of New York at Stony Book, was the first book exclusively dedicated to Sparse Matrices. Graduate courses using this as a textbook were offered at that University in the early 1980s).
Sparse Matrix Multiplication Package, Randolph E. Bank, Craig C. Douglas [1]
Pissanetzky, Sergio 1984, "Sparse Matrix Technology", Academic Press
R. A. Snay. Reducing the profile of sparse symmetric matrices. Bulletin Géodésique, 50:341–352, 1976. Also NOAA Technical Memorandum NOS NGS-4, National Geodetic Survey, Rockville, MD.

Contents