You solve a system of linear equations with matrices by writing it as Ax = b, then finding the vector x that satisfies every equation at once. For square systems you can use Gaussian elimination, which reduces A to row echelon form using three legal row operations. For a single right-hand side, elimination costs about 2n³/3 floating-point operations on an n-by-n matrix. For repeated solves with the same A, factor once and reuse. And when A is square and nonsingular, the inverse x = A⁻¹b is mathematically correct but computationally wasteful. The sparse matrix and matrix product capabilities in MATLAB are documented in MathWorks' own reference material (MathWorks, MATLAB documentation).
Mechanism: elimination as row operations that preserve the solution set
A linear system is a set of equations sharing the same unknowns. Matrix notation compresses that set into one object, but the rules for solving it do not change. Three operations leave the solution set untouched: swap two rows, multiply a row by a nonzero scalar, and add a multiple of one row to another. Gaussian elimination applies these to drive the matrix toward row echelon form. At that point back substitution recovers the unknowns from the bottom row upward. Partial pivoting is standard practice, choosing the largest available entry in each column as the pivot, because it keeps the process numerically stable.
Why matrix form works
Each row of Ax = b is one equation, and each column of A corresponds to one unknown. Solving the system means finding the intersection of the hyperplanes those rows define. Elimination changes the equations but not the intersection. LU factorization, where A = LU with L lower triangular and U upper triangular, takes this further: each solve becomes a forward substitution followed by a back substitution, both O(n²). Artificial intelligence has been applied in parts of this work, including automated discovery of matrix multiplication strategies (IEEE Spectrum, artificial intelligence for matrix multiplication). Critically, elimination is not guaranteed to finish cleanly. It fails when a zero pivot appears, signaling either a singular matrix or a dependent set of equations.
Quantified context: costs, thresholds, and what the numbers mean
The cost of Gaussian elimination scales as 2n³/3 operations, compared with n² for evaluating a defined formula. For a 10-by-10 system, elimination needs roughly 667 operations. For a 10,000-by-10,000 dense system, that climbs into the trillions. Sparse matrices change the calculus: an identity matrix of the same size solves instantly, since elimination only touches nonzero entries. This is why sparse solvers dominate large-scale engineering problems—they exploit zeros to skip most work.
Two thresholds deserve attention. First, a matrix is judged singular in practice when its pivot magnitude or condition number implies unreliable division; dense double-precision arithmetic usually carries about 16 significant decimal digits of precision. Second, the costs above assume dense storage and direct methods. Iterative solvers can beat elimination when A is sparse and well-conditioned, and they are often the only tractable choice for very large systems.
Comparison table: methods at a glance
| Method | Best for | Typical cost | Main caveat |
|---|---|---|---|
| Gaussian elimination with partial pivoting | Single square system, moderate size | ~2n³/3 operations | Fails on singular or nearly dependent rows |
| LU factorization | Many solves with the same matrix | ~2n³/3 once, then O(n²) per solve | Requires updating factors if A changes |
| Matrix inverse A⁻¹ | Theory and small, stable matrices | Higher overhead than elimination | Numerically unstable and wasteful in practice |
| Cramer's rule | 2×2 and classroom examples | Grows roughly factorially | Impractical beyond very small n |
| Iterative solvers (Gauss–Seidel, conjugate gradient) | Large, sparse, well-conditioned systems | Low per iteration | May not converge on some matrices |
Edge cases: singular systems, underdetermined systems, and ill-conditioning
Systems do not always have exactly one solution. If A is singular, the system has either no solution or infinitely many. Infinitely many solutions occur when the equations contain redundancy—one row a multiple of another, for instance. No solution occurs when the equations contradict each other.
Rectangular systems (more unknowns than equations, or more equations than unknowns) require least-squares fitting rather than direct inversion, and they always have a well-defined "best approximate" answer under standard conditions. The dangerous middle ground is the ill-conditioned matrix: it is technically invertible, but small errors in b produce large errors in x. Someone who inverts a nearly singular matrix may report a confident-looking solution that carries no real accuracy.
Misconceptions: what matrix solving does not promise
Common errors cluster around a few assumptions:
Assuming a non-symmetric matrix can always be inverted safely; symmetry does not guarantee stability. Assuming more equations than unknowns mean a unique solution; the count alone says nothing. Assuming decimal precision is exact; floating-point arithmetic introduces rounding at every step. Assuming Cramer's rule is fine for all sizes; at 5×5 it produces ten determinants, at 20×20 roughly 10¹⁸. Assuming x = A⁻¹b is the preferred computation; elimination and factorization are faster and stabler.
A frequent classroom claim says "inverse to solve." Textbooks are usually careful to say A⁻¹b is exact only in exact arithmetic. If A is singular, they add, no inverse exists. Careless reading then turns this into a rule for all cases.
A worked illustration of pivoting and conditioning
Consider a 2×2 system whose second row is nearly the first row scaled. Partial pivoting reorganizes rows to keep division safe, but when rows are nearly dependent, no row order fixes the problem. The condition number of A grows with the near-dependence, and each extra digit of error in b costs roughly one digit of accuracy in x. In exact arithmetic, elimination preserves the truth; in floating point, the answer's reliability comes from the pivot sizes the pivot search exposes and the accuracy the arithmetic supports.
Practical actions: how to actually solve a system
Before computing, run a short checklist. The steps below assume you have A and b and want x.
Write out Ax = b and check that dimensions match: a rows-vs-columns mismatch is the first failure mode. Test A for obvious dependence, such as duplicated rows or a row proportional to another, since this predicts the singular case. Choose the right routine: LU factorization with partial pivoting for repeated right-hand sides; iterative methods for large sparse A. On dense problems, use a library's solve function rather than forming the inverse explicitly. Estimate conditioning with a condition-number estimate, and flag results whose estimated error exceeds your tolerance. Where the system is rectangular, switch to least squares rather than forcing direct elimination.
For step 4, a threshold helps: if the reciprocal condition number falls below the floating-point epsilon, treat the result as singular. For iterative methods, set a convergence tolerance and a maximum iteration count up front.
Limits
The methods above assume arithmetic has enough precision and x is uniquely determined. When some unknown coefficients are uncertain, the conditioning of the system, not the algorithm's arithmetic, determines accuracy. Somewhat counterintuitively, Gaussian elimination without any pivot search can misreport toggle-like subtleties: it may run to completion and still yield a singular, non-unique system without any visible arithmetic error. Numerical routines that return a warning should therefore be trusted, and results worth relying on should be validated—for example, by substituting the computed vector back into every equation.
Software practice reinforces this: the general matrix solve functions in major numerical libraries report conditioning rather than assume it, and the matrix algebra recommendations in vendors' numerical-methods references also prefer factorization to inversion. Checking a computed x against Ax≈b is more valuable than trusting any promised digit count. When symptoms of failure appear, fall back to a reliable library routine.