LinearAlgebra Missing Support

[Nucs]

Overview

NumSharp implements only a small corner of NumPy's linear algebra surface. The
matrix/vector products exist, but the entire LAPACK-backed core of numpy.linalg
— decompositions, eigenvalues, equation-solving, matrix inversion, determinants, and
norms — is absent, and there is no np.linalg namespace at all. This issue tracks the
gap and proposes building np.linalg out.

Problem

NumPy's "linear algebra" is really two surfaces:

• numpy.linalg — 31 functions (+ the LinAlgError exception), almost all
  backed by LAPACK/BLAS (numpy/linalg/umath_linalg.cpp).
• Matrix/vector products in the main np.* namespace — dot, matmul, inner,
  outer, vdot, tensordot, einsum, kron, cross (NumPy 2.x re-exports several
  of these inside np.linalg for Array-API conformance).

NumSharp currently ships ~7 of ~40 of these routines and none of the
LAPACK-heavy core. Porting real ML / scientific code stalls immediately on norm,
inv, solve, lstsq, eig, svd, det — the most common calls — and there is no
np.linalg namespace for them to live in.

Background: the former NDArray.inv / qr / svd / lstqr / multi_dot methods were
non-functional stubs (each return null; / return default;), including a misspelled
lstqr (NumPy's is lstsq, returning a 4-tuple) and a wrong-shape multi_dot (NumPy
takes a sequence). They were mis-homed on NDArray (in NumPy these are np.linalg
module functions, never ndarray methods) and have been removed rather than left
as silent null-returning traps. This issue tracks implementing them correctly under
np.linalg.

Current coverage (final state)

Grouped by NumPy's own routines.linalg categories:

NumPy category	Implemented in NumSharp	Missing
Matrix & vector products	dot, matmul, outer	inner, vdot, vecdot, tensordot, einsum/einsum_path, kron, cross, linalg.multi_dot
Decompositions	—	cholesky, qr, svd, svdvals
Matrix eigenvalues	—	eig, eigh, eigvals, eigvalsh
Norms & other numbers	trace, diagonal	norm, matrix_norm, vector_norm, det, slogdet, cond, matrix_rank
Solving & inverting	—	solve, lstsq, inv, pinv, tensorsolve, tensorinv
Matrix functions / transpose	matrix_power, matrix_transpose, .T/transpose/swapaxes	—

Bottom line: every LAPACK-backed routine (all decompositions, all eigenvalues, all
solve/inverse/det, and all norms) is missing. What exists is only what can be expressed
with the existing elementwise/reduction/matmul kernels.

Proposal

1. Introduce a np.linalg namespace mirroring numpy.linalg (np.linalg.inv(a),
   np.linalg.svd(a), …), instead of bolting functions onto NDArray. Add the
   LinAlgError type (NumPy raises it for singular matrices, non-convergence, etc.).
2. Choose a backend for the LAPACK-heavy core — hand-writing SVD/QR/eig is a poor
   use of effort. Candidates (decide before implementing):
   • MathNet.Numerics — fully managed, easy, portable; slower.
   • OpenBLAS / LAPACKE via P/Invoke — native, fastest, matches NumPy; adds a native
     dependency + per-RID packaging.
3. Implement in dependency order: products (pure kernels) → norms/det (reductions +
   an LU) → decompositions (qr/svd/cholesky) → eigenvalues → solve/inverse/lstsq/pinv
   (built on the decompositions).

Task checklist

Infra

• Create the np.linalg static namespace + LinAlgError
• Select and wire the LAPACK backend (MathNet vs OpenBLAS P/Invoke)
• Alias existing dot/matmul/outer/trace/diagonal/matrix_power/matrix_transpose into np.linalg where NumPy 2.x also exposes them

Products & contractions (main np.* namespace)

• inner (Implement numpy.inner #106)
• vdot (Implement numpy.vdot #105)
• vecdot
• tensordot (Implement numpy.tensordot #108)
• kron
• cross
• einsum / einsum_path (large — may split into its own issue)
• linalg.multi_dot

Decompositions

• linalg.cholesky
• linalg.qr
• linalg.svd
• linalg.svdvals

Eigenvalues

• linalg.eig
• linalg.eigh
• linalg.eigvals
• linalg.eigvalsh

Norms & other numbers

• linalg.norm (np.linalg.norm #239, No numpy.linalg.norm() ? #441, np.linalg.norm not support #485)
• linalg.matrix_norm
• linalg.vector_norm
• linalg.det
• linalg.slogdet
• linalg.cond
• linalg.matrix_rank (How to calculate the rank of a matrix with NumSharp? #472)

Solving & inverting

• linalg.solve
• linalg.lstsq (NDArray.lstqr() doesn't work #454)
• linalg.inv (How to get the inverse of a 2D matrix？ #438)
• linalg.pinv (np.linalg.pinv not supported #497)
• linalg.tensorsolve
• linalg.tensorinv

Evidence

• numpy.linalg.__all__ = 31 functions + LinAlgError (verified against NumPy 2.4.2
  numpy/linalg/_linalg.py): matrix_power, solve, tensorsolve, tensorinv, inv, cholesky, eigvals, eigvalsh, pinv, slogdet, det, svd, svdvals, eig, eigh, lstsq, norm, qr, cond, matrix_rank, multi_dot, trace, diagonal, cross, outer, tensordot, matmul, matrix_transpose, matrix_norm, vector_norm, vecdot.
• These are LAPACK-backed in NumPy (umath_linalg.cpp), which is why they were never
  hand-ported to pure C#.
• Several standing issues already request individual pieces (see Related), showing real
  user demand.
• Real-world usage ranking (GitHub code search) — each missing routine was counted
  across public GitHub code to prioritize by demand (full table below). np.linalg.norm
  dominates by ~4×, and the top tier matches the "most common calls" named under
  Problem.

Usage across GitHub (prioritization signal)

Each missing routine was counted via GitHub REST code search
(gh api search/code → total_count) in its canonical call form — np.<fn>( for the
main-namespace products, np.linalg.<fn>( for the linalg routines. The trailing (
counts real call sites and disambiguates prefixes (otherwise np.linalg.eig also swallows
eigh/eigvals/eigvalsh, svd swallows svdvals, inv swallows pinv).

np.linalg.norm dominates — ~305K hits, ~4× the runner-up (inv) and more than the
entire bottom half of the list combined. The top tier — norm → inv → cross → einsum →
det → solve → svd — lines up with the "most common calls" named under Problem and
corroborates the standing norm requests (#239 / #441 / #485). This argues for an
impact-ordered build: norm first, then inv / det / solve / svd.

#	Function	Approx. GitHub hits
1	np.linalg.norm	304,640
2	np.linalg.inv	78,720
3	np.cross	56,736
4	np.einsum	35,456
5	np.linalg.det	29,584
6	np.linalg.solve	27,808
7	np.linalg.svd	25,312
8	np.linalg.lstsq	22,192
9	np.linalg.pinv	19,792
10	np.linalg.eig	17,936
11	np.kron	17,488
12	np.linalg.eigh	15,456
13	np.linalg.matrix_rank	10,680
14	np.linalg.matrix_norm	10,328
15	np.linalg.cholesky	10,160
16	np.linalg.eigvalsh	10,128
17	np.linalg.vector_norm	9,160
18	np.linalg.eigvals	8,288
19	np.tensordot	7,712
20	np.inner	7,688
21	np.linalg.qr	7,008
22	np.vdot	5,888
23	np.linalg.cond	4,800
24	np.linalg.multi_dot	4,720
25	np.linalg.slogdet	4,528
26	np.vecdot	2,436
27	np.linalg.tensorinv	1,672
28	np.linalg.tensorsolve	1,656
29	np.einsum_path	1,564
30	np.linalg.svdvals	1,392

_{Method: GitHub REST code-search total_count is an approximate estimate (noisy
±~10% on large result sets; re-queries drift) and counts only the dominant np. alias —
not numpy.linalg.… or from numpy.linalg import …. Those add roughly constant multipliers,
so the ranking is stable even though the absolute numbers are not exact. Measured 2026-07-19.}

Scope / Non-goals

• Not reintroducing the deleted null-returning stubs.
• FFT (numpy.fft) is out of scope — separate module (Implement numpy.fft.fft #114).
• einsum may be split into its own issue given its size/complexity.
• This issue is about API existence + NumPy-correct behavior; SIMD/perf tuning of the
  new routines is follow-up work.

Related issues

• How to get the inverse of a 2D matrix？ #438 inv · NDArray.lstqr() doesn't work #454 lstsq · np.linalg.pinv not supported #497 pinv · How to calculate the rank of a matrix with NumSharp? #472 matrix_rank · np.linalg.norm not support #485 / No numpy.linalg.norm() ? #441 / np.linalg.norm #239 norm
• Implement numpy.tensordot #108 tensordot · Implement numpy.inner #106 inner · Implement numpy.vdot #105 vdot
• Performance on np.matmul #427 matmul performance (related — existing op, not missing)

[Nucs]

Checked off what computes today, verified in src/NumSharp.Core (as of 297f883f):

• np.linalg namespace + LinAlgError — LinearAlgebra/linalg/np.linalg.cs, Exceptions/LinAlgError.cs.
• Array-API aliases — linalg.matmul, matrix_transpose, outer, trace, diagonal, matrix_power (np.linalg.arrayapi.cs, np.linalg.matrix_power.cs).
• Products (managed kernels, no backend needed): inner, vdot, vecdot, tensordot, linalg.multi_dot.
• Norm family: linalg.norm / matrix_norm / vector_norm compute for all standard orders. ⚠️ The matrix 2-/-2/nuclear orders throw NotSupportedException because they require SVD (np.linalg.norm.cs:210).

Left unchecked — not yet computing:

• All 18 LAPACK factorizations — cholesky, qr, svd, svdvals, eig, eigh, eigvals, eigvalsh, det, slogdet, cond, matrix_rank, solve, lstsq, inv, pinv, tensorsolve, tensorinv. The public API, NumPy-exact validation and verbatim error contract exist, but every one throws NotSupportedException via NoLapack(...) (Backends/TensorEngine.LinearAlgebra.cs). Core ships no managed LU/QR/SVD/eigensolver, and no shipping backend implements the IBlasBackend.Try* factorization members yet.
• LAPACK backend wiring — the seam exists (IBlasBackend.LinearAlgebra.cs: 15 Try* defaults + TensorEngine.Blas), and NumSharp.Interop.BLAS (OpenBLAS) delivers the products, but no shipping backend delivers the factorizations — so this stays open.
• kron — missing.
• cross — only np.linalg.cross (array-API form) exists; top-level np.cross is absent.
• einsum / einsum_path — subscripts are fully parsed & validated and the output shape resolved, but the contraction kernel is not implemented (throws NotSupportedException); einsum_path is absent.