R2NLS solver for unconstrained nonlinear least-squares (NLS) problems
#311
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduces the R2NLS solver, an inexact second-order quadratic regularization method for nonlinear least-squares, with support for multiple subproblem solvers including QRMumps. Updates documentation, tests, and dependencies to include R2NLS and its utilities, and adds comprehensive test coverage for the new solver and its subsolvers.
The finalizer that calls close! on QRMumpsSolver has been commented out due to crashes in libdqrm.dll during object finalization, likely caused by a double free or invalid pointer reference. Additional minor changes include switching from view to direct indexing for SparseMatrixCOO construction to avoid pointer issues, and reordering some operations in SolverCore.solve! for clarity.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #311 +/- ##
==========================================
+ Coverage 89.00% 89.40% +0.40%
==========================================
Files 7 8 +1
Lines 1200 1435 +235
==========================================
+ Hits 1068 1283 +215
- Misses 132 152 +20 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Eliminated a duplicate call to mul!(∇f, Jx', r) in the SolverCore.solve! function to avoid unnecessary computation.
tmigot
requested changes
Aug 21, 2025
Member
tmigot
left a comment
tmigot
left a comment
There was a problem hiding this comment.
Thanks @farhadrclass for the PR. I made a first batch of comments (but didn't check the new algorithm)
src/R2NLS.jl
Outdated
| ls_subsolver.irn[1:ls_subsolver.nnzj], | ||
| ls_subsolver.jcn[1:ls_subsolver.nnzj], | ||
| ls_subsolver.val[1:ls_subsolver.nnzj], | ||
| ) #For now till they fix the SparseMatrixCOO to accept pointers |
Member
There was a problem hiding this comment.
Is there an open issue for this?
Contributor
Author
There was a problem hiding this comment.
No, but @dpo and I thinking about this right now
tmigot
requested changes
Aug 21, 2025
| SolverCore = "0.3" | ||
| SolverParameters = "0.1" | ||
| SolverTools = "0.9" | ||
| SparseArrays = "1.11.0" |
Member
There was a problem hiding this comment.
Suggested change
| SparseArrays = "1.11.0" | |
| SparseArrays = "1.10.0" |
Contributor
Author
There was a problem hiding this comment.
why do we want 1.10?
Deleted utilities.jl and its tests, removing the opnorm/TSVD utilities and all related test coverage. Dropped Arpack and GenericLinearAlgebra from dependencies and compat, and updated code and tests to no longer use or include them. Fixed a typo in the CRLS subsolver dispatch in test_solvers.jl.
Contributor
Author
|
@tmigot thank you for the review, I applied most of what you recommend and asked some questions to confirm the rest |
Introduces the R2NLSParameterSet struct to encapsulate all algorithmic parameters for the R2NLS solver, replacing individual keyword arguments. Updates R2SolverNLS and R2NLS constructors and solve! logic to use the new parameter set, improving type safety, maintainability, and clarity of parameter handling.
Introduces a ShiftedOperator type for shifted linear systems, updates R2NSolver to support new subsolvers (cg, cr), and refactors subsolver dispatch logic. Adds non-positive curvature handling strategies, improves step selection, and updates logging for more detailed iteration information. Also adds a Project.toml for HSL_jll dependency and minor improvements to R2NLS.jl parameter handling.
Added comprehensive test suite for R2N and its subsolvers in my_R2N_dev/test_R2N.jl. Updated Project.toml files to include new dependencies and compat entries. Refactored R2N.jl to improve ShiftedOperator, subsolver dispatch, and negative curvature handling, and removed HSL_jll usage. Minor documentation and logic improvements throughout.
Uncommented and revised the MA97 solver test in test_R2N.jl, added HSL_jll as a dependency, and improved MA97 integration in R2N.jl. Adjusted Hessian-vector product logic to use hprod! for MA97Solver, and made minor code style and efficiency improvements throughout.
Redesigns MA97Solver to use index mappers for direct updates to the MA97 value buffer, avoiding repeated sparse matrix construction. This improves efficiency by mapping Hessian and diagonal entries directly to their positions in the CSC structure, and updates the subsolve! method accordingly. Also updates ShiftedOperator to support AbstractMatrix and improves opnorm handling for both operator and matrix cases.
Introduces a new test file for R2N error handling and updates existing test suites to include R2N and R2N_exact solvers. This ensures allocation, callback, consistency, restart, and unbounded tests now cover these solvers, improving test coverage and robustness for the R2N family.
Replaces all instances of 'subsolver_type = ShiftedLBFGSSolver' with 'subsolver= :shifted_lbfgs' in test and development files.
Replaces the MA97Solver with a generic HSLDirectSolver that can wrap both MA97 and MA57 HSL direct solvers. Updates all relevant logic, type dispatch, and subsolve! methods to support both solvers, and adds :ma57 as an allowed subsolver. This refactor simplifies the codebase and enables easier extension to other HSL direct solvers.
Introduces configurable line search parameters (ls_c, ls_increase, ls_decrease, ls_min_alpha, ls_max_alpha) to the R2N solver and parameter set. Refactors the Armijo negative curvature handling to use these parameters. Cleans up and expands test scripts, removes special handling for R2N_exact, and adds a placeholder for HSL subsolver tests.
Refactored and improved the integration of HSL direct solvers (MA97, MA57) in R2N, including more robust handling, logging, and test coverage. Enabled and updated tests for HSL solvers, improved output verbosity, and fixed minor issues in test and solver logic. Also adjusted test verbosity and temporarily disabled a problematic test include.
Refactors the main solver loop to better handle singular and indefinite systems when using HSL direct solvers (MA97/MA57), including robust inertia detection and sigma adjustment. Adds get_inertia methods for MA97 and MA57, centralizes HSL factorization/solve logic, and improves non-positive curvature handling. Also adds an example in test_HSL.jl for using MA97 and MA57 with singular systems.
Commented out unused test code in test_R2N.jl and adjusted test parameters for max_iter and verbose levels. Improved npcCount initialization and reset logic in R2N.jl, and added comments regarding HSL solver flag handling. These changes help clarify test output and address issues with negative curvature count tracking.
Introduces a new test file 'test_hsl_subsolver.jl' to verify the functionality of HSL subsolvers (ma97 and ma57, with and without Armijo handler) using a simple nonlinear problem. The new test is included in 'runtests.jl' and will be skipped if LIBHSL is not functional.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces the new
R2NLSsolver for unconstrained nonlinear least-squares (NLS) problems, expands documentation and test coverage for this solver.Major features and solver additions:
R2NLSsolver, an inexact second-order quadratic regularization method for unconstrained NLS, to the codebase and exported it for use. (src/JSOSolvers.jl,src/R2NLS.jl,README.md,docs/src/solvers.md) [1] [2] [3] [4]R2NLSas an available solver for unconstrained NLS problems. (docs/src/solvers.md,README.md) [1] [2] [3]Utilities and infrastructure:
opnormutility function inutilities.jlthat computes operator norms for both standard numeric and arbitrary element types, using LAPACK, ARPACK, or TSVD as appropriate. This is now included and exported in the main module. (src/utilities.jl,src/JSOSolvers.jl) [1] [2]opnormutility, covering multiple numeric types and matrix shapes. (test/test_Utilities.jl)Testing enhancements:
R2NLSand its subsolver variants in all relevant test suites, including allocation, callback, consistency, restart, and general solver tests. (test/allocs.jl,test/callback.jl,test/consistency.jl,test/restart.jl,test/runtests.jl,test/test_solvers.jl) [1] [2] [3] [4] [5] [6] [7]Dependency and import updates:
Arpack,SparseArrays, andKrylovin both the main module and test files to support the new solver and utilities. (src/JSOSolvers.jl,test/runtests.jl) [1] [2]Note: