Applications of Mathematics, Vol. 62, No. 2, pp. 121-134, 2017
New quasi-Newton method for solving systems of nonlinear equations
Ladislav Lukšan, Jan Vlček
Received September 13, 2016. First published March 3, 2017.
Abstract: We propose a new Broyden method for solving systems of nonlinear equations, which uses the first derivatives, but is more efficient than the Newton method (measured by the computational time) for larger dense systems. The new method updates QR or LU decompositions of nonsymmetric approximations of the Jacobian matrix, so it requires $O(n^2)$ arithmetic operations per iteration in contrast with the Newton method, which requires $O(n^3)$ operations per iteration. Computational experiments confirm the high efficiency of the new method.
Keywords: nonlinear equation; system of equations; trust-region method; quasi-Newton method; adjoint Broyden method; numerical algorithm; numerical experiment
References: [1] J. M. Bennett: Triangular factors of modified matrices. Numer. Math. 7 (1965), 217-221. DOI 10.1007/BF01436076 | MR 0177503 | Zbl 0132.36204 [2] C. G. Broyden: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19 (1965), 577-593. DOI 10.2307/2003941 | MR 0198670 | Zbl 0131.13905 [3] A. R. Conn, N. I. M. Gould, P. L. Toint: Trust-Region Methods. MPS/SIAM Series on Optimization 1, Society for Industrial and Applied Mathematics, Philadelphia; Mathematical Programming Society, Philadelphia (2000). DOI 10.1137/1.9780898719857 | MR 1774899 | Zbl 0958.65071 [4] J. E. Dennis, Jr., R. B. Schnabel: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Classics in Applied Mathematics 16, Society for Industrial and Applied Mathematics, Philadelphia (1996). DOI 10.1137/1.9781611971200 | MR 1376139 | Zbl 0847.65038 [5] D. M. Gay: Some convergence properties of Broyden's method. SIAM J. Numer. Anal. 16 (1979), 623-630. DOI 10.1137/0716047 | MR 0537276 | Zbl 0453.65034 [6] A. Griewank, A. Walther: Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation. Society for Industrial and Applied Mathematics, Philadelphia (2008). DOI 10.1137/1.9780898717761 | MR 2454953 | Zbl 1159.65026 [7] C. M. Ip, M. J. Todd: Optimal conditioning and convergence in rank one quasi-Newton updates. SIAM J. Numer. Anal. 25 (1988), 206-221. DOI 10.1137/0725015 | MR 0923935 | Zbl 0638.65041 [8] L. Lukšan: Inexact trust region method for large sparse systems of nonlinear equations. J. Optimization Theory Appl. 81 (1994), 569-590. DOI 10.1007/BF02193101 | MR 1281739 | Zbl 0803.65071 [9] L. Lukšan, M. Tůma, J. Vlček, N. Ramešová, M. Šiška, J, Hartman, C. Matonoha: UFO 2014. Interactive System for Universal Functional Optimization. Technical Report V-1218. Prague, Institute of Computer Science AS CR, Praha (2014). [10] L. Lukšan, J. Vlček: Truncated trust region methods based on preconditioned iterative subalgorithms for large sparse systems of nonlinear equations. J. Optimization Theory Appl. 95 (1997), 637-658. DOI 10.1023/A:1022678023392 | MR 1600609 | Zbl 0902.90146 [11] L. Lukšan, J. Vlček: Computational experience with globally convergent descent methods for large sparse systems of nonlinear equations. Optim. Methods Softw. 8 (1998), 201-223. DOI 10.1080/10556789808805677 | MR 1623249 | Zbl 0927.65068 [12] M. J. D. Powell: A new algorithm for unconstrained optimization. Nonlinear Programming, Proc. Sympos., Univ. of Wisconsin, Madison (J. B. Rosen, O. L. Mangasarian, K. Ritter, eds.) Academic Press, London (1970), 31-65. DOI 10.1016/b978-0-12-597050-1.50006-3 | MR 0272162 | Zbl 0228.90043 [13] M. J. D. Powell: On the global convergence of trust region algorithms for unconstrained minimization. Math. Program. 29 (1984), 297-303. DOI 10.1007/BF02591998 | MR 0753758 | Zbl 0569.90069 [14] S. Schlenkrich, A. Griewank, A. Walther: On the local convergence of adjoint Broyden methods. Math. Program. 121 (2010), 221-247. DOI 10.1007/s10107-008-0232-y | MR 2524890 | Zbl 1185.90207 [15] S. Schlenkrich, A. Walther: Global convergence of quasi-Newton methods based on adjoint Broyden updates. Appl. Numer. Math. 59 (2009), 1120-1136. DOI 10.1016/j.apnum.2008.05.007 | MR 2495142 | Zbl 1163.65025 [16] Y.-X. Yuan: Recent advances in numerical methods for nonlinear equations and nonlinear least squares. Numer. Algebra Control Optim. 1 (2011), 15-34. DOI 10.3934/naco.2011.1.15 | MR 2806290 | Zbl 1226.65045 [17] Y.-X. Yuan: Recent advances in trust region algorithms. Math. Program. 151 (2015), 249-281. DOI 10.1007/s10107-015-0893-2 | MR 3347556 | Zbl 1317.65141
Affiliations: Ladislav Lukšan, Jan Vlček, Institute of Computer Science, Czech Academy of Sciences, Pod Vodárenskou věží 2, 182 07 Praha 8, Czech Republic, e-mail: luksan@cs.cas.cz, vlcek@cs.cas.cz
