Applications of Mathematics, Vol. 67, No. 2, pp. 233-250, 2022
A new nonmonotone adaptive trust region algorithm
Ahmad Kamandi, Keyvan Amini
Received April 29, 2020. Published online April 30, 2021.
Abstract: We propose a new and efficient nonmonotone adaptive trust region algorithm to solve unconstrained optimization problems. This algorithm incorporates two novelties: it benefits from a radius dependent shrinkage parameter for adjusting the trust region radius that avoids undesirable directions and exploits a new strategy to prevent sudden increments of objective function values in nonmonotone trust region techniques. Global convergence of this algorithm is investigated under some mild conditions. Numerical experiments demonstrate the efficiency and robustness of the proposed algorithm in solving a collection of unconstrained optimization problems from the CUTEst package.
References: [1] M. Ahookhosh, K. Amini: A nonmonotone trust region method with adaptive radius for unconstrained optimization problems. Comput. Math. Appl. 60 (2010), 411-422. DOI 10.1016/j.camwa.2010.04.034 | MR 2665646 | Zbl 1201.90184
[2] M. Ahookhosh, K. Amini, M. R. Peyghami: A nonmonotone trust-region line search method for large-scale unconstrained optimization. Appl. Math. Modelling 36 (2012), 478-487. DOI 10.1016/j.apm.2011.07.021 | MR 2835025 | Zbl 1236.90077
[3] R. Ayanzadeh, S. Mousavi, M. Halem, T. Finin: Quantum annealing based binary compressive sensing with matrix uncertainty. Available at https://arxiv.org/abs/1901.00088 (2019), 15 pages.
[4] R. Chen, M. Menickelly, K. Scheinberg: Stochastic optimization using a trust-region method and random models. Math. Program. 169 (2018), 447-487. DOI 10.1007/s10107-017-1141-8 | MR 3800867 | Zbl 1401.90136
[5] A. R. Conn, N. I. M. Gould, P. L. Toint: Trust Region Methods. MPS/SIAM Series on Optimization 1. SIAM, Philadelphia (2000). DOI 10.1137/1.9780898719857 | MR 1774899 | Zbl 0958.65071
[6] N. Y. Deng, Y. Xiao, F. J. Zhou: Nonmonotonic trust region algorithm. J. Optim. Theory Appl. 76 (1993), 259-285. DOI 10.1007/BF00939608 | MR 1203903 | Zbl 0797.90088
[7] E. D. Dolan, J. J. Moré: Benchmarking optimization software with performance profiles. Math. Program. 91 (2002), 201-213. DOI 10.1007/s101070100263 | MR 1875515 | Zbl 1049.90004
[8] H. Esmaeili, M. Kimiaei: A trust-region method with improved adaptive radius for systems of nonlinear equations. Math. Methods Oper. Res. 83 (2016), 109-125. DOI 10.1007/s00186-015-0522-0 | MR 3464191 | Zbl 1333.90126
[9] N. I. M. Gould, S. Lucidi, M. Roma, P. L. Toint: Solving the trust-region subproblem using the Lanczos method. SIAM J. Optim. 9 (1999), 504-525. DOI 10.1137/S1052623497322735 | MR 1686795 | Zbl 1047.90510
[10] N. I. M. Gould, D. Orban, P. L. Toint: CUTEst: A constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60 (2015), 545-557. DOI 10.1007/s10589-014-9687-3 | MR 3320935 | Zbl 1325.90004
[11] L. Grippo, F. Lampariello, S. Lucidi: A nonmonotone line search technique for Newton's method. SIAM J. Numer. Anal. 23 (1986), 707-716. DOI 10.1137/0723046 | MR 0849278 | Zbl 0616.65067
[12] M. Hong, M. Razaviyayn, Z. Q. Luo, J.-S. Pang: A unified algorithmic framework for block-structured optimization involving big data: With applications in machine learning and signal processing. IEEE Signal Processing Magazine 33 (2016), 57-77. DOI 10.1109/MSP.2015.2481563
[13] A. Kamandi, K. Amini, M. Ahookhosh: An improved adaptive trust-region algorithm. Optim. Lett. 11 (2017), 555-569. DOI 10.1007/s11590-016-1018-4 | MR 3610242 | Zbl 1367.90102
[14] J. J. Moré, D. C. Sorensen: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4 (1983), 553-572. DOI 10.1137/0904038 | MR 0723110 | Zbl 0551.65042
[15] J. Nocedal, S. J. Wright: Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer, New York (2006). MR 2244940 | Zbl 1104.65059
[16] M. R. Peyghami, D. A. Tarzanagh: A relaxed nonmonotone adaptive trust region method for solving unconstrained optimization problems. Comput. Optim. Appl. 61 (2015), 321-341. DOI 10.1007/s10589-015-9726-8 | MR 3349838 | Zbl 1326.90081
[17] R. B. Schnabel, E. Eskow: A new modified Cholesky factorization. SIAM J. Sci. Stat. Comput. 11 (1990), 1136-1158. DOI 10.1137/0911064 | MR 1068501 | Zbl 0716.65023
[18] J. Shen, S. Mousavi: Least sparsity of $p$-norm based optimization problems with $p>1$. SIAM J. Optim. 28 (2018), 2721-2751. DOI 10.1137/17M1140066 | MR 3858810 | Zbl 06951736
[19] Z.-J. Shi, J. Guo: A new trust region method with adaptive radius. Comput. Optim. Appl. 41 (2008), 225-242. DOI 10.1007/s10589-007-9099-8 | MR 2447894 | Zbl 1216.90086
[20] Z. Shi, S. Wang: Nonmonotone adaptive trust region method. Eur. J. Oper. Res. 208 (2011), 28-36. DOI 10.1016/j.ejor.2010.09.007 | MR 2737860 | Zbl 1229.90206
[21] T. Steihaug: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20 (1983), 626-637. DOI 10.1137/0720042 | MR 0701102 | Zbl 0518.65042
[22] Y. Xue, H. Liu, Z. Liu: An improved nonmonotone adaptive trust region method. Appl. Math., Praha 64 (2019), 335-350. DOI 10.21136/AM.2019.0138-18 | MR 3956176 | Zbl 07088744
[23] X. Zhang, J. Zhang, L. Liao: An adaptive trust region method and its convergence. Sci. China, Ser. A 45 (2002), 620-631. MR 1911178 | Zbl 1105.90361
[24] Q. Zhou, D. Hang: Nonmonotone adaptive trust region method with line search based on new diagonal updating. Appl. Numer. Math. 91 (2015), 75-88. DOI 10.1016/j.apnum.2014.12.009 | MR 3312325 | Zbl 1310.65070
Affiliations: Ahmad Kamandi (corresponding author), Department of Mathematics, University of Science and Technology of Mazandaran, P.O. Box 48518-78195, Behshahr, Iran, e-mail: ahmadkamandi@mazust.ac.ir; Keyvan Amini, Department of Mathematics, Faculty of Science, Razi University, P.O. Box 67141-15111, Kermanshah, Iran, e-mail: kamini@razi.ac.ir
