On the Asymptotic Convergence and Acceleration of Gradient Methods | |
Huang, Yakui1; Dai, Yu-Hong3,4; Liu, Xin-Wei1; Zhang, Hongchao2 | |
刊名 | JOURNAL OF SCIENTIFIC COMPUTING |
2022 | |
卷号 | 90期号:1页码:29 |
关键词 | Gradient methods Asymptotic convergence Spectral property Acceleration of gradient methods Barzilai-Borwein method Unconstrained optimization Quadratic optimization |
ISSN号 | 0885-7474 |
DOI | 10.1007/s10915-021-01685-8 |
英文摘要 | We consider the asymptotic behavior of a family of gradient methods, which include the steepest descent and minimal gradient methods as special instances. It is proved that each method in the family will asymptotically zigzag between two directions. Asymptotic convergence results of the objective value, gradient norm, and stepsize are presented as well. To accelerate the family of gradient methods, we further exploit spectral properties of stepsizes to break the zigzagging pattern. In particular, a new stepsize is derived by imposing finite termination on minimizing two-dimensional strictly convex quadratic function. It is shown that, for the general quadratic function, the proposed stepsize asymptotically converges to the reciprocal of the largest eigenvalue of the Hessian. Furthermore, based on this spectral property, we propose a periodic gradient method by incorporating the Barzilai-Borwein method. Numerical comparisons with some recent successful gradient methods show that our new method is very promising. |
资助项目 | National Natural Science Foundation of China[11701137] ; National Natural Science Foundation of China[11631013] ; National Natural Science Foundation of China[12071108] ; National Natural Science Foundation of China[11671116] ; National Natural Science Foundation of China[11991021] ; National Natural Science Foundation of China[12021001] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA27000000] ; Beijing Academy of Artificial Intelligence (BAAI) ; Natural Science Foundation of Hebei Province[A2021202010] ; China Scholarship Council[201806705007] ; USA National Science Foundation[DMS-1819161] ; USA National Science Foundation[DMS-2110722] |
WOS研究方向 | Mathematics |
语种 | 英语 |
出版者 | SPRINGER/PLENUM PUBLISHERS |
WOS记录号 | WOS:000720653400002 |
内容类型 | 期刊论文 |
源URL | [http://ir.amss.ac.cn/handle/2S8OKBNM/59571] |
专题 | 中国科学院数学与系统科学研究院 |
通讯作者 | Zhang, Hongchao |
作者单位 | 1.Hebei Univ Technol, Inst Math, Tianjin 300401, Peoples R China 2.Louisiana State Univ, Dept Math, Baton Rouge, LA 70803 USA 3.Univ Chinese Acad Sci, Sch Math Sci, Beijing 100049, Peoples R China 4.Chinese Acad Sci, Acad Math & Syst Sci, ICMSEC, LSEC, Beijing 100190, Peoples R China |
推荐引用方式 GB/T 7714 | Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,et al. On the Asymptotic Convergence and Acceleration of Gradient Methods[J]. JOURNAL OF SCIENTIFIC COMPUTING,2022,90(1):29. |
APA | Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,&Zhang, Hongchao.(2022).On the Asymptotic Convergence and Acceleration of Gradient Methods.JOURNAL OF SCIENTIFIC COMPUTING,90(1),29. |
MLA | Huang, Yakui,et al."On the Asymptotic Convergence and Acceleration of Gradient Methods".JOURNAL OF SCIENTIFIC COMPUTING 90.1(2022):29. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论