Sparse Learning with Stochastic Composite Optimization | |
Zhang, Weizhong1; Zhang, Lijun2; Jin, Zhongming1; Jin, Rong3; Cai, Deng1; Li, Xuelong4; Liang, Ronghua5; He, Xiaofei1 | |
刊名 | ieee transactions on pattern analysis and machine intelligence
![]() |
2017-06-01 | |
卷号 | 39期号:6页码:1223-1236 |
关键词 | Sparse learning stochastic optimization stochastic composite optimization |
ISSN号 | 0162-8828 |
产权排序 | 4 |
英文摘要 | in this paper, we study stochastic composite optimization (sco) for sparse learning that aims to learn a sparse solution from a composite function. most of the recent sco algorithms have already reached the optimal expected convergence rate o(1/lambda t), but they often fail to deliver sparse solutions at the end either due to the limited sparsity regularization during stochastic optimization (so) or due to the limitation in online-to-batch conversion. even when the objective function is strongly convex, their high probability bounds can only attain o(root log(1/delta)/t with delta is the failure probability, which is much worse than the expected convergence rate. to address these limitations, we propose a simple yet effective two-phase stochastic composite optimization scheme by adding a novel powerful sparse online-to-batch conversion to the general stochastic optimization algorithms. we further develop three concrete algorithms, optimalsl, lastsl and averagesl, directly under our scheme to prove the effectiveness of the proposed scheme. both the theoretical analysis and the experiment results show that our methods can really outperform the existing methods at the ability of sparse learning and at the meantime we can improve the high probability bound to approximately o(log (log (t)/delta)/lambda t). |
WOS标题词 | science & technology ; technology |
类目[WOS] | computer science, artificial intelligence ; engineering, electrical & electronic |
研究领域[WOS] | computer science ; engineering |
关键词[WOS] | online ; algorithms ; recovery ; gradient |
收录类别 | SCI ; EI |
语种 | 英语 |
WOS记录号 | WOS:000401091200013 |
内容类型 | 期刊论文 |
源URL | [http://ir.opt.ac.cn/handle/181661/28919] ![]() |
专题 | 西安光学精密机械研究所_光学影像学习与分析中心 |
作者单位 | 1.Zhejiang Univ, Coll Comp Sci, State Key Lab CAD&CG, 388 Yuhang Tang Rd, Hangzhou 310058, Zhejiang, Peoples R China 2.Nanjing Univ, Natl Key Lab Novel Software Technol, Nanjing 210023, Jiangsu, Peoples R China 3.Alibaba Grp, Seattle, WA 98057 USA 4.Chinese Acad Sci, State Key Lab Transicent Opt & Photon, Xian Inst Opt & Precis Mech, Ctr OPT IMagery Anal & Learning OPTIMAL, Xian 710119, Shaanxi, Peoples R China 5.Zhejiang Univ Technol, Coll Informat Engn, 288 Liuhe Rd, Hangzhou 310058, Zhejiang, Peoples R China |
推荐引用方式 GB/T 7714 | Zhang, Weizhong,Zhang, Lijun,Jin, Zhongming,et al. Sparse Learning with Stochastic Composite Optimization[J]. ieee transactions on pattern analysis and machine intelligence,2017,39(6):1223-1236. |
APA | Zhang, Weizhong.,Zhang, Lijun.,Jin, Zhongming.,Jin, Rong.,Cai, Deng.,...&He, Xiaofei.(2017).Sparse Learning with Stochastic Composite Optimization.ieee transactions on pattern analysis and machine intelligence,39(6),1223-1236. |
MLA | Zhang, Weizhong,et al."Sparse Learning with Stochastic Composite Optimization".ieee transactions on pattern analysis and machine intelligence 39.6(2017):1223-1236. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论