统计学习理论及其应用系列报告
日期： 2025年10月26日（周日）
时间： 上午9:30-11:30，下午14:00-17:00
地点： 数理楼145
报告人1：张海樟 教授（中山大学）
报告题目：Approximation via Function Composition for Deep Neural Networks
摘要：Deep neural networks use target function in machine learning. A popular technique in analyzing the expressive power of ReLU networks is based on an identity that allows exponential approximation of the square function $x^2$ and the multiplication operator $xy$ by self-compositions of the sawtooth function. It is interesting to study whether such a technique can be built for neural networks with other activation functions and whether one can have a faster approximation order. To this end, we first discover a hidden functional composition equation between the square function and the sawtooth function. By exploring the functional equation, we propose new continuously differentiable activation functions and compositional neural networks that can approximate polynomials exponentially and analytic functions sub-exponentially.
报告人简介：张海樟，2003年本科毕业于北京师范大学数学系，2006年硕士毕业于中科院数学所，2009年博士毕业于美国雪城大学（Syracuse University）数学系，2009年6月-2010年5月 密歇根大学（University of Michigan）博士后。2010年6月起担任中山大学教授、博士生导师。研究兴趣包括学习理论、应用调和分析和函数逼近. 代表性成果有再生核的Weierstrass逼近定理, 深度神经网络的收敛性理论，以及再生核巴拿赫空间理论. 在Journal of Machine Learning Research、Applied and Computational Harmonic Analysis、Mathematics of Computation、 Neural Networks、Neural Computation、Neurocomputing、Constructive Approximation、IEEE Transactions系列等发表多篇原创性工作,主持包括优秀青年基金在内的多项国家和省部级基金.
报告人2：陈洪 教授（华中农业大学）
报告题目：Bilevel Manifold Learning
摘要：The manifold assumption states that high-dimensional ambient data usually possess an instinct geometric structure with a low-dimensional representation, which has promoted many manifold learning models achieving promising performance for wide applications. However, it is challenging to estimate the latent manifold when facing redundant and noisy input variables. To address the above issue, this report will introduce our preliminary explorations of bilevel optimization for additive models with manifold regularization and manifold fitting with Cycle-GAN. Generalization analysis and empirical evaluations are stated for the proposed bilevel learning schemes.
报告人简介：陈洪，华中农业大学教授，博士生导师。研究方向为机器学习理论，农业人工智能。主持国家级项目7项,其中国家自然科学基金面上项目3项。在人工智能顶会NeurIPS、ICML、ICLR和数学与信息知名期刊ACHA、JAT IEEE TPAMI/ TIT/TIP/TNNLS/TCYB等发表系列研究论文,在Nature Genetics等合作发表智慧农业应用论文。
报告人3：李洽 副教授（中山大学）
报告题目：First-order Algorithms for Fractional Programming
摘要：This talk introduces novel first-order algorithms for fractional optimization, with applications in sparse recovery. We first address single-ratio minimization problems, where the numerator combines a nonsmooth and a smooth nonconvex term, while the denominator is a nonsmooth convex function. To tackle this class of problems, we propose the proximal-gradient-subgradient algorithm (PGSA) and its line-search variant PGSA_L. Specially for the case where the nonsmooth term in the numerator is convex, we enhance PGSA by incorporating a backtracked extrapolation technique. Additionally, for block-separable structures, we propose a solving framework called multi-proximity gradient algorithm (MPGA), along with two specific algorithms: cyclic MPGA and randomized MPGA. To tackle more general fractional optimization problems, we propose a novel min-max reformulation and develop the alternating maximization proximal descent algorithm (AMPDA). Convergence guarantees are rigorously established for all proposed algorithms. Numerical experiments demonstrate the efficiency of our methods, highlighting significant improvements in computational performance over existing approaches.
报告人简介：李洽，中山大学计算机学院副教授，博士生导师，现任计算机学院数据科学系副主任，广东省计算数学学会常务理事兼副秘书长，广东省计算科学重点实验室成员。2013 年获中山大学数学（信息计算科学方向）博士学位，博士期间曾赴美国Syracuse University数学系访问一年。研究方向包括最优化理论与算法及在机器学习、大数据分析、图像处理等领域中的应用，研究成果发表于SIAM Journal on Optimization, Applied and Computational Harmonic Analysis, Mathematics of Operations Research, Inverse Problems等期刊。主持国家级科研项目四项（包括国基面上两项、国基青年与国防类一项），参与项目包括国基重大研究计划集成项目，科技部重大专项等。
报告人4：张娜 教授（华南农业大学）
报告题目：Proximal methods for structured nonsmooth fractional optimization over Riemannian submanifolds
摘要：In this talk, we consider a class of structured nonsmooth optimization problems over an embedded submanifold of a Euclidean space, where the first part of the objective is the sum of a difference-of-convex (DC) function and a smooth function, while the remaining part is a weakly convex function over a smooth function. This model problem has many important applications in machine learning and scientific computing, for example, the sparse Fisher discriminant analysis. We propose a manifold proximal-gradient-subgradient algorithm (MPGSA) and show that under mild conditions any accumulation point of the solution sequence generated by it is a critical point of the underlying problem. By assuming the Kurdyka-Lojasiewicz property of an auxiliary function, we further establish the convergence of the full sequence generated by MPGSA under some suitable conditions. When the second component of the DC function involved is the maximum of finite continuously differentiable convex functions, we also propose an enhanced MPGSA with guaranteed subsequential convergence to a lifted B-stationary points of the optimization problem. Finally, some preliminary numerical experiments are conducted to illustrate the efficiency of the proposed algorithms.
报告人简介：张娜，华南农业大学数学与信息学院教授。研究领域包括最优化理论及算法、信号及图像处理。论文发表在SIAM Journal on Optimization, Applied and Computational Harmonic Analysis, Inverse Problems等国际知名期刊。2017 年论文入选《Inverse Problems》Highlight论文，获得2015年中国计算数学年会优秀青年论文竞赛一等奖，获得2021年广东省计算数学学会优秀青年学术成果一等奖。主持多项国家级和省级科研项目。
报告人5：焦雨领 教授（武汉大学）
报告题目：Some Theory on Learning with Transformers
摘要：In this talk, we will focus on the approximation upper and lower bounds for transformers. As an application, we will derive the convergence rates for several tasks involving learning with transformers to understand LLM training, Incontext Learning.
报告人简介：焦雨领，武汉大学人工智能学院，教授博导，副院长。入选国家高层次青年人才，担任ACM Transactions on Probabilistic Machine Learning、Statistical Theory and Related Fields 副主编。主要研究机器学习、科学计算。近期关注深度学习数理基础，在计算数学、应用数学、统计学、电子工程、人工智能等领域的旗舰期刊和会议上发表论文四十多篇： SIAM 系列（5 篇）、Appl.Comput. Harmon. Anal.(2篇)、Inverse Probl. (2 篇)；Ann. Stat. （3 篇）、J.Amer. Statist. Assoc.（2篇）； IEEE Trans. Inf. Theory (5 篇)、IEEE Trans. Signal Process.（3篇）；J. Mach. Learn. Res. (7 篇)、ICML (3 篇)、NeurIPS (5 篇，其中一篇Oral、一篇Spotlight)；中国科学（2 篇）、Nat. Commun.。主持国家重点研发计划子课题、国家自然科学基金面上项目及一批同华为开展的校企合作项目。
