We study the problem of learning a single neuron under standard squared loss in the presence of arbitrary label noise and group-level distributional shifts, for a broad family of covariate distributions. Our goal is to identify a ''best-fit'' neuron parameterized by $\mathbf{w}_*$ that performs well under the most challenging reweighting of the groups. Specifically, we address a Group Distributionally Robust Optimization problem: given sample access to $K$ distinct distributions $\mathcal p_{[1]},\dots,\mathcal p_{[K]}$, we seek to approximate $\mathbf{w}_*$ that minimizes the worst-case objective over convex combinations of group distributions $\boldsymbolλ \in Δ_K$, where the objective is $\sum_{i \in [K]}λ_{[i]}\,\mathbb E_{(\mathbf x,y)\sim\mathcal p_{[i]}}(σ(\mathbf w\cdot\mathbf x)-y)^2 - νd_f(\boldsymbolλ,\frac{1}{K}\mathbf1)$ and $d_f$ is an $f$-divergence that imposes (optional) penalty on deviations from uniform group weights, scaled by a parameter $ν\geq 0$. We develop a computationally efficient primal-dual algorithm that outputs a vector $\widehat{\mathbf w}$ that is constant-factor competitive with $\mathbf{w}_*$ under the worst-case group weighting. Our analytical framework directly confronts the inherent nonconvexity of the loss function, providing robust learning guarantees in the face of arbitrary label corruptions and group-specific distributional shifts. The implementation of the dual extrapolation update motivated by our algorithmic framework shows promise on LLM pre-training benchmarks.
Adversarial dictionary learning for a robust analysis and modelling of spontaneous neuronal activity
University of Crete(克里特大学);Foundation for Research and Technology Hellas(希腊研究与技术基金会);Brigham and Women’s Hospital(布莱根妇女医院);Harvard Medical School(哈佛医学院);Boston VA Research Institute(波士顿退伍军人事务研究所);Jamaica Plain Veterans Administration Hospital(牙买加平原退伍军人管理局医院)
Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivity
Columbia University(哥伦比亚大学);Edmond and Lily Safra Center for Brain Sciences(埃德蒙和莉莉·萨弗拉脑科学中心);Racah Institute of Physics(拉卡物理研究所);The Hebrew University of Jerusalem(耶路撒冷希伯来大学);Center for Brain Science(脑科学中心);Harvard University(哈佛大学)
Robust Parameter and State Estimation in Multiscale Neuronal Systems Using Physics-Informed Neural Networks
Department of Mathematics, University of Iowa(数学系,爱荷华大学);Department of Mathematics, Volen National Center for Complex Systems, Brandeis University(数学系,布兰迪大学复杂系统国家中心)