Fundamental Limits and Tradeoffs in Invariant Representation Learning

鲁棒线性回归：一般范数的相变与精确权衡

Robust Linear Regression: Phase-Transitions and Precise Tradeoffs for General Norms

本文研究了线性回归模型在测试时对抗攻击下的表现，定量揭示了对抗鲁棒性与标准准确率之间的基本权衡，并给出了适用于任意特征协方差矩阵和攻击范数的精确相变刻画。

Fundamental tradeoffs between memorization and robustness in random features and neural tangent regimes

This work studies the (non)robustness of two-layer neural networks in various high-dimensional linearized regimes. We establish fundamental trade-offs between memorization and robustness, as measured by the Sobolev-seminorm of the model w.r.t the data distribution, i.e the square root of the average squared $L_2$-norm of the gradients of the model w.r.t the its input. More precisely, if $n$ is the number of training examples, $d$ is the input dimension, and $k$ is the number of hidden neurons in a two-layer neural network, we prove for a large class of activation functions that, if the model m…

Fundamental Limits of Obfuscation for Linear Gaussian Dynamical Systems: An Information-Theoretic Approach

In this paper, we study the fundamental limits of obfuscation in terms of privacy-distortion tradeoffs for linear Gaussian dynamical systems via an information-theoretic approach. Particularly, we obtain analytical formulas that capture the fundamental privacy-distortion tradeoffs when privacy masks are to be added to the outputs of the dynamical systems, while indicating explicitly how to design the privacy masks in an optimal way: The privacy masks should be colored Gaussian with power spectra shaped specifically based upon the system and noise properties.

Computational and Statistical Tradeoffs in Learning to Rank

For massive and heterogeneous modern datasets, it is of fundamental interest to provide guarantees on the accuracy of estimation when computational resources are limited. In the application of learning to rank, we provide a hierarchy of rank-breaking mechanisms ordered by the complexity in thus generated sketch of the data. This allows the number of data points collected to be gracefully traded off against computational resources available, while guaranteeing the desired level of accuracy. Theoretical guarantees on the proposed generalized rank-breaking implicitly provide such trade-offs, whic…

对抗鲁棒分类中可证明的权衡关系

Provable tradeoffs in adversarially robust classification

针对机器学习的对抗脆弱性问题，推导了类别不平衡的二/三分类高斯问题下ℓ₂、ℓ∞攻击的贝叶斯最优鲁棒分类器，揭示了标准与鲁棒准确率的权衡及数据不平衡的影响，还给出凸损失校准与鲁棒风险有限样本速率等结果。

等化几率（Equalized Odds）下的公平性-准确率内在权衡

Intrinsic Fairness-Accuracy Tradeoffs under Equalized Odds

该研究针对等化几率下的公平性-准确率权衡问题，提出了依赖数据统计特征的任意分类器准确率上界，并在三个真实数据集上验证，揭示了群体统计差异对低偏差高准确率的根本性限制。

The Role of Embedding Complexity in Domain-invariant Representations

Unsupervised domain adaptation aims to generalize the hypothesis trained in a source domain to an unlabeled target domain. One popular approach to this problem is to learn domain-invariant embeddings for both domains. In this work, we study, theoretically and empirically, the effect of the embedding complexity on generalization to the target domain. In particular, this complexity affects an upper bound on the target risk; this is reflected in experiments, too. Next, we specify our theoretical framework to multilayer neural networks. As a result, we develop a strategy that mitigates sensitivity…

受语义性约束：支配泛化-识别权衡的普适定律

Bound by semanticity: universal laws governing the generalization-identification tradeoff

该研究揭示了智能系统在泛化与识别之间的根本权衡极限，推导出普适帕累托前沿，并通过实验验证了有限语义分辨率作为基本约束的作用。

看似好得不真实：生成式复原模型中幻觉的信息论分析

Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models

本文从信息论角度分析生成式图像复原模型的幻觉问题，证明最小不确定性随感知质量增长，并通过超分辨率与修复实验揭示其可靠性和感知质量间的根本权衡。

光学ISAC：基本性能极限与收发器设计

Optical ISAC: Fundamental Performance Limits and Transceiver Design

本文刻画光学ISAC系统的最优容量-失真权衡，探索率-失真区域的界，提出渐近最优的目标距离估计器，证明可达率-克拉美-罗界的外界性质，提出两种优化输入分布的算法并适配DRT到光学场景。