An extended latent factor framework for ill-posed linear regression

The classical latent factor model for linear regression is extended by assuming that,
up to an unknown orthogonal transformation, the features consist of subsets that are
relevant and irrelevant for the response. Furthermore, a joint low-dimensionality is
imposed only on the relevant features vector and the response variable. This framework
allows for a comprehensive study of the partial-least-squares (PLS) algorithm under
random design. In particular, a novel perturbation bound for PLS solutions is proven
and the high-probability L2-estimation rate for the PLS estimator is obtained. This
novel framework also sheds light on the performance of other regularisation methods
for ill-posed linear regression that exploit sparsity or unsupervised projection. The
theoretical findings are confirmed by numerical studies on both real and simulated
data.