Do Better Embeddings Lead to Better Judgments? A User Study of PCA and UMAP for Single-Cell Data
abstract
Dimensionality-reduction visualizations are essential to modern biology, but the same data can yield different embeddings depending on algorithm and hyperparameter choices. Whether these choices influence analyst conclusions, and whether more faithful embeddings yield better conclusions, is unknown. We ran an exploratory within-subject study with 12 participants comparing PCA, UMAP set to default hyperparameters, and a tuned version of UMAP across clustering and trajectory tasks grounded in single-cell analysis workflows. Both UMAP conditions produced higher accuracy and confidence than PCA and were strongly preferred, and completion time did not differ significantly. However, tuning did not improve accuracy over default UMAP and higher confidence did not correspond to higher accuracy. These findings suggest that computational embedding fidelity and perceptual usefulness are related but distinct.