Annotate this: A comparative benchmark of MS/MS annotation algorithms across reference compounds and complex samples

Abstract

Introduction Metabolite annotation from LC-MS/MS data remains a bottleneck in untargeted metabolomics. A growing number of computational tools employing diverse strategies, from spectral library matching and in silico fragmentation to machine learning and \textit{de novo} structure generation, have been developed. However, the diversity of tools makes it difficult to determine which approaches provide reliable annotations. Direct comparisons under consistent, realistic conditions are needed to guide tool selection and improve annotation confidence. Objectives We benchmarked five popular annotation tools: MS2Query, SIRIUS, MSNovelist, MetFrag and ChemDistiller, under default parameters and without prior sample knowledge. Methods Tools were evaluated on two datasets: 357 in-house acquired MS/MS spectra from reference compounds with unambiguous ground truth, and 194 expert-annotated spectra from NIST SRM 1950 plasma. Performance was assessed across formula and structure accuracy, coverage, precision, Tanimoto similarity to ground truth, and score reliability. Results MS2Query and SIRIUS emerged as top performers with complementary profiles: MS2Query achieved the highest annotation coverage (100% on reference compounds) while SIRIUS yielded the highest conditional accuracy (78%). MSNovelist performed comparably to SIRIUS. MetFrag performance was database-dependent, with PubChem yielding near-complete annotation failure and KEGG enabling moderate precision. ChemDistiller excelled at formula annotation but underperformed structurally. Conclusion No single tool dominates across all metrics. Tool selection should be guided by balance between coverage and precision, with score-based filtering and multi-tool consensus strategies recommended to improve annotation confidence.

Abstract as supplied by the indexing source. © the authors / publisher.