Towards Next-Generation LLM Training: From the Data-Centric Perspective

Hao Liang, Zhengyang Zhao, Zhaoyang Han, Meiyi Qiang, Xiaochen Ma, Bohan Zeng, Qifeng Cai, Zhiyu Li, Linpeng Tang, Weinan E, Wentao Zhang
March, 2026
PDF arXiv

Abstract

This perspective outlines two complementary directions for next-generation LLM training: agent-based systems for automatic, reusable data preparation and unified data-model interaction systems that dynamically select, mix, and reweight training data.

Type
Preprint
Publication
arXiv preprint arXiv:2603.14712
Xiaochen Ma
Xiaochen Ma
Ph.D. Student in Computer Science and Engineering

I build data infrastructure and high-performance distributed systems that turn heterogeneous data into reliable inputs for large-scale LLM training, with a focus on Ray and Apache Spark.