Composition: In constructing BabyBabelLM our guiding principle was developmental plausibility — the idea that pretraining data should approximate the child's linguistic environment. To this end, we prioritized domains such as: child-directed speech, educational materials, children's books, and transcribed conversations. We considered a diverse set of sources, including publicly available datasets, open-access children books and wikis, and local experts in linguistics and language acquisition. This involved both the curation of available developmentally plausible resources, and the addition of original data.

For most languages, the collection effort was led by a researcher fluent or familiar with that language, responsible for gathering appropriate corpora and ensuring dataset quality. In addition to language-specific data, languages were supplemented with extensive multilingual resources (e.g., CHILDES, GlotStoryBooks), which also comprised the main dataset content for languages without a dedicated leader. Synthetic data were deliberately excluded due to their qualitative differences from naturally occurring language.

Preprocessing: After language-specific collection, all data were uniformly processed through a shared preprocessing pipeline developed as part of the project, ensuring comparability across languages. In addition, each dataset was annotated with rich document-level metadata in a standardized format, describing the document’s source, content, age estimate, license, and other information.

Language Tiers: Given the difference in availability of developmentally plausible corpora, we sorted languages into three tiers based on training set size, corresponding to the equivalent of respectively 100M, 10M, or 1M English words. Dataset sizes were calibrated by language-adjusted byte estimates (paper link), to maintain comparability of data budgets across languages with differing orthographic and morphological characteristics.

Licensing: Finally, throughout collection, we took great care in verifying that all data were released under licenses that permit academic research, and we release our data with document-level licensing information and data source attribution to ensure each resource is used ethically and responsibly.