Apple says that OpenELM is trained on publicly available datasets using the CoreNet library which includes RefinedWeb, deduplicated PILE, a subset of RedPajama, and a subset of Dolma v1.6 ...