RT Daniel van Strien: datatrove — the data-processing library behind FineWeb, FineWeb2, and FinePDFs — just shipped 0.10.0! – JobsPipelineExecutor: …
RT Daniel van Striendatatrove — the data-processing library behind FineWeb, FineWeb2, and FinePDFs — just shipped 0.10.0!- JobsPipelineExecutor: run pipelines on @huggingface Jobs: fan-out, multi-stage dependencies,…