Skip to content

mlm.papl_unconditional

PAPL-style unconditional MLM: fixed lengths (default 100..800 step 100), one length per batch.

PaplUnconditionalMLMDataset

Bases: IterableDataset

All-mask sequences; yields examples_per_node samples per length, in length order.

PaplUnconditionalCollator

Bases: Collator

Stacks equal-length PAPL examples (no BOS/EOS, no random MLM noise).