Skip to content

mlm.history_mlm

StepResults

Bases: TypedDict

Parameters:

Name Type Description Default
x

The input token ids.

required
attention_mask

The mask for the input token ids.

required
position_ids

The ids of the positions in the sequence.

required
logits

The logits for the next token.

required
t

The time step.

required
current_step

The current step.

required
change

The change in the logits.

required
input_end_positions

The end positions of the input tokens.

required

HistoryPluginBase

reset()

Called between examples.

HistoryTopKPlugin

Bases: HistoryPluginBase

We will dump the top k tokens and probs as tensors in separate files for each step.