mlm.history_mlm
StepResults
Bases: TypedDict
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
The input token ids. |
required | |
attention_mask
|
The mask for the input token ids. |
required | |
position_ids
|
The ids of the positions in the sequence. |
required | |
logits
|
The logits for the next token. |
required | |
t
|
The time step. |
required | |
current_step
|
The current step. |
required | |
change
|
The change in the logits. |
required | |
input_end_positions
|
The end positions of the input tokens. |
required |
HistoryPluginBase
reset()
Called between examples.
HistoryTopKPlugin
Bases: HistoryPluginBase
We will dump the top k tokens and probs as tensors in separate files for each step.