logitorch.data_collators.bertnot_collator

Classes

BERTNOTTextualEntailmentCollator	Collator for BERTNOTTextualEntailment dataset.
BERTNOTWiki20KCollator	Collator for BERTNOTWiki20K dataset.

Module Contents

class logitorch.data_collators.bertnot_collator.BERTNOTTextualEntailmentCollator(pretrained_tokenizer: str)

Collator for BERTNOTTextualEntailment dataset.

Args:

pretrained_tokenizer (str): Pretrained tokenizer name or path.

Returns:

Tuple[Dict[str, torch.Tensor], torch.Tensor]: Batch of tokenized premises, hypotheses, and labels.

tokenizer

class logitorch.data_collators.bertnot_collator.BERTNOTWiki20KCollator(pretrained_tokenizer: str)

Collator for BERTNOTWiki20K dataset.

Args:

pretrained_tokenizer (str): Pretrained tokenizer name or path.

Returns:

Tuple[Dict[str, torch.Tensor], torch.Tensor]: Batch of tokenized sentences and labels.

tokenizer