logitorch.pl_models.bertnot

Classes

PLBERTNOT

PyTorch Lightning module for BERTNOT model.

Module Contents

class logitorch.pl_models.bertnot.PLBERTNOT(pretrained_model: str, task: str = 'mlm', num_labels: int = 2, learning_rate: float = 1e-05, weight_decay: float = 0.1, batch_size: int = 32, gamma: float = 0.4)

Bases: lightning.pytorch.LightningModule

PyTorch Lightning module for BERTNOT model.

Args:

pretrained_model (str): Pretrained model name or path. task (str): Task type, either “mlm” (masked language modeling) or “te” (text entailment). num_labels (int): Number of labels for the classification task. learning_rate (float): Learning rate for the optimizer. weight_decay (float): Weight decay for the optimizer. batch_size (int): Batch size for the data loader. gamma (float): Gamma value for the loss calculation.

Attributes:

model (BERTNOT): BERTNOT model instance. pretrained_model (str): Pretrained model name or path. learning_rate (float): Learning rate for the optimizer. weight_decay (float): Weight decay for the optimizer. batch_size (int): Batch size for the data loader. gamma (float): Gamma value for the loss calculation. task (str): Task type, either “mlm” (masked language modeling) or “te” (text entailment).

configure_optimizers()

Configure the optimizer and learning rate scheduler.

Returns:

Tuple[List[torch.optim.Optimizer], List[torch.optim.lr_scheduler._LRScheduler]]: Optimizers and schedulers.

forward(x, y=None, loss='cross_entropy')

Forward pass of the PLBERTNOT model.

Args:

x: Input data. y: Target labels. loss (str): Loss function type.

Returns:

torch.Tensor: Model output.

predict(context: str, hypothesis: str = None, task='mlm', device='cpu')

Make predictions using the PLBERTNOT model.

Args:

context (str): Input context. hypothesis (str): Input hypothesis (optional). task (str): Task type, either “mlm” (masked language modeling) or “te” (text entailment). device (str): Device to run the model on.

Returns:

torch.Tensor: Model predictions.

train_dataloader()

Get the training data loader.

Returns:

Dict[str, DataLoader]: Dictionary of data loaders.

training_step(train_batch: Tuple[Dict[str, torch.Tensor], torch.Tensor], batch_idx: int)

Training step of the PLBERTNOT model.

Args:

train_batch: Batch of training data. batch_idx (int): Batch index.

Returns:

torch.Tensor: Loss value.

validation_step(val_batch: Tuple[Dict[str, torch.Tensor], torch.Tensor], batch_idx: int)

Validation step of the PLBERTNOT model.

Args:

val_batch: Batch of validation data. batch_idx (int): Batch index.

batch_size = 32

gamma = 0.4

learning_rate = 1e-05

model

pretrained_model

task = 'mlm'

weight_decay = 0.1