logitorch.pl_models.bertnot
===========================

.. py:module:: logitorch.pl_models.bertnot


Classes
-------

.. autoapisummary::

   logitorch.pl_models.bertnot.PLBERTNOT


Module Contents
---------------

.. py:class:: PLBERTNOT(pretrained_model: str, task: str = 'mlm', num_labels: int = 2, learning_rate: float = 1e-05, weight_decay: float = 0.1, batch_size: int = 32, gamma: float = 0.4)

   Bases: :py:obj:`lightning.pytorch.LightningModule`


   PyTorch Lightning module for BERTNOT model.

   Args:
       pretrained_model (str): Pretrained model name or path.
       task (str): Task type, either "mlm" (masked language modeling) or "te" (text entailment).
       num_labels (int): Number of labels for the classification task.
       learning_rate (float): Learning rate for the optimizer.
       weight_decay (float): Weight decay for the optimizer.
       batch_size (int): Batch size for the data loader.
       gamma (float): Gamma value for the loss calculation.

   Attributes:
       model (BERTNOT): BERTNOT model instance.
       pretrained_model (str): Pretrained model name or path.
       learning_rate (float): Learning rate for the optimizer.
       weight_decay (float): Weight decay for the optimizer.
       batch_size (int): Batch size for the data loader.
       gamma (float): Gamma value for the loss calculation.
       task (str): Task type, either "mlm" (masked language modeling) or "te" (text entailment).


   .. py:method:: configure_optimizers()

      Configure the optimizer and learning rate scheduler.

      Returns:
          Tuple[List[torch.optim.Optimizer], List[torch.optim.lr_scheduler._LRScheduler]]: Optimizers and schedulers.



   .. py:method:: forward(x, y=None, loss='cross_entropy')

      Forward pass of the PLBERTNOT model.

      Args:
          x: Input data.
          y: Target labels.
          loss (str): Loss function type.

      Returns:
          torch.Tensor: Model output.



   .. py:method:: predict(context: str, hypothesis: str = None, task='mlm', device='cpu')

      Make predictions using the PLBERTNOT model.

      Args:
          context (str): Input context.
          hypothesis (str): Input hypothesis (optional).
          task (str): Task type, either "mlm" (masked language modeling) or "te" (text entailment).
          device (str): Device to run the model on.

      Returns:
          torch.Tensor: Model predictions.



   .. py:method:: train_dataloader()

      Get the training data loader.

      Returns:
          Dict[str, DataLoader]: Dictionary of data loaders.



   .. py:method:: training_step(train_batch: Tuple[Dict[str, torch.Tensor], torch.Tensor], batch_idx: int)

      Training step of the PLBERTNOT model.

      Args:
          train_batch: Batch of training data.
          batch_idx (int): Batch index.

      Returns:
          torch.Tensor: Loss value.



   .. py:method:: validation_step(val_batch: Tuple[Dict[str, torch.Tensor], torch.Tensor], batch_idx: int)

      Validation step of the PLBERTNOT model.

      Args:
          val_batch: Batch of validation data.
          batch_idx (int): Batch index.



   .. py:attribute:: batch_size
      :value: 32



   .. py:attribute:: gamma
      :value: 0.4



   .. py:attribute:: learning_rate
      :value: 1e-05



   .. py:attribute:: model


   .. py:attribute:: pretrained_model


   .. py:attribute:: task
      :value: 'mlm'



   .. py:attribute:: weight_decay
      :value: 0.1