logitorch.models.bertnot
========================

.. py:module:: logitorch.models.bertnot


Classes
-------

.. autoapisummary::

   logitorch.models.bertnot.BERTNOT


Module Contents
---------------

.. py:class:: BERTNOT(pretrained_bert_model: str, num_labels: int = 2)

   Bases: :py:obj:`torch.nn.Module`


   
   BERTNOT model for fine-tuning BERT for various tasks.

   Args:
       pretrained_bert_model (str): Path or identifier of the pre-trained BERT model.
       num_labels (int, optional): Number of labels for the classification task. Defaults to 2.


   .. py:method:: forward(x, y=None, task='mlm', loss='cross_entropy')

      Forward pass of the BERTNOT model.

      Args:
          x (dict): Input dictionary containing the input tensors.
          y (torch.Tensor, optional): Target tensor. Defaults to None.
          task (str, optional): Task type. Defaults to "mlm".
          loss (str, optional): Loss type. Defaults to "cross_entropy".

      Returns:
          tuple: Tuple containing the loss and logits if y is not None, otherwise returns logits.



   .. py:method:: predict(context: str, hypothesis: str = None, task='mlm', device='cpu')

      Perform prediction using the BERTNOT model.

      Args:
          context (str): Input context string.
          hypothesis (str, optional): Input hypothesis string. Defaults to None.
          task (str, optional): Task type. Defaults to "mlm".
          device (str, optional): Device to run the model on. Defaults to "cpu".

      Returns:
          str or int: Predicted token or label.



   .. py:attribute:: cross_entopy_loss


   .. py:attribute:: dropout


   .. py:attribute:: kl_loss


   .. py:attribute:: log_softmax


   .. py:attribute:: losses
      :value: ['cross_entropy', 'unlikelihood', 'kl']



   .. py:attribute:: model


   .. py:attribute:: num_labels
      :value: 2



   .. py:attribute:: original_bert


   .. py:attribute:: original_bert_softmax


   .. py:attribute:: sequence_classifier


   .. py:attribute:: tasks
      :value: ['mlm', 'te']



   .. py:attribute:: tokenizer


   .. py:attribute:: unlikelihood_loss