How to Fix RuntimeError: 0D or 1D target tensor expected, multi-target not supported

RuntimeError: 0D or 1D target tensor expected, multi-target not supported error is raised when the target tensor (labels) has an incorrect shape. In PyTorch, the CrossEntropyLoss expects the target tensor to have a 1D shape.

To fix this error, make sure your labels tensor has the correct shape. The tensor should be (batch_size,), where batch_size is the number of samples in your input. The example assumes a single input sample, so the shape should be (1,).

Each element in this target tensor should be a class index in the range [0, C-1], where C is the number of classes.

If the target tensor is not in this format (for example, if it’s a 2D tensor or one-hot encoded), this error will occur.

import torch
from transformers import BertTokenizer, BertForSequenceClassification
from torch.nn import CrossEntropyLoss

# Loading the model and tokenizer
model_name = "bert-base-uncased"
tokenizer = BertTokenizer.from_pretrained(model_name)
model = BertForSequenceClassification.from_pretrained(model_name)

# Prepare your input
input_text = "Your input text goes here"
tokens = tokenizer(input_text, return_tensors="pt")
input_ids = tokens["input_ids"]
attention_mask = tokens["attention_mask"]

# Forward pass to get the logits
logits = model(input_ids, attention_mask=attention_mask).logits

# Prepare your labels
labels = torch.tensor([0]) # Assuming you have a binary classification task and a single input sample

# Compute the loss
loss_function = CrossEntropyLoss()
loss = loss_function(logits, labels)
print(loss)

Output

How to Fix RuntimeError - 0D or 1D target tensor expected, multi -target not supported

For nn.CrossEntropyLoss, the target has to be a single number from the interval [0, #classes] instead of a one-hot encoded target vector. Your target is [1, 0]. Thus, PyTorch thinks you want multiple labels per input, which is unsupported.

You can replace your one-hot-encoded targets like this:

[1, 0] --> 0

[0, 1] --> 1

Correct Target Tensor Shape for a Single Sample

If you have a single input sample, your target tensor should be of shape (1,) with the single element being the class index.

For example: target = torch.tensor([class_index]) where class_index is the integer representing the class.

Correct Target Tensor Shape for Batch:

For a batch of samples, the target tensor should be a 1D tensor, with each element corresponding to the class index of each sample in the batch.

For example: target = torch.tensor([class_index1, class_index2, …, class_indexN]) for a batch of N samples.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.