To load a pre-trained model from a disk using the Hugging Face Transformers library, save the pre-trained model and its tokenizer to your local disk, and then you can load them using the from_pretrained.
Follow the below step-by-step guide.
- Install the Hugging Face Transformers library using this command if you haven’t already.
pip install transformers
- Save the pre-trained model and its tokenizer to your local disk.
from transformers import BertModel, BertTokenizer # Choose a pre-trained model, e.g., bert-base-uncased model_name = "bert-base-uncased" # Download the pre-trained model and tokenizer model = BertModel.from_pretrained(model_name) tokenizer = BertTokenizer.from_pretrained(model_name) # Save the model and tokenizer to your local disk model.save_pretrained("path/to/your/local/directory/model") tokenizer.save_pretrained("path/to/your/local/directory/tokenizer")
Replace these two paths “path/to/your/local/directory/model” and “path/to/your/local/directory/tokenizer” with your desired directory paths.
- Load the pre-trained model and tokenizer from your local disk.
from transformers import BertModel, BertTokenizer # Load the model and tokenizer from the local disk model = BertModel.from_pretrained("path/to/your/local/directory/model") tokenizer = BertTokenizer.from_pretrained("path/to/your/local/directory/tokenizer")
Again, replace the “path/to/your/local/directory/model” and “path/to/your/local/directory/tokenizer” with the directory paths where you saved the model and tokenizer.
That’s it.