Python has a built-in module called “tempfile” that provides the .TemporaryFile() and .TemporaryNamedFile() methods that will create a temporary file (without or with name) efficiently.
Why do we need to use temporary folders and files? While executing a program, numerous things happen behind the scenes, and you need to store data temporarily. For that, we need to create a temporary file, which must be in a secure location and either unnamed or have an unpredictable name.
The tempfile.Temporary() function creates a file-like object that stores data without requiring a file system.
Always use the context manager like “with statement” for automatic cleanup. Always remember to close the tempfile unless you need to keep it open. It prevents orphaned files or directories from cluttering the system.
The permission of the temporary file is always 0o600, which allows you to read and write.
Unnamed temp file
If you want a temp file for a shorter amount of time and it is not necessary to have a name, you can use the tempfile.Temporary() method.
Before going further, you need to import tempfile at the head of your file. Then, we can use its methods.
import tempfile
with tempfile.TemporaryFile() as temp_file:
temp_file.write(b"Hello mate, from temp file!")
temp_file.seek(0)
content = temp_file.read()
print(content.decode())
# Output: Hello mate, from temp file!
Even though it is a temp file with no name, it will still occupy space in memory.
On macOS, I can see the tempfiles like this:
It temporarily stores in a complex location like this: /var/folders/8k/x52znv0n7r53eebnmngj4kmw0400gn/T
The file has no visible names, so it is safe for sensitive data.
If you are working with very large datasets, always use the TemporaryFile() method. It uses disk storage but is optimized for temporary data that does not persist.
Named temp file
The tempfile.NamedTemporaryFile() method creates a named file. A named temp file is helpful when another process wants to use it.
from tempfile import NamedTemporaryFile
with NamedTemporaryFile(mode='w+', delete=True, suffix='.log') as tmp:
print(f"Temporary file path: {tmp.name}")
# File is deleted when `delete=True` (default) after context exits.
# Output: Temporary file path: /var/folders/8k/x42znv0n7r5383bnmngk4kmw0900gn/T/tmpuvh9_37x.log
By default, when the “with” context is complete, the file will be automatically deleted. If you pass delete=False, you need to clear the temp file manually.
To customize the file name, you can use the “suffix/prefix” argument.
The above code will work fine on macOS or Linux, but it may have issues with Windows OS.
On Windows, when you create a temporary file using the above approach, the OS automatically locks it. To handle this, you need to pass delete=false and then manually delete the file using the OS.remove() method.
import tempfile
import os
with tempfile.NamedTemporaryFile(suffix=".txt", delete=False) as named_temp:
named_temp.write(b"Temporary file with a name!")
print("File name:", named_temp.name)
# Deleting the file manually if delete=False
os.remove(named_temp.name)
# Output: your windows path
Define a custom temp directory
You can use the “dir” parameter in the NamedTemporaryFile() method to specify a custom base directory.
NamedTemporaryFile(dir='/var/folders/8k/tmp')
Using SpooledTemporaryFile()
There is also another function from the tempfile module, which is SpooledTemporaryFile(). It optimizes small file handling by spooling data in memory.
from tempfile import SpooledTemporaryFile
with SpooledTemporaryFile(max_size=1024) as tmp:
tmp.write(b'Small data') # Stays in memory
tmp.rollover() # Force write to disk
print(tmp)
# Output: <tempfile.SpooledTemporaryFile object at 0x102a2fb50>
Using mkstemp()
If you want lower-level control over temporary files, use the tempfile.mkstemp() method. It returns a tuple containing an OS-level file descriptor and the path to the temporary file.
import os
from tempfile import mkstemp
# Create a temporary file
fd, path = mkstemp()
try:
with os.fdopen(fd, 'w+') as tmp:
tmp.write('Manual cleanup example')
print(path)
finally:
os.remove(path) # Manual cleanup
# Output: /var/folders/8k/x42znv0n7r5383bnnmgl4kmw0009gn/T/tmp7kcdwijz
Your temp file path will be different from mine, and it depends on the system’s configuration.
Creating a temporary directory
In Python3, you can use the tempfile.TemporaryDirectory() method to create a temporary directory. To automatically delete it, we should combine this function with the “with” context manager.
from tempfile import TemporaryDirectory
with TemporaryDirectory() as dirpath:
print(f"Temporary directory created at: {dirpath}")
# Work with files inside the directory here
with open(f"{dirpath}/temp_file.txt", "w") as f:
f.write("Hello from the temp directory!")
# Directory and all contents are deleted here
# Output: Temporary directory created at:
# /var/folders/8l/x42qnv0n7r5383bnmngj5kmw0000gm/T/tmp21xelljj
The above code will generate a temp folder, and when the “with” block exists, this folder will be deleted automatically.
You can control the output temp directory’s name using suffix, prefix, and dir arguments like this:
from tempfile import TemporaryDirectory
with TemporaryDirectory(
prefix="mydata_",
suffix="_tmp",
dir="/custom/path"
) as dirpath:
print(f"Custom directory: {dirpath}")
# Output: Custom directory: /custom/path/mydata_a8b3kfp2_tmp
If automatic cleanup fails somehow, you can use the .cleanup() method.
from tempfile import TemporaryDirectory
temp_dir = TemporaryDirectory()
dirpath = temp_dir.name
print(f"Directory: {dirpath}")
# Manually deleting the directory
temp_dir.cleanup()
# Directory is now gone
# Output: Directory: /var/folders/8k/x42zmn0n7r5383bnmnkb4kmw0000kl/T/tmpphe2bb4b
Using mkdtemp()
For getting lower-level control, you can use the tempfile.mkdtemp() method. It creates a temporary directory and returns its path.
from tempfile import mkdtemp
import os
# Create a temporary directory using mkdtemp()
dir_path = mkdtemp()
try:
print(f"Directory: {dir_path}")
finally:
os.rmdir(dir_path)
# Output: Directory: /var/folders/8m/x21znv0n7r5383bnmngj4kmw0000kb/T/tmpx0c8i6bc
That’s all!
