Skip to main content

Use Storage on Your Instance

Persistent Storage, powered by Backblaze, is in Beta. Account owners and admins can join under Settings → Profile, or by ticking Join the beta program at signup. They can leave it the same way, under Settings → Profile. See Overview.

Where it mounts

Each attached storage is a folder under /persistent:

ls /persistent
# datasets  checkpoints
  • Storage named datasets is at /persistent/datasets.
  • In login shells, $MC_PERSISTENT is set to /persistent.
  • Every user on the instance can read and write the folder.

Copy files in like any other folder:

cp -r ./checkpoints/. /persistent/checkpoints/

When it mounts

Storage mounts every time the instance starts. If a folder under /persistent is empty when you expect files, see FAQ and Troubleshooting.

What persists

  • Kept: everything you write under /persistent/<name>. It stays in the storage after you terminate the instance.
  • Not kept: everything else on the instance, including your home folder and /workspace. Terminating an instance destroys that data.

To use the files again, deploy a new instance and attach the same storage.

Network storage, not a local disk

Persistent Storage is object storage behind a filesystem layer. It has higher latency than the instance's local NVMe disk and only partial POSIX behavior. Keep hot data on local disk.

Persistent Storage is network object storage, mounted as a folder. It is not a disk inside the instance. See Limitations for what it doesn't support and what happens when storage can't be reached.

  • It works best for large files you read and write whole: datasets, model weights, checkpoints, archives.
  • It is slower than the instance's local disk for many small reads and writes. Keep databases, package caches and scratch files on the local disk, and copy results to /persistent when you're done.
  • Reading files from your storage counts as a download. Downloads are free up to an allowance, then charged. See Pricing and Billing.

Before you terminate

Files are uploaded to storage in the background after you write them. Flush your storage before you stop or terminate an instance. Pass the storage folder to sync; plain sync doesn't wait for these uploads:

sync /persistent/<name>

A file isn't safe in storage until its upload finishes. A file that is still being written or uploaded when the instance is stopped or terminated can be incomplete.

One instance at a time

Attaching the same storage to more than one instance at the same time isn't supported and can corrupt your data. Every mount is read-write, and there is no file locking between instances. Attach each storage to one instance at a time: stop the instance that has it and detach it before you attach it to another.

To move storage to another instance:

  1. Run sync /persistent/<name> on the instance that has it.
  2. Stop that instance.
  3. Open My Instances, click Storage on it, and click Detach. See Detach storage.
  4. Attach the storage to the other instance, at deploy or from My Instances. On an existing instance, stop it and start it again.

See Sharing and Permissions.

Mascot