Volumes

Volumes offer a high-performance alternative to cloud buckets for storing data. Unlike buckets, volumes are limited to a single region and cannot be accessed across regions and clouds.

Benefits of using volumes:

• Performance: Volumes are generally faster than object stores, and SkyPilot lets you choose from different storage classes based on your performance requirements.

• Data persistence: Volumes can persist data independently of task life cycles, making it easy to share data between different tasks (e.g., datasets, caches) or preserve results.

• Size control: You can set volume size limits to manage costs and limit storage usage.

SkyPilot supports creating and managing volumes directly through the sky CLI and the web dashboard.

Supported volume types:

• Kubernetes: Persistent Volume Claims (PVCs)

  • Tested storage backends: AWS EBS, GCP Persistent Disk, Nebius network SSD, JuiceFS, Nebius shared file system, GCP Filestore

• RunPod: Network Volumes

Volumes on Kubernetes

In Kubernetes clusters, PVCs (Persistent Volume Claims) request and bind to PV (Persistent Volume) resources. These persistent volumes can be backed by various storage backends, including block storage solutions (AWS EBS, GCP Persistent Disk) and distributed file systems (JuiceFS, Nebius shared file system, AWS EFS, GCP Filestore), etc.

SkyPilot supports creating and managing PVC volumes in Kubernetes clusters through three commands:

• sky volumes apply: Create a new volume

• sky volumes ls: List all volumes

• sky volumes delete: Delete a volume

Note

Volumes are shared across users on a SkyPilot API server. A user can mount volumes created by other users. This is useful for sharing caches and data across users.

Quickstart

1. Prepare a volume YAML file:

   # volume.yaml
   name: new-pvc
   type: k8s-pvc
   infra: kubernetes  # or k8s or k8s/context
   size: 10Gi
   labels:
     key: value
   config:
     namespace: default  # optional
     storage_class_name: csi-mounted-fs-path-sc  # optional
     access_mode: ReadWriteMany  # optional
   

2. Create the volume with sky volumes apply volume.yaml:

   $ sky volumes apply volume.yaml
   Proceed to create volume 'new-pvc'? [Y/n]: Y
   Creating PVC: new-pvc-73ec42f2-5c6c4e
   

3. Mount the volume in your task YAML:

   # task.yaml
   volumes:
     /mnt/data: new-pvc  # The volume new-pvc will be mounted to /mnt/data
   
   run: |
     echo "Hello, World!" > /mnt/data/hello.txt
   

Managing volumes

List all volumes with sky volumes ls:

$ sky volumes ls
NAME     TYPE     INFRA                         SIZE  USER   WORKSPACE  AGE   STATUS  LAST_USE     USED_BY
new-pvc  k8s-pvc  Kubernetes/nebius-mk8s-vol    1Gi   alice  default    8m    IN_USE  <timestamp>  <cluster_name>

Tip

Use -v to view detailed information about a volume.

$ sky volumes ls -v
NAME     TYPE     INFRA                         SIZE  USER   WORKSPACE  AGE   STATUS  LAST_USE             USED_BY   NAME_ON_CLOUD              STORAGE_CLASS           ACCESS_MODE
new-pvc  k8s-pvc  Kubernetes/nebius-mk8s-vol    1Gi   alice  default    8m    IN_USE  2025-06-24 10:18:32  training  new-pvc-73ec42f2-5c6c4e    csi-mounted-fs-path-sc  ReadWriteMany

Delete a volume with sky volumes delete:

$ sky volumes delete new-pvc
Proceed to delete volume 'new-pvc'? [Y/n]: Y
Deleting PVC: new-pvc-73ec42f2-5c6c4e

If the volume is in use, it will be marked as IN_USE and cannot be deleted.

You can also check the volumes in the SkyPilot dashboard.

Filesystem volume examples

This section demonstrates how to configure and use distributed filesystems as SkyPilot volumes. We’ll cover options like JuiceFS (a cloud-native distributed filesystem) and Nebius shared file system (a high-performance shared storage solution).

JuiceFS

To use JuiceFS as a SkyPilot volume:

1. Install the JuiceFS CSI driver on your Kubernetes cluster. Follow the official installation guide for detailed instructions.

2. Verify the driver installation - Confirm that the juicefs-sc storage class has been created successfully:

$ kubectl get storageclass
NAME           PROVISIONER         RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
juicefs-sc     csi.juicefs.com     Retain          Immediate           false                  10m

Note

If the juicefs-sc storage class is not available, refer to the JuiceFS storage class creation guide to set it up.

3. Create a SkyPilot volume for JuiceFS with a volume YAML:

# juicefs-volume.yaml
name: juicefs-pvc
type: k8s-pvc
infra: k8s
size: 100Gi
config:
  storage_class_name: juicefs-sc
  access_mode: ReadWriteMany

$ sky volumes apply juicefs-volume.yaml

4. Mount the volume to SkyPilot task in your SkyPilot YAML:

# task.yaml
num_nodes: 2

volumes:
  # Mount the JuiceFS volume to /mnt/data across all nodes
  /mnt/data: juicefs-pvc

run: |
  # Verify the volume is mounted and accessible
  df -h /mnt/data
  ls -la /mnt/data

# Launch the cluster with the JuiceFS volume
$ sky launch -c juicefs-cluster task.yaml

Nebius shared file system

To use Nebius shared file system as a SkyPilot volume:

1. Set up the Nebius filesystem infrastructure by following the official documentation:

   • Create a shared filesystem

   • Create a node group and mount the filesystem

   • Install the CSI driver

2. Verify the storage class - Confirm that the csi-mounted-fs-path-sc storage class has been created:

$ kubectl get storageclass
NAME                     PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
csi-mounted-fs-path-sc   mounted-fs-path.csi.nebius.ai  Delete          WaitForFirstConsumer   false                  10m

3. Create a SkyPilot volume for Nebius file system with a volume YAML:

# nebius-volume.yaml
name: nebius-pvc
type: k8s-pvc
infra: k8s
size: 100Gi
config:
  storage_class_name: csi-mounted-fs-path-sc
  access_mode: ReadWriteMany

$ sky volumes apply nebius-volume.yaml

4. Mount the volume to SkyPilot task in your SkyPilot YAML:

# task.yaml
num_nodes: 2

volumes:
  # Mount the Nebius shared filesystem to /mnt/data across all nodes
  /mnt/data: nebius-pvc

run: |
  # Verify the volume is mounted and accessible
  df -h /mnt/data
  ls -la /mnt/data

# Launch the cluster with the Nebius volume
$ sky launch -c nebius-cluster task.yaml

Volumes on RunPod

RunPod Network Volumes provide persistent storage that can be mounted into pods on RunPod. SkyPilot supports creating and managing RunPod network volumes via the same three commands:

• sky volumes apply: Create a new network volume

• sky volumes ls: List all volumes

• sky volumes delete: Delete a volume

Notes specific to RunPod:

• infra must specify the RunPod data center (zone), e.g. runpod/CA/CA-MTL-1.

• Volume name length is limited (max 30 characters).

• Labels are not currently supported for RunPod volumes.

Quickstart

1. Prepare a volume YAML file:

   # runpod-volume.yaml
   name: rpvol
   type: runpod-network-volume
   infra: runpod/CA/CA-MTL-1  # DataCenterId (zone)
   size: 100Gi                # GiB
   

2. Create the volume with sky volumes apply runpod-volume.yaml:

   $ sky volumes apply runpod-volume.yaml
   Proceed to create volume 'rpvol'? [Y/n]: Y
   Created RunPod network volume rpvol-43dbb4ab-15e906 (id=5w6ecp2w9n)
   

3. Mount the volume in your task YAML:

   # task.yaml
   volumes:
     /workspace: rpvol
   
   run: |
     echo "Hello, RunPod!" > /workspace/hello.txt
   

Managing volumes

Same as Kubernetes volumes, refer to Managing volumes for more details.