Configuration Sources and Overrides#

SkyPilot allows you to set configuration across multiple sources, providing flexibility in how you manage your configurations.

It also implements a override priority mechanism to merge configurations from different sources when layering configurations.

For example, you can have a project configuration storing default values for all jobs in a project, a user configuration to apply globally to all projects and Task YAML overrides for specific jobs.

Configuration sources#

Configuration Type	Configuration Location	Description
Server configuration	`~/.sky/config.yaml` on API server	Applies to all requests made to the SkyPilot API server.
User configuration	`~/.sky/config.yaml`	Applies to all SkyPilot invocations.
Project configuration	`$pwd/.sky.yaml`	Applies to all SkyPilot invocations in the current directory.
SkyPilot YAML	`config` field in the SkyPilot YAML	Applies to a specific SkyPilot task.
CLI flags	Using `--config` CLI flag	Override configuration for a specific command.

All configurations use the configuration syntax. Any new configuration changes do not affect existing clusters.

You can layer configurations from multiple sources. When multiple sources are specified, SkyPilot merges them.

Note

The following fields are ignored if specified on the client side:

Tip

If you are the admin of a SkyPilot API server, you can disable overrides or allow only certain fields to be overridden by enforcing an admin policy.

Server configuration#

If you are using a remote SkyPilot API server, it looks for ~/.sky/config.yaml in the API server instance/container to find the server configuration.

To specify a different file, set SKYPILOT_GLOBAL_CONFIG environment variable to the desired path.

If you are using a local API server, you can use User configuration to set global configuration.

User configuration#

SkyPilot client looks for ~/.sky/config.yaml to find the user configuration.

To specify a different file, set SKYPILOT_GLOBAL_CONFIG environment variable to the desired path.

Project configuration#

SkyPilot client looks for $pwd/.sky.yaml to find the current project configuration.

To specify a different file, set SKYPILOT_PROJECT_CONFIG environment variable to the desired path.

SkyPilot YAML#

You can specify inline configuration options in SkyPilot YAML files in the config field.

The following fields are supported in SkyPilot YAML inline configuration:

Example:

# In your SkyPilot YAML
config:
  docker:
    run_options: ...
  kubernetes:
    pod_config: ...
    provision_timeout: ...
  gcp:
    managed_instance_group: ...
  nvidia_gpus:
    disable_ecc: ...

CLI flag#

You can pass configuration arguments to the CLI using the --config flag.

The --config flag can either be a path to a config YAML file, or a dotlist of key-value pairs. If passing in a config file, only one --config flag can be provided.

Example:

# pass a config file
sky launch --config my_config.yaml ...
# pass individual config options
sky launch --config 'kubernetes.provision_timeout=600' --config 'kubernetes.pod_config.spec.priorityClassName=high-priority' ...
sky launch --config 'kubernetes.custom_metadata.annotations.myannotation1=myvalue1' --config 'kubernetes.custom_metadata.annotations.myannotation2=myvalue2' ...

Configuration overrides#

If the same configuration field is specified in multiple configuration sources, configuration is overridden based on the following priority order:

CLI flag (highest priority)
SkyPilot YAML
Project configuration
User configuration
Server configuration (lowest priority)

Merging rules:

Lists are overridden by config sources with higher priority.
- Exception: lists in kubernetes.pod_config appended to the existing list.
Dictionaries are merged, with individual keys overridden by config sources with higher priority.

Override example#

If the following is configured in the user config file:

kubernetes:
  allowed_contexts: [context1, context2]
  provision_timeout: 600
aws:
  labels:
    map-migrated: my-value
    Owner: user-unique-name

And the following in the project config file:

# project config overrides user config
kubernetes:
  allowed_contexts: [context3, context4]
  provision_timeout: 300
aws:
  labels:
    Owner: project-unique-name

The combined configuration is:

kubernetes:
  # lists are overridden by config sources with higher priority
  allowed_contexts: [context3, context4]
  provision_timeout: 300
  aws:
    # dicts are merged, with individual keys overridden by
    # config sources with higher priority
    labels:
      map-migrated: my-value
      Owner: project-unique-name