Welcome to SkyPilot!

Run AI on Any Infra — Unified, Faster, Cheaper

Star Watch Fork

SkyPilot is a framework for running AI and batch workloads on any infra, offering unified execution, high cost savings, and high GPU availability.

SkyPilot abstracts away infra burdens:

• Launch dev clusters, jobs, and serving on any infra

• Easy job management: queue, run, and auto-recover many jobs

SkyPilot supports multiple clusters, clouds, and hardware (the Sky):

• Bring your reserved GPUs, Kubernetes clusters, or 12+ clouds

• Flexible provisioning of GPUs, TPUs, CPUs, with auto-retry

SkyPilot cuts your cloud costs & maximizes GPU availability:

• Autostop: automatic cleanup of idle resources

• Managed Spot: 3-6x cost savings using spot instances, with preemption auto-recovery

• Optimizer: 2x cost savings by auto-picking the cheapest & most available infra

SkyPilot supports your existing GPU, TPU, and CPU workloads, with no code changes.

Current supported infra (Kubernetes; AWS, GCP, Azure, OCI, Lambda Cloud, Fluidstack, RunPod, Cudo, Paperspace, Cloudflare, Samsung, IBM, VMware vSphere):

Ready to get started?

Install SkyPilot in ~1 minute. Then, launch your first dev cluster in ~5 minutes in Quickstart.

Everything is launched within your cloud accounts, VPCs, and cluster(s).

Contact the SkyPilot team

You can chat with the SkyPilot team and community on the SkyPilot Slack.

Learn more

Runnable examples:

• LLMs on SkyPilot

  • Llama 3.2: lightweight and vision models

  • Pixtral

  • Llama 3.1 finetuning and serving

  • GPT-2 via llm.c

  • Llama 3

  • Qwen

  • Gemma

  • Mixtral 8x7B; Mistral 7B (from official Mistral team)

  • Code Llama

  • vLLM: Serving LLM 24x Faster On the Cloud (from official vLLM team)

  • SGLang: Fast and Expressive LLM Serving On the Cloud (from official SGLang team)

  • Vicuna chatbots: Training & Serving (from official Vicuna team)

  • Train your own Vicuna on Llama-2

  • Ollama: Quantized LLMs on CPUs

  • LoRAX

  • QLoRA

  • LLaMA-LoRA-Tuner

  • Tabby: Self-hosted AI coding assistant

  • LocalGPT

  • Add yours here & see more in llm/!

• Framework examples: PyTorch DDP, DeepSpeed, JAX/Flax on TPU, Stable Diffusion, Detectron2, Distributed TensorFlow, NeMo, programmatic grid search, Docker, Cog, Unsloth, Ollama, llm.c, Airflow and many more.

Case Studies and Integrations: Community Spotlights

Tutorials: SkyPilot Tutorials

Follow updates:

• Twitter

• Slack

• SkyPilot Blog (Introductory blog post)

Read the research:

• SkyPilot paper and talk (NSDI 2023)

• Sky Computing whitepaper

• Sky Computing vision paper (HotOS 2021)