Source: examples/stable_diffusion

Stable Diffusion#

Setup#

  1. Install skypilot package by following these instructions.

  2. Run git clone https://github.com/skypilot-org/skypilot.git && cd examples/stable_diffusion

  3. Run sky launch -c stable-diffusion stable_diffusion_docker.yaml

  4. Run ssh -L 7860:localhost:7860 stable-diffusion

  5. Open http://localhost:7860/ in browser. If the page doesn’t load, try again in a few minutes to allow the container to start.

  6. Type in text prompt and click “Generate”.

Stable Diffusion Web Tool UI

  1. Once you are done, run sky stop stable-diffusion to stop the VM.

  2. To restart VM, repeat steps 3 and 4.

Usage Tips#

  • Avoid exceeding 900x900 for image resolution due GPU memory constraints

  • You can toggle “Classifier Free Guidance Scale” to higher value to enforce adherence to prompt

  • Here are some good example text prompts (Classifier Free Guidance Scale = 7.5, sampling steps = 50):

    • “donkey playing poker”

    • “UC Berkeley student writing code on a laptop”

    • “Marvel vs. DC”

    • “corgi on Golden Gate Bridge”

    • “desert golf”

    • “Indian McDonald’s”

    • “Elon Musk robot”

    • “mechanical heart”

    • “Batman in San Francisco”

    • “futuristic city in the sky and clouds”

    • “bear ballroom dancing”

    • “wall-e terminator”

    • “psychedelic Yosemite”

    • “rap song album cover”

    • “Wall Street bull rodeo”

    • “Trump in minecraft”

Included files#

Dockerfile

# Note: this file is adapted from https://github.com/AbdBarho/stable-diffusion-webui-docker/blob/master/build/Dockerfile 
# <<EOF heredoc was removed from RUN git clone https://github.com/hlky/sd-enable-textual-inversion.git line
# This was to fix an error when the instance parses the Dockerfile.

# syntax=docker/dockerfile:1

FROM continuumio/miniconda3:4.12.0


RUN conda install python=3.8.5 && conda clean -a -y
RUN conda install pytorch==1.11.0 torchvision==0.12.0 cudatoolkit=11.3 -c pytorch && conda clean -a -y
RUN git clone https://github.com/hlky/stable-diffusion.git && cd stable-diffusion && git reset --hard ff8c2d0b709f1e4180fb19fa5c27ec28c414cedd
RUN conda env update --file stable-diffusion/environment.yaml --name base && conda clean -a -y

# fonts for generating the grid
RUN apt-get update && apt install fonts-dejavu-core rsync -y && apt-get clean

# Note: don't update the sha of previous versions because the install will take forever
# instead, update the repo state in a later step
RUN cd stable-diffusion && git pull && git reset --hard c5b2c86f1479dec75b0e92dd37f9357a68594bda && \
  conda env update --file environment.yaml --name base && conda clean -a -y

# Textual-inversion:
RUN git clone https://github.com/hlky/sd-enable-textual-inversion.git && \
  cd /sd-enable-textual-inversion && git reset --hard 08f9b5046552d17cf7327b30a98410222741b070 && \
  rsync -a /sd-enable-textual-inversion/ /stable-diffusion/

# add info
COPY info.py /info.py
RUN  python /info.py /stable-diffusion/frontend/frontend.py

WORKDIR /stable-diffusion
ENV TRANSFORMERS_CACHE=/cache/transformers TORCH_HOME=/cache/torch CLI_ARGS="" \
  GFPGAN_PATH=/stable-diffusion/src/gfpgan/experiments/pretrained_models/GFPGANv1.3.pth \
  RealESRGAN_PATH=/stable-diffusion/src/realesrgan/experiments/pretrained_models/RealESRGAN_x4plus.pth \
  RealESRGAN_ANIME_PATH=/stable-diffusion/src/realesrgan/experiments/pretrained_models/RealESRGAN_x4plus_anime_6B.pth
EXPOSE 7860
CMD \
  for path in "${GFPGAN_PATH}" "${RealESRGAN_PATH}" "${RealESRGAN_ANIME_PATH}"; do \
  name=$(basename "${path}"); \
  base=$(dirname "${path}"); \
  test -f "/models/${name}" && mkdir -p "${base}" && ln -sf "/models/${name}" "${path}" && echo "Mounted ${name}";\
  done;\
  # force facexlib cache
  mkdir -p /cache/weights/ && rm -rf /stable-diffusion/src/facexlib/facexlib/weights && \
  ln -sf  /cache/weights/ /stable-diffusion/src/facexlib/facexlib/ && \
  # run, -u to not buffer stdout / stderr
  python3 -u scripts/webui.py --outdir /output --ckpt /models/model.ckpt --save-metadata ${CLI_ARGS}

docker-compose.yml

version: '3.9'

services:
  model:
    image: berkeleyskypilot/stable-diffusion
    restart: on-failure
    ports:
      - "7860:7860"
    volumes:
      - ./cache:/cache
      - ./output:/output
      - ./models:/models
    environment:
      - CLI_ARGS=--extra-models-cpu --optimized-turbo
    deploy:
      resources:
        reservations:
          devices:
              - driver: nvidia
                device_ids: ['0']
                capabilities: [gpu]

pushing_docker_image.md

Pushing Docker Image

GCR

  1. Install skypilot package by following these instructions.

  2. Run git clone https://github.com/skypilot-org/skypilot.git .

  3. Steps 4-6 are based on these GCR setup instructions

  4. First enable container registry in your project

  5. Next follow these steps to install and authenticate Docker on your CLI

  6. Ensure that you have Storage Admin permission in your project

  7. Run cd skypilot/examples/stable_diffusion and docker build Dockerfile

  8. Run docker tag stable-diffusion-webui-docker_model gcr.io/(my-project)/stable-diffusion.

  9. Run docker push gcr.io/(my-project)/stable-diffusion and verify on GCR that the image is there.

DockerHub

  1. Make a free DockerHub account.

  2. In your CLI, run docker login and use the login credentials from your DockerHub account.

  3. Run cd skypilot/examples/stable_diffusion and docker build Dockerfile

  4. Run docker tag stable-diffusion-webui-docker_model (my-dockerhub-username)/stable-diffusion.

  5. Run docker push (my-dockerhub-username)/stable-diffusion and verify on Dockerhub that the image is there.

stable_diffusion_docker.yaml

#SkyPilot YAML to run stable diffusion web tool on 1 V100 GPU.

resources:
  accelerators: V100:1

file_mounts:
  /stable_diffusion: .

setup: |
  cd stable-diffusion-webui-docker  
  sudo rm -r stable-diffusion-webui-docker
  git clone https://github.com/AbdBarho/stable-diffusion-webui-docker.git
  cd stable-diffusion-webui-docker
  git reset --hard 0d8b7d4ac8f9ba99e041ca332547eab9d65e6360
  wget https://huggingface.co/CompVis/stable-diffusion-v-1-4-original/resolve/main/sd-v1-4.ckpt -P models
  mv models/sd-v1-4.ckpt models/model.ckpt
  docker pull berkeleyskypilot/stable-diffusion

run: |
  cd stable-diffusion-webui-docker
  docker run -d \
    --name model \
    --restart on-failure \
    -p 7860:7860 \
    -v $(pwd)/cache:/cache \
    -v $(pwd)/output:/output \
    -v $(pwd)/models:/models \
    -e CLI_ARGS='--extra-models-cpu --optimized-turbo' \
    --gpus all \
    berkeleyskypilot/stable-diffusion