Exposing Services on Kubernetes#

Note

This is a guide on how to configure an existing Kubernetes cluster (along with the caveats involved) to successfully expose ports and services externally through SkyPilot.

If you are a SkyPilot user and your cluster has already been set up to expose ports, Opening Ports explains how to expose services in your task through SkyPilot.

SkyServe and SkyPilot clusters can open ports to expose services. For SkyPilot clusters running on Kubernetes, we support either of two modes to expose ports:

LoadBalancer Service (default)
Nginx Ingress

By default, SkyPilot creates a LoadBalancer Service on your Kubernetes cluster to expose the port.

If your cluster does not support LoadBalancer services, SkyPilot can also use an existing Nginx IngressController to create an Ingress to expose your service.

LoadBalancer Service#

This mode exposes ports through a Kubernetes LoadBalancer Service. This is the default mode used by SkyPilot.

To use this mode, you must have a Kubernetes cluster that supports LoadBalancer Services:

On Google GKE, Amazon EKS or other cloud-hosted Kubernetes services, this mode is supported out of the box and no additional configuration is needed.
On bare metal and self-managed Kubernetes clusters, MetalLB can be used to support LoadBalancer Services.

When using this mode, SkyPilot will create a single LoadBalancer Service for all ports that you expose on a cluster. Each port can be accessed using the LoadBalancer’s external IP address and the port number. Use sky status --endpoints <cluster> to view the external endpoints for all ports.

In cloud based Kubernetes clusters, this will automatically create an external Load Balancer. GKE creates a Pass-through Load Balancer and AWS creates a Network Load Balancer. These load balancers will be automatically terminated when the cluster is deleted.

Note

LoadBalancer services are not supported on kind clusters created using sky local up.

Note

The default LoadBalancer implementation in EKS selects a random port from the list of opened ports for the LoadBalancer’s health check. This can cause issues if the selected port does not have a service running behind it.

For example, if a SkyPilot task exposes 5 ports but only 2 of them have services running behind them, EKS may select a port that does not have a service running behind it and the LoadBalancer will not pass the healthcheck. As a result, the service will not be assigned an external IP address.

To work around this issue, make sure all your ports have services running behind them.

Internal Load Balancers#

To restrict your services to be accessible only within the cluster, you can set all SkyPilot services to use internal load balancers.

Depending on your cloud, set the appropriate annotation in the SkyPilot config file (~/.sky/config.yaml):

# ~/.sky/config.yaml
kubernetes:
  custom_metadata:
    annotations:
      # For GCP/GKE
      networking.gke.io/load-balancer-type: "Internal"
      # For AWS/EKS
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"
      # For Azure/AKS
      service.beta.kubernetes.io/azure-load-balancer-internal: "true"

Nginx Ingress#

This mode exposes ports by creating a Kubernetes Ingress backed by an existing Nginx Ingress Controller.

To use this mode:

Install the Nginx Ingress Controller on your Kubernetes cluster. Refer to the documentation for installation instructions specific to your environment.
Verify that the ingress-nginx-controller service has a valid external IP:

$ kubectl get service ingress-nginx-controller -n ingress-nginx

# Example output:
# NAME                             TYPE                CLUSTER-IP    EXTERNAL-IP     PORT(S)
# ingress-nginx-controller         LoadBalancer        10.24.4.254   35.202.58.117   80:31253/TCP,443:32699/TCP

Note

If the EXTERNAL-IP field is <none>, you can manually specify the Ingress IP or hostname through the skypilot.co/external-ip annotation on the ingress-nginx-controller service. In this case, having a valid EXTERNAL-IP field is not required.

For example, if your ingress-nginx-controller service is NodePort:

# Add skypilot.co/external-ip annotation to the nginx ingress service.
# Replace <IP> in the following command with the IP you select.
# Can be any node's IP if using NodePort service type.
$ kubectl annotate service ingress-nginx-controller skypilot.co/external-ip=<IP> -n ingress-nginx

If the EXTERNAL-IP field is <none> and the skypilot.co/external-ip annotation does not exist, SkyPilot will use localhost as the external IP for the Ingress, and the endpoint may not be accessible from outside the cluster.

Update the SkyPilot config at ~/.sky/config to use the ingress mode.

kubernetes:
  ports: ingress

Tip

For RKE2 and K3s, the pre-installed Nginx ingress is not correctly configured by default. Follow the bare-metal installation instructions to set up the Nginx ingress controller correctly.

When using this mode, SkyPilot creates an ingress resource and a ClusterIP service for each port opened. The port can be accessed externally by using the Ingress URL plus a path prefix of the form /skypilot/{pod_name}/{port}.

Use sky status --endpoints <cluster> to view the full endpoint URLs for all ports.

$ sky status --endpoints mycluster
8888: http://34.173.152.251/skypilot/test-2ea4/8888

Note

When exposing a port under a sub-path such as an ingress, services expecting root path access, (e.g., Jupyter notebooks) may face issues. To resolve this, configure the service to operate under a different base URL. For Jupyter, use –NotebookApp.base_url flag during launch. Alternatively, consider using LoadBalancer mode.