Image as Volumes

Learn how to use container images as volumes in Kubernetes v1.33, a powerful feature that allows you to mount OCI images directly as volumes in your pods.

Overview

Image as Volumes is a feature introduced in Kubernetes v1.33 that allows you to mount container images directly as volumes. This is particularly useful for:

• Immutable artifacts: Distribute read-only data, configurations, or binaries packaged as container images
• AI/ML models: Package and deploy machine learning models as container images
• Static assets: Serve static content like web assets, documentation, or datasets
• Configuration bundles: Deploy complex configurations as versioned images

Key Benefits

📦 Unified Distribution

• Use the same container registry infrastructure for both applications and data
• Leverage existing image pull secrets and access controls

🔒 Security & Immutability

• Content-addressable storage ensures data integrity
• Images are pulled and verified using standard registry authentication
• Read-only by default, preventing accidental modifications

🚀 Version Control

• Tag and version your data assets just like application images
• Easy rollbacks and updates using image tags
• Leverage container registry features like image scanning and lifecycle policies

⚡ Efficient Caching

• Images are cached on nodes, reducing pull times for subsequent pods
• Layer-based caching similar to container images

Prerequisites

Before you begin, ensure you have:

• Kubernetes cluster version 1.33 or later
• kubectl configured to communicate with your cluster
• Access to a container registry (Docker Hub, GCR, ECR, etc.)
• Basic understanding of Kubernetes pods and volumes

Verify Feature Availability

First, verify that your Kubernetes cluster supports the Image as Volumes feature:

# Check Kubernetes version
kubectl version --short

# Verify the feature gate is enabled
kubectl get --raw /metrics | grep image_volume

The feature should be enabled by default in v1.33+, but if you're running an earlier version with feature gates, ensure ImageVolumes=true is set.

Creating an Image to Use as a Volume

Step 1: Prepare Your Content

Create a directory with the content you want to distribute:

mkdir -p mydata
echo "Hello from image volume!" > mydata/greeting.txt
echo "version: 1.0.0" > mydata/config.yaml

Step 2: Create a Dockerfile

Create a simple Dockerfile that copies your content:

FROM scratch
COPY mydata /data

This creates a minimal image containing only your data files.

Step 3: Build and Push the Image

Build and push your image to a container registry:

# Build the image
docker build -t myregistry.com/myapp/data:v1.0.0 .

# Push to registry
docker push myregistry.com/myapp/data:v1.0.0

For a complete example with an ML model:

FROM scratch
COPY model.pkl /models/
COPY config.json /models/

docker build -t myregistry.com/ml/model:v2.1.0 .
docker push myregistry.com/ml/model:v2.1.0

Using Image as Volume in Pods

Basic Example

Here's a simple pod that mounts an image as a volume:

apiVersion: v1
kind: Pod
metadata:
  name: image-volume-demo
spec:
  containers:
  - name: app
    image: nginx:alpine
    volumeMounts:
    - name: data-volume
      mountPath: /usr/share/nginx/html
      readOnly: true
  volumes:
  - name: data-volume
    image:
      reference: myregistry.com/myapp/data:v1.0.0

Apply the pod:

kubectl apply -f pod.yaml

Verify the volume is mounted:

# Check pod status
kubectl get pod image-volume-demo

# Verify content
kubectl exec image-volume-demo -- ls -la /usr/share/nginx/html
kubectl exec image-volume-demo -- cat /usr/share/nginx/html/greeting.txt

Using Image Pull Secrets

If your image is in a private registry, specify pull secrets:

apiVersion: v1
kind: Pod
metadata:
  name: private-image-volume
spec:
  containers:
  - name: app
    image: myapp:latest
    volumeMounts:
    - name: private-data
      mountPath: /data
      readOnly: true
  volumes:
  - name: private-data
    image:
      reference: private-registry.com/data:v1.0.0
      pullSecret:
        name: registry-credentials

Create the pull secret first:

kubectl create secret docker-registry registry-credentials \
  --docker-server=private-registry.com \
  --docker-username=myuser \
  --docker-password=mypassword \
  --docker-email=myemail@example.com

Multiple Image Volumes

You can mount multiple images in the same pod:

apiVersion: v1
kind: Pod
metadata:
  name: multi-image-volumes
spec:
  containers:
  - name: ml-app
    image: python:3.11-slim
    command: ["python", "/app/inference.py"]
    volumeMounts:
    - name: model-volume
      mountPath: /models
      readOnly: true
    - name: config-volume
      mountPath: /config
      readOnly: true
    - name: app-code
      mountPath: /app
      readOnly: true
  volumes:
  - name: model-volume
    image:
      reference: myregistry.com/ml/model:v2.1.0
  - name: config-volume
    image:
      reference: myregistry.com/ml/config:latest
  - name: app-code
    image:
      reference: myregistry.com/ml/app:v1.0.0

Use Cases

Use Case 1: AI/ML Model Deployment

Package machine learning models as container images for easy distribution:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ml-inference
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ml-inference
  template:
    metadata:
      labels:
        app: ml-inference
    spec:
      containers:
      - name: inference-server
        image: tensorflow/serving:latest
        ports:
        - containerPort: 8501
        volumeMounts:
        - name: model
          mountPath: /models/mymodel
          readOnly: true
        env:
        - name: MODEL_BASE_PATH
          value: /models
        - name: MODEL_NAME
          value: mymodel
      volumes:
      - name: model
        image:
          reference: myregistry.com/models/sentiment-analysis:v3.2.0

Use Case 2: Configuration Management

Distribute complex configuration bundles:

apiVersion: v1
kind: Pod
metadata:
  name: app-with-configs
spec:
  containers:
  - name: app
    image: myapp:latest
    volumeMounts:
    - name: configs
      mountPath: /etc/config
      readOnly: true
  volumes:
  - name: configs
    image:
      reference: myregistry.com/configs/prod:v2024.10

Use Case 3: Static Website Content

Serve static website content from container images:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: website
spec:
  replicas: 2
  selector:
    matchLabels:
      app: website
  template:
    metadata:
      labels:
        app: website
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
        volumeMounts:
        - name: website-content
          mountPath: /usr/share/nginx/html
          readOnly: true
      volumes:
      - name: website-content
        image:
          reference: myregistry.com/website/static:v2024.10.18

Use Case 4: Shared Libraries and Binaries

Distribute compiled binaries or shared libraries:

apiVersion: v1
kind: Pod
metadata:
  name: app-with-libraries
spec:
  containers:
  - name: app
    image: ubuntu:22.04
    command: ["/shared-libs/myapp"]
    volumeMounts:
    - name: shared-libs
      mountPath: /shared-libs
      readOnly: true
    env:
    - name: LD_LIBRARY_PATH
      value: /shared-libs/lib
  volumes:
  - name: shared-libs
    image:
      reference: myregistry.com/libs/opencv:v4.8.0

Best Practices

1. Use Semantic Versioning

Tag your images with semantic versions for better tracking:

docker tag mydata:latest myregistry.com/data:v1.2.3
docker push myregistry.com/data:v1.2.3

2. Keep Images Small

Use multi-stage builds or FROM scratch to minimize image size:

# Build stage
FROM alpine:latest AS builder
COPY source/ /build/
RUN process-data.sh

# Final minimal image
FROM scratch
COPY --from=builder /build/output /data

3. Leverage Image Digests

For production, use image digests for immutability:

volumes:
- name: critical-data
  image:
    reference: myregistry.com/data@sha256:abc123...

4. Implement Proper Access Controls

Use RBAC and image pull secrets appropriately:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa
imagePullSecrets:
- name: registry-secret
---
apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  serviceAccountName: app-sa
  containers:
  - name: app
    image: myapp:latest
    volumeMounts:
    - name: sensitive-data
      mountPath: /secrets
  volumes:
  - name: sensitive-data
    image:
      reference: private-registry.com/secrets:latest

5. Monitor Image Pull Performance

Large images can impact pod startup time. Monitor and optimize:

# Check image pull metrics
kubectl get events --sort-by='.lastTimestamp' | grep -i pull

# Check pod startup time
kubectl describe pod <pod-name> | grep -A 5 "Events:"

6. Use Image Pull Policies Wisely

Configure appropriate pull policies:

volumes:
- name: cached-data
  image:
    reference: myregistry.com/data:v1.0.0
    pullPolicy: IfNotPresent  # Default, uses cached image if available

Available options:

• IfNotPresent: Pull only if not cached locally (default)
• Always: Always pull the latest version
• Never: Never pull, fail if not cached

Troubleshooting

Issue 1: Image Pull Failures

Symptom: Pod stuck in ImagePullBackOff or ErrImagePull

Solutions:

# Check pod events
kubectl describe pod <pod-name>

# Verify image exists
docker pull myregistry.com/data:v1.0.0

# Check pull secret
kubectl get secret registry-credentials -o yaml

# Test registry connectivity
kubectl run test --image=myregistry.com/data:v1.0.0 --restart=Never

Issue 2: Mount Path Conflicts

Symptom: Volume not mounting correctly or application errors

Solution: Ensure mount paths don't conflict:

# Correct: Different mount paths
volumeMounts:
- name: data1
  mountPath: /data/source1
- name: data2
  mountPath: /data/source2

Issue 3: Read-Only Violations

Symptom: Application fails trying to write to image volume

Solution: Image volumes are read-only by default. If you need writable storage, use an emptyDir overlay:

volumeMounts:
- name: data-overlay
  mountPath: /data
volumes:
- name: base-data
  image:
    reference: myregistry.com/data:v1.0.0
- name: data-overlay
  emptyDir: {}

Issue 4: Slow Pod Startup

Symptom: Pods take a long time to start

Solutions:

1. Reduce image size:

   FROM scratch
   # Only include necessary files
   COPY --from=builder /output /data

2. Pre-pull images on nodes:

   # Use DaemonSet to pre-pull on all nodes
   kubectl create -f prepull-daemonset.yaml

3. Use image digests for faster verification:

   reference: myregistry.com/data@sha256:abc123...

Comparison with ConfigMaps and Secrets

Feature	Image Volumes	ConfigMaps	Secrets
Size Limit	Limited by registry	1 MB	1 MB
Versioning	Native (tags/digests)	Manual	Manual
Distribution	Registry	API Server	API Server
Caching	Node-level	etcd	etcd
Best For	Large datasets, binaries	Small configs	Sensitive data
Immutability	Strong	Optional	Optional

Advanced Configuration

Setting Resource Limits for Image Pulls

Control resource usage during image pulls:

apiVersion: v1
kind: Pod
metadata:
  name: resource-limited
spec:
  containers:
  - name: app
    image: myapp:latest
    resources:
      limits:
        ephemeral-storage: 2Gi
    volumeMounts:
    - name: large-dataset
      mountPath: /data
  volumes:
  - name: large-dataset
    image:
      reference: myregistry.com/datasets/large:v1.0.0

Using with Init Containers

Process image content during initialization:

apiVersion: v1
kind: Pod
metadata:
  name: init-container-example
spec:
  initContainers:
  - name: setup
    image: busybox:latest
    command: ['sh', '-c', 'cp /source/* /destination/']
    volumeMounts:
    - name: source-data
      mountPath: /source
      readOnly: true
    - name: working-data
      mountPath: /destination
  containers:
  - name: app
    image: myapp:latest
    volumeMounts:
    - name: working-data
      mountPath: /data
  volumes:
  - name: source-data
    image:
      reference: myregistry.com/data:v1.0.0
  - name: working-data
    emptyDir: {}

Migration from Other Volume Types

From ConfigMap to Image Volume

Before:

volumes:
- name: config
  configMap:
    name: app-config

After:

# Create image from ConfigMap data
kubectl get configmap app-config -o jsonpath='{.data}' > config-data.txt
# Build image with this data
docker build -t myregistry.com/config:v1 .

volumes:
- name: config
  image:
    reference: myregistry.com/config:v1

From hostPath to Image Volume

Before:

volumes:
- name: data
  hostPath:
    path: /data/files

After:

# Package hostPath content as image
docker build -t myregistry.com/data:v1 /data/files

volumes:
- name: data
  image:
    reference: myregistry.com/data:v1

Security Considerations

1. Image Scanning

Always scan images for vulnerabilities:

# Using Trivy
trivy image myregistry.com/data:v1.0.0

# Using Docker Scout
docker scout cves myregistry.com/data:v1.0.0

2. Registry Security

Implement proper registry security:

• Use TLS for registry connections
• Enable image signing and verification
• Implement registry access controls
• Audit image pull activities

3. Runtime Security

Apply security contexts:

apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 2000
  containers:
  - name: app
    image: myapp:latest
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
    volumeMounts:
    - name: data
      mountPath: /data
      readOnly: true
  volumes:
  - name: data
    image:
      reference: myregistry.com/data:v1.0.0

Monitoring and Observability

Tracking Image Volume Usage

Monitor image volume metrics:

# Get pod metrics
kubectl top pod <pod-name>

# Check volume usage
kubectl exec <pod-name> -- df -h /data

# Monitor image pull events
kubectl get events --field-selector involvedObject.kind=Pod --watch

Prometheus Metrics

Key metrics to monitor:

• kubelet_image_pull_duration_seconds: Image pull duration
• kubelet_image_volume_size_bytes: Size of image volumes
• kubelet_image_pull_errors_total: Failed image pulls

Resources

• Kubernetes Documentation: Image Volumes
• KEP-4639: Image Volumes Proposal
• OCI Image Spec: https://github.com/opencontainers/image-spec
• Container Registry Options: Docker Hub, Google Container Registry, Amazon ECR, Azure Container Registry

Conclusion

Image as Volumes is a powerful feature in Kubernetes v1.33 that simplifies the distribution of data, models, and configurations using the familiar container image format. By leveraging existing container registry infrastructure, you can:

• Distribute large datasets efficiently
• Version control your data assets
• Implement strong immutability guarantees
• Leverage existing security and access controls

Start experimenting with Image as Volumes in your development clusters before rolling out to production, and always follow security best practices when working with container images.