Kubernetes 1.28 Release Notes: What Changed

Kubernetes 1.28, codenamed "Planternetes," was released in August 2023. This release focuses on planning for the future with improved job management, native sidecar support, and enhanced resource management.

Release Overview

Release Date: August 15, 2023
Codename: Planternetes
Theme: Planning for the future of Kubernetes

Major Features and Enhancements

1. Sidecar Containers (Alpha)

Native support for sidecar containers that automatically start and stop with the main container.

What Changed:

apiVersion: v1
kind: Pod
spec:
  containers:
  - name: main-app
    image: my-app:latest
  - name: sidecar
    image: sidecar-proxy:latest
    restartPolicy: Always  # Sidecar restart policy

Benefits:

• Automatic lifecycle management of sidecar containers
• Proper shutdown ordering (sidecars stop after main container)
• Better integration with service mesh and logging sidecars

Status: Alpha in 1.28, graduated to Beta in 1.29

2. Enhanced Job Management

Improvements to Job and CronJob APIs for better batch workload management.

New Features:

• Job Pod Failure Policy: Fine-grained control over job failure handling
• Pod Replacement Policy: Control when pods are replaced in jobs
• Backoff Limit Per Index: Per-index backoff limits for Indexed Jobs

Example:

apiVersion: batch/v1
kind: Job
metadata:
  name: indexed-job
spec:
  completions: 5
  parallelism: 2
  completionMode: Indexed
  podFailurePolicy:
    rules:
    - action: FailJob
      onExitCodes:
        containerName: main
        operator: In
        values: [1, 2, 3]

3. Node Resource Management Improvements

Better resource allocation and management at the node level.

Enhancements:

• Improved Topology Manager
• Better resource slice management
• Enhanced resource allocation reporting

4. Service Account Token Improvements

Enhanced security for service account tokens.

Changes:

• Better token management
• Improved security defaults
• Enhanced token rotation support

5. API Enhancements

New Stable APIs:

• Various APIs promoted from beta to stable
• Better validation and error messages
• Improved API consistency

Deprecations and Removals

Deprecated Features

1. Beta API Versions: Several beta APIs marked for deprecation

   • Check kubectl get --raw /apis for current API versions
   • Plan migration to stable APIs

2. Legacy Features: Some older features marked as deprecated

   • Will be removed in future releases (typically 2 releases notice)

Removed Features

Note: Features deprecated in previous releases have been removed in 1.28. Check release notes for specific removals.

Breaking Changes

API Version Changes

Some beta APIs removed. Update your manifests:

# Check for deprecated API usage
kubectl convert --help

# Review your resources
kubectl get all --all-namespaces -o yaml | grep -i "apiVersion"

Behavior Changes

1. Scheduler: Improved scheduling algorithms may result in different pod placement
2. Resource Allocation: Changes to resource management may affect resource limits
3. Security: Enhanced security defaults may require configuration updates

Upgrade Considerations

Before Upgrading

1. Review Deprecations: Check for any deprecated APIs or features you're using
2. Update Manifests: Migrate to stable API versions
3. Test Compatibility: Verify your applications work with 1.28
4. Update Tools: Ensure kubectl and other tools support 1.28

Upgrade Steps

# 1. Backup cluster configuration
kubectl get all --all-namespaces -o yaml > backup.yaml

# 2. Check current version
kubectl version --short

# 3. Update kubectl client (if using managed service, this may be automatic)
# Follow your cloud provider's upgrade process

# 4. Verify cluster health after upgrade
kubectl get nodes
kubectl get pods --all-namespaces

Post-Upgrade

1. Verify Applications: Ensure all workloads are running correctly
2. Check Logs: Review component logs for warnings or errors
3. Monitor Metrics: Watch cluster metrics for anomalies
4. Test Features: Verify any new features you plan to use

Notable Fixes and Improvements

Bug Fixes

• Fixed issues with pod scheduling under certain conditions
• Resolved resource allocation edge cases
• Fixed service account token handling
• Improved error handling in various components

Performance Improvements

• Faster API server response times
• Improved scheduler performance
• Better resource utilization
• Enhanced etcd performance

Security Enhancements

• Improved service account token security
• Enhanced RBAC performance
• Better secret management
• Improved network policy enforcement

Migration Guide

Migrating from 1.27 to 1.28

1. Update API Versions:

   # Find deprecated APIs
   kubectl get all --all-namespaces -o yaml | grep -E "apiVersion.*beta"
   
   # Update to stable versions
   # Example: apps/v1beta1 -> apps/v1
   

2. Update Sidecar Patterns:

   • If using sidecars, consider migrating to native sidecar support
   • Review restart policies for sidecar containers

3. Review Job Configurations:

   • Take advantage of new job management features
   • Update job failure policies if needed

Tool Compatibility

Ensure these tools support Kubernetes 1.28:

• kubectl 1.28+
• Helm 3.x (check specific version)
• Your CI/CD tools
• Monitoring and logging solutions

New kubectl Features

# Enhanced job management
kubectl get jobs -o wide
kubectl describe job <job-name>

# Better resource inspection
kubectl top nodes
kubectl top pods

# Improved debugging
kubectl debug <pod-name>

Best Practices for 1.28

1. Use Sidecar Support: Migrate to native sidecar containers where applicable
2. Leverage Job Improvements: Use new job management features for batch workloads
3. Stay on Stable APIs: Use stable API versions in all manifests
4. Monitor Deprecations: Regularly check for deprecated features
5. Test Before Production: Always test upgrades in non-production first

Related Resources

• Kubernetes Version History Overview
• Kubernetes 1.29 Release Notes
• Kubernetes Cluster Upgrades
• Kubernetes Best Practices

Conclusion

Kubernetes 1.28 introduces important features like native sidecar support and enhanced job management while maintaining stability and backward compatibility. Plan your upgrade carefully, test thoroughly, and take advantage of the new features to improve your Kubernetes operations.

Key takeaways:

• Native sidecar support (alpha) for better lifecycle management
• Enhanced job management capabilities
• Improved resource management
• Various API improvements and deprecations
• Focus on planning for future Kubernetes evolution