Tech Bytes Logo

Tech Bytes Engineering

October 13, 2025

SECURITY DEEP DIVE

Kubernetes Security Hardening: 10 Critical Steps to Secure Your Cluster

From zero to production-ready security: RBAC, Pod Security Standards, Network Policies, and runtime protection with battle-tested configurations

10 Security Layers Production Tested Copy-Paste Ready
Dillip Chowdary

Dillip Chowdary

Senior DevOps Engineer • 6+ years Security & K8s expertise

Wake-Up Call

Real incident: A misconfigured Kubernetes cluster at a Fortune 500 company led to exposed customer data for 72 hours. The attack vector? Default RBAC permissions and no network policies.

Cost: $4.2M in fines + immeasurable reputation damage.

Kubernetes security isn't optional—it's critical infrastructure. After securing dozens of production clusters and passing SOC 2 and ISO 27001 audits, I've distilled the essential security practices into 10 actionable steps that every team should implement.

This isn't a theoretical guide. These are production-tested configurations currently protecting clusters handling millions of requests per day. Let's get started.

10 Security Layers

RBAC & Service Accounts

Least privilege access control

Pod Security Standards

Restrict dangerous pod configurations

Network Policies

Zero-trust pod communication

Secrets Management

External secrets & encryption

Image Security

Scanning & signing

Runtime Security

Falco & behavioral monitoring

Audit Logging

Comprehensive activity tracking

API Server Security

Hardening control plane

etcd Encryption

Secrets at rest encryption

Compliance & Scanning

CIS benchmarks & vulnerability scans

Step 1: Implement Strict RBAC

Problem: Default Kubernetes gives service accounts overly broad permissions. Attackers exploiting a pod can escalate privileges cluster-wide.

Create Least-Privilege Service Account YAML
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-reader
  namespace: production
automountServiceAccountToken: false  # Disable auto-mounting

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: app-reader-role
  namespace: production
rules:
  # Only allow reading ConfigMaps
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list"]

  # Only allow reading Secrets (specific names)
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["app-config", "db-credentials"]
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: app-reader-binding
  namespace: production
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: app-reader-role
subjects:
  - kind: ServiceAccount
    name: app-reader
    namespace: production

RBAC Best Practices

  • Use automountServiceAccountToken: false by default
  • Create role-specific service accounts (no shared accounts)
  • Scope roles to specific namespaces (avoid ClusterRoles unless necessary)
  • Use resourceNames to restrict access to specific resources
  • Never grant * permissions in production
Audit Existing RBAC Permissions Bash
# List all service accounts with cluster-admin rights (dangerous!)
kubectl get clusterrolebindings -o json | jq -r '
  .items[] |
  select(.roleRef.name=="cluster-admin") |
  .metadata.name
'

# Find service accounts with wildcard permissions
kubectl get roles,clusterroles --all-namespaces -o json | jq -r '
  .items[] |
  select(.rules[]?.verbs[]? == "*") |
  "\(.metadata.namespace // "cluster")/\(.metadata.name)"
'

# List all service accounts and their bound roles
kubectl get rolebindings,clusterrolebindings --all-namespaces \
  -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,\
ROLE:.roleRef.name,SUBJECT:.subjects[0].name

Step 2: Enforce Pod Security Standards

Pod Security Standards (PSS) replaced Pod Security Policies in Kubernetes 1.25. Three enforcement levels: privileged, baseline, and restricted.

Enforce Restricted PSS at Namespace Level YAML
apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    # Enforce restricted standard (most secure)
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest

    # Audit mode for monitoring violations
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/audit-version: latest

    # Warn mode for user feedback
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: latest

Restricted PSS Requirements

The restricted standard enforces:

  • No privileged containers
  • No host namespaces (network, PID, IPC)
  • No host ports
  • Must drop ALL capabilities
  • Must run as non-root user
  • Read-only root filesystem recommended
  • No privilege escalation
Compliant Pod Configuration YAML
apiVersion: v1
kind: Pod
metadata:
  name: secure-app
  namespace: production
spec:
  serviceAccountName: app-reader  # Dedicated SA
  automountServiceAccountToken: false  # Explicit disable

  securityContext:
    runAsNonRoot: true
    runAsUser: 10000
    runAsGroup: 10000
    fsGroup: 10000
    seccompProfile:
      type: RuntimeDefault  # Seccomp for syscall filtering

  containers:
    - name: app
      image: myapp:1.2.3

      securityContext:
        allowPrivilegeEscalation: false
        runAsNonRoot: true
        runAsUser: 10000
        capabilities:
          drop:
            - ALL  # Drop all Linux capabilities
        readOnlyRootFilesystem: true  # Immutable filesystem

      resources:
        limits:
          cpu: "1000m"
          memory: "512Mi"
        requests:
          cpu: "100m"
          memory: "128Mi"

      volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: cache
          mountPath: /app/cache

  volumes:
    - name: tmp
      emptyDir: {}  # Writable /tmp
    - name: cache
      emptyDir: {}  # Writable cache dir

Step 3: Implement Network Policies

By default, all pods in a Kubernetes cluster can talk to each other. Network Policies implement zero-trust networking at the pod level.

Default Deny All Traffic YAML
---
# Deny all ingress traffic by default
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production
spec:
  podSelector: {}  # Apply to all pods
  policyTypes:
    - Ingress

---
# Deny all egress traffic by default
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-egress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Egress
Allow Specific Traffic Only YAML
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-server-network-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api-server

  policyTypes:
    - Ingress
    - Egress

  # Allowed incoming traffic
  ingress:
    # Allow traffic from ingress controller only
    - from:
        - namespaceSelector:
            matchLabels:
              name: ingress-nginx
        - podSelector:
            matchLabels:
              app: nginx-ingress
      ports:
        - protocol: TCP
          port: 8080

  # Allowed outgoing traffic
  egress:
    # Allow DNS lookups
    - to:
        - namespaceSelector:
            matchLabels:
              name: kube-system
        - podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53

    # Allow database access
    - to:
        - podSelector:
            matchLabels:
              app: postgres
      ports:
        - protocol: TCP
          port: 5432

    # Allow external API calls (specific CIDR)
    - to:
        - ipBlock:
            cidr: 52.1.2.0/24  # External API IP range
      ports:
        - protocol: TCP
          port: 443

Testing Network Policies

Always test before enforcing:

# Test connectivity from pod to pod
kubectl exec -it api-server-pod -- curl http://postgres-service:5432

# Expected: Connection allowed (policy permits it)

kubectl exec -it api-server-pod -- curl http://redis-service:6379

# Expected: Connection refused (no policy allows it)

Step 4: Proper Secrets Management

Never store secrets in Kubernetes Secrets without encryption. Use External Secrets Operator + AWS Secrets Manager/HashiCorp Vault.

External Secrets Operator with AWS Secrets Manager YAML
# Install External Secrets Operator first:
# helm repo add external-secrets https://charts.external-secrets.io
# helm install external-secrets external-secrets/external-secrets -n external-secrets-system --create-namespace

---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: aws-secretsmanager
  namespace: production
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: external-secrets-sa  # SA with IAM role

---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: app-database-credentials
  namespace: production
spec:
  refreshInterval: 1h  # Sync every hour

  secretStoreRef:
    name: aws-secretsmanager
    kind: SecretStore

  target:
    name: db-credentials  # Creates this K8s Secret
    creationPolicy: Owner

  data:
    - secretKey: username
      remoteRef:
        key: prod/database/credentials
        property: username

    - secretKey: password
      remoteRef:
        key: prod/database/credentials
        property: password

    - secretKey: connection-string
      remoteRef:
        key: prod/database/credentials
        property: connection_string

Secrets Best Practices

  • Store secrets in external vaults (AWS Secrets Manager, Vault)
  • Enable etcd encryption at rest (see Step 9)
  • Rotate secrets automatically (use ESO refresh)
  • Audit secret access with RBAC logs
  • Never commit secrets to Git
  • Never use Secrets as environment variables (prefer mounted files)

Step 5: Container Image Security

Image Scanning with Trivy

# Scan image for vulnerabilities
trivy image --severity HIGH,CRITICAL myapp:1.2.3

# Scan and fail CI if HIGH+ found
trivy image --exit-code 1 \
  --severity HIGH,CRITICAL \
  myapp:1.2.3

Image Signing with Cosign

# Sign image
cosign sign --key cosign.key \
  myregistry.io/myapp:1.2.3

# Verify signature before deployment
cosign verify --key cosign.pub \
  myregistry.io/myapp:1.2.3
Enforce Image Policies with OPA Gatekeeper YAML
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8sallowedrepos
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedRepos
      validation:
        openAPIV3Schema:
          properties:
            repos:
              type: array
              items:
                type: string

  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sallowedrepos

        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          not startswith(container.image, input.parameters.repos[_])
          msg := sprintf("Image %v not from approved registry", [container.image])
        }

---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
  name: allowed-docker-registries
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    namespaces:
      - production
  parameters:
    repos:
      - "myregistry.io/"
      - "public.ecr.aws/mycompany/"

Quick Wins: Do These Now

5 Minutes

  • Enable PSS restricted mode on namespace
  • Add default deny network policies
  • Set automountServiceAccountToken: false

30 Minutes

  • Audit existing RBAC permissions
  • Integrate Trivy image scanning in CI/CD
  • Deploy External Secrets Operator

Conclusion

Security is a journey, not a destination. These 10 steps provide a solid foundation, but remember:

  • Defense in depth: Multiple layers catch what one misses
  • Continuous improvement: Audit quarterly, patch promptly
  • Team awareness: Security is everyone's responsibility
  • Stay updated: K8s security evolves constantly

Master Kubernetes Security

Get weekly DevOps & security insights in your inbox