Running Kubernetes in Scope: Container Orchestration Practices That Pass a PCI-DSS Audit

Every new cluster you add to a payment platform is a new boundary question for your Cardholder Data Environment, whether or not anyone on the platform team thought of it that way. A namespace that shares a control plane with cardholder-data workloads, a shared ingress controller, a CNI that lets pods route to each other by default: each of these pulls infrastructure into scope that the security architecture diagram may not even show. Kubernetes makes this worse than a traditional VM estate because the abstractions move fast. A developer adds a sidecar, a platform engineer swaps a service mesh, and the CDE boundary has shifted without a change ticket that says so.

A dim data center aisle with rows of server racks, one group of racks enclosed in a locked wire mesh cage with a padlock and amber warning light, visually separating it from the open racks beside it, cool blue ambient lighting.

This matters because PCI-DSS scope is not defined by what you intend to isolate. It is defined by what can actually reach cardholder data, directly or through a connected system. A cluster running mixed workloads, where a marketing microservice and a payment-processing service share a control plane, puts the entire cluster in scope unless you can prove segmentation that an assessor will accept. That proof has to come from the orchestration layer itself, not from a network diagram someone updated for the audit.

Why Adding a Cluster Expands Your Cardholder Data Environment Scope

The core problem is that Kubernetes defaults toward flat networking and shared control planes, which is exactly what PCI-DSS Requirement 1 and the segmentation guidance ask you to avoid. Out of the box, any pod can resolve and reach any other pod’s service unless a NetworkPolicy says otherwise. The API server has a single set of credentials and a single audit log for every namespace it manages, cardholder-data workloads included. If you run a multi-tenant cluster without strict policy enforcement, you have built a flat network with better marketing.

Adding a cluster also adds a new identity boundary, a new secrets store, a new set of RBAC bindings, and a new CI/CD path that can reach production. Every one of those is a control an assessor will ask about. Treat each new cluster as a scope decision, not an infrastructure decision: decide before deployment whether it will host CDE workloads, document the segmentation controls that keep it isolated if it does not, and build those controls into the cluster’s configuration before the first workload lands, not after the first finding.

What a PCI-DSS Assessor Actually Wants From Your GitOps Pipeline

A QSA reviewing a Kubernetes-based payment platform is not impressed by a fast deployment pipeline. They are looking for an answer to one question: for any given state of the production cluster, can you show who approved that state, what it changed, and when it took effect, using records that nobody could have altered after the fact.

This is where most GitOps setups fall short, not because the workflow is weak but because the evidence trail was never designed as evidence. A pipeline that deploys from a Git repository is a good start, but if commits aren’t signed, if merges don’t require a second reviewer, or if the deployment tool can apply changes that never went through Git at all, you have a system that looks auditable and isn’t. An assessor will ask for the commit history, the approval record, and the actual cluster state side by side, and if those three don’t line up without manual reconciliation, that’s a finding.

The GitOps pipeline an assessor wants to see has these properties:

  • Every production change originates from a pull request with a recorded reviewer, not a direct kubectl apply.
  • The reconciliation tool, whether Argo CD, Flux, or similar, is configured to reject any manifest that didn’t come through the Git source of truth.
  • Commit signing is enforced at the repository level, not just recommended in a wiki page.
  • The pipeline’s own service accounts have no standing write access to production outside the reconciliation loop.

None of this is exotic. It’s the difference between a pipeline built for velocity and one built to produce a record a stranger can follow six months later.

Policy as Code: Enforcing CDE Segmentation at the Admission Controller

Network diagrams and firewall rules tell an assessor what you intended. Admission control tells them what the cluster actually enforces. This distinction matters because PCI-DSS Requirement 1 asks for evidence that segmentation controls are in place and operating, not just documented.

Tools like OPA Gatekeeper or Kyverno let you write segmentation rules as policy objects that the API server evaluates before any resource is created or updated. A policy can reject any pod spec that doesn’t carry the correct namespace label for CDE workloads, reject any NetworkPolicy that allows unrestricted egress from a CDE namespace, or reject any deployment that doesn’t request a non-root security context. Because these policies live in version control alongside the manifests they govern, they produce the same Git history and review trail as the workloads themselves.

A secure data center access turnstile with a badge reader, one indicator glowing red denying entry to a technician while another gate beside it glows green, concrete floor and cable trays in the background.

The practical payoff is that segmentation stops depending on someone remembering to apply a NetworkPolicy correctly. If a developer’s manifest tries to put a pod in the CDE namespace without the required policy annotations, the admission controller rejects it at the API server, before it ever runs. That rejection is logged, timestamped, and tied to the identity that attempted it. An assessor can ask to see the policy bundle, the version it was at on a given date, and the admission logs showing it rejecting a noncompliant manifest, and you can produce all three without reconstructing anything.

Write these policies to map directly to the segmentation claims in your Report on Compliance. If you tell an assessor that CDE namespaces cannot initiate outbound connections to non-CDE namespaces, have a Kyverno or Gatekeeper policy that enforces exactly that claim, and keep the policy name or comment referencing the requirement it satisfies.

Signing Commits and Container Images So Every Deploy Has a Verifiable Origin

A Git commit without a verified signature is a claim about authorship, not proof of it. The same is true of a container image pulled from a registry: a tag tells you what someone named it, not who built it or whether it matches the source it claims to come from. For a payment workload, both gaps are unacceptable, and PCI-DSS Requirement 6’s guidance on secure development expects you to close them.

Commit signing, using GPG or a modern alternative like Sigstore’s gitsign, should be enforced at the repository level so unsigned commits are rejected outright rather than flagged after the fact. Pair this with branch protection rules that require signed commits from a known set of keys before a merge to the branch your GitOps tool watches. This turns “who approved this change” into a cryptographic fact rather than a matter of checking a pull request’s author field, which can be spoofed or misattributed.

A compact hardware security module appliance mounted in an open server rack, small status LEDs lit, a thin fiber cable running to a nearby switch, rack door ajar in cool server-room light.

Image signing closes the second gap. Sigstore’s Cosign lets you sign container images at build time and verify that signature at admission time, so the cluster will refuse to run an image that wasn’t built by your CI pipeline, even if someone manages to push a tag directly to the registry. Combine this with an admission policy that checks the signature and the provenance attestation (what source commit, what build pipeline, what base image) before allowing the pod to start. At that point, every running workload in the CDE has a verifiable chain back to a signed commit and a signed build, which is precisely the kind of origin verification an assessor is trained to ask for and rarely gets.

Key management for all of this signing matters as much as the signing itself; a signing key stored in a developer’s laptop keychain undermines the whole chain. If you’re evaluating hardware-backed key storage for signing infrastructure, our guide to hardware security modules for fintech servers covers what to look for in a buyer’s comparison.

Mapping Kubernetes RBAC to PCI-DSS Requirements 7 and 8

Requirement 7 asks you to restrict access to cardholder data based on job function. Requirement 8 asks you to identify and authenticate access to system components. Kubernetes RBAC can satisfy both, but only if it’s built around those requirements from the start rather than retrofitted onto a cluster where every engineer has cluster-admin because it was faster to grant during an incident.

Start by mapping roles, not people. Define ClusterRoles and Roles that correspond to actual job functions: a deployment role that can create and update workloads in a given namespace but not read secrets, a read-only role for observability tooling, a break-glass role for incident response that’s time-bound and logged separately. Bind these roles to groups managed through your identity provider, not to individual user accounts, so that access changes when someone changes teams without anyone touching the cluster directly.

Requirement 8’s authentication expectations mean service accounts need the same scrutiny as human users. A service account with a long-lived token and broad permissions is a standing credential that never expires and rarely gets reviewed, which is the opposite of what an assessor wants to see. Use short-lived, workload-identity-based authentication wherever your cloud provider supports it, and audit service account bindings on the same cycle you audit human access.

The audit evidence here is the RoleBinding and ClusterRoleBinding objects themselves, stored in Git alongside everything else, plus the API server’s audit log showing actual access patterns. When an assessor asks who can read secrets in the payment-processing namespace, the answer should be a query against version-controlled RBAC manifests, not a conversation with whoever remembers setting it up.

Capturing Tamper-Evident Logs From etcd, the API Server, and the CNI

Logs assembled after an assessor asks for them are the weakest form of evidence you can offer. PCI-DSS Requirement 10 expects audit trails that are protected from unauthorized modification, which means the logging architecture has to be designed for tamper-evidence before anything happens, not aggregated under deadline.

A windowless archival storage room with rows of write-once optical and tape drives on steel shelving behind a heavy vault-style door, warm amber indicator lights, dust-free and quiet.

Three sources matter most in a Kubernetes cluster handling cardholder data. The Kubernetes API server audit log records every request made to the API, including who made it, what resource it touched, and whether it was allowed. Etcd, as the cluster’s backing store, should have its own access logging and, where supported, encryption at rest for the secrets and configuration it holds. The CNI plugin’s flow logs or NetworkPolicy logs show what traffic the segmentation rules actually allowed or denied, which is the evidence that your admission-controller segmentation claims are operating in practice, not just configured.

Ship all three to a log store the cluster’s own administrators cannot modify, using write-once storage or a dedicated logging pipeline with its own access controls separate from cluster RBAC. If the same credentials that can modify a deployment can also modify its audit trail, the trail isn’t tamper-evident, it’s just another log file. Retain these logs for the period your QSA specifies, and index them in a way that lets you query by namespace, by identity, and by time range without exporting raw files and grepping under pressure.

Assembling an Audit Trail a Reviewer Can Actually Follow

The individual controls above are only as useful as the story they tell together. An assessor doesn’t want five separate systems to check. They want one trail: a commit, signed by a known identity, reviewed and merged by a second known identity, reconciled into the cluster by a GitOps tool with no standing write access of its own, admitted by a policy engine that checked signature, provenance, and segmentation rules, running under an RBAC role that matches the workload’s actual job function, with every step logged somewhere the people who made the change cannot edit afterward.

Build a single index that ties these records together by change identifier, so that given a deployment timestamp, you can pull the commit, the approvers, the policy evaluation result, and the relevant audit log entries in one query rather than five. This is the difference between an audit that takes an afternoon and one that takes a week of reconstructing what happened from memory and scattered dashboards.

Run your current manifests and pipeline logs against the checklist above before your next assessment, and if any part of your cluster, CI/CD, or key management runs through a third party, vet that provider against our fintech vendor risk framework.

Leave a Comment