Kubernetes Data Services¶
The shared, in-cluster stateful backing stores that platform applications consume instead of
bundling their own databases, caches and object stores. They live under
kubernetes/infrastructure/services
(plus one controller under
kubernetes/infrastructure/controllers)
and are reconciled by Flux onto the staging-cpt-aws EKS cluster. The design intent is a single
shared CloudNativePG Postgres cluster and a single shared Valkey cache in the database
namespace, with MinIO object storage as a durable write buffer for the capture pipeline.
All four subsystems are Kustomize base + overlays/<env> trees driven by Flux
Kustomization CRs. Everything currently wired for reconciliation targets staging only —
prod is a separate cluster (see the per-environment notes and the legacy warnings below).
Subsystem inventory¶
| Subsystem | Kind | Namespace | Deployed today? | Source |
|---|---|---|---|---|
| CloudNativePG operator | Helm operator (chart cloudnative-pg 0.23.*) |
cloudnative-pg |
Yes (staging) | services/cloudnative-pg |
Postgres (shared CNPG Cluster + backups) |
CNPG Cluster + ScheduledBackup |
database |
Yes (staging) | services/postgres |
| Valkey (shared cache/broker) | StatefulSet + PVC + headless Service |
database |
Yes (staging) | services/valkey |
| MinIO tenant (object storage) | Helm tenant chart 7.1.* |
minio-ingest |
No — flux-kustomization commented out | services/minio |
| MinIO operator | Helm operator chart 7.1.* |
minio-operator |
Yes (via controllers) | controllers/minio-operator |
| CNPG backup bucket + IRSA | Terraform module | (AWS) | Yes (staging) | terraform/aws/_modules/cnpg-backups |
Reconciliation order¶
The Flux Kustomizations form a dependency chain so CRDs and namespaces always exist before the
resources that reference them. From the staging root
kubernetes/overlays/staging-cpt-aws/kustomization.yaml:
cloudnative-pg (wait: true — operator Ready + CRDs installed)
│ dependsOn
▼
postgres (healthChecks: Cluster postgres healthy)
│ dependsOn
▼
valkey (database namespace already created by postgres)
│
▼
openreplay / defectdojo / dependency-track / sonarqube / observability(grafana)
(each declares a Database CR + managed role on the shared cluster)
The minio-operator controller reconciles as part of
infrastructure/controllers
(applied before the app GitOps wiring). The MinIO tenant is intentionally not yet wired — its
Flux Kustomization is commented out in both the staging and prod roots.
CloudNativePG operator¶
The CNPG operator is the Postgres control plane. It installs the postgresql.cnpg.io/v1 CRDs
(Cluster, Database, ScheduledBackup, managed roles, …) that the shared Postgres service and
every consuming app declare against.
What it deploys¶
Source:
services/cloudnative-pg/base
| Resource | Detail |
|---|---|
Namespace |
cloudnative-pg, label app.kubernetes.io/part-of: data-stores |
HelmRepository |
cloudnative-pg in flux-system, URL https://cloudnative-pg.github.io/charts, interval 1h |
HelmRelease |
chart cloudnative-pg version 0.23.*, interval 1h, install/upgrade remediation retries: 3, tests disabled |
PriorityClass |
openreplay-low, value -20, preemptionPolicy: Never |
Key HelmRelease values (helmrelease.yaml):
values:
crds:
create: true # operator installs/owns the CNPG CRDs
resources:
requests: { cpu: 50m, memory: 100Mi }
limits: { memory: 200Mi }
The operator Deployment gets priorityClassName: openreplay-low stamped on via a
postRenderers Kustomize patch (chart-independent) rather than a chart value:
postRenderers:
- kustomize:
patches:
- target: { kind: Deployment }
patch: |
- op: add
path: /spec/template/spec/priorityClassName
value: openreplay-low
The openreplay-low PriorityClass¶
Defined here — in the foundation service that reconciles first — so it always exists before any
pod references it (otherwise dependent pods would be admission-rejected with
no PriorityClass openreplay-low and the deploy would deadlock). It is shared by the CNPG
operator, the CNPG cluster pods, and Valkey.
| Field | Value | Rationale |
|---|---|---|
value |
-20 |
Below ingest-besteffort (-10) and default priority (0) — evicted first under node memory pressure |
preemptionPolicy |
Never |
These pods never preempt anything else |
globalDefault |
false |
Opt-in only |
Why so low
The shared data stores run on the shared capture nodes. Keeping them below every capture tier means that under memory pressure the capture camera/capture workloads are protected and the data stores are shed first. Their EBS PVCs survive a reschedule, so eviction is recoverable.
Flux wiring¶
overlays/staging/flux-kustomization.yaml
uses wait: true (interval 10m, timeout: 10m) so the Kustomization only reports Ready once
the operator is running and the CRDs are installed. The postgres Kustomization dependsOn
this, so its Cluster CR never applies into a cluster that lacks the CNPG CRDs.
Postgres — the shared CloudNativePG cluster¶
A single postgresql.cnpg.io/v1 Cluster named postgres in the database namespace. Apps do
not bundle their own Postgres; instead each declares a Database CR and a managed role
against this one cluster. Source:
services/postgres/base.
Cluster spec¶
From cluster.yaml:
| Setting | Value | Notes |
|---|---|---|
instances |
1 |
Staging = single instance, no failover. Comment: bump to 3 for HA in prod |
imageName |
(unset) | Operator pins its version-matched default image — PostgreSQL 17 |
priorityClassName |
openreplay-low |
Same eviction tier as the operator/Valkey |
storage |
5Gi, storageClass: gp3 |
Encrypted EBS |
resources |
requests 100m / 256Mi, limit 768Mi |
|
bootstrap.initdb |
database app, owner app |
Default DB; per-app DBs created via Database CRs |
monitoring.enablePodMonitor |
true |
Scraped by the staging kube-prometheus-stack |
Postgres tuning parameters (spec.postgresql.parameters):
max_connections: "200"
shared_buffers: "128MB"
effective_cache_size: "512MB"
work_mem: "4MB"
maintenance_work_mem: "64MB"
Managed roles¶
CNPG reconciles only the roles declared in spec.managed.roles, and each attribute is
reconciled to exactly what is declared. Every role is login: true, superuser: false,
createdb/createrole: false, inherit: true, connectionLimit: -1, with its password enforced
from an ESO-synced kubernetes.io/basic-auth secret in the database namespace.
| Role | Owns database | Password secret | Consumer |
|---|---|---|---|
openreplay |
openreplay |
openreplay-db-owner |
OpenReplay session replay |
defectdojo |
defectdojo |
defectdojo-db-owner |
DefectDojo |
dependencytrack |
dependencytrack |
dependencytrack-db-owner |
Dependency-Track |
sonarqube |
sonarqube |
sonarqube-db-owner |
SonarQube |
grafana |
grafana |
grafana-db-owner |
Grafana (observability) — stores state here instead of sqlite |
How apps consume the shared cluster¶
Apps connect over the CNPG-generated read-write service
postgres-rw.database.svc.cluster.local:5432 (no in-cluster auth beyond the role password). Each
app creates a Database CR — it never creates a new Cluster. Example
(openreplay's database.yaml):
apiVersion: postgresql.cnpg.io/v1
kind: Database
metadata:
name: openreplay
namespace: database
spec:
cluster:
name: postgres
name: openreplay
owner: openreplay
databaseReclaimPolicy: retain # keep the DB if the CR is pruned
The pattern per consumer:
- The role and its
passwordSecretare declared centrally incluster.yaml. - The password
ExternalSecretis co-located in the postgres staging overlay (see below) so it exists when theClusterfirst reconciles its managed roles. - The
DatabaseCR lives in the consuming service's own directory (defectdojo, dependency-track, sonarqube, grafana), all withdatabaseReclaimPolicy: retain.
Password lockstep
Each *-db-owner ExternalSecret reads the same Secrets Manager value the app itself
connects with, so the CNPG-enforced role password and the app's connection password stay in
lockstep. They are deliberately co-located in the postgres overlay rather than the downstream
app overlay — otherwise CNPG could not set the role password until the app reconciled much
later.
Password ExternalSecrets¶
The staging overlay
(overlays/staging/kustomization.yaml)
adds five ExternalSecrets, all refreshInterval: 1h against the aws-secrets-manager
ClusterSecretStore, rendering a basic-auth secret:
| ExternalSecret | Secrets Manager key | Property |
|---|---|---|
openreplay-db-owner |
staging-openreplay-app-secrets |
postgresql-password |
defectdojo-db-owner |
appsec/defectdojo |
db-password |
dependencytrack-db-owner |
appsec/dependency-track |
db-password |
sonarqube-db-owner |
appsec/sonarqube |
db-password |
grafana-db-owner |
grafana |
db-password |
StorageClass gp3¶
storageclass.yaml
defines the cluster-scoped gp3 class used by both Postgres and Valkey (the cluster default is
gp2). Defined once here as the foundational data service.
provisioner: ebs.csi.aws.com
parameters:
type: gp3
encrypted: "true"
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer # volume lands in the pod's AZ
allowVolumeExpansion: true
Flux wiring¶
overlays/staging/flux-kustomization.yaml
dependsOn cloudnative-pg and gates readiness on a healthCheck for the Cluster being
actually healthy (not just the CR applied), so dependents (valkey, openreplay) don't start their
migration/app pods before Postgres accepts connections. It uses wait: false.
Postgres backups — barman-cloud + S3 + IRSA¶
Continuous backup to S3 via CNPG's built-in barman-cloud. Setting barmanObjectStore makes CNPG
configure WAL archiving (archive_command) automatically, so base backups plus streamed WAL give
point-in-time recovery (PITR).
In-cluster backup config¶
From spec.backup in
cluster.yaml:
backup:
barmanObjectStore:
destinationPath: s3://staging-cnpg-backups-444455556666/
endpointURL: https://s3.af-south-1.amazonaws.com # af-south-1 is opt-in; pin it explicitly
s3Credentials:
inheritFromIAMRole: true # web-identity via IRSA, no static keys
wal: { compression: gzip, maxParallel: 2 }
data: { compression: gzip }
retentionPolicy: "30d" # barman prunes base backups + no-longer-needed WAL; NOT an S3 lifecycle expiry
IRSA is wired through the serviceAccountTemplate on the Cluster, which annotates the
ServiceAccount CNPG generates (named postgres) so the barman sidecar inherits the pod's
web-identity credentials:
serviceAccountTemplate:
metadata:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::444455556666:role/staging-cnpg-backups-role
Account-specific values
The destinationPath bucket name and the IRSA role-arn both embed the staging account
id 444455556666. There is no prod overlay yet — a prod deployment must update both alongside
the Terraform-provisioned role ARN.
Daily base backup¶
scheduled-backup.yaml
drives a daily base backup; WAL is archived continuously in between.
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
name: postgres-daily
namespace: database
spec:
schedule: "0 0 2 * * *" # 6-field cron (leading seconds) → 02:00 daily
immediate: true # first base backup on CR create, proving backups work at apply time
backupOwnerReference: self
cluster: { name: postgres }
Terraform: _modules/cnpg-backups¶
The S3 bucket and IRSA role are provisioned by
terraform/aws/_modules/cnpg-backups,
mirroring the S3+IRSA pattern in _modules/openreplay. It is invoked via Terragrunt at
terraform/aws/staging/af-south-1/cnpg-backups,
which dependencies on ../eks (the module reads the EKS OIDC issuer via a data source).
What it creates (main.tf):
| Resource | Detail |
|---|---|
aws_s3_bucket |
${environment}-cnpg-backups-${account_id} (globally-unique, account-suffixed) |
aws_s3_bucket_public_access_block |
All four blocks true |
aws_s3_bucket_versioning |
Enabled |
aws_s3_bucket_server_side_encryption_configuration |
SSE-S3 AES256 (no CMK — accepted for staging) |
aws_s3_bucket_lifecycle_configuration |
Reaps noncurrent versions after 30d + aborts incomplete multipart uploads after 3d; does not expire current objects |
aws_iam_role |
${environment}-cnpg-backups-role, trust sts:AssumeRoleWithWebIdentity |
aws_iam_policy + attachment |
barman-cloud S3 permissions |
The trust policy pins the subject with StringEquals (not a wildcard) to exactly one
ServiceAccount — system:serviceaccount:database:postgres
(from vars cluster_namespace = database, cluster_service_account = postgres).
Retention is barman's job, not S3's
The lifecycle rule intentionally only expires noncurrent object versions. barman manages
backup retention itself via spec.backup.retentionPolicy (30d). An S3 lifecycle expiry on
current objects could delete base backups / WAL segments barman still needs to recover, so it
is deliberately omitted.
barman IAM permissions granted by the policy:
BucketAccess: s3:ListBucket, s3:GetBucketLocation, s3:ListBucketMultipartUploads (on the bucket)
ObjectAccess: s3:GetObject, s3:PutObject, s3:DeleteObject,
s3:AbortMultipartUpload, s3:ListMultipartUploadParts (on bucket/*)
Module outputs (bucket_name, destination_path, role_arn, service_account_annotations) are
documented to be wired back into the Cluster manifest's barmanObjectStore.destinationPath and
serviceAccountTemplate annotation. Today those values are hand-copied into cluster.yaml
(Kubernetes manifests and Terraform state are separate), so the two must be kept in sync manually.
Valkey — shared cache / broker¶
A single-instance, Redis-compatible cache/broker in the database namespace. Source:
services/valkey/base.
What it deploys¶
| Resource | Detail |
|---|---|
StatefulSet valkey |
replicas: 1, image valkey/valkey:8.1-alpine, imagePullPolicy: IfNotPresent |
PersistentVolumeClaim valkey |
2Gi, gp3, ReadWriteOnce (static claim, not a volumeClaimTemplate) |
Service valkey |
headless (clusterIP: None), TCP 6379 |
Container args and sizing
(statefulset.yaml):
args: ["--appendonly", "yes", "--maxmemory", "256mb", "--maxmemory-policy", "noeviction"]
resources:
requests: { cpu: 25m, memory: 64Mi }
limits: { cpu: 500m, memory: 320Mi }
Security / scheduling hardening:
priorityClassName: openreplay-low(same eviction tier as Postgres).automountServiceAccountToken: false— a cache needs no API access.- Pod
securityContext:runAsNonRoot,runAsUser: 999,fsGroup: 999(the image runs as uid/gid 999 and can't chown a fresh EBS volume;fsGroupmakes/datagroup-writable),seccompProfile: RuntimeDefault. - Container
securityContext:allowPrivilegeEscalation: false,capabilities: drop [ALL]. - Liveness + readiness probes both exec
valkey-cli ping.
Do not scale beyond 1 replica
The StatefulSet mounts one shared RWO PVC via a static claimName (not a
volumeClaimTemplate), so multiple replicas would contend for the same EBS volume. It is
explicitly a single-instance cache.
The StatefulSet carries checkov.io/skip* annotations documenting accepted security-gate/Checkov
findings (tag-pinned image with IfNotPresent, fixed non-root uid 999, writable working dir on the
mounted PVC, tag 8.1-alpine rather than a digest).
How apps consume Valkey¶
Clients resolve valkey.database.svc.cluster.local:6379 (no auth, matching how the shared CNPG
tier is reachable in-cluster). OpenReplay uses it as its Redis via
global.redis.redisHost: valkey.database.svc.cluster.local
(helmrelease-openreplay.yaml).
Flux wiring¶
overlays/staging/flux-kustomization.yaml
dependsOn postgres (which creates the database namespace), interval 10m, timeout: 10m.
MinIO — object storage (operator + tenant)¶
S3-compatible object storage intended as the durable write buffer for the capture pipeline. It is split into a cluster-wide operator (a controller) and a per-tenant MinIO cluster.
Not yet reconciled
The MinIO tenant is not deployed. Its Flux Kustomization is commented out in both
root overlays with the note we are not ready for this yet
(staging,
prod).
The operator is reconciled (it is listed in
infrastructure/controllers/kustomization.yaml),
so today there is a running operator with no tenant.
MinIO operator (controller)¶
Source:
controllers/minio-operator.
| Resource | Detail |
|---|---|
Namespace |
minio-operator |
HelmRepository |
minio in flux-system, URL https://operator.min.io, interval 1h |
HelmRelease minio-operator |
chart operator version 7.1.* |
Values (helmrelease.yaml):
operator:
replicaCount: 2 # HA; chart spreads replicas across nodes by default
resources:
requests: { cpu: 100m, memory: 128Mi }
limits: { memory: 256Mi }
Both install and upgrade use crds: CreateReplace and remediation.retries: 3. The
HelmRepository named minio is shared by the tenant HelmRelease below.
MinIO tenant (service)¶
Source:
services/minio/base.
The HelmRelease minio-tenant (dependsOn the operator) deploys the tenant minio-ingest into the
minio-ingest namespace via chart tenant version 7.1.*.
| Setting | Base (staging) value | Notes |
|---|---|---|
| Tenant name | minio-ingest |
|
| Image | quay.io/minio/minio:RELEASE.2025-04-08T15-41-24Z |
IfNotPresent |
| Credentials | configSecret: minio-env-configuration (existing) |
Must pre-exist in minio-ingest with a config.env key |
Pool pool-0 |
servers: 4, volumesPerServer: 2 |
8 drives → MinIO auto-selects EC:4 (4 data + 4 parity), tolerates loss of any 4 drives |
| Drive size | 50Gi |
8 × 50Gi = 400Gi raw, ~200Gi usable after EC:4 |
| StorageClass | minio-gp3 |
|
| Anti-affinity | requiredDuringScheduling on kubernetes.io/hostname |
No two MinIO pods on one node — requires ≥4 schedulable workers |
priorityClassName |
ingest-critical |
Protected from eviction (critical write buffer) |
mountPath / subPath |
/export / /data |
|
exposeServices |
minio: false, console: false |
ClusterIP only; console exposed via Ingress in overlays |
metrics |
enabled: true, port 9000, http |
Prometheus-compatible on the API port |
Security context: runs as uid/gid 1000, runAsNonRoot, allowPrivilegeEscalation: false,
drop: [ALL], seccompProfile: RuntimeDefault, fsGroupChangePolicy: OnRootMismatch (avoids
recursive chown on large volumes). Base resource requests are 500m CPU / 1Gi memory, limit
2Gi.
StorageClass minio-gp3¶
Dedicated gp3 class
(storageclass.yaml),
separate from the database tier's gp3:
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Retain # EBS volumes are NOT deleted with the PVC — durable write buffer
allowVolumeExpansion: true
parameters:
type: gp3
iops: "3000"
throughput: "125"
csi.storage.k8s.io/fstype: xfs # recommended for MinIO on EBS
Retain reclaim policy
reclaimPolicy: Retain means removing a PVC leaves the EBS volume behind. Decommissioning
requires manual PV/EBS cleanup. This is deliberate — MinIO is treated as a durable store
where data loss is unacceptable.
Root credentials secret¶
secret.yaml
is a placeholder Secret named minio-env-configuration whose header instructs sealing it
with external-secrets/sealed-secrets before committing. It carries shell exports MinIO reads at
startup:
export MINIO_ROOT_USER=REPLACE_ME_ROOT_USER
export MINIO_ROOT_PASSWORD=REPLACE_ME_ROOT_PASSWORD
export MINIO_SITE_OBJECT_LOCK_ENABLED=on # site-wide WORM (Write Once Read Many)
WORM is irreversible
MINIO_SITE_OBJECT_LOCK_ENABLED=on enables object locking (WORM) for all buckets site-wide.
Per the file's own warning it cannot be disabled once data is written without potential data
loss — appropriate for a durable write buffer but a one-way door.
MinIO per-environment overlays¶
Both overlays add an ESO-synced google-oidc-secret (Google OIDC creds, also used by Grafana,
synced into minio-ingest so the AWS Load Balancer Controller can configure OIDC auth for the
Console Ingress) and a minio-console Ingress fronted by an OIDC-authenticated ALB. The Console
sits behind Google SSO (auth-type: oidc, 28800s session, authenticate on unauthenticated).
| Aspect | Staging (overlay) | Prod (overlay) |
|---|---|---|
| Drive size / resources | Base defaults (50Gi, small) — no patch | Patched to 500Gi per drive (8 × 500Gi = 4TiB raw, ~2TiB usable), requests 2 CPU / 4Gi, limits 4 CPU / 8Gi |
| Console host | minio.staging.cpt.aws.example.net |
minio.prod.cpt.aws.example.net |
| ALB | staging-alb (shared IngressGroup group.name: staging-alb with public-api metrics), backend-protocol: HTTPS, console port 9443 |
prod-alb, static EIP allocations + subnet mappings, console port 9001 |
| ACM cert account | 444455556666 (staging) |
777788889999 (prod) |
| Extra RBAC | aws-load-balancer-controller-secret-reader Role/RoleBinding granting the kube-system ALB controller SA read on secrets in minio-ingest |
(inherited via base + prod externalsecret) |
Shared staging ALB re-homing
The staging Console Ingress joins the staging-alb IngressGroup so it shares one ALB with the
public-api metrics Ingress. Both Ingresses must carry the same group.name or the second
CreateLoadBalancer fails with DuplicateLoadBalancerName. Adding the group re-homes the ALB,
so the controller recreates staging-alb once (new DNS suffix; external-dns repoints).
Per-environment summary¶
Staging (staging-cpt-aws) |
Prod (prod-cpt-aws) |
|
|---|---|---|
| CNPG operator | Deployed | No overlay / not wired |
| Postgres cluster | 1 instance, 5Gi gp3, backups → staging-cnpg-backups-444455556666 |
Not present (comments say scale to 3 instances) |
| Valkey | Deployed, 1 replica, 2Gi | Not present |
| MinIO operator | Deployed (via controllers) | Deployed (via controllers) |
| MinIO tenant | Overlay exists, commented out in root | Overlay exists (500Gi), commented out in root |
| CNPG backup Terraform | terraform/aws/staging/af-south-1/cnpg-backups |
No prod Terragrunt unit found |
The database-namespace data stores (CNPG operator, Postgres, Valkey) are staging-only today.
The cluster.yaml and scheduled-backup.yaml live in base but only a staging overlay wires
them, and account-specific backup values are hard-coded to the staging account.
Operational notes¶
- Namespaces:
cloudnative-pg(operator),database(Postgres cluster + Valkey),minio-operator(operator),minio-ingest(tenant). Thedatabasenamespace is created by the postgres service; Valkey depends on it. - In-cluster endpoints: Postgres RW
postgres-rw.database.svc.cluster.local:5432; Valkeyvalkey.database.svc.cluster.local:6379. Neither enforces network-level auth in-cluster; Postgres role passwords are enforced by CNPG. - Backups / DR: PITR via barman base backups (daily 02:00) + continuous WAL to S3, IRSA-scoped,
30-day retention managed by barman. To restore, use CNPG's
barmanObjectStorerecovery bootstrap againsts3://staging-cnpg-backups-444455556666/. - Eviction ordering: Postgres, Valkey and the CNPG operator all sit at
openreplay-low(-20) and are shed first under node memory pressure; MinIO tenant sits atingest-criticaland is protected. EBS PVCs survive reschedules. - Monitoring: the CNPG cluster exposes a
PodMonitor(enablePodMonitor: true) scraped by the staging kube-prometheus-stack; the MinIO tenant exposes Prometheus metrics on port 9000. - CRDs: owned by the operators (
crds.create: truefor CNPG,crds: CreateReplacefor MinIO). Removing an operator can strand CRs; the reconciliation ordering exists to avoidno matches for kind Cluster-style deadlocks.
Legacy / cleanup flags¶
Committed OIDC credentials in the staging MinIO ExternalSecret
overlays/staging/externalsecret.yaml
has a real-looking Google OIDC client id and client secret hard-coded in leading comment
lines (a GOCSPX-… value). The manifest body correctly pulls these from Secrets Manager, so
the comment is dead but leaks live-looking credentials into git history. It should be scrubbed
and the credential rotated.
MinIO tenant is orphaned mid-migration
The tenant Flux Kustomization is commented out in both root overlays (we are not ready for
this yet) while the operator runs. A full MinIO base/overlay tree (including a hand-copied
prod overlay and a placeholder root-credentials secret) exists but reconciles nothing.
Placeholder root-credentials secret committed
base/secret.yaml
is a plaintext Secret with REPLACE_ME_ROOT_USER / REPLACE_ME_ROOT_PASSWORD placeholders
and a REPLACE: seal with external-secrets… header, yet it is committed and part of the base
kustomization. There is no ExternalSecret for MinIO root creds — the tenant can't stand up
without a real secret being supplied.
No prod path for the shared data stores
Postgres, Valkey and the CNPG operator have staging overlays only. cluster.yaml hard-codes
the staging account id in the backup destinationPath and IRSA role ARN, and no prod
Terragrunt unit for cnpg-backups exists. The in-code comments (bump to 3 for HA in prod)
describe an unbuilt prod deployment.
Prod MinIO Ingress TLS host mismatch
In the prod ingress.yaml
the rule host is minio.prod.cpt.aws.example.net but the tls.hosts block lists
minio-console.prod.cpt.aws.example.net, and the prod console backend port (9001) differs from
staging (9443). Worth reconciling before the prod tenant is enabled.
Terraform ↔ manifest values are hand-synced
_modules/cnpg-backups outputs (bucket_name, role_arn, service_account_annotations) are
meant to feed the CNPG Cluster manifest, but the manifest hard-codes them. Bucket name and
role ARN must be kept in sync manually across Terraform and Kubernetes.