Kubernetes Deployment

Complete Kubernetes manifests for Billerang multi-tenant deployment.

Namespace Structure

billerang-prod      # API pods
billerang-auth      # Keycloak
billerang-cache     # Redis session cache
billerang-db        # CloudNativePG clusters
billerang-jobs      # CronJobs per provider

Backend Deployment

Stateless API Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: billerang-api
  namespace: billerang-prod
spec:
  replicas: 3
  selector:
    matchLabels:
      app: billerang-api
  template:
    metadata:
      labels:
        app: billerang-api
    spec:
      containers:
        - name: billerang-api
          image: ghcr.io/billerang/billerang-api:latest
          ports:
            - containerPort: 8080
          env:
            - name: PROVIDER_CONFIG
              valueFrom:
                configMapKeyRef:
                  name: provider-connections
                  key: providers.json
            - name: REDIS_URL
              value: "redis://redis-session.billerang-cache:6379"
          envFrom:
            - secretRef:
                name: provider-db-credentials
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
            limits:
              memory: "4Gi"
              cpu: "2000m"
          readinessProbe:
            httpGet:
              path: /health/ready
              port: 8080
            initialDelaySeconds: 30
          livenessProbe:
            httpGet:
              path: /health/live
              port: 8080
            initialDelaySeconds: 60

Horizontal Pod Autoscaler

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: billerang-api-hpa
  namespace: billerang-prod
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: billerang-api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

Redis Session Cache

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-session
  namespace: billerang-cache
spec:
  serviceName: redis-session
  replicas: 3
  selector:
    matchLabels:
      app: redis-session
  template:
    metadata:
      labels:
        app: redis-session
    spec:
      containers:
        - name: redis
          image: redis:7-alpine
          ports:
            - containerPort: 6379
          command:
            - redis-server
            - --appendonly yes
            - --cluster-enabled yes
          volumeMounts:
            - name: redis-data
              mountPath: /data
  volumeClaimTemplates:
    - metadata:
        name: redis-data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: linode-block-storage
        resources:
          requests:
            storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
  name: redis-session
  namespace: billerang-cache
spec:
  type: ClusterIP
  ports:
    - port: 6379
  selector:
    app: redis-session

CloudNativePG Per-Provider

PostgreSQL Cluster

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: pg-pluxee-fr
  namespace: billerang-db
spec:
  instances: 3  # 1 primary + 2 replicas

  postgresql:
    parameters:
      max_connections: "400"
      shared_buffers: "2GB"
      effective_cache_size: "6GB"
      wal_level: "replica"
      hot_standby: "on"
      hot_standby_feedback: "on"

  bootstrap:
    initdb:
      database: opencell
      owner: opencell
      postInitSQL:
        - CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

  storage:
    size: 100Gi
    storageClassName: linode-block-storage-retain

  backup:
    barmanObjectStore:
      destinationPath: s3://billerang-backups/pluxee-fr/
      endpointURL: https://eu-central-1.linodeobjects.com
      s3Credentials:
        accessKeyId:
          name: linode-obj-credentials
          key: ACCESS_KEY
        secretAccessKey:
          name: linode-obj-credentials
          key: SECRET_KEY
      wal:
        compression: gzip
      data:
        compression: gzip
    retentionPolicy: "30d"

  scheduledBackups:
    - name: daily-backup
      schedule: "0 2 * * *"
      backupOwnerReference: self

R/W and RO Services

apiVersion: v1
kind: Service
metadata:
  name: pg-pluxee-fr-rw
  namespace: billerang-db
spec:
  type: ClusterIP
  ports:
    - port: 5432
  selector:
    cnpg.io/cluster: pg-pluxee-fr
    role: primary
---
apiVersion: v1
kind: Service
metadata:
  name: pg-pluxee-fr-ro
  namespace: billerang-db
spec:
  type: ClusterIP
  ports:
    - port: 5432
  selector:
    cnpg.io/cluster: pg-pluxee-fr
    role: replica

Provider Connection ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: provider-connections
  namespace: billerang-prod
data:
  providers.json: |
    {
      "providers": [
        {
          "code": "PLUXEE_FR",
          "organization": "pluxee-group",
          "environment": "prod",
          "database": {
            "rwService": "pg-pluxee-fr-rw.billerang-db",
            "roService": "pg-pluxee-fr-ro.billerang-db",
            "name": "opencell"
          }
        },
        {
          "code": "PLUXEE_DE",
          "organization": "pluxee-group",
          "environment": "prod",
          "database": {
            "rwService": "pg-pluxee-de-rw.billerang-db",
            "roService": "pg-pluxee-de-ro.billerang-db",
            "name": "opencell"
          }
        }
      ]
    }

Job Scheduling

Monthly Billing CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: billing-run-pluxee-fr
  namespace: billerang-jobs
spec:
  schedule: "0 2 1 * *"  # 1st of month, 2am
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      backoffLimit: 3
      template:
        spec:
          serviceAccountName: job-runner
          containers:
            - name: billing-job
              image: ghcr.io/billerang/job-runner:latest
              env:
                - name: JOB_TYPE
                  value: "MONTHLY_BILLING"
                - name: PROVIDER_CODE
                  value: "PLUXEE_FR"
                - name: DB_RW_URL
                  valueFrom:
                    secretKeyRef:
                      name: provider-pluxee-fr-db
                      key: rw-url
              resources:
                requests:
                  memory: "1Gi"
                  cpu: "500m"
          restartPolicy: OnFailure

Daily Usage Rating CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: usage-rating-pluxee-fr
  namespace: billerang-jobs
spec:
  schedule: "0 */4 * * *"  # Every 4 hours
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: rating-job
              image: ghcr.io/billerang/job-runner:latest
              env:
                - name: JOB_TYPE
                  value: "USAGE_RATING"
                - name: PROVIDER_CODE
                  value: "PLUXEE_FR"
          restartPolicy: OnFailure

Ingress Configuration

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: billerang-api
  namespace: billerang-prod
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
spec:
  tls:
    - hosts:
        - api.billerang.com
      secretName: billerang-api-tls
  rules:
    - host: api.billerang.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: billerang-api
                port:
                  number: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keycloak
  namespace: billerang-auth
spec:
  tls:
    - hosts:
        - auth.billerang.com
      secretName: keycloak-tls
  rules:
    - host: auth.billerang.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: keycloak
                port:
                  number: 8080

Monitoring Read-Replica Lag

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: cnpg-alerts
spec:
  groups:
    - name: cnpg.rules
      rules:
        - alert: ReplicaLagHigh
          expr: cnpg_pg_replication_lag_seconds > 30
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "PostgreSQL replica lag > 30s"

        - alert: ReplicaLagCritical
          expr: cnpg_pg_replication_lag_seconds > 60
          for: 2m
          labels:
            severity: critical
          annotations:
            summary: "PostgreSQL replica lag > 60s"

Installation Order

1. Install CloudNativePG operator
2. Install cert-manager
3. Create namespaces
4. Deploy Redis cluster
5. Deploy Keycloak
6. Create CloudNativePG clusters per provider
7. Apply ConfigMaps and Secrets
8. Deploy API
9. Configure Ingress
10. Create CronJobs per provider