Multi-Tenant SaaS Architecture

Billerang is designed as a stateless, multi-tenant SaaS platform on Kubernetes.

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                              BILLERANG SaaS                                  │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ┌──────────────┐    ┌──────────────────────────────────────────────────┐   │
│  │   CDN        │    │           Akamai/Linode Kubernetes               │   │
│  │  (Akamai)    │    │                                                  │   │
│  │              │    │  ┌────────────────────────────────────────────┐  │   │
│  │ React SPA   │────│  │     Shared Services Namespace              │  │   │
│  │ billerang-  │    │  │  ┌─────────────┐  ┌───────────────────┐   │  │   │
│  │ portal      │    │  │  │  Keycloak   │  │  Ingress/Gateway  │   │  │   │
│  │             │    │  │  │ (all orgs)  │  │  (route by org)   │   │  │   │
│  └──────────────┘    │  │  └─────────────┘  └───────────────────┘   │  │   │
│                      │  └────────────────────────────────────────────┘  │   │
│                      │                                                  │   │
│                      │  ┌────────────────────────────────────────────┐  │   │
│                      │  │     Backend Pool (Stateless, Shared)       │  │   │
│                      │  │                                            │  │   │
│                      │  │  ┌─────────┐ ┌─────────┐ ┌─────────┐      │  │   │
│                      │  │  │ OC Pod  │ │ OC Pod  │ │ OC Pod  │ ...  │  │   │
│                      │  │  │  (HPA)  │ │  (HPA)  │ │  (HPA)  │      │  │   │
│                      │  │  └─────────┘ └─────────┘ └─────────┘      │  │   │
│                      │  │       ↓ Dynamic connection routing ↓       │  │   │
│                      │  └────────────────────────────────────────────┘  │   │
│                      │                                                  │   │
│  ┌────────────────┐  │  ┌──────────────┐  ┌──────────────┐            │   │
│  │ Organization:  │  │  │  Provider:   │  │  Provider:   │  ...       │   │
│  │ Pluxee Group   │  │  │  PLUXEE_FR   │  │  PLUXEE_DE   │            │   │
│  │                │  │  │              │  │              │            │   │
│  │ SSO Realm:     │  │  │ CloudNativePG│  │ CloudNativePG│            │   │
│  │ pluxee-group   │  │  │ ┌──────────┐ │  │ ┌──────────┐ │            │   │
│  │                │  │  │ │Primary RW│ │  │ │Primary RW│ │            │   │
│  │ Providers:     │  │  │ │Replica RO│ │  │ │Replica RO│ │            │   │
│  │ - PLUXEE_FR    │  │  │ └──────────┘ │  │ └──────────┘ │            │   │
│  │ - PLUXEE_DE    │  │  └──────────────┘  └──────────────┘            │   │
│  │ - PLUXEE_ES    │  │                                                  │   │
│  └────────────────┘  └──────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘

Key Concepts

Concept	Description	Example
Organization	Top-level customer (Keycloak realm/org)	Pluxee Group
Provider	Billing tenant (separate DB)	PLUXEE_FR, PLUXEE_DE
Environment	sbx (sandbox) vs prod	pluxee-sbx

Authentication: Cookie Session with Redis

Session Flow

1. User accesses app.billerang.com
2. Redirect to auth.billerang.com/realms/billerang/login
3. Keycloak authenticates → Sets secure HttpOnly cookie
4. Cookie contains session ID → Redis stores session data
5. Backend validates session via Keycloak adapter
6. Session contains: organization, provider, environment, roles

Session Data in Redis

{
  "sessionId": "abc123-session-id",
  "userId": "user-uuid",
  "email": "user@pluxee.com",
  "organization": "pluxee-group",
  "provider": "PLUXEE_FR",
  "environment": "prod",
  "roles": ["billing_admin", "invoice_viewer"],
  "expiresAt": "2024-01-15T18:00:00Z"
}

Component Matrix

Component	Scaling	State	Location
Frontend (React)	CDN	Stateless	Akamai CDN
Backend (WildFly)	HPA (3-20 pods)	Stateless	K8s billerang-prod
Redis	3 replicas	Session cache	K8s billerang-cache
Keycloak	3 replicas	Shared DB	K8s billerang-auth
PostgreSQL	CloudNativePG	Per-provider	K8s billerang-db
Jobs	CronJob	Per-provider	K8s billerang-jobs

Stateless Backend Design

Challenge

Opencell uses WildFly with:

• Stateful EJBs
• Infinispan cache
• Timer-based jobs

Solution

1. Dynamic Datasource Resolution - Route to correct DB based on session
2. Redis Session Cache - Distributed session storage
3. Decoupled Jobs - K8s CronJobs per provider

Dynamic Datasource Resolution

@ApplicationScoped
public class ProviderDataSourceResolver {

    @Inject
    private ProviderConnectionRegistry registry;

    public DataSource getDataSource(String providerCode, boolean readOnly) {
        ConnectionInfo info = registry.getConnection(providerCode);
        if (readOnly) {
            return createDataSource(info.getReadOnlyUrl());
        }
        return createDataSource(info.getReadWriteUrl());
    }
}

Provider Context from Cookie Session

@RequestScoped
public class ProviderContext {

    @Inject
    private RedisSessionService redisSessionService;

    @PostConstruct
    void init() {
        Cookie sessionCookie = findCookie(request, "KEYCLOAK_SESSION");
        if (sessionCookie != null) {
            SessionData session = redisSessionService.getSession(sessionCookie.getValue());
            this.organizationCode = session.getOrganization();
            this.providerCode = session.getProvider();
            this.environment = session.getEnvironment();
        }
    }

    public String getConnectionKey() {
        return String.format("%s_%s_%s", organizationCode, providerCode, environment);
    }
}

Scaling Strategy

API Pods: Auto-scale based on CPU/Memory (independent of providers)
         └── All pods can serve any provider (stateless)
         └── Connection routing via session claims

DB Clusters: Scale independently per provider
         └── PLUXEE_FR: 1 primary + 2 replicas
         └── PLUXEE_DE: 1 primary + 1 replica (smaller)
         └── Add replicas as load increases

Jobs: Dedicated CronJobs per provider
         └── Run on separate node pool if needed
         └── Scale job parallelism for large providers

Adding New Provider Checklist

1. ☐ Create CloudNativePG cluster (pg-{provider-code})
2. ☐ Create K8s secrets for DB credentials
3. ☐ Add provider to ConfigMap (provider-connections)
4. ☐ Configure Keycloak organization with provider attribute
5. ☐ Create CronJobs for billing/rating
6. ☐ Run Liquibase migrations on new DB
7. ☐ Test with sample data

Key Files

• ProviderContext.java
• ProviderDataSourceResolver.java
• RedisSessionService.java