Skip to main content

API Gateway & Ingress (Apache APISIX)

Okustera API Gateway and Ingress is powered by Apache APISIX, providing ultra-low latency, dynamic traffic routing, automated SSL/TLS termination, Web Application Firewall (WAF) threat inspection, rate limiting, and canary traffic shifting.

APISIX operates directly at the edge of the Okustera tenant Kubernetes cluster, dynamically synchronizing routing rules via ApisixRoute CRDs and the official Terraform provider.

Okustera API Gateway & Traffic Ingress Console

Key Capabilities​

  • Dynamic Ingress Routing: Zero-reload rule updates without dropping existing TCP sockets or in-flight requests.
  • Automated SSL/TLS Termination: Automatic certificate lifecycle management through cert-manager (Let's Encrypt or private CA ClusterIssuers) paired with ApisixTls, enforcing HTTP-to-HTTPS (308 Permanent Redirect) and HSTS.
  • Web Application Firewall (WAF): Deep packet inspection powered by ModSecurity and OWASP Core Rule Set (CRS 3.3+) mitigating SQL Injection (SQLi), Cross-Site Scripting (XSS), and malicious payloads with DetectionOnly and Blocking modes.
  • Canary & Blue/Green Deployments: Weight-based traffic splitting, HTTP header matching (X-Canary: true), and cookie-based sticky sessions.
  • Enterprise Security & Governance: Granular token-bucket rate limiting, IP restriction, CORS management, forward authentication, and Prometheus metrics telemetry.

Web Application Firewall (WAF) & Threat Inspection​

Okustera incorporates OWASP ModSecurity Core Rule Set protection at both the public edge controller and APISIX ingress gateway layers:

  • DetectionOnly Mode: Inspects incoming HTTP request URIs, query strings, headers, and request bodies against OWASP CRS attack signatures. Violations are recorded to the central audit log without interrupting traffic, allowing safe tuning and baseline traffic profiling.
  • Blocking Mode: Actively defends endpoints by immediately terminating matched attack vectors with an HTTP 403 Forbidden response and an attack correlation ID.

Automated SSL/TLS & Ingress Routing​

When TLS is enabled on a route:

  1. The control plane requests an X.509 certificate from the specified ClusterIssuer (e.g., letsencrypt-prod or internal staging CA).
  2. The issued certificate is synchronized into a Kubernetes TLS secret.
  3. An ApisixTls resource binds the certificate to the domain's SNI, while edge ingress enforces TLS 1.2/1.3 and HSTS (max-age=31536000).

Canary Traffic Splitting​

The APISIX traffic-split plugin allows gradual releases of new application versions to mitigate operational risk:

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: order-service-route
  namespace: default
spec:
  http:
    - name: canary-traffic
      match:
        hosts:
          - api.okustera.com
        paths:
          - /orders*
      backends:
        - serviceName: order-service-stable
          servicePort: 8080
          weight: 90
        - serviceName: order-service-canary
          servicePort: 8080
          weight: 10
      plugins:
        - name: limit-count
          enable: true
          config:
            count: 1000
            time_window: 60
            rejected_code: 429
            key: remote_addr

AI Gateway & LLM Streaming Extensions (Phase 8 Alignment)​

The Apache APISIX gateway is extended in Phase 8: AI Inference PaaS to serve as a high-throughput AI Gateway:

  • Unbuffered Streaming SSE: Configures proxy_buffering: "off" and extended client read timeouts (600s) to stream token deltas in real time with sub-50ms Time-to-First-Token (TTFT).
  • Intelligent Model Payload Routing: Inspects incoming JSON request bodies ("model": "...") and routes requests dynamically across autoscaled vLLM RayService instances and embedding endpoints.
  • Fast Admission & Guardrail Routing (Kev-0.8B): Synchronously queries the Kev admission router in under 15ms, dropping prompt injections immediately (HTTP 403 Forbidden) and tagging request headers with X-AI-Route (iac_devops, deep_reasoning, openstack_api).
  • Semantic Prompt Caching: Integrates with Valkey DBaaS (omc-valkey) to return identical or semantically equivalent prompt completions in under 5ms with zero GPU overhead.
  • Token-Level FinOps: Automatically extracts usage.prompt_tokens and usage.completion_tokens from response bodies and streams billing records to FinOps, Metering & Billing.
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: ai-inference-route
  namespace: default
spec:
  http:
    - name: ai-chat-stream
      match:
        hosts:
          - ai.okustera.com
        paths:
          - /v1/chat/completions
      backends:
        - serviceName: vllm-llama-70b-head-svc
          servicePort: 8000
      plugins:
        - name: proxy-rewrite
          enable: true
          config:
            headers:
              set:
                X-Accel-Buffering: "no"
        - name: key-auth
          enable: true
        - name: limit-count
          enable: true
          config:
            count: 600
            time_window: 60
            rejected_code: 429
            key: consumer_name

Multi-Tenant Ingress Architecture & Tenant Gateways​

In Okustera, API Gateway routing enforces strict multi-tenant boundary isolation between the central cloud control plane and individual tenant environments:

Multi-Tenant Scoping Rules​

  • Strict Project Scoping: Every route declared in the portal or via Terraform is explicitly tagged with project_id.
  • Zero Leakage: Admin users logged into the platform portal only view platform control-plane routes by default, while tenant users strictly view their own isolated ingress routes.
  • Dual-Layer Edge Protection: Edge Ingress terminates public TLS and applies OWASP CRS inspection before proxying traffic directly into the tenant's dedicated APISIX cluster.

Declarative Management via Terraform​

The official Okustera Terraform provider manages APISIX ingress routes, automated TLS certificates, and WAF inspection rules directly in IaC:

resource "okustera_apisix_route" "api_endpoint" {
  name   = "public-customer-api"
  domain = "api.okustera.com"
  path   = "/api/v1/*"

  upstream = {
    service_name = "customer-service"
    port         = 8080
    timeout      = 15
  }

  # Automated SSL/TLS with cert-manager
  tls_enabled    = true
  cluster_issuer = "letsencrypt-prod"

  # OWASP Core Rule Set WAF Protection
  waf_enabled = true
  waf_mode    = "DetectionOnly" # Or "Blocking" for enforcement

  # Rate Limiting & CORS
  rate_limit = {
    requests_per_minute = 300
    burst               = 50
  }

  cors_enabled = true
}

Learn more in the Terraform Resource Reference.