okustera_ai_ray_deployment (Resource)

The okustera_ai_ray_deployment resource deploys and manages a KubeRay RayService vLLM inference cluster with dynamic Multi-LoRA multiplexing, fractional GPU profiles (mig-1g.10gb, mig-2g.20gb, mig-3g.40gb, mps-timeslice-4x, cpu-avx512), read-only CephFS model mounts, and concurrency-driven scale-to-zero autoscaling.

Example Usage

resource "okustera_ai_ray_deployment" "multi_lora_serving" {
  name          = "qwen-secops-serving"
  base_model    = "Qwen/Qwen2.5-1.5B-Instruct"
  enable_lora   = true
  max_loras     = 64
  max_lora_rank = 64
  gpu_profile   = "mig-1g.10gb"

  lora_adapters = [
    {
      name          = "secops-lora"
      oci_reference = okustera_ai_oci_artifact.secops_lora.oci_reference
      cephfs_path   = "/models/foundry/oci-artifacts"
      rank          = 16
    }
  ]

  autoscaling = {
    enable_in_tree_autoscaling         = true
    metric                             = "vllm:num_requests_waiting"
    target_concurrency                 = 5
    min_replicas                       = 0
    max_replicas                       = 4
    scale_to_zero_idle_timeout_seconds = 300
  }
}