Infrastructure
Network Topology
Multi-region AWS topology — click any node card to inspect its infrastructure specifications.
edge-global-router
Global edge routing layer — terminates TLS, applies WAF rules and steers traffic to the nearest healthy regional gateway.
us-east-1
us-east-1a
api-gw-us-east-1
API Gateway
Regional REST gateway fronting inference endpoints. Request validation, throttling and auth at the edge of the VPC.
- Service
- API Gateway (regional)
- Requests / sec
- 8.2k sustained
- Throttle limit
- 12k rps burst
- p50 / p99
- 6 ms / 24 ms
- Integration
- ALB → EKS inference pods
- Auth
- JWT authorizer · mTLS internal
ec2-p4de-us-east-1a
EC2 Compute Cluster
Primary A100 inference cluster. Serving production ranking and PINN solvers; running hot on utilization.
- Instance
- p4d.24xlarge
- Nodes
- 6 / 8 in service
- GPUs
- 24× A100 40GB (2 idle)
- vCPU / RAM
- 768 vCPU · 4.6 TB
- EBS
- 64 TB gp3 · 12,000 IOPS
- Util / Temp
- 92% · 74°C
- Networking
- 100 Gbps EFA enabled
us-east-1b
ec2-p4de-us-east-1b
EC2 Compute Cluster
Mirrored A100 inference cluster in the second AZ. Takes overflow traffic and handles batch scoring jobs.
- Instance
- p4d.24xlarge
- Nodes
- 4 / 4 in service
- GPUs
- 16× A100 40GB
- vCPU / RAM
- 512 vCPU · 3.1 TB
- EBS
- 40 TB gp3 · 12,000 IOPS
- Util / Temp
- 81% · 66°C
- Networking
- 100 Gbps EFA enabled
vec-index-us-east-1
Vector Index
Primary HNSW vector index backing RAG retrieval for production assistants. Cross-region replicated to eu-west-1.
- Engine
- OpenSearch 2.11 (k-NN)
- Vectors
- 148M · 768-dim
- Index type
- HNSW · ef_search 128
- Storage
- 2.4 TB gp3
- p95 query latency
- 18 ms
- Replication
- async → eu-west-1 · 2.1s lag
eu-west-1
eu-west-1a
api-gw-eu-west-1
API Gateway
European regional gateway serving EMEA traffic with data residency pinned to EU infrastructure.
- Service
- API Gateway (regional)
- Requests / sec
- 4.1k sustained
- Throttle limit
- 8k rps burst
- p50 / p99
- 7 ms / 26 ms
- Integration
- ALB → EKS inference pods
- Auth
- JWT authorizer · mTLS internal
ec2-p5-eu-west-1a
EC2 Compute Cluster
European GPU cluster for EMEA inference plus fine-tuning workloads scheduled in off-peak windows.
- Instance
- p5.48xlarge
- Nodes
- 3 / 3 in service
- GPUs
- 24× H100 80GB
- vCPU / RAM
- 576 vCPU · 9.6 TB
- EBS
- 72 TB io2 · 32,000 IOPS
- Util / Temp
- 74% · 61°C
- Networking
- 100 Gbps EFA enabled
eu-west-1b
ec2-p5-eu-west-1b
EC2 Compute Cluster
Second European AZ. Currently offline pending capacity restoration after a power event; traffic re-routed to 1a.
- Instance
- p5.48xlarge
- Nodes
- 0 / 3 in service
- GPUs
- 24× H100 80GB (halted)
- vCPU / RAM
- 576 vCPU · 9.6 TB
- EBS
- 72 TB io2 · detached
- Last seen
- 14h 22m ago
- Incident
- INFRA-2291 · power event
vec-index-eu-west-1
Vector Index
Read replica of the primary vector index for EMEA retrieval queries, keeping EU user data in-region.
- Engine
- OpenSearch 2.11 (k-NN)
- Vectors
- 148M · 768-dim (replica)
- Index type
- HNSW · ef_search 128
- Storage
- 2.4 TB gp3
- p95 query latency
- 21 ms
- Replication lag
- 2.1s behind primary