Project · core infrastructure
Model Serving / Inference Layer
Unified model-serving interface for apps and agent systems.
Status
Integration
Runtime
Local and remote model endpoints
Systems
Router, registry, clients
Overview
Inference integration layer providing common interfaces across local and remote providers.
Technology Stack
OllamaAPI providersNode gateways
Capabilities
- Provider abstraction
- Fallback routes
- Model policies
Implementation
Unified model clients + routing contracts.
Problem
Projects needed consistent model access patterns without duplicated integration logic.
Architecture
- Client Layer
- Apps
- Agents
- Inference Router
- policy
- fallback
- selection
- Model Backends
- local models
- remote APIs
Outcome
Lower integration friction and stronger control over latency/cost.
What's Next
- Benchmark dashboard
- Version pinning