AI infrastructure you own.
Self-hosted inference, vector database, monitoring, and API layer on your VPS. No per-token bills. No vendor lock-in. Nobody can pull it out from under you. The smartest model you can economically own and operate.
Sovereignty, not savings
This is not "cheaper than API." It is AI infrastructure you control. Your data stays on your metal. Your inference runs on your GPU. Your costs are flat and predictable. And nobody can change the pricing, throttle your access, or deprecate your model out from under you.
Data never leaves
Your business data, your customer data, your prompts. Nothing routes through a third-party API. Full privacy by architecture, not by policy.
Flat monthly cost
No per-token surprises. No usage spikes. Your GPU runs at a fixed rate whether you send 10 requests or 10,000. Budget once, run forever.
Swap anything
Models are interchangeable services behind an OpenAI-compatible endpoint. Swap Qwen for DeepSeek without changing application code. No lock-in at any layer.
Twelve layers. One Docker Compose.
Every component is open-source or Apache 2.0 licensed. Battle-tested. Replaceable. The stack deploys as a single Docker Compose on any GPU VPS provider.
| Layer | Component | Why |
|---|---|---|
| OS | Ubuntu 24.04 LTS | Massive driver/tooling ecosystem. Boring. Correct. |
| Container | Docker + Compose | The lingua franca. Every component ships a Dockerfile. |
| PaaS | Coolify v4 | 280+ templates, managed DBs, auto-SSL. Free, Apache 2.0. |
| Edge | Cloudflare | DNS, CDN, DDoS, WAF. Non-negotiable at this price point. |
| Proxy | Caddy | Auto-TLS, clean config. Coolify-native. |
| Database | PostgreSQL + pgvector | Relational + vector search in one engine. No extra dependency. |
| Cache | Redis | Session, queue, rate-limiting, KV cache coordination. |
| Inference | vLLM v0.6+ | OpenAI-compatible API. PagedAttention v2, speculative decoding. |
| Model | Qwen3-Coder-Next (default) | Best self-hosted coding model. Apache 2.0. Swappable. |
| API | FastAPI | Agent/API layer, business logic, auth, rate limiting. |
| Gateway | LiteLLM | Routes local + external APIs. Fallback, cost tracking, unified interface. |
| Monitor | Prometheus + Grafana | GPU utilization, inference latency, error rates, cost per request. |
Pick your GPU. We configure the rest.
Infrastructure cost depends on which GPU tier fits your workload. You pay your VPS provider directly for the hardware; Faction deploys and manages the stack on top of it.
RTX 4090
24GB VRAM. Fits Qwen 27B dense at Q4. Good for general-purpose inference at lower volume.
$255-$505/mo provider cost
L40S
48GB VRAM. Fits Qwen3-Coder-Next full precision. Larger MoE models at Q4.
$525-$630/mo provider cost
A100 80GB
80GB VRAM. Most MoE models, 70B dense at Q4. The workhorse for production agent workloads.
$440-$780/mo provider cost
H100 SXM5
80GB HBM3. Any single-GPU-feasible model at full speed. For throughput-critical production.
$750-$1,825/mo provider cost
Provider pricing from neo-cloud GPU providers (Vast.ai, RunPod, Spheron), paid directly to them. Hyperscaler pricing (AWS, Azure) runs 2-5x higher. Stack certified on both. Add $20-50/month for the non-GPU application VPS.
Managed by Faction, or run it yourself
Managed deployment
Deployed and managed on the GPU tier that fits your workload. You pay your VPS provider directly for the hardware; the managed engagement is quoted fixed at scoping.
- Full stack deployed on your chosen GPU VPS
- Model selection and configuration
- Security hardening and Cloudflare setup
- Prometheus + Grafana monitoring dashboards
- Monthly patching and model updates
- 4-hour response SLA
- Quarterly model evaluation and swap recommendations
Self-serve kit
The same architecture in the hands of builders who can run it themselves: the complete Docker Compose deployment package, hardened configuration files, a step-by-step guide, and provider selection guidance, with a guided-deployment option. The kit and its companion course live at mattmurphy.ai. Deploy independently and upgrade to Faction managed at any time.
When self-hosting makes sense
Can't send data to external APIs
Regulated industries, sensitive customer data, proprietary business logic. If the data can't leave your infrastructure, self-hosting is the only option.
Flat monthly vs. variable per-token
Production agent workloads with high sustained utilization. Break-even against frontier APIs around 160-256M tokens/month at 60-70% utilization.
Nobody can pull the rug
Model deprecation, pricing changes, rate limits, terms-of-service shifts. Your stack runs the same tomorrow regardless of what any provider announces today.
The stack completes the Faction story
The Conveyor Belt
Your team vibe-codes on the stack's inference layer. The Belt finishes, stages, and audits those builds into production. Self-hosted AI meets managed engineering.
Faction AIOS
Full AI operating platform builds can deploy on this infrastructure instead of cloud APIs. Domain services, agents, and evaluation loops running on hardware you own.
API-as-a-service vs. infrastructure you own
API-as-a-service
Zero setup. Pay per token. Data leaves your network on every call. Provider controls pricing, model availability, rate limits, and terms. Fast to start, impossible to control at scale.
Infrastructure you own
Your GPU, your model, your data. Flat monthly cost. Swap models without changing code. No rate limits, no per-token bills, no vendor lock-in. Takes a week to deploy. Runs as long as you want it to.
Hand yourself the keys.
Tell us what you're running. We'll recommend the GPU tier, configure the stack, and hand you the keys to AI infrastructure nobody can take away.
Hosted AI Infrastructure runs on the same methodology as every Faction build. The Belt finishes your code. AIOS builds the platform. The stack provides the foundation.