Models

Models are launched with a template: LLMs with vLLM or SGLang, embeddings and rerankers with Embeddings, images and video with ComfyUI. You can also run anything else yourself (Ollama library, your own weights from Hugging Face) with the other templates.

How model files get to your server

• Model weights are downloaded by the host machine and mounted read-only at /models/<model-id>. They don’t count toward your disk.
• On-demand cache. The first launch of a model anywhere on the platform downloads it from Hugging Face; at the same time we copy it to our storage. Every later launch — by anyone — downloads it from our cache, which is much faster. Hosts also keep models on their disks, so a relaunch on the same machine is almost instant. The catalog marks cached models ⚡.
• Models that nobody launched for 30 days are removed from the cache and are downloaded again on the next launch.
• Gated models (Llama, Gemma) are not cached: add your own Hugging Face token as the HF_TOKEN environment variable (the token’s account must have accepted the model’s license). The model server downloads the weights itself.
• ComfyUI is configured automatically (extra_model_paths): the chosen model’s files appear in the checkpoint / diffusion model / text encoder / VAE / LoRA loaders.

Tiers

Tier	Total VRAM	Typical GPUs
light	up to 12 GB	RTX 3060, 3080 Ti, 4060 Ti, any bigger card
standard	up to 24 GB	RTX 3090, 4090, A5000, A10, L4, or 2×16 GB
pro	up to 80 GB	A100, H100, RTX PRO 6000, 1–2×48 GB, 2–4×24 GB
cluster	more than 80 GB	several 80+ GB GPUs in one server (H100, H200, B200)

The requirement is the total VRAM of the GPUs in one server: a 40 GB model fits one A100 40GB, one 48 GB card, or two 24 GB cards (vLLM splits it with tensor parallelism). The dashboard only offers models that fit your selection.

Text and code (LLM)

Use with vllm or sglang. The model name in API requests is the Hugging Face id.

id	Model	Params	Min VRAM	License	Notes
qwen3-0.6b	Qwen3 0.6B	0.6B	6 GB	Apache-2.0	Classification, routing, tests
qwen3-1.7b	Qwen3 1.7B	1.7B	8 GB	Apache-2.0	Simple chat and extraction
smollm3-3b	SmolLM3 3B	3B	10 GB	Apache-2.0	Long context, reasoning mode
qwen3-4b-instruct	Qwen3 4B Instruct 2507	4B	12 GB	Apache-2.0	Best quality per GB on cheap GPUs
phi-4-mini	Phi-4 mini	3.8B	12 GB	MIT	Math and logic
llama-3.2-3b	Llama 3.2 3B	3B	12 GB	Llama 3.2	Gated, needs HF_TOKEN
gemma-3-4b	Gemma 3 4B	4B	12 GB	Gemma	Vision; gated
qwen3-8b	Qwen3 8B	8B	24 GB	Apache-2.0	All-rounder, thinking mode, 100+ languages
gpt-oss-20b	gpt-oss 20B	21B MoE	24 GB	Apache-2.0	OpenAI open weights, reasoning, tools
qwen2.5-7b-instruct	Qwen2.5 7B Instruct	7B	24 GB	Apache-2.0	Assistants and RAG
qwen2.5-coder-7b	Qwen2.5 Coder 7B	7B	24 GB	Apache-2.0	Code completion
qwen2.5-vl-7b	Qwen2.5 VL 7B	7B	24 GB	Apache-2.0	Images, documents, OCR
deepseek-r1-qwen-7b	DeepSeek-R1 Distill Qwen 7B	7B	24 GB	MIT	Step-by-step reasoning
deepseek-r1-llama-8b	DeepSeek-R1 Distill Llama 8B	8B	24 GB	MIT	Reasoning
mistral-7b-instruct	Mistral 7B Instruct v0.3	7B	24 GB	Apache-2.0	Function calling
llama-3.1-8b	Llama 3.1 8B	8B	24 GB	Llama 3.1	Gated
qwen3-14b	Qwen3 14B	14B	40 GB	Apache-2.0	Smarter than 8B
phi-4	Phi-4 14B	14B	40 GB	MIT	Math, code, structured answers
deepseek-r1-qwen-14b	DeepSeek-R1 Distill Qwen 14B	14B	40 GB	MIT	Deeper reasoning
mistral-nemo-12b	Mistral NeMo 12B	12B	40 GB	Apache-2.0	128k context, multilingual
gemma-3-12b	Gemma 3 12B	12B	40 GB	Gemma	Vision; gated
qwen3-32b	Qwen3 32B	32B	80 GB	Apache-2.0	Flagship dense Qwen3
qwen3-30b-a3b-instruct	Qwen3 30B-A3B Instruct 2507	30B MoE	80 GB	Apache-2.0	Big-model quality, high speed
qwen3-coder-30b	Qwen3 Coder 30B-A3B	30B MoE	80 GB	Apache-2.0	Agentic coding (Cline, Roo, Continue)
qwen2.5-coder-32b	Qwen2.5 Coder 32B	32B	80 GB	Apache-2.0	Top dense coding model
qwen2.5-vl-32b	Qwen2.5 VL 32B	32B	80 GB	Apache-2.0	Documents and charts
deepseek-r1-qwen-32b	DeepSeek-R1 Distill Qwen 32B	32B	80 GB	MIT	Strongest R1 distill
mistral-small-24b	Mistral Small 3.2 24B	24B	80 GB	Apache-2.0	Vision and function calling
gpt-oss-120b	gpt-oss 120B	117B MoE	80 GB	Apache-2.0	Largest OpenAI open model on one 80 GB GPU
gemma-3-27b	Gemma 3 27B	27B	80 GB	Gemma	Vision; gated
llama-3.3-70b	Llama 3.3 70B	70B	160 GB	Llama 3.3	2×80 GB; gated
glm-4.5-air	GLM-4.5 Air	106B MoE	320 GB	MIT	Agents and code; 4×80 GB
qwen3-235b-fp8	Qwen3 235B-A22B FP8	235B MoE	320 GB	Apache-2.0	Frontier level; 4×80 GB Hopper
qwen3-coder-480b-fp8	Qwen3 Coder 480B FP8	480B MoE	640 GB	Apache-2.0	Best open agentic coder; 8×80 GB Hopper
deepseek-r1-0528	DeepSeek-R1 0528	671B MoE	1000 GB	MIT	Full R1; 8×H200
deepseek-v3.1	DeepSeek-V3.1	671B MoE	1000 GB	MIT	Hybrid thinking; 8×H200

FP8 models need Hopper (H100/H200) or newer GPUs for full speed.

Embeddings and rerankers

Use with the embeddings template: /v1/embeddings for embedding models, /v1/rerank for rerankers.

id	Model	Params	Min VRAM	License
qwen3-embedding-0.6b	Qwen3 Embedding 0.6B	0.6B	6 GB	Apache-2.0
qwen3-embedding-8b	Qwen3 Embedding 8B	8B	24 GB	Apache-2.0
bge-m3	BGE-M3	568M	6 GB	MIT
multilingual-e5-large	Multilingual E5 Large	560M	6 GB	MIT
bge-reranker-v2-m3	BGE Reranker v2 M3	568M	6 GB	Apache-2.0

Images

Use with the comfyui template.

id	Model	Params	Min VRAM	License	Notes
sdxl-base	Stable Diffusion XL 1.0	3.5B	8 GB	OpenRAIL++	Huge LoRA ecosystem
juggernaut-xl-v9	Juggernaut XL v9	3.5B	8 GB	CreativeML OpenRAIL-M	Photorealism, portraits
lumina-2	Lumina Image 2.0	2.6B	12 GB	Apache-2.0	Good prompt following
flux1-schnell	FLUX.1 schnell (FP8)	12B	16 GB	Apache-2.0	Top quality in 4 steps, commercial use
hidream-i1-fast	HiDream-I1 Fast (FP8)	17B	24 GB	MIT	State-of-the-art open image model
qwen-image	Qwen-Image (FP8)	20B	40 GB	Apache-2.0	Best text rendering (posters, UI)

Video

Use with the comfyui template.

id	Model	Params	Min VRAM	License	Notes
wan2.1-t2v-1.3b	Wan 2.1 Text-to-Video 1.3B	1.3B	12 GB	Apache-2.0	Video even on 12 GB cards (480p)
wan2.2-ti2v-5b	Wan 2.2 TI2V 5B	5B	24 GB	Apache-2.0	720p text/image-to-video on one 24 GB GPU
mochi-preview	Mochi 1 Preview (FP8)	10B	24 GB	Apache-2.0	Smooth motion
wan2.2-t2v-14b	Wan 2.2 Text-to-Video 14B (FP8)	14B MoE	40 GB	Apache-2.0	Cinematic 720p, 4-step LoRAs included
wan2.2-i2v-14b	Wan 2.2 Image-to-Video 14B (FP8)	14B MoE	40 GB	Apache-2.0	Animate any image

Audio

Speech models are built into the audio templates: speaches (faster-whisper + TTS), whisper (large-v3), kokoro (TTS). See Templates → Audio.

Your own models

• Any Hugging Face model with vLLM: rent pytorch or cuda, then pip install vllm and vllm serve <repo>; or use the custom template with the vLLM image and your own command.
• Ollama library (hundreds of GGUF models): open-webui or ollama templates.
• LoRAs and extra checkpoints for ComfyUI: ComfyUI-Manager, or download into /workspace.