Key Insight
| Sizes | Latency target | API compatibility | Context window |
|---|---|---|---|
| 8B · 70B · 405B | <500ms | OpenAI-compatible | 128K |
Call Llama 3.1 in three lines
OneInfer's endpoint is OpenAI-compatible, so the official OpenAI SDKs work unchanged — just point the base URL at OneInfer.
from openai import OpenAI
client = OpenAI(base_url="https://api.oneinfer.ai/v1", api_key="YOUR_ONEINFER_KEY")
resp = client.chat.completions.create(
model="llama-3.1-70b",
messages=[{"role": "user", "content": "Explain kernel fusion in one paragraph."}],
stream=True,
)
for chunk in resp:
print(chunk.choices[0].delta.content or "", end="")Also available: llama-3.1-8b for lean, low-latency paths and llama-3.1-405b for frontier reasoning. Streaming, multi-turn, system prompts and function calling are supported — see the Chat Completions docs.
Which Llama 3.1 size should you use?
| Model | Best for | Price / 1M in | Price / 1M out |
|---|---|---|---|
| llama-3.1-8b | High-volume, low-latency, classification, routing | $0.10 | $0.20 |
| llama-3.1-70b | General reasoning, chat, RAG, agents | $0.60 | $0.80 |
| llama-3.1-405b | Frontier reasoning, hardest tasks | $2.70 | $2.70 |
Illustrative pricing for layout — replace with live OneInfer rates before publishing. Estimate your own spend in the cost calculator.
Why run Llama 3.1 on OneInfer
Kernel-optimized for the model, not just the framework
OneInfer forges Triton and CUDA kernels tuned to Llama 3.1's exact tensor shapes and hot paths, then benchmarks and ships the fastest — throughput you don't get from a generic serving stack.
Intelligent routing and failover
Short prompts take a lean, low-latency path; heavy reasoning is elevated automatically. If a provider slows or fails, traffic rebalances without you paging anyone at 3AM.
One API, every modality
The same key that serves Llama 3.1 also serves vision, audio and video models — and multi-model Chains stitch them together with under 1ms of overhead.
Ship Llama 3.1 today
Free credits to start. OpenAI-compatible, so migration is a base-URL change.
Frequently Asked Questions
+How do I get a Llama 3.1 API key?
Create a free OneInfer account, generate a key in the console, and set it as your API key with the base URL https://api.oneinfer.ai/v1.
+Is the Llama 3.1 API OpenAI-compatible?
Yes. Existing OpenAI SDK code works with only an endpoint and key change, including streaming and function calling.
+Which Llama 3.1 sizes are available?
8B, 70B and 405B. Use 8B for high-volume low-latency work, 70B for general reasoning, and 405B for the hardest tasks.
+Can I get dedicated capacity for Llama 3.1?
Yes. Move any model to a dedicated deployment — cloud, hybrid or self-hosted — for guaranteed capacity and data isolation, on the same API.
Related: All Model APIs · OpenRouter alternative · Best LLM inference API 2026 · Chat Completions API · Cost calculator