GroqCloud
A cloud API that runs popular open-weight models, such as Llama and OpenAI's gpt-oss, plus Whisper speech-to-text, on Groq's own LPU chips for very high output speed. It is OpenAI-compatible, so existing client code works by changing the base URL, and has a free plan with rate limits.
Platform
AI & Machine Learning
Groq
Under an hour to a first call
Intermediate
Free plan; pay per token beyond it
What GroqCloud is
GroqCloud is Groq's hosted inference platform. Instead of GPUs, Groq runs models on its own chips, which it calls LPUs, and is known for very high output speed. It serves a curated set of open-weight models rather than its own proprietary model family.
What it serves
As of October 2026 the production models listed in the docs include Llama 3.1 8B, Llama 3.3 70B, OpenAI's open-weight gpt-oss 120B and 20B, and Whisper models for speech-to-text, with context windows of about 131K tokens for the text models. The docs list speeds of several hundred to about a thousand tokens per second depending on the model. The list changes, so check the models page before you plan around a specific one.
Why it matters when choosing a stack
- Speed: for voice assistants, live autocomplete or any feature where users wait, a much faster host can matter more than a slightly smarter model.
- Open-weight licences without GPUs: you get models such as gpt-oss (Apache 2.0) without running servers yourself.
- Easy switching: the API is OpenAI-compatible at
https://api.groq.com/openai/v1, so you can point existing code at it and compare it with other providers using the same golden test set.
Remember that the same open model can behave slightly differently on different hosts, depending on precision and settings. Test the exact endpoint you will use.
Plans and limits
There is a Free plan with base rate limits that vary by model, and a Developer plan with higher limits and access to Batch and Flex processing. Rate limits apply to the whole organisation, not to each user, so a team sharing one account shares the limits.
Prerequisites and time
An account, an API key and any OpenAI-compatible SDK. A first call takes minutes.
Free or paid?
Free to start within the free plan's limits; beyond that you pay per token at the prices on its pricing page.
Common questions
Is faster always better? Only when a person is waiting. For overnight batch jobs, price matters more than speed, so compare cost per task as well.
Best for: latency-sensitive features on open-weight models, and quick speed comparisons against other hosts.
Ready to start?
Opens on Groq in a new tab.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.