LLM Hub
On-demand hosting of open-weight models on NCSA HPC, with a streaming chat interface and an OpenAI-compatible API — so your data never leaves University infrastructure.
What it is
LLM Hub is a model-hosting service built at NCSA. Instead of relying on a commercial vendor, you spin up the open-weight model you need on NCSA hardware, use it, and shut it down when you’re done — with no servers of your own to run.
You work through a friendly dashboard: launch a model, chat with it in your browser or point your code at its private endpoint, and watch its status the whole time.
When to use it
Good fit
- You want private, self-hosted models with a friendly dashboard.
- You need both an interactive chat UI and a programmatic endpoint.
- Your data must stay on University / NCSA infrastructure.
- You want to try several open-weight models on demand.
Look elsewhere if
- You need commercial models (GPT, Claude) — use a gateway like Lumen.
- You need dollar-based budgets or cost enforcement (not tracked here).
- You want document Q&A with citations — see IllinoisChat.
- You need massive offline batch throughput — see LLMFlux.
Key capabilities
| Models | Deploy the open-weight model you choose, on demand, as a GPU job on NCSA hardware. Each deployment gets its own private endpoint you can chat with or call from code, and can optionally be reached from off-campus without a VPN. |
|---|---|
| Access | Sign in with your Illinois NetID; generate API keys (valid 30 days) for programmatic access. Share any deployment with colleagues by email — including an invite for those not yet registered. |
| Monitoring | Watch each deployment’s status and live logs in real time, and see per-deployment usage metrics. Rate chat responses to flag quality. |
| Cost | Free to use — no per-token charges. Compute draws on your NCSA/HPC allocation, and idle deployments shut down automatically when their time is up. |
| Where it runs | NCSA HPC infrastructure. Your prompts and data stay within University control. |
Data & hosting note
LLM Hub runs models on NCSA hardware, so prompts stay on University infrastructure — a good choice for sensitive work.
What’s recorded: chats in the web interface are saved to your history and can be deleted; requests sent directly to a deployment’s API endpoint pass straight through and aren’t recorded.