Skip to content

Xinference Example

Xorbits Inference(Xinference) is an open-source project to run language models on your own machine. You can use it to serve open-source LLMs like Llama-2 locally.


Follow the instructions at Using Xinference to setup Xinference and run the llama-2-chat model.


  • API Endpoit:
  • API Key: random strings
  • Model: llama-2-chat

You can find all the available models at


  • Only models with chat in their name are supported.