Anyscale Example
Anyscale provides fast, cost-efficient, serverless APIs for LLM serving and fine tuning.
Preparation
Sign up to Anyscale and generate an API key from https://app.endpoints.anyscale.com/console/credentials.
Configuration
We're using the mistralai/Mixtral-8x7B-Instruct-v0.1
model in this example.
You can find all the available models from Anyscale website: https://docs.endpoints.anyscale.com/category/supported-models