I am still using llama 70b chat via perplexyty API because of the free included api credits in the perplexity subscription. But 1 euro per million tokens seems overpriced nowadays for this model. But I cannot find a new price list from perplexity with all available chat models, where do I find them? I can only find a list of the available ONLINE model but I dont need web search for my api calls.
Perplexity’s API has been pivoting recently, they no longer offer offline models via their API. I don’t imagine they will offer offline models in the future either; they have been slowly moving away from hosting open-source models to hosting their own custom online search models over the past year. They have a list of their currently supported models here.
If you’re looking to run the latest offline Llama models, Meta has partnered with several different cloud providers that you can use that have serverless APIs, most of which are OpenAI API compatible. They have a guide here. I have personally used AWS Bedrock, Azure AI Studio, and Groq. I would recommend Azure AI Studio if you wanted a similar experience to Perplexity’s API, as it is also OpenAI API compatible. I’d recommend Groq if you want lightning-fast inference times.
I hope this helps!