Using model='llama-3.1-sonar-large-128k-online', for a functionality to deliver insights and recommendations about the most updated salaries by region and role, and recent or upcoming labour regulations. That model is 70B and have tried with and without # frequency_penalty=0.2,. With the ‘frequency_penalty’ parameter, the response is perfect and does not repeat a specific word at the end. What should I do? Is the problem my instructions? Why might that be?
My recommendation is to not specify the sampling parameters such as frequency_penalty, our defaults are tuned for sanity to avoid this problem