textgen-ui is (a lot!) faster and gives better answers than llama-cpp-python

[vmajor]

Well this sucks because my code relies on llapa-cpp-python directly, not through an API and I have an entire roadmap for how I am going to build an agent system using the llama-cpp-python as the provider.

1. textgen-ui gives better answers than llama-cpp-python while using the same model and same settings that are available to both textgen and llama-cpp-python. Textgen has more settings, but I am not sure how relevant they are. I fixed the same seed.
2. Textgen is approximately 10x faster

But textgen uses llama-cpp-python for some of its functionality, it is the requirement, so I am confused and kind of exhausted. And I am hoping there is an easy answer that would allow me to get the same performance from llama-cpp-python as I get with what is mostly its GUI.

[vmajor]

I just noticed that when I use "chat" mode in textgen-ui I get the same poor response as when I prompt the model with llama-cpp-python. I guess now I need to find out why does textgen 'default' mode work better than 'chat' mode and native llama-cpp-python

[vmajor]

never mind... thank you for participating in my rubber ducky moment. The difference was due to the prompting. I changed the default prompt structure from my formerly trusty 'alpaca' to just asking the question, and the model gained 50 IQ points.