Is sampling different to llama.cpp?

[ee-lang]

I set the parameters to be identical (and of course temperature to be 0 and the same seed), but I get slightly different responses to exactly the same prompt when running llama.cpp vs llama_cpp.py.
Is there a difference in sampling? (i.e. in logits/probabilities calculation)

[rlleshi]

@ee-lang do you mind mentioning which parameters you are setting (repetition_penalty, top_k, top_p, max_tokens, temperature etc.), and their exact values?

The responses that I'm getting are namely widely different.

[ee-lang]

Every parameter that I could set via the llama_cpp.py interface I set the same. With llama.cpp there a few more variables, but mostly everything was left on the default value. As far as I saw there was no difference between the two models parameters'.

[rlleshi]

Exactly, I also see no differences. This leaves me to wonder why the responses through this wrapper are so off.

Just one example below:

Prompt: What is 10+10 - 100?

llama.cpp: The calculation is as follows:\n\n10 + 10 = 20\n\n20 - 100 = -80
llama-cpp-python: 20