antoinelyset/openhermes-2-mistral-7b-awq

When decoding text, samples from the top k most likely tokens; lower to ignore less likely tokens

When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens

The JSON stringified of the messages (array of objects with role/content like OpenAI) to predict on

Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.

Max new tokens

Whether to use beam search instead of sampling