JaumePrats commited on
Commit
8de97bb
·
1 Parent(s): f5e42c3

add sampling params recommendation in usage section

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -219,6 +219,7 @@ At what temperature does water boil?<|im_end|>
219
  <|im_start|>assistant
220
  Water turns into vapor at 100°C.<|im_end|>
221
  ```
 
222
 
223
  ---
224
 
 
219
  <|im_start|>assistant
220
  Water turns into vapor at 100°C.<|im_end|>
221
  ```
222
+ Loading the model with transformers' `AutoModelForCausalLM` guarantees that adequate sampling parameters are used during generation. If using alternative inference libraries such as vLLM, Ollama, or SGLang, it is crucial to verify that optimal parameters are used. To this end, in order to ensure optimal results, we recommend using **temperatures around 0-0.2** without any type of repetition penalties applied.
223
 
224
  ---
225