JaumePrats
commited on
Commit
·
8de97bb
1
Parent(s):
f5e42c3
add sampling params recommendation in usage section
Browse files
README.md
CHANGED
|
@@ -219,6 +219,7 @@ At what temperature does water boil?<|im_end|>
|
|
| 219 |
<|im_start|>assistant
|
| 220 |
Water turns into vapor at 100°C.<|im_end|>
|
| 221 |
```
|
|
|
|
| 222 |
|
| 223 |
---
|
| 224 |
|
|
|
|
| 219 |
<|im_start|>assistant
|
| 220 |
Water turns into vapor at 100°C.<|im_end|>
|
| 221 |
```
|
| 222 |
+
Loading the model with transformers' `AutoModelForCausalLM` guarantees that adequate sampling parameters are used during generation. If using alternative inference libraries such as vLLM, Ollama, or SGLang, it is crucial to verify that optimal parameters are used. To this end, in order to ensure optimal results, we recommend using **temperatures around 0-0.2** without any type of repetition penalties applied.
|
| 223 |
|
| 224 |
---
|
| 225 |
|