vLLM MoE config for RTX 5090

#5
by j4ys0n - opened

Generated the MoE config for this model for use with an RTX 5090 GPU. There's also a script so you can create your own for different GPUs.
https://github.com/MissionSquad/vllm-moe-configs

Sign up or log in to comment