(Optional) If you are running decoding with gemma-2 models, you will also need to install flashinfer. python -m pip install flashinfer -i https://flashinfer.ai/whl ...