Docker
Docker builds
Check out prebuilt docker containers for different targets:
| Target | Container | Latest Release v1.0 |
|---|---|---|
| NVIDIA TURING | gdiamos/scalarlm-nvidia-7.5:latest | gdiamos/scalarlm-nvidia-7.5:v1.0 |
| NVIDIA AMPERE | gdiamos/scalarlm-nvidia-8.0:latest | gdiamos/scalarlm-nvidia-8.0:v1.0 |
| NVIDIA AMPERE | gdiamos/scalarlm-nvidia-8.6:latest | gdiamos/scalarlm-nvidia-8.6:v1.0 |
| NVIDIA HOPPER | gdiamos/scalarlm-nvidia-9.0:latest | gdiamos/scalarlm-nvidia-9.0:v1.0 |
| NVIDIA BLACKWELL | gdiamos/scalarlm-nvidia-12.0:latest | gdiamos/scalarlm-nvidia-12.0:v1.0 |
| ARM | gdiamos/scalarlm-arm:latest | gdiamos/scalarlm-arm:v1.0 |
| AMD | gdiamos/scalarlm-amd:latest | gdiamos/scalarlm-amd:v1.0 |
| x86 | gdiamos/scalarlm-cpu:latest | gdiamos/scalarlm-cpu:v1.0 |
For example, to launch a development server on a modern macbook, e.g. m2
docker run -it -p 8000:8000 --entrypoint /app/cray/scripts/start_one_server.sh gdiamos/scalarlm-arm:latestPassing arguments
You can change most ScalarLM arguments using environment variables. For example, to change the model name on Hopper:
docker run --gpus all -it -p 8000:8000 -e SCALARLM_MODEL=EssentialAI/rnj-1-instruct --entrypoint /app/cray/scripts/start_one_server.sh gdiamos/scalarlm-nvidia-9.0:latest