01
Overview
North Small Translate can be self-hosted because Cohere publishes official weights, but it is not a lightweight local model. The base checkpoint contains 218 billion total parameters, even though only 25 billion are active per token.
02
Start with the license
The official public weights are governed by CC BY-NC 4.0 plus Cohere Labs' Acceptable Use Policy. That means research and non-commercial deployment are the default public-weight use case. Commercial deployment requires a separate Cohere commercial path.
Do not build a commercial product around the downloaded checkpoint without checking those terms.
03
Official checkpoints
Cohere publishes the base model plus quantized variants on Hugging Face. The official FP8 checkpoint is W8A8. Cohere's launch material also references lower-precision deployment options including W4A4/NVFP4-style configurations.
Prefer Cohere-published checkpoints when you need the clearest provenance. Community conversions can be useful, but they should be labeled as community artifacts rather than official Cohere releases.
04
Hardware guidance
Cohere's launch post lists minimum hardware for its W4A4 deployment path as one B200 or two H100 GPUs. Its model documentation lists suggested hardware of one B200 or two H100s.
Those are official reference configurations, not a promise that every workload will fit with the same headroom. Batch size, framework overhead, context length, concurrency and KV cache can materially change memory use.
05
Why active parameters do not equal memory footprint
The model activates about 25B parameters per token, but the full sparse expert set still has to be stored somewhere. MoE therefore reduces compute per token more directly than it reduces total model-storage requirements.
This is why a 218B-total MoE can have lower per-token compute than a dense 218B model while still requiring substantial accelerator memory.
06
Choosing a precision
Use higher precision when fidelity and reproducibility matter most and you have the hardware. Use FP8 or lower-precision official checkpoints when memory and serving efficiency matter more.
Quantization can reduce memory and improve throughput, but may change translation quality. Benchmark your actual language pairs rather than assuming near-lossless behavior across every domain.
07
Deployment workflow
- Choose an official checkpoint and verify its license.
- Confirm your runtime supports the checkpoint's precision and architecture.
- Provision the documented GPU class with enough headroom for your target context and concurrency.
- Load the model and run simple source→target translation tests.
- Benchmark representative long documents and high-value language pairs.
- Compare quality with your current translation system before migration.
- Add observability for latency, throughput, errors and quality-review feedback.
08
Long-context considerations
The model supports 16K input and 16K output tokens. Translating long documents in fewer chunks can preserve context, but it also increases memory pressure and may complicate retries or quality review.
For production localization, chunk boundaries should follow semantic structure—such as sections or paragraphs—rather than arbitrary token counts whenever possible.
09
Commercial self-hosting
The Hugging Face weights are non-commercial by default. Organizations that need licensed commercial deployment should evaluate Cohere Model Vault or another Cohere-approved commercial path instead of assuming public weights can be used in revenue-generating production.
10
What not to claim
Do not state that North Small Translate runs on a typical consumer GPU. Do not publish a universal VRAM number unless Cohere documents one for the exact checkpoint/runtime combination. Do not present a community GGUF or conversion as an official Cohere model. Do not treat Cohere's benchmark tables as independent validation.
11
When self-hosting makes sense
Self-hosting is most attractive when data sovereignty, network isolation, predictable infrastructure, research access or custom serving controls matter. If operational simplicity is the priority, Cohere's hosted API is likely the easier starting point.
Sources
Primary and supporting sources
Facts were rechecked against the linked sources immediately before publication. Pricing, product availability and rollout status can change.