module
0.0.0-20241115211940-95d2488c9f64
Repository: https://github.com/maruel/n-bits-go.git
Documentation: pkg.go.dev
# README
n-bits
Algorithms to better understand DNN (deep neural networks) weights.
This tool gathers bit usage in a ML model and prints statistics. It is quite performance optimized, processing 405 billions weights (over 800GiB) in slightly more than 3 minutes.
Installation
go install github.com/maruel/n-bits-go/cmd/n-bits@latest
Usage
Analyze popular models in increasingly large size:
# LLM
n-bits analyze -hf-repo Qwen/Qwen2.5-0.5B
n-bits analyze -hf-repo facebook/MobileLLM-1B
n-bits analyze -hf-repo meta-llama/Llama-3.2-1B
n-bits analyze -hf-repo HuggingFaceTB/SmolLM2-1.7B
n-bits analyze -hf-repo google/gemma-2-2b
n-bits analyze -hf-repo microsoft/Phi-3.5-mini-instruct
n-bits analyze -hf-repo mistralai/Mistral-7B-v0.3 -hf-glob model*.safetensors
n-bits analyze -hf-repo meta-llama/Llama-3.1-70B-Instruct
n-bits analyze -hf-repo meta-llama/Llama-3.1-405B-Instruct
# Image
n-bits analyze -hf-repo Qwen/Qwen2-VL-2B-Instruct
n-bits analyze -hf-repo stabilityai/stable-diffusion-3.5-large
n-bits analyze -hf-repo stabilityai/stable-diffusion-3.5-large -hf-glob text_encoder_3/*.safetensors
n-bits analyze -hf-repo stabilityai/stable-fast-3d
n-bits analyze -hf-repo meta-llama/Llama-3.2-11B-Vision
n-bits analyze -hf-repo meta-llama/Llama-Guard-3-11B-Vision
n-bits analyze -hf-repo black-forest-labs/FLUX.1-dev -hf-glob flux1-dev.safetensors
# Audio
n-bits analyze -hf-repo openai/whisper-large-v3-turbo
n-bits analyze -hf-repo openai/whisper-large-v3 -hf-glob model.safetensors
n-bits analyze -hf-repo openai/whisper-large-v3 -hf-glob model.fp32*.safetensors
The results range from 6.3% (openai/whisper-large-v3-turbo in float16), with SD3.5 being a close second at 6.4% to (openai/whisper-large-v3 in float32) 50% wasted. The median is around 17%.
Metadata
Dump the metadata for each of the models you downloaded up to now:
find ~/.cache/huggingface/hub/models--*/snapshots -type l -name '*.safetensors' -exec n-bits metadata -name {} \;