Onnx Webgpu Converter

Tags

Install Options

Works With

Source

Skill Preview

# ONNX WebGPU Model Converter

Convert any HuggingFace model to ONNX and run it in the browser with Transformers.js + WebGPU.

## Workflow Overview

1. **Check if ONNX version already exists** on HuggingFace
2. **Set up Python environment** with optimum
3. **Export model to ONNX** with optimum-cli
4. **Quantize** for target deployment (WebGPU vs WASM)
5. **Upload to HuggingFace Hub** (optional)
6. **Use in Transformers.js** with WebGPU

## Step 1: Check for Existing ONNX Models

Before converting, check if the model already has an ONNX version:

- Search `onnx-community/<model-name>` on HuggingFace Hub
- Check the model repo for an `onnx/` folder
- Browse https://huggingface.co/models?library=transformers.js (1200+ pre-converted)

If found, skip to Step 6.

## Step 2: Environment Setup

```bash
# Create venv (recommended)
python -m venv onnx-env && source onnx-env/bin/activate

# Install optimum with ONNX support
pip install "optimum[onnx]" onnxruntime

# For GPU-accelerated export (optional)
pip install onnxruntime-gpu
```

**Verify installation:**
```bash
optimum-cli export onnx --help
```

## Step 3: Export to ONNX

### Basic Export (auto-detect task)

```bash
optimum-cli export onnx --model <model_id_or_path> ./output_dir/
```

### With Explicit Task

```bash
optimum-cli export onnx \
  --model <model_id> \
  --task <task> \
  ./output_dir/
```

**Common tasks:** `text-generation`, `text-classification`, `feature-extraction`, `image-classification`, `automatic-speech-recognition`, `object-detection`, `image-segmentation`, `question-answering`, `token-classification`, `zero-shot-classification`

For decoder models, append `-with-past` for KV cache reuse (default behavior):
`text-generation-with-past`, `text2text-generation-with-past`, `automatic-speech-recognition-with-past`

### Full CLI Reference

| Flag | Description |
|------|-------------|
| `-m MODEL, --model MODEL` | HuggingFace model ID or local path (required) |
| `--task TASK` | Export task (auto-detected if on Hub) |
| `--opset OPSET` | ONNX opset version (default: auto) |
| `--device DEVICE` | Export device, `cpu` (default) or `cuda` |
| `--optimize {O1,O2,O3,O4}` | ONNX Runtime optimization level |
| `--monolith` | Force single ONNX file (vs split encoder/decoder) |
| `--no-post-process` | Skip post-processing (e.g., decoder merging) |
| `--trust-remote-code` | Allow custom model code from Hub |
| `--pad_token_id ID` | Override pad token (needed for some models) |
| `--cache_dir DIR` | Cache directory for downloaded models |
| `--batch_size N` | Batch size for dummy inputs |
| `--sequence_length N` | Sequence length for dummy inputs |
| `--framework {pt}` | Source framework |
| `--atol ATOL` | Absolute tolerance for validation |

### Optimization Levels

| Level | Description |
|-------|-------------|
| O1 | Basic general optimizations |
| O2 | Basic + extended + transformer fusions |
| O3 | O2 + GELU approximation |
| O4 | O3 + mixed precision fp16 (GPU only, requires `--device cuda`) |

## Step 4: Quantize for Web Deployment

### Quantization Types for Transformers.js

| dtype | Precision | Best For | Size Reduction |
|-------|-----------|----------|----------------|
| `fp32` | Full 32-bit | Maximum accuracy | None (baseline) |
| `fp16` | Half 16-bit | WebGPU default quality | ~50% |
| `q8` / `int8` | 8-bit | WASM default, good balance | ~75% |
| `q4` / `bnb4` | 4-bit | Maximum compression | ~87% |
| `q4f16` | 4-bit weights, fp16 compute | WebGPU + small size | ~87% |

### Using optimum-cli quantization

```bash
# Dynamic quantization (post-export)
optimum-cli onnxruntime quantize \
  --onnx_model ./output_dir/ \
  --avx512 \
  -o ./quantized_dir/
```

### Using Python API for finer control

```python
from optimum.onnxruntime import ORTQuantizer, ORTModelForSequenceClassification
from optimum.onnxruntime.configuration import AutoQuantizationConfig

model = ORTModelForSequenceClassification.from_pretrained("./output_dir/")
quantizer = ORTQuantizer.from_pretrai