Skip to content

Nemo ASR Parakeet RNNT 1.1B Pipeline Tuning with Torch Tweak

This example demonstrates how to use Torch Tweak to tune the Nemo ASR with Parakeet RNNT 1.1B model.

Environment Setup

You can use either of the following options to set up the environment:

Option 1 - virtual environment managed by you

Activate your virtual environment and install the dependencies:

pip install --index-url https://download.pytorch.org/whl/xpu \
        --extra-index-url https://pypi.org/simple \
        .

Option 2 - virtual environment managed by uv

Install dependencies:

uv sync

Usage

Sample audio file

The example uses a sample audio file that is downloaded automatically when you run the commands below without an explicit --audio_path. You can also download it manually:

wget https://dldata-public.s3.us-east-2.amazonaws.com/2086-149220-0033.wav

Tuning and inference the model

To tune the ASR model, run:

tune

or

uv run tune

To infer the ASR model, run:

inference

or

uv run inference

To benchmark the ASR model, run

benchmark

or

uv run benchmark

Dynamic batching

The service uses dynamic batching - requests are grouped and processed together for efficiency. Currently, there is one frontend and one worker. To support multiple workers, move batching to a separate service that handles request grouping.

Model Details

Can be found in following pages: * https://huggingface.co/nvidia/parakeet-rnnt-1.1b * https://docs.nvidia.com/nemo-framework/user-guide/24.09/nemotoolkit/asr/models.html