Repository navigation

Add TDT loss kernel - #46048

Draft

ebezzam wants to merge 95 commits into

huggingface:mainfrom

ebezzam:tdt_loss_kernel

Draft

Add TDT loss kernel#46048
ebezzam wants to merge 95 commits into
huggingface:mainfrom
ebezzam:tdt_loss_kernel

Commits on Feb 20, 2026
- parakeet tdt intergration
  
  Hainan Xu
  authored and
  lmaksym
  committed
  fa7d6e0
- Add TDT decoder support for Parakeet ASR models
  lmaksym
  committed
  f2b4938
Commits on Feb 25, 2026
- Add expected outputs for TDT, small fixes.
  ebezzam
  committed
  fa36657
- Separate CTC and TDT generate outputs.
  ebezzam
  committed
  05e2e34
- Work with auto device, better init,
  ebezzam
  committed
  bb5ff33
Commits on Feb 26, 2026
- Test timestamps and expose token duration.
  ebezzam
  committed
  9ec79b0
- Add reproducer link.
  ebezzam
  committed
  33f128e
Commits on Feb 27, 2026
- fix: align TDT training and decoding with NeMo implementation
  lmaksym
  committed
  760b4b6
- revert: restore lasr generated files to original state
  lmaksym
  committed
  b33002f
- warn: torchaudio rnnt_loss does not train duration head
  lmaksym
  committed
  48b39dd
Commits on Mar 2, 2026
- Relax timestamp test, and test nits.
  ebezzam
  committed
  e9f23ab
Commits on Mar 3, 2026
- feat: TDT training
  lmaksym
  committed
  e2b97aa
- chore: for cuda detection and run without patching
  lmaksym
  committed
  6b9fc73
- Equivalent timestamp processing as Nemo, and various nits/cleanup.
  ebezzam
  committed
  6c879bc
- Merge branch 'parakeet-tdt' of github.com:lmaksym/transformers into parakeet-tdt
  ebezzam
  committed
  149e17f
- Simplify durations config.
  ebezzam
  committed
  36bfa63
- Update training examples.
  ebezzam
  committed
  2df0ccc
- chore: enable parralelism
  lmaksym
  committed
  388c6d3
Commits on Mar 4, 2026
- chore: performance optimization
  lmaksym
  committed
  08b2b55
- fix: formatting
  lmaksym
  committed
  0c4e05a
Commits on Mar 5, 2026
- Doc and testing nits
  ebezzam
  committed
  1ddd804
Commits on Mar 6, 2026
- Use active mask from current step, and nits.
  ebezzam
  committed
  f512670
- Better pre-allocate.
  ebezzam
  committed
  07d8e35
- TDT has separate pad token and blank token.
  ebezzam
  committed
  fab050a
- Merge branch 'main' into parakeet-tdt
  ebezzam
  authored
  c438565
- Regenerate lasr.
  ebezzam
  committed
  86d980c
- Merge branch 'parakeet-tdt' of github.com:lmaksym/transformers into parakeet-tdt
  ebezzam
  committed
  895c4a0
Commits on Mar 7, 2026
- Style checks and nits
  ebezzam
  committed
  ab21380
- Nits, put back ctc loss test
  ebezzam
  committed
  d0141d5
Commits on Mar 10, 2026
- More standard model output.
  ebezzam
  committed
  f7529d4
- Style
  ebezzam
  committed
  77b95d7
Commits on Mar 23, 2026
- Remove compute_loss flag and allow monkey patching to tdt loss
  ebezzam
  committed
  94eae66
- Update src/transformers/models/parakeet/modular_parakeet.py
  
  ebezzam
  and
  eustlb
  authored
  f7d4067
- Address various comments.
  ebezzam
  committed
  f75c17b
Commits on Mar 24, 2026
- More compatible with Transformers forward/generate approach
  ebezzam
  committed
  5a49b65
- compile option for generation and decoder cache
  ebezzam
  committed
  881233f
- Cleaner, better conventions.
  ebezzam
  committed
  b41a8ee
- Merge branch 'main' into parakeet-tdt
  ebezzam
  authored
  897753a
- Update with main.
  ebezzam
  committed
  6c914db
Commits on Mar 26, 2026
- doc nits
  ebezzam
  committed
  756cee1
- Imitate whisper for encoder outputs as input
  ebezzam
  committed
  f30c536
- Address tests and nits.
  ebezzam
  committed
  fa95fc8
- Inherit from GenerateMixIn for get_compiled_call
  ebezzam
  committed
  5df7f28
- Comment nit
  ebezzam
  committed
  cd706d4
Commits on Apr 15, 2026
- forward cleanup
  eustlb
  committed
  a47ed8a
- generate cleanup + separate generation file
  eustlb
  committed
  13b68ce
- generate: add _supported_generation_modes
  eustlb
  committed
  72c1ad0
- automatic init of the loss
  eustlb
  committed
  8e23b3d
- modular cleanups
  eustlb
  committed
  1cc39fd
- use is_encoder_decoder
  eustlb
  committed
  531f297
- timestamp processing fully from tokens + durations
  eustlb
  committed
  2c0f23a
- convertion script update
  eustlb
  committed
  cef6639
- test update
  eustlb
  committed
  fd3cf9b
- make
  eustlb
  committed
  e63a5bf
- Merge branch 'main' into parakeet-tdt
  eustlb
  authored
  f9d1a4f
- test update
  eustlb
  committed
  43ee7cd
- test update
  eustlb
  committed
  c2a0f78
Commits on Apr 16, 2026
- ensure correct loss computation
  eustlb
  committed
  1fd7ed7
- kernel loss
  eustlb
  committed
  7cc9d2e
- test loss integration
  eustlb
  committed
  e753eab
- push to hub pr
  eustlb
  committed
  ed3fa4d
- integration tests to rely fully on transcripts
  eustlb
  committed
  ab66b23
- udpate fixtures
  eustlb
  committed
  a5ba0c6
- we don't need to monkey patch with numba anymore!
  eustlb
  committed
  48279a6
- fix pipeline usage
  eustlb
  committed
  1d7680d
- nit
  eustlb
  committed
  59ddced
- fix usage
  eustlb
  committed
  31490d1
Commits on Apr 17, 2026
- Pass through tests and examples: improve kernel fallback, update with nvidia checkpoint, style checks.
  ebezzam
  committed
  d8eb1b6
- Update checkpoint
  ebezzam
  committed
  1f1b912
- Merge branch 'main' into parakeet-tdt
  ebezzam
  authored
  9ab08d1
- Add TDT to mapping after merge.
  ebezzam
  committed
  fd9f8b1
Commits on Apr 20, 2026
- Fix lasr generate test.
  ebezzam
  committed
  136f676
- Output attention mask if labels provided for computing loss.
  ebezzam
  committed
  833d289
Commits on May 13, 2026
- Apply suggestion from @ArthurZucker
  
  ebezzam
  and
  ArthurZucker
  authored
  a1c62a1
- Improve ParakeetTDTDecoderCache definition and usage.
  ebezzam
  committed
  8683570
- Remove tuple parsing.
  ebezzam
  committed
  1d4b0f4
- processor refactor
  ebezzam
  committed
  a418eca
- Merge branch 'parakeet-tdt' of github.com:lmaksym/transformers into parakeet-tdt
  ebezzam
  committed
  5d0c631
Commits on May 18, 2026
- Update conversion.
  ebezzam
  committed
  5c603c1
Commits on May 19, 2026
- Merge branch 'main' into tdt_loss_kernel
  ebezzam
  authored
  09ba99c
- Modular after merge.
  ebezzam
  committed
  e743b2d
- Don't allow all kernels.
  ebezzam
  committed
  8d09cb6
Commits on Sep 28, 2026
- Merge remote-tracking branch 'origin/main' into tdt-loss-kernel
  Deep-unlearning
  committed
  5176486
- Use the kernels-community/tdt-loss kernel for the TDT loss
  
  Deep-unlearning
  and
  claude
  committed
  1cbb74f
- Run the TDT integration loss checks before the train-mode backward
  
  Deep-unlearning
  and
  claude
  committed
  12bbc03
- Fall back to PyTorch when the tdt-loss kernel cannot be loaded, fix integration test loops
  
  Deep-unlearning
  and
  claude
  committed
  475304e
- Use the kernels exchange pattern for the TDT loss
  Deep-unlearning
  committed
  28b0a40
- Skip the kernel checks when Hub kernels are disabled
  Deep-unlearning
  committed
  d6e739c
Commits on Sep 29, 2026
- Address review: mode flags for the tdt_loss kernel mapping, keep only the kernel exchange test
  Deep-unlearning
  committed
  3f02948
Commits on Oct 2, 2026
Commits on Oct 5, 2026
- Address review: drop the kernels-disabled skip in test_use_kernels
  Deep-unlearning
  committed
  8c0d977
Commits on Oct 10, 2026
- Only kernelize tdt_loss in test_use_kernels, fix the kernel Hub link
  Deep-unlearning
  committed
  3321659
Commits on Oct 13, 2026
- Address review: drop the kernel note in the docstring and the test mapping restriction
  Deep-unlearning
  committed
  3d23b40

Commits

Commits on Feb 20, 2026

Commits on Feb 25, 2026

Commits on Feb 26, 2026

Commits on Feb 27, 2026

Commits on Mar 2, 2026

Commits on Mar 3, 2026

Commits on Mar 4, 2026

Commits on Mar 5, 2026

Commits on Mar 6, 2026

Commits on Mar 7, 2026

Commits on Mar 10, 2026

Commits on Mar 23, 2026

Commits on Mar 24, 2026

Commits on Mar 26, 2026

Commits on Apr 15, 2026

Commits on Apr 16, 2026

Commits on Apr 17, 2026

Commits on Apr 20, 2026

Commits on May 13, 2026

Commits on May 18, 2026

Commits on May 19, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Oct 2, 2026

Commits on Oct 5, 2026

Commits on Oct 10, 2026

Commits on Oct 13, 2026