Repository navigation
Conversation
| # The models we train have hidden dim up to 8k anyway (e.g. Llama 70B), so this is fine. | ||
|
|
||
| import math | ||
| from typing import List, Optional |
There was a problem hiding this comment.
Quite a bit of rewrite going on in this file with the porting tool. Some of them are formatting-related, though.
There was a problem hiding this comment.
yea I think we must have ran a formatter and made some small changes manually in the past. its kinda nice that this surfaced those divergence
There was a problem hiding this comment.
But I would argue that it makes it difficult to catch the actual changes significantly more, especially on the existing kernels that we maintain in kernels-community. It becomes difficult to validate if those changes are purely formatting or if we should validate them with additional testing.
The repo ignores all dotfiles (`.*`), which silently swallows `.ports/` recipes and the generated `.port-provenance.json`. Negate both so a new port directory is visible to git rather than quietly dropped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
771626e to
22c1b8a
Compare
| dresidual.reshape(-1, dresidual.shape[-1]) | ||
| ) | ||
| dresidual = maybe_contiguous_lastdim(dresidual.reshape(-1, dresidual.shape[-1])) | ||
| assert dresidual.shape == x.shape |
There was a problem hiding this comment.
This one seems out of the ordinary.
There was a problem hiding this comment.
I think we must have run some formatter at some point since I think the changed version is what matches upstream on the pinned commit https://github.com/Dao-AILab/flash-attention/blob/b02b07e1a10238fe12831b80a8937ed59b1353a5/flash_attn/ops/triton/layer_norm.py#L1196
| torch.cuda.get_device_properties(torch.cuda.current_device()), | ||
| "warp_size", | ||
| 32, | ||
| ) |
There was a problem hiding this comment.
Removes duplicate. So, that is a nice side-effect!
There was a problem hiding this comment.
yea thats a nice catch and this aligns with upstream too https://github.com/Dao-AILab/flash-attention/blob/b02b07e1a10238fe12831b80a8937ed59b1353a5/flash_attn/ops/triton/layer_norm.py#L37
|
this was a great exploration and its nice that it uncovered some divergences with upstream! its a good argument for us to use the tool on flash-ops once stabilized (the only thing this PR would need to change is the location of the port folder) - so we can either close this PR and reopen or make those changes to this PR |
2 participants
Similar to #1092.