More (optional) load time verification #715

Activity

1. vasqu

   added

   enhancementNew feature or requestNew feature or request

   on Jul 14, 2026
2. vasqu commented on Jul 14, 2026

   

   vasqu

   on Jul 14, 2026

   ContributorAuthor

   More actions

   cc @sayakpaul @drbh @danieldk
3. sayakpaul commented on Jul 15, 2026

   

   sayakpaul

   on Jul 15, 2026

   Member

   More actions

   Sharing some internal brainstorming messages:

   these are good points but some of these problems are hard to solve with a generic solution. this is because:

   1. kernel authors can add conditional logic for specific archs in their source code, which may only check arch at runtime and throw an error if some specific operation/function is not supported
   2. the JIT kernels require some tool to compile/search and this information is also checked at runtime (and this may look for specific libs or nvcc or etc..)

   there are only two ways for us to know if a kernel will run in a given instance - 1. we try and it succeeds or fails (and record this) 2. the kernel dev tells us (records this somewhere)

   I think the suggestions above are solid and we should improve the ability to specify deps - and more validation so we can throw explicit errors from kernels.

   specifically

   1. I think we should be able to add versions to deps
   2. maybe we can check the default location for nvcc and add a message to specify a path if in a non standard location
   3. I think we can use the existing cuda-capabilities field in build.toml for sonic moe, and then we can update the _validate_variant_dependencies function to check the listed cuda caps at runtime?

   👍React with 👍1Reacted by Anton Vlasjuk
4. danieldk commented on Jul 17, 2026

   

   danieldk

   on Jul 17, 2026

   Member

   More actions

   I think we should be able to add versions to deps

   I am not sure if this is a step in the right direction if this concerns Python dependencies. Kernels were largely standalone, but we are slowly introducing more and more of our own homegrown dependency management with a subpar experience. At some point we might as well start shipping kernels as wheels, because installing kernels and then getting runtime exceptions that certain packages are missing is bad. The additional issue we'll start to run into is that if two kernels require different versions of CuTe-DSL that are not compatible, kernels are not seamless anymore, because we've brought in dependency hell.

   The only direction I can see this work is if we removed Python dependencies completely and started shipping einops, etc. as kernels and provided a way for kernels to depend on other kernels (in a better way than just using get_kernel), but that has some issues for CuTe-DSL, since it contains proprietary code.

   👀React with 👀1Reacted by Anton Vlasjuk
5. sayakpaul commented on Jul 18, 2026

   

   sayakpaul

   on Jul 18, 2026

   Member

   More actions

   started shipping einops, etc. as kernels

   Not sure how that would be received by the community but I am keen on learning more about how you'd envision it.
6. sayakpaul commented on Sep 29, 2026

   

   sayakpaul

   on Sep 29, 2026

   Last edited by sayakpaul

   Member

   More actions

   Since this issue, we have worked on a bunch of things:

   • Load-time validation of kernel metadata (both the following PRs had adjacent PRs that I am not listing for the sake of brevity)):
     • feat: implement capability check in main #798
     • kernels: make kernels version check a validator #808
   • Upcoming: Support fetching for particular CUDA capabilities #698
   • kernels-kernels dependency

   ❤️React with ❤️1Reacted by Anton Vlasjuk