Consider evaluating Wenmode as a replacement for Mistune

[lepture]

Summary

Would the nbconvert maintainers be open to evaluating Wenmode as a possible replacement for Mistune in nbconvert's Markdown rendering path?

This is not a request to switch immediately. I am opening this to ask whether an exploratory spike or prototype PR would be useful.

Current context

nbconvert currently depends on mistune>=2.0.3,<4 and has a dedicated markdown_mistune.py integration layer with compatibility handling for Mistune 2 and 3.

Wenmode is a newer Markdown toolkit by the same author as Mistune. It was designed around explicit rule composition, mdast-compatible AST output, plugins, renderer separation, and safer HTML defaults.

Why this might be relevant to nbconvert

A Wenmode-based Markdown integration might make some parts of nbconvert's Markdown pipeline easier to reason about:

• explicit rule/preset selection for the Markdown dialect nbconvert needs;
• plugins for features such as tables, task lists, strikethrough, definition lists, math, and other extension syntax;
• clearer separation between parsing, AST transforms, and HTML rendering;
• a structured AST path that may be useful for future conversion or filtering work;
• potentially less compatibility code around Mistune 2 vs 3 over time.

Possible spike scope

If maintainers are interested, I can prepare a small exploratory PR that does not commit nbconvert to switching. The spike could:

1. Add a Wenmode-based Markdown renderer path behind an internal helper or optional experiment.
2. Map nbconvert's current Mistune plugin behavior to Wenmode presets/plugins.
3. Compare rendered HTML for existing Markdown cell fixtures.
4. Check behavior for:
   • tables;
   • task lists;
   • strikethrough;
   • definition lists;
   • URL autolinks;
   • inline and block math;
   • raw HTML passthrough/escaping expectations;
   • notebook-specific HTML output.
5. Run the existing nbconvert test suite and report compatibility gaps.

Open questions

Before spending time on a prototype, I would like to know whether maintainers think this direction is worth exploring.

Some questions I expect would need maintainer input:

• Is exact rendered HTML compatibility with the current Mistune path required?
• Would a replacement need to support both trusted and sanitized output modes?
• Is adding a new Markdown dependency acceptable if it reduces nbconvert-specific compatibility code?
• Should this wait until Wenmode reaches a stable 1.0 release?

If this sounds useful, I am happy to work on a minimal exploratory PR and document the differences.