Should we show syntax errors by default?

[yangdanny97]

Pyrefly has 4 type checking modes for the IDE:

• force-off (no diagnostics)
• default (show diagnostics only if you have a pyrefly.toml or a pyrefly section in your pyproject.toml)
• error-missing-imports (only show diagnostics for missing imports)
• force-on (show diagnostics always)

We're considering making default and error-missing-imports also show diagnostics for syntax errors, even on projects w/o a pyrefly.toml.

The pros are:

• syntax errors are useful
• match pyright's behavior (there is no way to disable or configure syntax errors, to the best of my knowledge)

The cons are:

• if the user has 3.12 syntax and we somehow accidentally guess that they are on 3.9, their experience may be noisy unless they put a pyrefly.toml

Collecting feedback till 2/1

[jorenham]

• if the user has 3.12 syntax and we somehow accidentally guess that they are on 3.9, their experience may be noisy unless they put a pyrefly.toml

For that it might help to include the Python version in the syntax error messages, so that the user immediately knows that it might be a config thing, and not a syntax thing. So something like "... is not valid in Python 3.9".

[yangdanny97]

https://pyrefly.org/sandbox/?project=N4IgZglgNgpgziAXKOBDAdgEwEYHsAeAdAA4CeSImMYABGLrgNoAqAugBT6I3MCUNAWgB8PRAB10NKTQBOMAC4BXGZPwgANCDJywUUoXm4AtlAoBiGgAVSOvTTRY8%2BGgGNc6SAHNlqeRHeEEhYAyjAwNAAW8vLEcIgA9PHa1HqEuDKe8TDo8Zi4LnDxbh4Q3jK%2B-jl06TSoAG6o0KjYsK7uXj5%2B7jS4xF3ocIHoZPIR7gJ1MDJwlTQAvDRiIADMhACMAAxLEiAAvpqoLn6TAGLQMBQOOAQk5LtAA

Yep, we already do this: "Cannot use type parameter lists on Python 3.10 (syntax was added in Python 3.12)"

[lolpack]

Another vote for showing Python syntax errors by default. https://kiro.dev/blog/empowering-kiro-with-ide-diagnostics/

On the efficiency side, we observed a 29% reduction in command executions because of cutting back on build/test commands.

[yangdanny97]

Starting in 0.52.0 next week, Pyrefly will show syntax errors in default and error-missing-imports modes. Thanks for the feedback and input everyone!

[lolpack]

Awesome! Should we keep this discussion open in case the community has feedback? ie. more on by default or issues?

[kinto0]

Starting in 0.60.0, we show invalid-annotation, missing-import, and unknown-name as warnings when in default mode