[Feel free to edit this first message]
This issue is there to try to summarize one of the issue we currently have with writing tooling to get some information out of PyPI and what could be better way to solve them.
Side note: the server behind pypi.org is https://github.com/pypi/warehouse.
Currently to gather some information about PyPI packages we need to scrape Pypi HTML (are are often blocked by fastly), which leads to inaccuracies and issues. This is mainly due to 2 large issues.
- Not all information available in HTML is available in the API:
- Some information are not available at all and needs inference
(which repository/workflow triggered a given release, for example)
The second part is that available information in the UI is often difficult to get access to at a glance.
Assuming one if a maintainer/organization owner: https://pypi.org/org/jupyter/ only lists packages, and release date. I think many things like version number, is it using trusted publishing, do all the maintainer have 2FA, is the project archived... etc should be visible. Likely in the form of a searchable/filterable table.
Instead of doing all that as separate tooling, I would like to investigate the possibility of getting funding via or with the PSF, and pushing this kind of things upstream.
This is of course not limited by our ability to write code, but by our ability to have warehouse maintainer time, and review of the code, so this should be structured as a way to help push warehouse forward potentially decreasing their load.
IMHO I much prefer having an unstable, and complete API even just read-only, than the current state.
[Feel free to edit this first message]
This issue is there to try to summarize one of the issue we currently have with writing tooling to get some information out of PyPI and what could be better way to solve them.
Side note: the server behind pypi.org is https://github.com/pypi/warehouse.
Currently to gather some information about PyPI packages we need to scrape Pypi HTML (are are often blocked by fastly), which leads to inaccuracies and issues. This is mainly due to 2 large issues.
(which repository/workflow triggered a given release, for example)
The second part is that available information in the UI is often difficult to get access to at a glance.
Assuming one if a maintainer/organization owner: https://pypi.org/org/jupyter/ only lists packages, and release date. I think many things like version number, is it using trusted publishing, do all the maintainer have 2FA, is the project archived... etc should be visible. Likely in the form of a searchable/filterable table.
Instead of doing all that as separate tooling, I would like to investigate the possibility of getting funding via or with the PSF, and pushing this kind of things upstream.
This is of course not limited by our ability to write code, but by our ability to have warehouse maintainer time, and review of the code, so this should be structured as a way to help push warehouse forward potentially decreasing their load.
IMHO I much prefer having an unstable, and complete API even just read-only, than the current state.