Merge branch 'main' into urllib-parse-allow-none

Author: serhiy-storchaka

Doc/library/compression.zstd.rst

@@ -73,7 +73,7 @@ Reading and writing compressed files
    argument is not None, a :exc:`!TypeError` will be raised.
 
    When writing, the *options* argument can be a dictionary
-   providing advanced decompression parameters; see
+   providing advanced compression parameters; see
    :class:`CompressionParameter` for detailed information about supported
    parameters. The *level* argument is the compression level to use when
    writing compressed data. Only one of *level* or *options* may be non-None.
@@ -117,7 +117,7 @@ Reading and writing compressed files
    argument is not None, a :exc:`!TypeError` will be raised.
 
    When writing, the *options* argument can be a dictionary
-   providing advanced decompression parameters; see
+   providing advanced compression parameters; see
    :class:`CompressionParameter` for detailed information about supported
    parameters. The *level* argument is the compression level to use when
    writing compressed data. Only one of *level* or *options* may be passed. The

Doc/library/importlib.metadata.rst

@@ -418,6 +418,16 @@ Distributions
    equal, even if they relate to the same installed distribution and
    accordingly have the same attributes.
 
+   .. method:: discover(cls, *, context=None, **kwargs)
+
+      Returns an iterable of :class:`Distribution` instances for all packages.
+
+      The optional argument *context* is a :class:`DistributionFinder.Context`
+      instance, used to modify the search for distributions. Alternatively,
+      *kwargs* may contain keyword arguments for constructing a new
+      :class:`!DistributionFinder.Context`.
+
+
 While the module level API described above is the most common and convenient usage,
 you can get all of that information from the :class:`!Distribution` class.
 :class:`!Distribution` is an abstract object that represents the metadata for
@@ -466,6 +476,61 @@ This metadata finder search defaults to ``sys.path``, but varies slightly in how
 - ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
 - ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.
 
+.. class:: DistributionFinder
+
+   A :class:`~importlib.abc.MetaPathFinder` subclass capable of discovering
+   installed distributions.
+
+   Custom providers should implement this interface in order to
+   supply metadata.
+
+    .. class:: Context(**kwargs)
+
+       A :class:`!Context` gives a custom provider a means to
+       solicit additional details from the callers of distribution discovery
+       functions like :func:`distributions` or :meth:`Distribution.discover`
+       beyond :attr:`!.name` and :attr:`!.path` when searching
+       for distributions.
+
+       For example, a provider could expose suites of packages in either a
+       "public" or "private" ``realm``. A caller of distribution discovery
+       functions may wish to query only for distributions in a particular realm
+       and could call ``distributions(realm="private")`` to signal to the
+       custom provider to only include distributions from that
+       realm.
+
+       Each :class:`!DistributionFinder` must expect any parameters and should
+       attempt to honor the canonical parameters defined below when
+       appropriate.
+
+       See the section on :ref:`implementing-custom-providers` for more details.
+
+       .. attribute:: name
+
+          Specific name for which a distribution finder should match.
+
+          A :attr:`!.name` of ``None`` matches all distributions.
+
+       .. attribute:: path
+
+          A property providing the sequence of directory paths that a
+          distribution finder should search.
+
+          Typically refers to Python installed package paths such as
+          "site-packages" directories and defaults to :attr:`sys.path`.
+
+
+.. function:: distributions(**kwargs)
+
+   Returns an iterable of :class:`Distribution` instances for all packages.
+
+   The *kwargs* argument may contain either a keyword argument ``context``, a
+   :class:`DistributionFinder.Context` instance, or pass keyword arguments for
+   constructing a new :class:`!DistributionFinder.Context`. The
+   :class:`!DistributionFinder.Context` is used to modify the search for
+   distributions.
+
+.. _implementing-custom-providers:
 
 Implementing Custom Providers
 =============================
@@ -493,7 +558,7 @@ interface expected of finders by Python's import system.
 ``importlib.metadata`` extends this protocol by looking for an optional
 ``find_distributions`` callable on the finders from
 :data:`sys.meta_path` and presents this extended interface as the
-``DistributionFinder`` abstract base class, which defines this abstract
+:class:`DistributionFinder` abstract base class, which defines this abstract
 method::
 
     @abc.abstractmethod
@@ -502,9 +567,11 @@ method::
         loading the metadata for packages for the indicated ``context``.
         """
 
-The ``DistributionFinder.Context`` object provides ``.path`` and ``.name``
-properties indicating the path to search and name to match and may
-supply other relevant context sought by the consumer.
+The :class:`DistributionFinder.Context` object provides
+:attr:`~DistributionFinder.Context.path` and
+:attr:`~DistributionFinder.Context.name` properties indicating the path to
+search and name to match and may supply other relevant context sought by the
+consumer.
 
 In practice, to support finding distribution package
 metadata in locations other than the file system, subclass
@@ -529,7 +596,7 @@ Imagine a custom finder that loads Python modules from a database::
 That importer now presumably provides importable modules from a
 database, but it provides no metadata or entry points. For this
 custom importer to provide metadata, it would also need to implement
-``DistributionFinder``::
+:class:`DistributionFinder`::
 
     from importlib.metadata import DistributionFinder
 

Doc/reference/datamodel.rst

@@ -2617,7 +2617,7 @@ Notes on using *__slots__*:
 * :exc:`TypeError` will be raised if *__slots__* other than *__dict__* and
   *__weakref__* are defined for a class derived from a
   :c:member:`"variable-length" built-in type <PyTypeObject.tp_itemsize>` such as
-  :class:`int`, :class:`bytes`, and :class:`tuple`.
+  :class:`int`, :class:`bytes`, and :class:`type`, except :class:`tuple`.
 
 * Any non-string :term:`iterable` may be assigned to *__slots__*.
 
@@ -2642,6 +2642,7 @@ Notes on using *__slots__*:
 
 .. versionchanged:: 3.15
    Allowed defining the *__dict__* and *__weakref__* *__slots__* for any class.
+   Allowed defining any *__slots__* for a class derived from :class:`tuple`.
 
 
 .. _class-customization:

Doc/whatsnew/3.15.rst

@@ -398,6 +398,10 @@ Other language changes
   for any class.
   (Contributed by Serhiy Storchaka in :gh:`41779`.)
 
+* Allowed defining any :ref:`__slots__ <slots>` for a class derived from
+  :class:`tuple` (including classes created by :func:`collections.namedtuple`).
+  (Contributed by Serhiy Storchaka in :gh:`41779`.)
+
 
 New modules
 ===========
@@ -1424,7 +1428,7 @@ that may require changes to your code.
   :func:`resource.setrlimit` and :func:`resource.prlimit` is now deprecated.
   (Contributed by Serhiy Storchaka in :gh:`137044`.)
 
-* :meth:`~mmap.mmap.resize` has been removed on platforms that don't support the
+* :meth:`mmap.mmap.resize` has been removed on platforms that don't support the
   underlying syscall, instead of raising a :exc:`SystemError`.
 
 * A resource warning is now emitted for an unclosed

Include/descrobject.h

@@ -80,10 +80,14 @@ struct PyMemberDef {
 #define _Py_T_NONE     20 // Deprecated. Value is always None.
 
 /* Flags */
-#define Py_READONLY            1
-#define Py_AUDIT_READ          2 // Added in 3.10, harmless no-op before that
-#define _Py_WRITE_RESTRICTED   4 // Deprecated, no-op. Do not reuse the value.
-#define Py_RELATIVE_OFFSET     8
+#define Py_READONLY            (1 << 0)
+#define Py_AUDIT_READ          (1 << 1) // Added in 3.10, harmless no-op before that
+#define _Py_WRITE_RESTRICTED   (1 << 2) // Deprecated, no-op. Do not reuse the value.
+#define Py_RELATIVE_OFFSET     (1 << 3)
+
+#ifndef Py_LIMITED_API
+#  define _Py_AFTER_ITEMS      (1 << 4) // For internal use.
+#endif
 
 PyAPI_FUNC(PyObject *) PyMember_GetOne(const char *, PyMemberDef *);
 PyAPI_FUNC(int) PyMember_SetOne(char *, PyMemberDef *, PyObject *);

Include/internal/pycore_descrobject.h

@@ -22,6 +22,8 @@ typedef propertyobject _PyPropertyObject;
 
 extern PyTypeObject _PyMethodWrapper_Type;
 
+extern void *_PyMember_GetOffset(PyObject *, PyMemberDef *);
+
 #ifdef __cplusplus
 }
 #endif

Lib/_pyrepl/_module_completer.py

@@ -108,6 +108,18 @@ def _find_modules(self, path: str, prefix: str) -> list[str]:
                 return []
 
         modules: Iterable[pkgutil.ModuleInfo] = self.global_cache
+        imported_module = sys.modules.get(path.split('.')[0])
+        if imported_module:
+            # Filter modules to those who name and specs match the
+            # imported module to avoid invalid suggestions
+            spec = imported_module.__spec__
+            if spec:
+                modules = [mod for mod in modules
+                           if mod.name == spec.name
+                           and mod.module_finder.find_spec(mod.name, None) == spec]
+            else:
+                modules = []
+
         is_stdlib_import: bool | None = None
         for segment in path.split('.'):
             modules = [mod_info for mod_info in modules
@@ -196,7 +208,6 @@ def global_cache(self) -> list[pkgutil.ModuleInfo]:
         """Global module cache"""
         if not self._global_cache or self._curr_sys_path != sys.path:
             self._curr_sys_path = sys.path[:]
-            # print('getting packages')
             self._global_cache = list(pkgutil.iter_modules())
         return self._global_cache
 

Lib/pdb.py

@@ -364,7 +364,7 @@ def __init__(self, completekey='tab', stdin=None, stdout=None, skip=None,
         bdb.Bdb.__init__(self, skip=skip, backend=backend if backend else get_default_backend())
         cmd.Cmd.__init__(self, completekey, stdin, stdout)
         sys.audit("pdb.Pdb")
-        if stdin:
+        if stdin is not None and stdin is not sys.stdin:
             self.use_rawinput = False
         self.prompt = '(Pdb) '
         self.aliases = {}

Lib/test/test_descr.py

@@ -1320,6 +1320,18 @@ class X(object):
         with self.assertRaisesRegex(AttributeError, "'X' object has no attribute 'a'"):
             X().a
 
+    def test_slots_after_items(self):
+        class C(tuple):
+            __slots__ = ['a']
+        x = C((1, 2, 3))
+        self.assertNotHasAttr(x, "__dict__")
+        self.assertNotHasAttr(x, "a")
+        x.a = 42
+        self.assertEqual(x.a, 42)
+        del x.a
+        self.assertNotHasAttr(x, "a")
+        self.assertEqual(x, (1, 2, 3))
+
     def test_slots_special(self):
         # Testing __dict__ and __weakref__ in __slots__...
         class D(object):
@@ -1422,6 +1434,9 @@ class W(base):
         self.assertIs(weakref.ref(a)(), a)
         self.assertEqual(a, base(arg))
 
+    @support.subTests('base', [int, bytes] +
+                      ([_testcapi.HeapCCollection] if _testcapi else []))
+    def test_unsupported_slots(self, base):
         with self.assertRaises(TypeError):
             class X(base):
                 __slots__ = ['x']

Lib/test/test_os/test_os.py

@@ -2627,6 +2627,8 @@ def test_execve_invalid_env(self):
     # See https://github.com/python/cpython/issues/137934 and the other
     # related issues for the reason why we cannot test this on Windows.
     @unittest.skipIf(os.name == "nt", "POSIX-specific test")
+    @unittest.skipUnless(unix_shell and os.path.exists(unix_shell),
+                        "requires a shell")
     def test_execve_env_concurrent_mutation_with_fspath_posix(self):
         # Prevent crash when mutating environment during parsing.
         # Regression test for https://github.com/python/cpython/issues/143309.
@@ -2648,14 +2650,17 @@ def __len__(self): return 1
             def keys(self): return KEYS
             def values(self): return VALUES
 
-        args = [sys.executable, '-c', "print({message!r})"]
+        args = [{unix_shell!r}, '-c', 'echo \"{message!s}\"']
         os.execve(args[0], args, MyEnv())
-        """.format(message=message)
+        """.format(unix_shell=unix_shell, message=message)
 
-        # Use '__cleanenv' to signal to assert_python_ok() not
-        # to do a copy of os.environ on its own.
-        rc, out, _ = assert_python_ok('-c', code, __cleanenv=True)
-        self.assertEqual(rc, 0)
+        # Make sure to forward "LD_*" variables so that assert_python_ok()
+        # can run correctly.
+        minimal = {k: v for k, v in os.environ.items() if k.startswith("LD_")}
+        with os_helper.EnvironmentVarGuard() as env:
+            env.clear()
+            env.update(minimal)
+            _, out, _ = assert_python_ok('-c', code, **env)
         self.assertIn(bytes(message, "ascii"), out)
 
     @unittest.skipUnless(sys.platform == "win32", "Win32-specific test")