gh-142349: Implement PEP 810 #142351

pablogsal · 2025-12-06T15:45:47Z

Issue: Implement PEP 810 #142349

⚠️ Dear reviewers, to avoid collapsing the GitHub API with a lot of comments, please open PRs against the base branch with any suggestions or fixes if you are sure are not controversial ⚠️

📚 Documentation preview 📚: https://cpython-previews--142351.org.readthedocs.build/

…s on bad usage of lazy

Highlight lazy imports in the new REPL

picnixz · 2025-12-06T18:25:43Z

@pablogsal PTAL LazyImportsCabal#25

Doc/using/cmdline.rst

Doc/library/types.rst

Include/import.h

Include/internal/pycore_magic_number.h

Lib/importlib/_bootstrap.py

Lib/rlcompleter.py

picnixz · 2025-12-06T18:33:10Z

I'll be doing small reviews because I'm not on my dev session and it takes too long for creating a PR :') I promise I won't be too nitpicky!

StanFromIreland · 2025-12-06T18:53:06Z

Doc/reference/simple_stmts.rst

   pair: name; binding
   pair: keyword; from
   pair: keyword; as
+   pair: keyword; lazy


I suggest you move this to the below section on lazy specifically.

picnixz

I'll review more tomorrow when I'm on my dev session, it'll be easier. But here are a few comments and questions about import.c

picnixz · 2025-12-06T18:39:42Z

Python/import.c

+PyImport_GetLazyImportsFilter(void)
+{
+    PyInterpreterState *interp = _PyInterpreterState_GET();
+    return Py_XNewRef(FT_ATOMIC_LOAD_PTR_RELAXED(LAZY_IMPORTS_FILTER(interp)));


If the import filter is not set, this may return NULL without an exception set. So either you want it to be documented or you want to return None instead.

picnixz · 2025-12-06T18:40:41Z

Python/import.c

+    PyObject *old = _Py_atomic_exchange_ptr(&LAZY_IMPORTS_FILTER(interp), Py_XNewRef(filter));
+    Py_XDECREF(old);
+#else
+    Py_XSETREF(LAZY_IMPORTS_FILTER(interp), Py_XNewRef(filter));


This makes me wondeer whether we couldn't have a FT helper for this. It's probably not the only place that will need this in the future.

picnixz · 2025-12-06T18:42:11Z

Python/import.c

+        if (PyDict_GetItemRef(globals, &_Py_ID(__name__), &modname) < 0) {
+            return NULL;
+        } else if (modname == NULL) {
+            modname = Py_None;
+        }


This should be two separate ifs right? because you might have modname == NULL if the name doesn't exist.

picnixz · 2025-12-06T18:42:49Z

Python/import.c

+    if (filter != NULL) {
+        PyObject *modname;
+        if (PyDict_GetItemRef(globals, &_Py_ID(__name__), &modname) < 0) {
+            return NULL;


You need to decref abs_name here (you can also have a goto error maybe?)

picnixz · 2025-12-06T18:44:02Z

Python/import.c

+        if (!PyObject_IsTrue(res)) {
+            Py_DECREF(abs_name);
+            return PyImport_ImportModuleLevelObject(


Just me thinking aloud but is it possible to have a UAF here because filter is an arbitrary callable (so, we could alter fromlist)?

picnixz · 2025-12-06T18:45:24Z

Python/import.c

+
+    // Add the module name to sys.lazy_modules set (PEP 810)
+    PyObject *lazy_modules_set = interp->imports.lazy_modules_set;
+    if (lazy_modules_set != NULL) {


When is it possible to have lazy_modules_set NULL?

picnixz · 2025-12-06T18:46:56Z

Python/import.c

+        }
+
+        child_dict = get_mod_dict(child_module);
+        if (child_dict == NULL || !PyDict_CheckExact(child_dict)) {


child_dict can be NULL and we must go to error here

Parser/action_helpers.c

StanFromIreland · 2025-12-06T19:00:18Z

Doc/reference/simple_stmts.rst

+
+   lazy import json


Suggested change

lazy import json

lazy import json

lazy import sys

For 'json' in sys.modules to run.

pablogsal · 2025-12-06T19:02:36Z

@DinoV @colesbury I haven't paid attention to FT when I updated the draft implementation so please take a special look at that

pablogsal · 2025-12-06T19:07:44Z

@DinoV @colesbury I haven't paid attention to FT when I updated the draft implementation so please take a special look at that

In particular ./python -m test test_import.test_lazy_imports -v shows some tsan problems.

I think I got them in 470b9e4 but not sure if that's the best way to fix this.

Include/internal/pycore_magic_number.h

DinoV and others added 30 commits October 2, 2025 13:22

Lazy imports grammar / AST changes

1c691ea

Add __lazy_import__, check sys.modules before import

3b0d745

Export lazy_import in imp module

9eef03c

Add lazy import filter

de281fd

Add compatiblity mode

41ab092

Flow import func through to lazy imports object and __lazy_import__

058bc6e

Reify lazy objects when accessed via the module object

6d7c87a

Import lazy.get

f992ee7

Implement better error

6a91132

Fix reference and global dict in spezializing LOAD_GLOBAL

03a419a

Remove copyright

e0878be

More fixes

f9880bf

Fix recursive lazy imports and error path in bytecodes.c

f3f5795

fix offset in addr2line

44a3e46

Add global flag

73de8d0

Draft: force * imports to be eager

07a633f

Syntax restrictions for lazy imports

20b14d9

Fix submodules crash

164423b

Implement disabling imports in try/except and * imports, report error…

00e7800

…s on bad usage of lazy

Add PyExc_ImportCycleError and raise it when a cycle is detected

781eedb

Publish lazy imported packages on parent

9be59ec

Re-enable eagerly returning imported module

b179da2

Fix __lazy_modules__

9078f57

Add sys.set_lazy_imports_filter

f67310c

Simplify grammar

39c33df

Move __lazy_imports__ check into the interpreter

c8c8838

Don't allow __lazy_imports__ to work in try/except

7c49405

Make imports in with blocks syntax errors

5d6026a

Highlight lazy imports in the new REPL

c0c0d80

Merge pull request #2 from bswck/lazy-import-pyrepl-highlight

5ff0dd2

Highlight lazy imports in the new REPL

pablogsal added 2 commits December 6, 2025 18:22

PeRfoRManCe

2d3681e

PeRfoRManCe

4cc6905

pablogsal force-pushed the lazy branch from 2d3681e to 4cc6905 Compare December 6, 2025 18:23

Amend return types for PyImport_SetLazy* functions

4f675fd

pablogsal and others added 2 commits December 6, 2025 18:28

moar fixes much refleaks wow

4c3477b

Merge pull request #25 from picnixz/patch-1

2f7d223

picnixz reviewed Dec 6, 2025

View reviewed changes

Doc updates

0985e2a

pablogsal requested a review from willingc as a code owner December 6, 2025 18:34

pablogsal and others added 5 commits December 6, 2025 18:37

Address feedback

610af78

More doc fixes

917b92e

Make error for invalid lazy from __future__ ... import prettier

e777866

Regen limited ABI

952ac21

Merge branch 'lazy' into lazy-future-error

33dc19e

StanFromIreland reviewed Dec 6, 2025

View reviewed changes

picnixz reviewed Dec 6, 2025

View reviewed changes

StanFromIreland reviewed Dec 6, 2025

View reviewed changes

Parser/action_helpers.c Outdated Show resolved Hide resolved

Fix error paths and edge cases

80133a5

StanFromIreland reviewed Dec 6, 2025

View reviewed changes

Free threading fixes

b0adc30

picnixz reviewed Dec 6, 2025

View reviewed changes

Include/internal/pycore_magic_number.h Outdated Show resolved Hide resolved

pablogsal and others added 5 commits December 6, 2025 19:16

TSAN fixes

470b9e4

Update magic number

41761e5

Merge pull request #26 from StanFromIreland/lazy-future-error

74b3efe

Regen frozenmain

006ca9c

Fix WASI failure

b5121b0

Uh oh!

gh-142349: Implement PEP 810 #142351

Are you sure you want to change the base?

gh-142349: Implement PEP 810 #142351

Conversation

pablogsal commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz commented Dec 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

picnixz commented Dec 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pablogsal commented Dec 6, 2025

Uh oh!

pablogsal commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pablogsal commented Dec 6, 2025 •

edited

Loading

picnixz Dec 6, 2025 •

edited

Loading

pablogsal commented Dec 6, 2025 •

edited

Loading