Calling PyList_GetItemRef after PyList_New segfaults

[lysnikolaou]

Crash report

What happened?

While working on Pillow, I came across the following issue: I'm getting a crash when calling PyList_GetItemRef immediately after calling PyList_New since list item is initialized to NULL.

PyObject *list = PyList_New(1);
return PyList_GetItemRef(list, 0);

In the docs for PyList_New, we include the following note:

If len is greater than zero, the returned list object’s items are set to NULL. Thus you cannot use abstract API functions such as PySequence_SetItem() or expose the object to Python code before setting all items to a real object with PyList_SetItem().

Not sure if this includes calling PyList_GetItemRef.

The problem is there on both builds, cause they both eventually try to call some variation of Py_NewRef / _Py_NewRefWithLock. PyList_GetItem does not crash and returns NULL.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

Python 3.13.0b2+ (heads/3.13:7302855, Jun 21 2024, 10:05:10) [Clang 15.0.0 (clang-1500.3.9.4)]

Linked PRs

• gh-121403: Add notes for PyList_GetXXX APIs about the need for init #121626
• [3.13] gh-121403: Add notes for PyList_GetXXX APIs about the need for init (gh-121626) #121827

[gaogaotiantian]

Maybe PyList_GetItemRef should check for the size of the list, instead of the limit?. Made a mistake. Personally I think this belongs to a misusage - trying to work with an uninitialized list.

[corona10]

I think that we should update the documentation, @colesbury WDYT?

[vstinner]

Thus you cannot use abstract API functions such as PySequence_SetItem() or expose the object to Python code before setting all items to a real object with PyList_SetItem().

Maybe the doc must only list functions which are safe to use: only PyList_SET_ITEM() is safe until the list is fully initialized.

[corona10]

only PyList_SET_ITEM() is safe until the list is fully initialized.

+1

[colesbury]

Yeah, I think we should update the docs to make the restrictions more clear instead of changing the behavior of PyList_GetItemRef.