Skip to content

os.sync() in test_posix can uninterruptibly hang tests for minutes #102184

Description

@izbyshev

I've been investigating why CPython test suite apparently hangs when run on a Linux server with 512 GB RAM and traced it to os.sync() call in test_posix.testNoArgFunctions(). If there are many dirty pages cached in RAM and the storage device is relatively slow (HDD in my case), sync() can take a lot of time. Worse, it puts the process into "uninterruptible sleep" state, so it's not possible to terminate the test worker process even with SIGKILL.

I think that grouping innocuous functions like os.getpid() and os.getuid() together with os.sync() as "not having side effects" is unwise, since os.sync() can have huge side effects (on a lower abstraction level). I propose to test os.sync() only if some resource is explicitly granted to the test system via --use option.

I don't see an exact match in the list of currently supported resources. The most appropriate one seems to be largefile: even though os.sync() doesn't create any files, it might cause heavy blocking I/O. I'm going to submit a PR making os.sync() test conditional on largefile.

Alternatively, we could add another resource (io?), but I'm not sure it's worth adding a new resource just for this case.

Linked PRs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance or resource usagetestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions