[asyncio] bpo-30423 bug: orphan future close loop and cause "RuntimeError: Event loop stopped before Future completed."

[jimmylai]

@1st1 this is the fix we did to fix the orphan future problem we saw in Instagram web service.

Problem:
"RuntimeError: Event loop stopped before Future completed." throws when calling run_until_complete().

We investigate and find out some orphan futures stay in the event loop before we run another run_until_complete(another_async_func()).
The orphan future has pending state and is attached with _run_until_complete_cb from previous run_until_complete.
It happens because the orphan future thrown Exception and then raise, thus remove_done_callback(_run_until_complete_cb) didn't called.
Move it to finally section can fix it.
With this patch, we stop seeing the Runtime Error.

Thanks @ambv for brainstorming the problem together.

[the-knights-who-say-ni]

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately we couldn't find an account corresponding to your GitHub username on bugs.python.org (b.p.o) to verify you have signed the CLA. This is necessary for legal reasons before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

Thanks again to your contribution and we look forward to looking at it!

[AraHaan]

Mind making an bpo issue for this as well? Also did you sign the CLA?

[jimmylai]

@AraHaan I just created an issue 30423. CLA was signed yesterday, still waiting for status update.

[ambv]

Contribution also covered by Facebook CCLA.

[1st1]

Is it possible to add a unittest for this (especially important for other loops like uvloop etc)?

[1st1]

This also needs backports to 3.6 and 3.5.

[ambv]

@jimmylai Thanks for the PR, Yury is right, let's add a test for this.

@1st1, I'll backport it to 3.6 now with cherry-picker. 3.5 might be trickier, I'll look.

[1st1]

@jimmylai Thanks for the PR, Yury is right, let's add a test for this.
@1st1, I'll backport it to 3.6 now with cherry-picker. 3.5 might be trickier, I'll look.

Thanks! Will you guys make a separate PR for the tests?

[jimmylai]

@1st1 @ambv I'll create a separate PR for unit test.