Skip to content

gh-110038: KqueueSelector must count all read/write events#110039

Merged
vstinner merged 3 commits intopython:mainfrom
sorcio:fix/kqueue-max-events
Sep 28, 2023
Merged

gh-110038: KqueueSelector must count all read/write events#110039
vstinner merged 3 commits intopython:mainfrom
sorcio:fix/kqueue-max-events

Conversation

@sorcio
Copy link
Contributor

@sorcio sorcio commented Sep 28, 2023 •
edited by bedevere-app bot
Loading

With this change, each registered read or write filter is counted separately, even if it shares a fd.

As a side effect, a call to KqueueSelector.select() might return a separate entry with the same key, one with EVENT_READ and the other with EVENT_WRITE (and never both). This was already the case before, but it might become more evident now.

@bedevere-app bedevere-app bot mentioned this pull request Sep 28, 2023
Closed
vstinner
vstinner reviewed Sep 28, 2023
View reviewed changes
Lib/test/test_selectors.py
result = s.select()
# We get the read and write either in the same result entry or in two
# distinct entries with the same key.
self.assertLessEqual(len(result), 2)
Copy link
Member

@vstinner vstinner Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to move this check at the end, and make sure that len == 2.

Copy link
Contributor Author

@sorcio sorcio Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might get 1 or 2 items, depending on the implementation. I think it's only 2 for KqueueSelector (which is not ideal, but see note in the PR).

Copy link
Member

@vstinner vstinner Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it's possible that a single entry contains both events, I see! Ok.

@vstinner vstinner added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Sep 28, 2023
vstinner
vstinner approved these changes Sep 28, 2023
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@vstinner vstinner enabled auto-merge (squash) September 28, 2023 17:19
@vstinner
Copy link
Member

vstinner commented Sep 28, 2023

Thanks for the test!

@vstinner vstinner merged commit b14f0ab into python:main Sep 28, 2023
@miss-islington
Copy link
Contributor

miss-islington commented Sep 28, 2023

Thanks @sorcio for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

miss-islington commented Sep 28, 2023

Sorry, @sorcio and @vstinner, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker b14f0ab51cb4851b25935279617e388456dcf716 3.12

@miss-islington
Copy link
Contributor

miss-islington commented Sep 28, 2023

Sorry, @sorcio and @vstinner, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker b14f0ab51cb4851b25935279617e388456dcf716 3.11

sorcio added a commit to sorcio/cpython that referenced this pull request Sep 28, 2023
@sorcio
[3.12] pythongh-110038: KqueueSelector must count all read/write even…
5012cf7 
…ts (pythonGH-110039).

(cherry picked from commit b14f0ab)

Co-authored-by: Davide Rizzo <sorcio@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Sep 28, 2023

GH-110043 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Sep 28, 2023
@bedevere-app
Copy link

bedevere-app bot commented Sep 28, 2023

GH-110044 is a backport of this pull request to the 3.11 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.11 only security fixes label Sep 28, 2023
sorcio added a commit to sorcio/cpython that referenced this pull request Sep 28, 2023
@sorcio
[3.11] pythongh-110038: KqueueSelector must count all read/write even…
01b23d0 
…ts (pythonGH-110039).

(cherry picked from commit b14f0ab)

Co-authored-by: Davide Rizzo <sorcio@gmail.com>
@bedevere-bot
Copy link

bedevere-bot commented Sep 28, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.x has failed when building commit b14f0ab.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/469/builds/6126) and take a look at the build logs.
  4. Check if the failure is related to this commit (b14f0ab) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/469/builds/6126

Failed tests:

  • test.test_asyncio.test_waitfor

Failed subtests:

  • test_wait_for_timeout_less_then_0_or_0 - test.test_asyncio.test_waitfor.AsyncioWaitForTest.test_wait_for_timeout_less_then_0_or_0

Summary of the results of the build (if available):

==

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_asyncio/test_waitfor.py", line 124, in test_wait_for_timeout_less_then_0_or_0
    self.assertLess(t1 - t0, 0.1)
AssertionError: 0.13320737797766924 not less than 0.1

vstinner pushed a commit that referenced this pull request Sep 28, 2023
@sorcio
[3.11] gh-110038: KqueueSelector must count all read/write events (GH…
d6e8001 
…-110039) (#110044)

[3.11] gh-110038: KqueueSelector must count all read/write events (GH-110039).
(cherry picked from commit b14f0ab)
csm10495 pushed a commit to csm10495/cpython that referenced this pull request Sep 29, 2023
@sorcio @csm10495
pythongh-110038: KqueueSelector must count all read/write events (pyt…
eb292de 
…hon#110039)
Yhg1s pushed a commit that referenced this pull request Oct 2, 2023
@sorcio
[3.12] gh-110038: KqueueSelector must count all read/write events (GH…
c794103 
…-110039) (#110043)

[3.12] gh-110038: KqueueSelector must count all read/write events (GH-110039).
(cherry picked from commit b14f0ab)
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
@sorcio @Glyphack
pythongh-110038: KqueueSelector must count all read/write events (pyt…
4b73ee7 
…hon#110039)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes

Assignees

@vstinner vstinner

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sorcio @vstinner @miss-islington @bedevere-bot