-
-
Notifications
You must be signed in to change notification settings - Fork 34.8k
bpo-17561: add socket.create_server_sock() and socket.has_dual_stack() #11215
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
6956916
37d867b
89dfd70
7ecfca6
a5867fa
5d3c5cc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -595,6 +595,26 @@ The following functions all create :ref:`socket objects <socket-objects>`. | |
| .. versionchanged:: 3.2 | ||
| *source_address* was added. | ||
|
|
||
| .. function:: create_server_sock(address, *, dual_stack=True, queue_size=5, reuse_addr=True) | ||
|
|
||
| Convenience function which creates a TCP server bound to *address* (a | ||
| 2-tuple ``(host, port)``) and return a socket object. If *host* is an empty | ||
| string or ``None`` all network interfaces are used. | ||
| If dual-stack is supported by kernel and *dual_stack* parameter | ||
| is ``True`` the socket will be able to serve both :data:`AF_INET` and | ||
| :data:`AF_INET6` (IPv4 / IPv6) connections. If not the right address family | ||
| will be chosen based on the host specified in *address*. | ||
| *queue_size* is the argument passed to :meth:`socket.listen`. | ||
| If *reuse_addr* is ``True`` :data:`socket.SO_REUSEADDR` socket flag is set. | ||
|
|
||
| .. versionadded:: 3.8 | ||
|
|
||
| .. function:: has_dual_stack() | ||
|
|
||
| Return ``True`` if the kernel supports creating a socket which can handle | ||
| both :data:`AF_INET` and :data:`AF_INET6` (IPv4 / IPv6) connections. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the name of this function is pretty confusing. A "dual stack host" is one that has both IPv4 and IPv6 support. A host can have dual stack support without being able to support both of them in a single socket. Also, for Python docs it should probably be "the platform", not "the kernel".
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree the name is not the best. Actually, on a second thought, I don't think it's necessary to expose this function after all. |
||
|
|
||
| .. versionadded:: 3.8 | ||
|
|
||
| .. function:: fromfd(fd, family, type, proto=0) | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -728,6 +728,76 @@ def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT, | |
| else: | ||
| raise error("getaddrinfo returns an empty list") | ||
|
|
||
| def has_dual_stack(): | ||
| """Return True if the kernel supports creating a socket which | ||
| can handle both AF_INET and AF_INET6 (IPv4 / IPv6) connections. | ||
| """ | ||
| if not has_ipv6 \ | ||
| or not hasattr(_socket, 'AF_INET6') \ | ||
| or not hasattr(_socket, 'IPV6_V6ONLY'): | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You must also test for
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And note that due to a bug in the socket module, |
||
| return False | ||
| try: | ||
| with socket(AF_INET6, SOCK_STREAM) as sock: | ||
|
asvetlov marked this conversation as resolved.
|
||
| if not sock.getsockopt(IPPROTO_IPV6, IPV6_V6ONLY): | ||
| return True | ||
| else: | ||
| sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, False) | ||
| return True | ||
| except error: | ||
| return False | ||
|
|
||
| def create_server_sock(address, *, dual_stack=has_dual_stack(), queue_size=5, | ||
| reuse_addr=os.name == 'posix' and sys.platform != 'cygwin'): | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In
Also
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. By the way, it would be useful to find out if
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. asyncio implementation is here: https://github.com/python/cpython/blob/master/Lib/asyncio/base_events.py#L1300-L1415
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
No because internally
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Mmm. I think that for an asyncio app this function is fundamentally useless because it will use
I will rename it to backlog
Good point. I took a look at some projects. Tornado uses 128, Twisted uses 50, curio uses 100, Trio (and Golang) uses a big size (but never exceding 65536):
Interesting. I never needed |
||
| """Convenience function which creates a TCP server bound to | ||
| *address* (a 2-tuple (host, port)) and return a socket object. | ||
|
|
||
| If *host* is an empty string or None all network interfaces are assumed. | ||
| If dual-stack is supported by kernel and *dual_stack* parameter | ||
| is True the socket will be able to serve both AF_INET and AF_INET6 | ||
| connections. If not the right address family will be chosen | ||
| based on the host specified in *address*. | ||
| *queue_size* is the argument passed to socket.listen() method, | ||
| *reuse_addr* dictates whether to set SO_REUSEADDR flag. | ||
|
|
||
| >>> with create_server_sock((None, 8000)) as server: | ||
| ... while True: | ||
| ... sock, addr = server.accept() | ||
| ... # handle new sock connection | ||
| """ | ||
| AF_INET6_ = getattr(_socket, "AF_INET6", 0) | ||
| host, port = address | ||
| if host == "": | ||
| # http://mail.python.org/pipermail/python-ideas/2013-March/019937.html | ||
| host = None | ||
| # If dual stack is supported and no host was specified '::' will | ||
| # accept AF_INET and AF_INET6 connections and all interfaces. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "on all interfaces" |
||
| if host is None and dual_stack: | ||
| host = "::" | ||
| err = None | ||
| info = getaddrinfo(host, port, AF_UNSPEC, SOCK_STREAM, 0, AI_PASSIVE) | ||
| for res in info: | ||
| af, socktype, proto, canonname, sa = res | ||
| sock = None | ||
| try: | ||
| sock = socket(af, socktype, proto) | ||
| if reuse_addr: | ||
| sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) | ||
| if af == AF_INET6_ and dual_stack: | ||
| if sock.getsockopt(IPPROTO_IPV6, IPV6_V6ONLY): | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it worth calling getsockopt() before setsockopt()?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good question. I'm not sure. My reasoning here was: being |
||
| sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, False) | ||
| sock.bind(sa) | ||
| sock.listen(queue_size) | ||
| return sock | ||
| except error as _: | ||
| err = _ | ||
| if sock is not None: | ||
| sock.close() | ||
|
|
||
| if err is not None: | ||
| raise err | ||
| else: | ||
| raise error("getaddrinfo() returned an empty list") | ||
|
|
||
| def getaddrinfo(host, port, family=0, type=0, proto=0, flags=0): | ||
| """Resolve host and port into list of address info entries. | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6048,9 +6048,67 @@ def test_new_tcp_flags(self): | |
| self.assertEqual([], unknown, | ||
| "New TCP flags were discovered. See bpo-32394 for more information") | ||
|
|
||
| class CreateServerSockTest(unittest.TestCase): | ||
|
|
||
| def echo_server(self, sock, port=None): | ||
| def run(): | ||
| with sock: | ||
| conn, _ = sock.accept() | ||
| with conn: | ||
| msg = conn.recv(1024) | ||
| if not msg: | ||
| return | ||
| conn.sendall(msg) | ||
|
|
||
| sock.settimeout(2) | ||
| t = threading.Thread(target=run) | ||
| t.start() | ||
| time.sleep(.1) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure this is reliable. @vstinner will probably object.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I object. Don't do that.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree. I will use a sync primitive. |
||
|
|
||
| def test_base(self): | ||
| port = support.find_unused_port() | ||
| with socket.create_server_sock((None, port)) as sock: | ||
| self.assertEqual(sock.getsockname()[1], port) | ||
| self.assertEqual(sock.type, socket.SOCK_STREAM) | ||
| if socket.has_dual_stack(): | ||
| self.assertEqual(sock.family, socket.AF_INET6) | ||
| else: | ||
| self.assertEqual(sock.family, socket.AF_INET) | ||
| self.echo_server(sock) | ||
| with socket.create_connection(('localhost', port), timeout=2) as cl: | ||
| cl.sendall(b'foo') | ||
| self.assertEqual(cl.recv(1024), b'foo') | ||
|
|
||
| def test_dual_stack(self): | ||
| with socket.create_server_sock((None, 0)) as sock: | ||
| self.echo_server(sock) | ||
| port = sock.getsockname()[1] | ||
| with socket.create_connection(("127.0.0.1", port), timeout=2) as cl: | ||
| cl.sendall(b'foo') | ||
| self.assertEqual(cl.recv(1024), b'foo') | ||
|
|
||
| with socket.create_server_sock((None, 0)) as sock: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You'll probably want to skip this if the testing platform doesn't support IPv6 at all (there has to be a flag for this already :-)).
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, will look into it |
||
| self.echo_server(sock) | ||
| port = sock.getsockname()[1] | ||
| if socket.has_dual_stack(): | ||
| with socket.create_connection(("::1", port), timeout=2) as cl: | ||
| cl.sendall(b'foo') | ||
| self.assertEqual(cl.recv(1024), b'foo') | ||
| else: | ||
| self.assertRaises(ConnectionRefusedError, | ||
| socket.create_connection, ("::1", port)) | ||
| # just stop server | ||
| with socket.create_connection(("127.0.0.1", port), timeout=2) \ | ||
| as cl: | ||
| cl.sendall(b'foo') | ||
| cl.recv(1024) | ||
| unittest.skip('dual stack cannot be tested as not supported') | ||
|
|
||
|
|
||
| def test_main(): | ||
| tests = [GeneralModuleTests, BasicTCPTest, TCPCloserTest, TCPTimeoutTest, | ||
| TestExceptions, BufferIOTest, BasicTCPTest2, BasicUDPTest, UDPTimeoutTest ] | ||
| TestExceptions, BufferIOTest, BasicTCPTest2, BasicUDPTest, | ||
| UDPTimeoutTest, CreateServerSockTest] | ||
|
|
||
| tests.extend([ | ||
| NonBlockingTCPTests, | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| Add socket.create_server_sock() function returning a TCP socket able to serve | ||
| both IPv4 and IPv6 connections. Also add socket.has_dual_stack() function, | ||
| which tells whether the kernel supports creating a socket which can handle | ||
| both IPv4 a IPv6 connections. (patch by Giampaolo Rodola) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could the function be extended to support UDP sockets or that would require a different implementation?
If the latter, perhaps you want to rename the function to
create_tcp_server?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right. I forgot about UDP. =) Yes, I think we can support it. I can simply add a
type=SOCK_STREAMparameter.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you want to support UDP please consider adding
allow_broadcastargument as we have in asyncio: https://github.com/python/cpython/blob/master/Lib/asyncio/base_events.py#L1154-L1158There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is worth adding because one can simply set
SO_BROADCASTagainst the returned socket.reuse_addrandreuse_portparameters are different because they necessarily must be set beforebind(). I would rather keep the API as simple as possible.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok