Setting up an HTTP server that listens over a file-socket

Question

How can I use HTTPServer (or some other class) to set up an HTTP server that listens to a filesystem socket instead of an actual network socket? By "filesystem socket" I mean sockets of the AF_UNIX type.

Kennethkennett

Answer 1

HTTPServer inherits from SocketServer.TCPServer, so I think it's fair to say that it isn't intended for that use-case, and even if you try to work around it, you may run into problems since you are kind of "abusing" it.

That being said, however, it would be possible per se to define a subclass of HTTPServer that creates and binds Unix sockets quite simply, as such:

class UnixHTTPServer(HTTPServer):
    address_family = socket.AF_UNIX

    def server_bind(self):
        SocketServer.TCPServer.server_bind(self)
        self.server_name = "foo"
        self.server_port = 0

Then, just pass the path you want to bind to by the server_address argument to the constructor:

server = UnixHTTPServer("/tmp/http.socket", ...)

Again, though, I can't guarantee that it will actually work well. You may have to implement your own HTTP server instead.

Teel

I'm surprised that the http servers were designed so tightly coupled with the network socket idea rather than a simple blocking stream which could possibly be a socket underneath. This seems like a bad design. Is there no standard HTTP parsing library that is independent of the source of the HTTP request? – Kennethkennett

It's not very strange if you consider that other parts of the HTTP infrastructure has become tightly coupled with TCP mechanics. For instance, CGI environments are specified to contain variables with the "server port", loggers frequently use IP addresses, and so on. I can agree on one level that it is a bit ugly, but seeing as how HTTP, in practice, is pretty much only used over TCP, it's not overly unexpected that such conventions arise. – Teel

If you think about it, even the usage of an http-schemed URL in itself implies the usage of TCP. There's no standard way of "addressing" a Unix-socket HTTP server. – Teel

No, all HTTP client libraries would try to resolve file as a DNS name if you do that. – Teel

Ah I see. And file:/// is not standard? – Kennethkennett

It is standard, but it would open a regular file, not connect to a Unix socket. – Teel

Oh hmm... maybe my approach is not sound then. Thanks. – Kennethkennett

Answer 2

Overview

In case it help anyone else, I have created a complete example (made for Python 3.8) based on Roger Lucas's example:

Server

import socketserver

from http.server import BaseHTTPRequestHandler

class myHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-type','text/html')
        self.end_headers()
        self.wfile.write(b"Hello world!")
        return

class UnixSocketHttpServer(socketserver.UnixStreamServer):
    def get_request(self):
        request, client_address = super(UnixSocketHttpServer, self).get_request()
        return (request, ["local", 0])

server = UnixSocketHttpServer(("/tmp/http.socket"), myHandler)
server.serve_forever()

This will listen on the unix socket and respond with "Hello World!" for all GET requests.

Client Request

You can send a request with:

curl --unix-socket /tmp/http.socket http://any_path/abc/123

Troubleshooting

If you run into this error:

OSError: [Errno 98] Address already in use

Then delete the socket file:

rm /tmp/http.socket 

Preventive

Answer 3

I followed the example from @Dolda2000 above in Python 3.5 and ran into an issue with the HTTP handler falling over with an invalid client address. You don't have a client address with Unix sockets in the same way that you do with TCP, so the code below fakes it.

import socketserver

...

class UnixSocketHttpServer(socketserver.UnixStreamServer):
    def get_request(self):
        request, client_address = super(UnixSocketHttpServer, self).get_request()
        return (request, ["local", 0])

...

server = UnixSocketHttpServer((sock_file), YourHttpHandler)
server.serve_forever()

With these changes, you can perform an HTTP request against the Unix socket with tools such as cURL.

curl --unix-socket /run/test.sock http:/test

Steamtight

Thanks for this answer - it works great. But when I try to run script second time it returns me following: OSError: [Errno 98] Address already in use, I just wonder how to allow reconnect to existing socket file, and how to set 0777 to socket file on a fly to allow socket connections from different users? – Masuria