Python's lack of design

Many years ago, my dad was going on and on about a new programming language.  He was talking about how simple it is to create an http server and serve up content.  I was very dismissive because to me, having an easy to use standard library doesn't make a language great.  That language was Python.

I have never been a fan of Python.  One of the areas that I felt it lacked was in the design (or lack of) surrounding the Python standard library.  Lets go back to the http server example from before.  We will use the Python3 example, since the Python2 example was even worse.  It is super easy to create an http server.  You just have to instantiate http.server.HTTPServer and pass in an http.server.SimpleHTTPRequestHandler object.  This object will serve up files in the current working directory.  Seems simple enough.  My project needs it to serve files in a sandbox directory, however.  So, we should be able to override the web root of the bundled http server, right?  Actually, no.  The SimpleHTTPRequestHandler object hard codes the webroot to be the current working directory.  In order to change that, you have to override the SimpleHTTPRequestHandler.translate_path() method.  You have to copy and paste the entire contents of that method and change one line to add a custom path.

Once you do that, you decide to test http streaming of large files.  While a large file is streaming, you decide to test something else and you notice something.  Your second connection to the http server freezes.  Woops.  You can't just instantiate HTTPServer.  That is a single threaded server.  You have to instantiate socketserver.ThreadingTCPServer instead.

Once your server supports multiple threads, lets add an XMLRPC server in there as well.  Python supports XMLRPC so it should be as simple as invoking some XMLRPC code from an overridden method of SimpleHTTPRequestHandler, right?  Well, no.  The XMLRPC server code is written as a class that extends http.server.BaseHTTPRequestHandler.  That implementation assumes you are using the xmlrpc.server.SimpleXMLRPCServer class.  If you aren't using the SimpleXMLRPCServer class, then things get weird really fast.

It seems natural that since Python supports an http server that can serve up static files and an xmlrpc server, that it should be able to handle a single server that does both things.  These two sub-libraries don't seem to intermingle, though.  Without any type of api or underlying design, you have to hack them together.  This is what I mean by Python's lack of design.  The standard library supports many different protocols and features, but they are not written as building blocks.  It is not easy to use these libraries inside of a larger application.

For my project, I ended up subclassing http.server.SimpleHTTPRequestHandler.  I created an instance of xmlrpc.server.SimpleXMLRPCDispatcher.  I started overriding methods to delegate to the SimpleXMLRPCDispatcher when a post to /RPC2 occurred.  Then I started the copy game.  I kept copying methods from http/server.py and xmlrpc/server.py into my program until both classes worked together.  I finally got XMLRPC and serving up static files working in the same server.

To get all of this to work, I had to use "private" methods.  I consider the methods private because the Python documentation doesn't list the methods.  Python doesn't have a concept of private vs protected vs public.  To me, this means future versions of Python could break my code, since I had to program to the implementation, not the interface.  This is a design anti-pattern.

My next step is to modify the code to support the Accept-Ranges header since the stock Python http server doesn't support it.  Until next time!