REST vs XML-RPC vs SOAP
This is an excerpt from a mail conversation with someone who was a bit surprised to learn that I really don’t like XML-based RPC protocols. Slightly edited for readability, and to protect the innocent.
I was just surprised after all of your work with xmlrpclib and soaplib. I mean you did do that work long before I came to the conclusion that RPC isn’t scalable so I would hardly blame you for changing your mind.
Well, I spent about 30 minutes on the first xmlrpclib (dead link) version, mostly because Dave Winer flamed me over some irrelevant topic, and told me I couldn’t program my way out of a brown paper bag (or something similar)…
The first version was client only, just enough to talk to a Frontier echo server. It took a few more hours to prepare the second release, and I’ve probably spent 2-3 days since then on bug fixes, documentation, and support. Plus a couple of days on _xmlrpclib, our commercial XML-RPC accelerator.
About a week in total. Mostly by accident.
:::
In contrast, SOAP was development hell, with moving specs, no working server to test against (we spent over a week trying to get IBM’s first Java implementation to run on a server at the labs, only to find that they had implemented an earlier version of the specification). The first version was based on the xmlrpclib design, but that architecture broke down, and I did at least two complete overhauls on the way to 0.9 (which we still haven’t released; nobody can motivate themselves to fix the last few bugs…).
:::
The first piece of the REST-puzzle came from the second overhaul — to hold the SOAP body during decoding, we designed a light-weight DOM-like data structure. Only took a day before I realized that it was an excellent way to work with almost any kind of XML data:
http://effbot.org/zone/element.htm
The second piece came from an XML-RPC based project where we needed to work with large chunks of binary data. Base64 just won’t cut it when you have to transport 1-100 megabyte entities mixed with metadata records (and other structured data). We solved this by letting an XML-RPC service return dynamically generated URLs, which the client used to fetch the resulting data via HTTP:
rpc = ServerProxy(...)
ticket = rpc.login(...)
result = rpc.search(ticket, "meteosat")
for scene in result:
url = rpc.geturl(ticket, scene)
async_http(url, file_consumer(scene + ".im"))
Moving to pure HTTP allowed us to throw away the code for the XML-RPC service. (I love throwing away old code ;-)
The third piece was a deeper understanding of XML, with emphasis on how to think in Infoset terms.
:::
So when we started working on the design for a large image distribution and processing system, we already had a simple and scalable design, and the tools to support it. Just send XML documents representing objects back and forth over HTTP, and use the lightweight DOM structure to hold parsed versions of them inside the application. Add some glue code to let application code access the DOM structures as ordinary Python objects, and you have a complete and scalable system.
The result was a much nicer specification (very few buzzwords) that anyone can understand, far less code, and most importantly, a much more robust design.
And we got the contract.
(now that I’m writing this, I realize that the result is not that different from an HTTP-based image archive I built back when Mosaic was state of the art in browing. IIRC, we shipped INI- files back and forth… oh well, one never learns ;-)
Plus, I still think [RPC] is incredibly easy for small apps…
Oh, sure — XML-RPC gives you a lot of power, and anyone can understand how it works, and understand what the limitations are (Dave W. might not know the limitations, but that’s another story ;-)
SOAP is something completely different; lots of additional complexity, but very few additional benefits. Some people love complexity (especially if they see a chance to make a living out of it, like Don Box ;-). But I don’t. Wouldn’t use Python if I did.
Cheers /F
rendered by a django application. hosted by webfaction.