Using the ElementTree Module to Generate SOAP Messages, Part 2: Talking to Google

November 17, 2003 | Fredrik Lundh

Note: A distribution kit containing the source code for this article is available from the effbot.org downloads site (look for ElementSOAP 0.1 or later).

Talking to Google #

The Google Web API is a SOAP interface to Google’s search service, and a few other services.

Note: To run the examples in this article, you need ElementTree 1.2a5 or later. Earlier versions may create invalid XML for request structures that contain more than one typed value.

<soap:Envelope>
  <soap:Body>
    <google:doGoogleSearch soap:encodingStyle="...">
      <key xsd:type="xsi:string">... license key ...</key>
      <q xsd:type="xsi:string">query</q>
      <start xsd:type="xsi:int">0</start>
      <maxResults xsd:type="xsi:int">10</maxResults>
      ... more fields ...
    </google:doGoogleSearch>
  </soap:Body>
</soap:Envelope>

The q field contains the query. You can download up to ten results per request; to get more results, you have to issue the same request again, with a different start setting.

The request structure must contain a few more mandatory fields; see the sample code below for a list.

The doGoogleSearch method takes a query string, and optional start and count arguments. It returns a complete response structure, which contains information about the search, as well as elements describing each individual hit. To get a list of hit descriptors, use findall(“.//item”).

Response subelements include estimatedTotalResultsCount, searchTime, startIndex, endIndex, and searchQuery (the original query string).

Handling Server Errors #

>>> g = GoogleService("... license key ...")

>>> g.doGoogleSearch("hello", maxResults=10)
<Element return at 9dd83c>

>>> g.doGoogleSearch("hello", maxResults=100)
Traceback (most recent call last):
  File "myprogram.py", line 24, in doGoogleSearch
  File "ElementSOAP.py", line 64, in call
    extra_headers=[("SOAPAction", action)]
  File "HTTPClient.py", line 114, in do_request
    raise HTTPError(errcode, errmsg, headers, h.getfile())
HTTPClient.HTTPError: (500, 'Internal Server Error')

Oops. Looks like someone didn’t like that request.

This error isn’t really an HTTP server error in the usual sense; rather, it’s the error code used by SOAP 1.1 to signal that a request couldn’t be processed. When this happens, the HTTP response body may contain a SOAP response with a special SOAP Fault element.

If you disable the errcode check in HTTPClient, and print out the response structure, you’ll get something like:

<soap:Envelope>
  <soap:Body>
    <soap:Fault>
      <faultcode>SOAP-ENV:Server</faultcode>
      <faultstring>Exception from service object:
      maxResults must be 10 or less.</faultstring>
      <faultactor>/search/beta2</faultactor>
      <detail>
         ... additional elements ...
      </detail>
      ...
    </soap:Fault>
  </soap:Body>
</soap:Envelope>

This looks like an ordinary response, but instead of the return element, it contains a soap:Fault element, with more information about what really happened.

It would be nice if our SOAP code raised some kind of SOAP-specific exception, instead of hiding the error message down in the HTTP client layer.

To deal with this, I’ve modified the HTTPClient slightly; if the status code isn’t 200, the client will still raise an exception, but the exception object contains not only the HTTP status code, but also the HTTP header dictionary and the file handle. This allows the exception handler to look at the header, and if necessary, parse the response body.

With this in place, it’s time to teach the SoapService class to look for SOAP fault responses. Here’s the updated server class:

from HTTPClient import HTTPClient, HTTPError

class SoapService:
    def __init__(self, url=None):
        self.__client = HTTPClient(url or self.url)
    def call(self, action, request):
        # build SOAP envelope
        envelope = Element(NS_SOAP_ENV + "Envelope")
        body = SubElement(envelope, NS_SOAP_ENV + "Body")
        body.append(request)
        # call the server
        try:
            response = self.__client.do_request(
                tostring(envelope),
                extra_headers=[("SOAPAction", action)]
                )
        except HTTPError, v:
            if v[0] == 500:
                # might be a SOAP fault
                response = ElementTree.parse(v[3])
        response = response.getroot().find(body.tag)[0]
        if response.tag == NS_SOAP_ENV + "Fault":
            raise SoapFault(
                response.findtext("faultcode"),
                response.findtext("faultstring"),
                response.findtext("faultactor"),
                response.find("detail")
                )
        return response

And here’s the exception class used for SOAP fault responses: