Posts Tagged Google

Handling Google Web Services Throttles

Introduction

On a recent project I was working on some integration with the Google Web Services static map API (documented here: http://code.google.com/apis/maps/documentation/staticmaps/) as part of an overall BizTalk solution. The integration was being used to record the latitude and longitude of coordinates of stores.

The calls to the Google Web Service were going through a custom .NET method due to some extra business logic being executed before making the call to the web service. When testing the integration on one or two geographic locations, everything worked successfully. But when the integration was tested for hundreds of geographic locations that all needed to be updated in a short period of time, the web service would just return empty data after a period of time. This was a notoriously hard bug to resolve because there was not any additional contextual information about why the web service calls were just returning empty information.

It took some digging but eventually I realized that the Google web service throttles itself to limit usage and prevent a Denial-of-Service (DoS). This limitation is documented here: http://code.google.com/apis/maps/documentation/staticmaps/#Limits. When the web service throttled the results came back as 0 Latitude and 0 Longitude (0,0) so this is confusing and misleading because this is actually a location on the globe, just right in the middle of the Atlantic ocean. A fault with a descriptive error message would have been better.

Workaround

Due to this limitation, I had to add some customized delays and locking to the .NET method so that the calls to the web service would not overwhelm the Google web service’s throttle. This scenario was even more complicated because we did not want more than one BizTalk orchestration instance to overflow the web service throttle either. Because the .NET assembly is running in the BizTalk process custom use of threading  is suspect – you do not know if it will work properly. It was difficult to determine the right interval of polling the Google web service that gave us the maximum thoroughput without causing the service to throttle. But following code worked for us in handling this scenario:

public static object LockObject = new object();

public static string GetLatLonCoordinates(string addressLine)
{
string ret = "";

lock(LockObject) {

// delay for 10 seconds - to avoid the Google request rate limitation.
// For more information on the limits of the Google Static Maps API,
// see: http://code.google.com/apis/maps/documentation/staticmaps/#Limits
Thread.Sleep(10000);

if (addressLine == null) { return "0,0"; }
if (addressLine.Trim().Length == 0) { return "0,0"; }

try
{

GoogleCoordinates coor = GoogleWebServiceWrapper.GetCoordinates(addressLine);
ret = coor.Latitude.ToString() + "," + coor.Longitude.ToString();

}
catch (Exception e)
{
System.Diagnostics.EventLog.WriteEntry("Application", e.Message, EventLogEntryType.Error);
}
}
return ret;
}

Leave a comment