Making HTTP requests beyond the 32MB urlfetch limit on Google App Engine
Since Google App Engine’s
urlfetch service imposes a 32MB response size limit on each request, one has to roll their own http.Client implementation using
appengine/socket to make requests exceeding that limit.
appengine/socket implementation is not working with HTTP calls inside the local development server, such that one can not use the same solution and code in development and production.
The two issues made me create the
fetchall package, a simple drop-in replacement for
urlfetch that gives you a
http.Client implementation that handles arbitrarily large HTTP requests (within the time limits of Google App Engine).
When running in the development server, the implementation will simply use Go’s
net package instead of
appengine/socket in order for request to work locally as well as in the cloud.
To use this package simply replace
fetchall.Client in your Go code.