Squid Expect: 100-continue header issues

As I mentioned on my other post about tuning Squid, this article is about the ignore_expect_100 setting and why it’s set on our Squid proxies.

We had and issue where requests for a certain site failed with an HTTP 417 error generated by Squid. You would find entries like these in the Squid access.log for the site  that wasn’t accessible:
1329478677.344      0 192.168.1.22 NONE/417 4480 POST http://somewebsite.com/webservice/data.asmx – NONE/- text/html
(Note: It’s actually an application communicating via HTTP and not manual browser-based access by human beings.)

Turned out the problem is that this application sends requests with an “Expect: 100-continue” HTTP-header and Squid doesn’t have a proper implementation of the HTTP 1.1 Expect-mechanism.

The purpose of the 100 (Continue) status (see section 10.1.1) is to allow a client that is sending a request message with a request body to determine if the origin server is willing to accept the request (based on the request headers) before the client sends the request body. In some cases, it might either be inappropriate or highly inefficient for the client to send the body if the server will reject the message without looking at the body.

In layman’s terms, this means if a client that wants to send a request (with content) to a webserver , e.g. upload a file with an HTTP-POST, it can first ask “Hey webserver, I’m about to POST you a file, here are the HTTP-Headers of my request. If that’s all right with you, send me a 100-continue response and I’ll transmit the file.” (kind of similar to an HTTP-HEAD , which is just an exchange of headers too). Now why is this useful? For example, it allows a webserver to check the Content-Type, Content-Length or any other header of the request before the client attempts to send any actual data. If the webserver doesn’t accept this MIME-type, decides the file is too large or anything else, it can simply respond with a 417 Expectation Failed and no resources would have been wasted by sending a file the web application wouldn’t process anyways.

Pre-Squid 2.7 this header was simply being ignored and filtered (while still forwarding the request without this header). Starting with 2.7, while still not supporting Expect-headers, Squid acts RFC-compliant by not forwarding the request at all.

The issue could be easily reproduced and compared with our 2 versions of Squid:
$ curl -v http://whateversite.com –header “Expect: 100-continue” -x 192.168.1.22:3128
=> New Squid 3.1 throws HTTP-error 417 to client.

The same on the old 2.5 based instance:
$ curl -v http://whateversite.com –header “Expect: 100-continue” -x 192.168.1.11:3128
=> Old Squid forwards request and client receives a reply.

The old behaviour can be forced on 2.7+ by setting ignore_expect_100 on in the squid.conf. Though this formally violates the RFC:
>However, the Expect request-header itself is end-to-end; it MUST be forwarded if the request is forwarded. Many older HTTP/1.0 and HTTP/1.1 applications do not understand the Expect header.

Especially the latter part is true, huh?

Squid is supposed to have a proper Expect-header implementation with version 3.2, which is currently in beta:
http://www.squid-cache.org/mail-archive/squid-users/201101/0284.html
>3.2 supports the expect-100 feature and has no known problems with this.
http://wiki.squid-cache.org/Features/HTTP11
>UPDATE: 1xx forwarding has been implemented in Squid-3.2.

Advertisements

One thought on “Squid Expect: 100-continue header issues

  1. Pingback: Squid Proxy Optimizations « alpacapowered

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s