h2: Add support for HTTP/2 CONNECT Fast Open

SpdyProxyClientSocket waits for 200 OK before returning OK for Connect.

Change that behavior to returning OK immediately after CONNECT header.

This feature is enabled by default. It should probably be turned on
through an interface but that implies passing a flag through deep
interface chains right now requiring intrusive changes to multiple
places.

Design notes:

The current approach is better than the obvious TCP Fast Open style fake
Connect().

Fast Open should not be used for preconnects as preconnects need actual
connections set up. The Naive client does not use preconnects per se
(using "...RawConnect") but the user agent will use preconnects and the
Naive client has to infer that. Hence there is a need to check the
incoming socket for available bytes right before Connect() and configure
whether a socket should be connected with Fast Open. But fake Connect()
make it difficult to check the incoming socket because it immediately
returns and there is not enough time for the first read of the incoming
socket to arrive.

To check for preconnects it is best to push the first read of the
incoming socket to as late as possible. The other (wrong) way of doing
that is to pass in an early read callback and call it immediately after
sending HEADERS and then send the available bytes right there. This way
is wrong because it does not work with late binding, which assumes
Connect() is idempotent and causes sockets opened in this way to be
potentially bound to the wrong socket requests.

The current approach is to return OK in Connect() right after sending
HEADERS before getting the reply, which is to be received later. If the
reply is received during a subsequent Read() and the reply indicates an
error, the error is returned to the callback of the Read(); otherwise
the error is ignored with the connection disconnected and subsequent
Read() and Write() should discover the disconnection.

net/spdy/spdy_proxy_client_socket.cc

@@ -46,6 +46,10 @@ SpdyProxyClientSocket::SpdyProxyClientSocket(
       write_buffer_len_(0),
       was_ever_used_(false),
       redirect_has_load_timing_info_(false),
+      // This is a hack to avoid messing up higher APIs.
+      // Should be false by default officially.
+      use_fastopen_(true),
+      read_headers_pending_(false),
       net_log_(NetLogWithSource::Make(spdy_stream->net_log().net_log(),
                                       NetLogSourceType::PROXY_CLIENT_SOCKET)),
       source_dependency_(source_net_log.source()),
@@ -308,6 +312,20 @@ int SpdyProxyClientSocket::DoLoop(int last_io_result) {
         rv = DoReadReplyComplete(rv);
         net_log_.EndEventWithNetErrorCode(
             NetLogEventType::HTTP_TRANSACTION_TUNNEL_READ_HEADERS, rv);
+        if (use_fastopen_ && read_headers_pending_) {
+          read_headers_pending_ = false;
+          if (rv < 0) {
+            // read_callback_ cannot be called.
+            if (!read_callback_)
+              rv = ERR_IO_PENDING;
+            // read_callback_ will be called with this error and be reset.
+            // Further data after that will be ignored.
+            next_state_ = STATE_DISCONNECTED;
+          } else {
+            // Does not call read_callback_ from here if headers are OK.
+            rv = ERR_IO_PENDING;
+          }
+        }
         break;
       default:
         NOTREACHED() << "bad state";
@@ -364,6 +382,12 @@ int SpdyProxyClientSocket::DoSendRequestComplete(int result) {
   if (result < 0)
     return result;
 
+  if (use_fastopen_) {
+    read_headers_pending_ = true;
+    next_state_ = STATE_OPEN;
+    return OK;
+  }
+
   // Wait for HEADERS frame from the server
   next_state_ = STATE_READ_REPLY_COMPLETE;
   return ERR_IO_PENDING;
@@ -427,6 +451,10 @@ void SpdyProxyClientSocket::OnHeadersSent() {
 void SpdyProxyClientSocket::OnHeadersReceived(
     const spdy::SpdyHeaderBlock& response_headers,
     const spdy::SpdyHeaderBlock* pushed_request_headers) {
+  if (use_fastopen_ && read_headers_pending_ && next_state_ == STATE_OPEN) {
+    next_state_ = STATE_READ_REPLY_COMPLETE;
+  }
+
   // If we've already received the reply, existing headers are too late.
   // TODO(mbelshe): figure out a way to make HEADERS frames useful after the
   //                initial response.
@@ -453,7 +481,7 @@ void SpdyProxyClientSocket::OnDataReceived(std::unique_ptr<SpdyBuffer> buffer) {
                                   NULL);
   }
 
-  if (!read_callback_.is_null()) {
+  if (!read_callback_.is_null() && user_buffer_ != nullptr) {
     int rv = PopulateUserReadBuffer(user_buffer_->data(), user_buffer_len_);
     user_buffer_ = NULL;
     user_buffer_len_ = 0;

net/spdy/spdy_proxy_client_socket.h

@@ -172,6 +172,9 @@ class NET_EXPORT_PRIVATE SpdyProxyClientSocket : public ProxyClientSocket,
   bool redirect_has_load_timing_info_;
   LoadTimingInfo redirect_load_timing_info_;
 
+  bool use_fastopen_;
+  bool read_headers_pending_;
+
   const NetLogWithSource net_log_;
   const NetLogSource source_dependency_;