Class HttpWebConnection

java.lang.Object
org.htmlunit.HttpWebConnection
All Implemented Interfaces:
AutoCloseable, WebConnection
public class HttpWebConnection extends Object implements WebConnection
Default implementation of WebConnection, using the HttpClient library to perform HTTP requests.

  • Field Details

    • LOG

      private static final org.apache.commons.logging.Log LOG

    • httpClientBuilder_

      private final Map<Thread, org.apache.http.impl.client.HttpClientBuilder> httpClientBuilder_

    • webClient_

      private final WebClient webClient_

    • virtualHost_

      private String virtualHost_

    • htmlUnitCookieSpecProvider_

      private final HtmlUnitCookieSpecProvider htmlUnitCookieSpecProvider_

    • usedOptions_

      private final WebClientOptions usedOptions_

    • connectionManager_

      private org.apache.http.impl.conn.PoolingHttpClientConnectionManager connectionManager_

    • sharedAuthCache_

      private final org.apache.http.client.AuthCache sharedAuthCache_
      Authentication cache shared among all threads of a web client.

    • httpClientContextByThread_

      private final Map<Thread, org.apache.http.client.protocol.HttpClientContext> httpClientContextByThread_
      Maintains a separate HttpClientContext object per HttpWebConnection and thread.

  • Constructor Details

    • HttpWebConnection

      public HttpWebConnection(WebClient webClient)
      Creates a new HTTP web connection instance.
      Parameters:
      webClient - the WebClient that is using this connection

  • Method Details

    • getResponse

      public WebResponse getResponse(WebRequest webRequest) throws IOException
      Submits a request and retrieves a response.
      Specified by:
      getResponse in interface WebConnection
      Parameters:
      webRequest - the request
      Returns:
      the response to the request defined by the specified request
      Throws:
      IOException - if an IO error occurs

    • onResponseGenerated

      protected void onResponseGenerated(org.apache.http.client.methods.HttpUriRequest httpMethod)
      Called when the response has been generated. Default action is to release the HttpMethod's connection. Subclasses may override.
      Parameters:
      httpMethod - the httpMethod used (can be null)

    • getHttpContext

      private org.apache.http.protocol.HttpContext getHttpContext()
      Returns the HttpClientContext for the current thread. Creates a new one if necessary.

    • setProxy

      private void setProxy(org.apache.http.client.methods.HttpRequestBase httpRequest, WebRequest webRequest)

    • makeHttpMethod

      private org.apache.http.client.methods.HttpUriRequest makeHttpMethod(WebRequest webRequest, org.apache.http.impl.client.HttpClientBuilder httpClientBuilder) throws URISyntaxException
      Creates an HttpMethod instance according to the specified parameters.
      Parameters:
      webRequest - the request
      httpClientBuilder - the httpClientBuilder that will be configured
      Returns:
      the HttpMethod instance constructed according to the specified parameters
      Throws:
      URISyntaxException - in case of syntax problems

    • escapeQuery

      private static String escapeQuery(String query)

    • getCharset

      private static Charset getCharset(Charset charset, List<NameValuePair> pairs)

    • buildFilePart

      void buildFilePart(KeyDataPair pairWithFile, org.apache.http.entity.mime.MultipartEntityBuilder builder)

    • buildHttpMethod

      private static org.apache.http.client.methods.HttpRequestBase buildHttpMethod(HttpMethod submitMethod, URI uri)
      Creates and returns a new HttpClient HTTP method based on the specified parameters.
      Parameters:
      submitMethod - the submit method being used
      uri - the uri being used
      Returns:
      a new HttpClient HTTP method based on the specified parameters

    • getHttpClientBuilder

      protected org.apache.http.impl.client.HttpClientBuilder getHttpClientBuilder()
      Lazily initializes the internal HTTP client.
      Returns:
      the initialized HTTP client

    • getTimeout

      protected int getTimeout(WebRequest webRequest)
      Returns the timeout to use for socket and connection timeouts for HttpConnectionManager. Is overridden to 0 by StreamingWebConnection which keeps reading after a timeout and must have long running connections explicitly terminated.
      Parameters:
      webRequest - the request might have his own timeout
      Returns:
      the WebClient's timeout

    • createHttpClientBuilder

      protected org.apache.http.impl.client.HttpClientBuilder createHttpClientBuilder()
      Creates the HttpClientBuilder that will be used by this WebClient. Extensions may override this method in order to create a customized HttpClientBuilder instance (e.g. with a custom ClientConnectionManager to perform some tracking; see feature request 1438216).
      Returns:
      the HttpClientBuilder that will be used by this WebConnection

    • configureTimeout

      private void configureTimeout(org.apache.http.impl.client.HttpClientBuilder builder, int timeout)

    • createRequestConfigBuilder

      private static org.apache.http.client.config.RequestConfig.Builder createRequestConfigBuilder(int timeout, InetAddress localAddress)

    • createSocketConfigBuilder

      private static org.apache.http.config.SocketConfig.Builder createSocketConfigBuilder(int timeout)

    • reconfigureHttpClientIfNeeded

      private org.apache.http.impl.client.HttpClientBuilder reconfigureHttpClientIfNeeded(org.apache.http.impl.client.HttpClientBuilder httpClientBuilder, WebRequest webRequest)
      React on changes that may have occurred on the WebClient settings. Registering as a listener would be probably better.

    • configureHttpsScheme

      private void configureHttpsScheme(org.apache.http.impl.client.HttpClientBuilder builder)

    • configureHttpProcessorBuilder

      private void configureHttpProcessorBuilder(org.apache.http.impl.client.HttpClientBuilder builder, WebRequest webRequest)

    • setVirtualHost

      public void setVirtualHost(String virtualHost)
      Sets the virtual host.
      Parameters:
      virtualHost - the virtualHost to set

    • getVirtualHost

      public String getVirtualHost()
      Gets the virtual host.
      Returns:
      virtualHost The current virtualHost

    • makeWebResponse

      protected WebResponse makeWebResponse(org.apache.http.HttpResponse httpResponse, WebRequest webRequest, DownloadedContent responseBody, long loadTime)
      Converts an HttpMethod into a WebResponse.
      Parameters:
      httpResponse - the web server's response
      webRequest - the WebRequest
      responseBody - the DownloadedContent
      loadTime - the download time
      Returns:
      a wrapper for the downloaded body.

    • downloadResponse

      protected WebResponse downloadResponse(org.apache.http.client.methods.HttpUriRequest httpMethod, WebRequest webRequest, org.apache.http.HttpResponse httpResponse, long startTime) throws IOException
      Downloads the response. This calls downloadResponseBody(HttpResponse) and constructs the WebResponse.
      Parameters:
      httpMethod - the HttpUriRequest
      webRequest - the WebRequest
      httpResponse - the web server's response
      startTime - the download start time
      Returns:
      a wrapper for the downloaded body.
      Throws:
      IOException - in case of problem reading/saving the body

    • downloadResponseBody

      protected DownloadedContent downloadResponseBody(org.apache.http.HttpResponse httpResponse) throws IOException
      Downloads the response body.
      Parameters:
      httpResponse - the web server's response
      Returns:
      a wrapper for the downloaded body.
      Throws:
      IOException - in case of problem reading/saving the body

    • downloadContent

      public static DownloadedContent downloadContent(InputStream is, int maxInMemory, File tempFileDirectory) throws IOException
      Reads the content of the stream and saves it in memory or on the file system.
      Parameters:
      is - the stream to read
      maxInMemory - the maximumBytes to store in memory, after which save to a local file
      tempFileDirectory - the directory to be used or null for the system default
      Returns:
      a wrapper around the downloaded content
      Throws:
      IOException - in case of read issues

    • newWebResponseInstance

      protected WebResponse newWebResponseInstance(WebResponseData responseData, long loadTime, WebRequest webRequest)
      Constructs an appropriate WebResponse. May be overridden by subclasses to return a specialized WebResponse.
      Parameters:
      responseData - Data that was send back
      loadTime - How long the response took to be sent
      webRequest - the request used to get this response
      Returns:
      the new WebResponse

    • getHttpRequestInterceptors

      private List<org.apache.http.HttpRequestInterceptor> getHttpRequestInterceptors(WebRequest webRequest)

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface WebConnection

    • createConnectionManager

      private static org.apache.http.impl.conn.PoolingHttpClientConnectionManager createConnectionManager(org.apache.http.impl.client.HttpClientBuilder builder)
      Has the exact logic in HttpClientBuilder.build() which sets the connManager part, but with the ability to configure socketFactory.

    • split

      private static String[] split(String s)

    • getField

      private static <T> T getField(Object target, String fieldName) throws IllegalAccessException
      Throws:
      IllegalAccessException