Package org.htmlunit

Class HttpWebConnection

  • All Implemented Interfaces:
    java.lang.AutoCloseable, WebConnection
    public class HttpWebConnection
extends java.lang.Object
implements WebConnection
    Default implementation of WebConnection, using the HttpClient library to perform HTTP requests.

    • Field Detail

      • LOG

        private static final org.apache.commons.logging.Log LOG

      • HACKED_COOKIE_POLICY

        private static final java.lang.String HACKED_COOKIE_POLICY
        See Also:
        Constant Field Values

      • httpClientBuilder_

        private final java.util.Map<java.lang.Thread,​org.apache.http.impl.client.HttpClientBuilder> httpClientBuilder_

      • webClient_

        private final WebClient webClient_

      • virtualHost_

        private java.lang.String virtualHost_

      • connectionManager_

        private org.apache.http.impl.conn.PoolingHttpClientConnectionManager connectionManager_

      • sharedAuthCache_

        private final org.apache.http.client.AuthCache sharedAuthCache_
        Authentication cache shared among all threads of a web client.

      • httpClientContextByThread_

        private final java.util.Map<java.lang.Thread,​org.apache.http.client.protocol.HttpClientContext> httpClientContextByThread_
        Maintains a separate HttpClientContext object per HttpWebConnection and thread.

    • Constructor Detail

      • HttpWebConnection

        public HttpWebConnection​(WebClient webClient)
        Creates a new HTTP web connection instance.
        Parameters:
        webClient - the WebClient that is using this connection

    • Method Detail

      • getResponse

        public WebResponse getResponse​(WebRequest webRequest)
                        throws java.io.IOException
        Submits a request and retrieves a response.
        Specified by:
        getResponse in interface WebConnection
        Parameters:
        webRequest - the request
        Returns:
        the response to the request defined by the specified request
        Throws:
        java.io.IOException - if an IO error occurs

      • onResponseGenerated

        protected void onResponseGenerated​(org.apache.http.client.methods.HttpUriRequest httpMethod)
        Called when the response has been generated. Default action is to release the HttpMethod's connection. Subclasses may override.
        Parameters:
        httpMethod - the httpMethod used (can be null)

      • getHttpContext

        private org.apache.http.protocol.HttpContext getHttpContext()
        Returns the HttpClientContext for the current thread. Creates a new one if necessary.

      • setProxy

        private void setProxy​(org.apache.http.client.methods.HttpRequestBase httpRequest,
                      WebRequest webRequest)

      • makeHttpMethod

        private org.apache.http.client.methods.HttpUriRequest makeHttpMethod​(WebRequest webRequest,
                                                                     org.apache.http.impl.client.HttpClientBuilder httpClientBuilder)
                                                              throws java.net.URISyntaxException
        Creates an HttpMethod instance according to the specified parameters.
        Parameters:
        webRequest - the request
        httpClientBuilder - the httpClientBuilder that will be configured
        Returns:
        the HttpMethod instance constructed according to the specified parameters
        Throws:
        java.net.URISyntaxException - in case of syntax problems

      • escapeQuery

        private static java.lang.String escapeQuery​(java.lang.String query)

      • getCharset

        private static java.nio.charset.Charset getCharset​(java.nio.charset.Charset charset,
                                                   java.util.List<NameValuePair> pairs)

      • buildFilePart

        void buildFilePart​(KeyDataPair pairWithFile,
                   org.apache.http.entity.mime.MultipartEntityBuilder builder)

      • buildHttpMethod

        private static org.apache.http.client.methods.HttpRequestBase buildHttpMethod​(HttpMethod submitMethod,
                                                                              java.net.URI uri)
        Creates and returns a new HttpClient HTTP method based on the specified parameters.
        Parameters:
        submitMethod - the submit method being used
        uri - the uri being used
        Returns:
        a new HttpClient HTTP method based on the specified parameters

      • getHttpClientBuilder

        protected org.apache.http.impl.client.HttpClientBuilder getHttpClientBuilder()
        Lazily initializes the internal HTTP client.
        Returns:
        the initialized HTTP client

      • getTimeout

        protected int getTimeout​(WebRequest webRequest)
        Returns the timeout to use for socket and connection timeouts for HttpConnectionManager. Is overridden to 0 by StreamingWebConnection which keeps reading after a timeout and must have long running connections explicitly terminated.
        Parameters:
        webRequest - the request might have his own timeout
        Returns:
        the WebClient's timeout

      • createHttpClientBuilder

        protected org.apache.http.impl.client.HttpClientBuilder createHttpClientBuilder()
        Creates the HttpClientBuilder that will be used by this WebClient. Extensions may override this method in order to create a customized HttpClientBuilder instance (e.g. with a custom ClientConnectionManager to perform some tracking; see feature request 1438216).
        Returns:
        the HttpClientBuilder that will be used by this WebConnection

      • configureTimeout

        private void configureTimeout​(org.apache.http.impl.client.HttpClientBuilder builder,
                              int timeout)

      • createRequestConfigBuilder

        private static org.apache.http.client.config.RequestConfig.Builder createRequestConfigBuilder​(int timeout,
                                                                                              java.net.InetAddress localAddress)

      • createSocketConfigBuilder

        private static org.apache.http.config.SocketConfig.Builder createSocketConfigBuilder​(int timeout)

      • reconfigureHttpClientIfNeeded

        private org.apache.http.impl.client.HttpClientBuilder reconfigureHttpClientIfNeeded​(org.apache.http.impl.client.HttpClientBuilder httpClientBuilder,
                                                                                    WebRequest webRequest)
        React on changes that may have occurred on the WebClient settings. Registering as a listener would be probably better.

      • configureHttpsScheme

        private void configureHttpsScheme​(org.apache.http.impl.client.HttpClientBuilder builder)

      • configureHttpProcessorBuilder

        private void configureHttpProcessorBuilder​(org.apache.http.impl.client.HttpClientBuilder builder,
                                           WebRequest webRequest)

      • setVirtualHost

        public void setVirtualHost​(java.lang.String virtualHost)
        Sets the virtual host.
        Parameters:
        virtualHost - the virtualHost to set

      • getVirtualHost

        public java.lang.String getVirtualHost()
        Gets the virtual host.
        Returns:
        virtualHost The current virtualHost

      • makeWebResponse

        protected WebResponse makeWebResponse​(org.apache.http.HttpResponse httpResponse,
                                      WebRequest webRequest,
                                      DownloadedContent responseBody,
                                      long loadTime)
        Converts an HttpMethod into a WebResponse.
        Parameters:
        httpResponse - the web server's response
        webRequest - the WebRequest
        responseBody - the DownloadedContent
        loadTime - the download time
        Returns:
        a wrapper for the downloaded body.

      • downloadResponse

        protected WebResponse downloadResponse​(org.apache.http.client.methods.HttpUriRequest httpMethod,
                                       WebRequest webRequest,
                                       org.apache.http.HttpResponse httpResponse,
                                       long startTime)
                                throws java.io.IOException
        Downloads the response. This calls downloadResponseBody(HttpResponse) and constructs the WebResponse.
        Parameters:
        httpMethod - the HttpUriRequest
        webRequest - the WebRequest
        httpResponse - the web server's response
        startTime - the download start time
        Returns:
        a wrapper for the downloaded body.
        Throws:
        java.io.IOException - in case of problem reading/saving the body

      • downloadResponseBody

        protected DownloadedContent downloadResponseBody​(org.apache.http.HttpResponse httpResponse)
                                          throws java.io.IOException
        Downloads the response body.
        Parameters:
        httpResponse - the web server's response
        Returns:
        a wrapper for the downloaded body.
        Throws:
        java.io.IOException - in case of problem reading/saving the body

      • downloadContent

        public static DownloadedContent downloadContent​(java.io.InputStream is,
                                                int maxInMemory,
                                                java.io.File tempFileDirectory)
                                         throws java.io.IOException
        Reads the content of the stream and saves it in memory or on the file system.
        Parameters:
        is - the stream to read
        maxInMemory - the maximumBytes to store in memory, after which save to a local file
        tempFileDirectory - the directory to be used or null for the system default
        Returns:
        a wrapper around the downloaded content
        Throws:
        java.io.IOException - in case of read issues

      • newWebResponseInstance

        protected WebResponse newWebResponseInstance​(WebResponseData responseData,
                                             long loadTime,
                                             WebRequest webRequest)
        Constructs an appropriate WebResponse. May be overridden by subclasses to return a specialized WebResponse.
        Parameters:
        responseData - Data that was send back
        webRequest - the request used to get this response
        loadTime - How long the response took to be sent
        Returns:
        the new WebResponse

      • getHttpRequestInterceptors

        private java.util.List<org.apache.http.HttpRequestInterceptor> getHttpRequestInterceptors​(WebRequest webRequest)

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface WebConnection

      • createConnectionManager

        private static org.apache.http.impl.conn.PoolingHttpClientConnectionManager createConnectionManager​(org.apache.http.impl.client.HttpClientBuilder builder)
        Has the exact logic in HttpClientBuilder.build() which sets the connManager part, but with the ability to configure socketFactory.

      • split

        private static java.lang.String[] split​(java.lang.String s)

      • getField

        private static <T> T getField​(java.lang.Object target,
                              java.lang.String fieldName)
                       throws java.lang.IllegalAccessException
        Throws:
        java.lang.IllegalAccessException