Package io.netty.handler.codec.http

Class HttpObjectDecoder

java.lang.Object
io.netty.channel.ChannelHandlerAdapter
io.netty.channel.ChannelInboundHandlerAdapter
io.netty.handler.codec.ByteToMessageDecoder
io.netty.handler.codec.http.HttpObjectDecoder
All Implemented Interfaces:
ChannelHandler, ChannelInboundHandler
Direct Known Subclasses:
HttpRequestDecoder, HttpResponseDecoder, RtspDecoder, RtspObjectDecoder
public abstract class HttpObjectDecoder extends ByteToMessageDecoder
Decodes ByteBufs into HttpMessages and HttpContents.

Parameters that prevents excessive memory consumption

NameDefault valueMeaning
maxInitialLineLength 4096 The maximum length of the initial line (e.g. "GET / HTTP/1.0" or "HTTP/1.0 200 OK") If the length of the initial line exceeds this value, a TooLongHttpLineException will be raised.
maxHeaderSize 8192 The maximum length of all headers. If the sum of the length of each header exceeds this value, a TooLongHttpHeaderException will be raised.
maxChunkSize 8192 The maximum length of the content or each chunk. If the content length (or the length of each chunk) exceeds this value, the content or chunk will be split into multiple HttpContents whose length is maxChunkSize at maximum.

Parameters that control parsing behavior

NameDefault valueMeaning
allowDuplicateContentLengths false When set to false, will reject any messages that contain multiple Content-Length header fields. When set to true, will allow multiple Content-Length headers only if they are all the same decimal value. The duplicated field-values will be replaced with a single valid Content-Length field. See RFC 7230, Section 3.3.2.
allowPartialChunks true If the length of a chunk exceeds the ByteBufs readable bytes and allowPartialChunks is set to true, the chunk will be split into multiple HttpContents. Otherwise, if the chunk size does not exceed maxChunkSize and allowPartialChunks is set to false, the ByteBuf is not decoded into an HttpContent until the readable bytes are greater or equal to the chunk size.

Chunked Content

If the content of an HTTP message is greater than maxChunkSize or the transfer encoding of the HTTP message is 'chunked', this decoder generates one HttpMessage instance and its following HttpContents per single HTTP message to avoid excessive memory consumption. For example, the following HTTP message: 
 GET / HTTP/1.1
 Transfer-Encoding: chunked

 1a
 abcdefghijklmnopqrstuvwxyz
 10
 1234567890abcdef
 0
 Content-MD5: ...
 [blank line]
triggers HttpRequestDecoder to generate 3 objects:
  1. An HttpRequest,
  2. The first HttpContent whose content is 'abcdefghijklmnopqrstuvwxyz',
  3. The second LastHttpContent whose content is '1234567890abcdef', which marks the end of the content.
If you prefer not to handle HttpContents by yourself for your convenience, insert HttpObjectAggregator after this decoder in the ChannelPipeline. However, please note that your server might not be as memory efficient as without the aggregator.

Extensibility

Please note that this decoder is designed to be extended to implement a protocol derived from HTTP, such as RTSP and ICAP. To implement the decoder of such a derived protocol, extend this class and implement all abstract methods properly.

Header Validation

It is recommended to always enable header validation.

Without header validation, your system can become vulnerable to CWE-113: Improper Neutralization of CRLF Sequences in HTTP Headers ('HTTP Response Splitting') .

This recommendation stands even when both peers in the HTTP exchange are trusted, as it helps with defence-in-depth.

  • Field Details

    • DEFAULT_MAX_INITIAL_LINE_LENGTH

      public static final int DEFAULT_MAX_INITIAL_LINE_LENGTH
      See Also:

    • DEFAULT_MAX_HEADER_SIZE

      public static final int DEFAULT_MAX_HEADER_SIZE
      See Also:

    • DEFAULT_CHUNKED_SUPPORTED

      public static final boolean DEFAULT_CHUNKED_SUPPORTED
      See Also:

    • DEFAULT_ALLOW_PARTIAL_CHUNKS

      public static final boolean DEFAULT_ALLOW_PARTIAL_CHUNKS
      See Also:

    • DEFAULT_MAX_CHUNK_SIZE

      public static final int DEFAULT_MAX_CHUNK_SIZE
      See Also:

    • DEFAULT_VALIDATE_HEADERS

      public static final boolean DEFAULT_VALIDATE_HEADERS
      See Also:

    • DEFAULT_INITIAL_BUFFER_SIZE

      public static final int DEFAULT_INITIAL_BUFFER_SIZE
      See Also:

    • DEFAULT_ALLOW_DUPLICATE_CONTENT_LENGTHS

      public static final boolean DEFAULT_ALLOW_DUPLICATE_CONTENT_LENGTHS
      See Also:

    • DEFAULT_STRICT_LINE_PARSING

      public static final boolean DEFAULT_STRICT_LINE_PARSING

    • THROW_INVALID_CHUNK_EXTENSION

      private static final Runnable THROW_INVALID_CHUNK_EXTENSION

    • THROW_INVALID_LINE_SEPARATOR

      private static final Runnable THROW_INVALID_LINE_SEPARATOR

    • maxChunkSize

      private final int maxChunkSize

    • chunkedSupported

      private final boolean chunkedSupported

    • allowPartialChunks

      private final boolean allowPartialChunks

    • validateHeaders

      @Deprecated protected final boolean validateHeaders
      Deprecated.
      This field is no longer used. It is only kept around for backwards compatibility purpose.

    • headersFactory

      protected final HttpHeadersFactory headersFactory

    • trailersFactory

      protected final HttpHeadersFactory trailersFactory

    • allowDuplicateContentLengths

      private final boolean allowDuplicateContentLengths

    • parserScratchBuffer

      private final ByteBuf parserScratchBuffer

    • defaultStrictCRLFCheck

      private final Runnable defaultStrictCRLFCheck

    • headerParser

      private final HttpObjectDecoder.HeaderParser headerParser

    • lineParser

      private final HttpObjectDecoder.LineParser lineParser

    • message

      private HttpMessage message

    • chunkSize

      private long chunkSize

    • contentLength

      private long contentLength

    • chunked

      private boolean chunked

    • isSwitchingToNonHttp1Protocol

      private boolean isSwitchingToNonHttp1Protocol

    • resetRequested

      private final AtomicBoolean resetRequested

    • name

      private AsciiString name

    • value

      private String value

    • trailer

      private LastHttpContent trailer

    • currentState

      private HttpObjectDecoder.State currentState

    • SP_LENIENT_BYTES

      private static final boolean[] SP_LENIENT_BYTES

    • LATIN_WHITESPACE

      private static final boolean[] LATIN_WHITESPACE

    • ISO_CONTROL_OR_WHITESPACE

      private static final boolean[] ISO_CONTROL_OR_WHITESPACE

    • SKIP_CONTROL_CHARS_BYTES

      private static final ByteProcessor SKIP_CONTROL_CHARS_BYTES

  • Constructor Details

    • HttpObjectDecoder

      protected HttpObjectDecoder()
      Creates a new instance with the default maxInitialLineLength (4096), maxHeaderSize (8192), and maxChunkSize (8192).

    • HttpObjectDecoder

      @Deprecated protected HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported)
      Deprecated.
      Use HttpObjectDecoder(HttpDecoderConfig) instead.
      Creates a new instance with the specified parameters.

    • HttpObjectDecoder

      @Deprecated protected HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders)
      Deprecated.
      Use HttpObjectDecoder(HttpDecoderConfig) instead.
      Creates a new instance with the specified parameters.

    • HttpObjectDecoder

      @Deprecated protected HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders, int initialBufferSize)
      Deprecated.
      Use HttpObjectDecoder(HttpDecoderConfig) instead.
      Creates a new instance with the specified parameters.

    • HttpObjectDecoder

      @Deprecated protected HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders, int initialBufferSize, boolean allowDuplicateContentLengths)
      Deprecated.
      Use HttpObjectDecoder(HttpDecoderConfig) instead.
      Creates a new instance with the specified parameters.

    • HttpObjectDecoder

      @Deprecated protected HttpObjectDecoder(int maxInitialLineLength, int maxHeaderSize, int maxChunkSize, boolean chunkedSupported, boolean validateHeaders, int initialBufferSize, boolean allowDuplicateContentLengths, boolean allowPartialChunks)
      Deprecated.
      Use HttpObjectDecoder(HttpDecoderConfig) instead.
      Creates a new instance with the specified parameters.

    • HttpObjectDecoder

      protected HttpObjectDecoder(HttpDecoderConfig config)
      Creates a new instance with the specified configuration.

  • Method Details

    • handlerRemoved0

      protected void handlerRemoved0(ChannelHandlerContext ctx) throws Exception
      Description copied from class: ByteToMessageDecoder
      Gets called after the ByteToMessageDecoder was removed from the actual context and it doesn't handle events anymore.
      Overrides:
      handlerRemoved0 in class ByteToMessageDecoder
      Throws:
      Exception

    • isValidating

      protected boolean isValidating(HttpHeadersFactory headersFactory)

    • decode

      protected void decode(ChannelHandlerContext ctx, ByteBuf buffer, List<Object> out) throws Exception
      Description copied from class: ByteToMessageDecoder
      Decode the from one ByteBuf to an other. This method will be called till either the input ByteBuf has nothing to read when return from this method or till nothing was read from the input ByteBuf.
      Specified by:
      decode in class ByteToMessageDecoder
      Parameters:
      ctx - the ChannelHandlerContext which this ByteToMessageDecoder belongs to
      buffer - the ByteBuf from which to read data
      out - the List to which decoded messages should be added
      Throws:
      Exception - is thrown if an error occurs

    • decodeLast

      protected void decodeLast(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception
      Description copied from class: ByteToMessageDecoder
      Is called one last time when the ChannelHandlerContext goes in-active. Which means the ByteToMessageDecoder.channelInactive(ChannelHandlerContext) was triggered. By default, this will just call ByteToMessageDecoder.decode(ChannelHandlerContext, ByteBuf, List) but sub-classes may override this for some special cleanup operation.
      Overrides:
      decodeLast in class ByteToMessageDecoder
      Throws:
      Exception

    • userEventTriggered

      public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception
      Description copied from class: ChannelInboundHandlerAdapter
      Calls ChannelHandlerContext.fireUserEventTriggered(Object) to forward to the next ChannelInboundHandler in the ChannelPipeline. Sub-classes may override this method to change behavior.
      Specified by:
      userEventTriggered in interface ChannelInboundHandler
      Overrides:
      userEventTriggered in class ByteToMessageDecoder
      Throws:
      Exception

    • addCurrentMessage

      private void addCurrentMessage(List<Object> out)

    • isContentAlwaysEmpty

      protected boolean isContentAlwaysEmpty(HttpMessage msg)

    • isSwitchingToNonHttp1Protocol

      protected boolean isSwitchingToNonHttp1Protocol(HttpResponse msg)
      Returns true if the server switched to a different protocol than HTTP/1.0 or HTTP/1.1, e.g. HTTP/2 or Websocket. Returns false if the upgrade happened in a different layer, e.g. upgrade from HTTP/1.1 to HTTP/1.1 over TLS.

    • reset

      public void reset()
      Resets the state of the decoder so that it is ready to decode a new message. This method is useful for handling a rejected request with Expect: 100-continue header.

    • resetNow

      private void resetNow()

    • invalidMessage

      private HttpMessage invalidMessage(HttpMessage current, ByteBuf in, Exception cause)

    • checkChunkExtensions

      private static void checkChunkExtensions(ByteBuf line)

    • invalidChunk

      private HttpContent invalidChunk(ByteBuf in, Exception cause)

    • readHeaders

      private HttpObjectDecoder.State readHeaders(ByteBuf buffer)

    • handleTransferEncodingChunkedWithContentLength

      protected void handleTransferEncodingChunkedWithContentLength(HttpMessage message)
      Invoked when a message with both a "Transfer-Encoding: chunked" and a "Content-Length" header field is detected. The default behavior is to remove the Content-Length field, but this method could be overridden to change the behavior (to, e.g., throw an exception and produce an invalid message).

      See: https://tools.ietf.org/html/rfc7230#section-3.3.3 

           If a message is received with both a Transfer-Encoding and a
     Content-Length header field, the Transfer-Encoding overrides the
     Content-Length.  Such a message might indicate an attempt to
     perform request smuggling (Section 9.5) or response splitting
     (Section 9.4) and ought to be handled as an error.  A sender MUST
     remove the received Content-Length field prior to forwarding such
     a message downstream.
      Also see: https://github.com/apache/tomcat/blob/b693d7c1981fa7f51e58bc8c8e72e3fe80b7b773/ java/org/apache/coyote/http11/Http11Processor.java#L747-L755 https://github.com/nginx/nginx/blob/0ad4393e30c119d250415cb769e3d8bc8dce5186/ src/http/ngx_http_request.c#L1946-L1953

    • readTrailingHeaders

      private LastHttpContent readTrailingHeaders(ByteBuf buffer)

    • isPermittedTrailingHeader

      private static boolean isPermittedTrailingHeader(AsciiString name)
      Checks whether the given trailer field name is permitted per RFC 9110 section 6.5

    • isDecodingRequest

      protected abstract boolean isDecodingRequest()

    • createMessage

      protected abstract HttpMessage createMessage(String[] initialLine) throws Exception
      Throws:
      Exception

    • createInvalidMessage

      protected abstract HttpMessage createInvalidMessage()

    • skipWhiteSpaces

      private static int skipWhiteSpaces(byte[] hex, int start, int length)
      It skips any whitespace char and return the number of skipped bytes.

    • getChunkSize

      private static int getChunkSize(byte[] hex, int start, int length)

    • splitInitialLine

      private String[] splitInitialLine(ByteBuf asciiBuffer)

    • splitFirstWordInitialLine

      protected String splitFirstWordInitialLine(byte[] asciiContent, int start, int length)

    • splitSecondWordInitialLine

      protected String splitSecondWordInitialLine(byte[] asciiContent, int start, int length)

    • splitThirdWordInitialLine

      protected String splitThirdWordInitialLine(byte[] asciiContent, int start, int length)

    • langAsciiString

      private static String langAsciiString(byte[] asciiContent, int start, int length)
      This method shouldn't exist: look at https://bugs.openjdk.org/browse/JDK-8295496 for more context

    • splitHeader

      private void splitHeader(byte[] line, int start, int length)

    • splitHeaderName

      protected AsciiString splitHeaderName(byte[] sb, int start, int length)

    • findNonSPLenient

      private static int findNonSPLenient(byte[] sb, int offset, int end)

    • findSPLenient

      private static int findSPLenient(byte[] sb, int offset, int end)

    • isSPLenient

      private static boolean isSPLenient(byte c)

    • isWhitespace

      private static boolean isWhitespace(byte b)

    • findNonWhitespace

      private static int findNonWhitespace(byte[] sb, int offset, int end)

    • findEndOfString

      private static int findEndOfString(byte[] sb, int start, int end)

    • isOWS

      private static boolean isOWS(byte ch)

    • isControlOrWhitespaceAsciiChar

      private static boolean isControlOrWhitespaceAsciiChar(byte b)