K40243113: Overview of the HTTP profile

Non-Diagnostic

Original Publication Date: Dec 20, 2018

Update Date: Jun 8, 2021

Topic

The BIG-IP system provides the HTTP profile as an option for processing HTTP traffic. The HTTP profile allows the virtual server to operate in full Layer 7 (L7) inspection mode and use features such as the following:

Full HTTP iRules logic OneConnect functionality (including OneConnect transformations) L7 persistence (cookie, hash, universal, and iRules) HTTP pipelining Virtual Server Authentication Cookie Encryption Request / Response Chunking

Description

A virtual server with an associated HTTP profile processes connections using the BIG-IP system's full proxy architecture for the purpose of making requests on behalf of clients. In this mode, the BIG-IP system processes the three-way TCP handshake and initial data packet on the client-side connection before initiating the TCP handshake on the server-side connection; the client's data packet triggers the BIG-IP system to initiate the server-side connection.

Important: HTTP profiles are incompatible with encrypted pass-through traffic, such as SSL and require a Client SSL profile to decrypt the traffic for L7 HTTP inspection. If the virtual server processing the encrypted traffic is configured with an HTTP profile and no Client SSL profile, the connection fails.

The HTTP profile provides settings to configure full L7 functionality for your virtual sever traffic. You can use the default profile settings or configure custom profile settings when you create a profile. The following table contains the settings and definitions for the HTTP profile.

Settings

Setting Default Description Name None Specifies the name of the profile. The proxy mode setting determines whether the virtual server operates in reverse, explicit, or transparent mode. The proxy mode offers different HTTP enforcement options for the profile and allows you to configure the system to act as a gateway in the case of explicit proxy mode. The following proxy mode options are available:

Reverse - Sets enforcement options that are suitable for many deployments and enables the BIG-IP system to manage responses from multiple servers. Explicit - Changes the enforcement options for the profile and enables the Proxy Mode Reverse BIG-IP system to process HTTP proxy requests and function as a gateway. By configuring browser traffic to use the proxy, you can control whether to allow or deny a requested connection, based on configured policies. The Explicit Proxy Mode requires a DNS resolver, specified in the Explicit Proxy area of the screen. Transparent - Changes the enforcement options for the profile and enables the BIG-IP system to forward invalid HTTP traffic to a specified server, instead of dropping the connection. By configuring an HTTP profile to forward invalid HTTP traffic, you can manage various atypical service provider scenarios, such as HTTP traffic from non-browser clients that function as web browsers.

Parent Profile http Indicates the parent profile from which this profile inherits settings. Indicates the realm that is sent to the client when basic HTTP authentication to Basic Auth None the virtual server fails. From the server perspective, the realm allows a Realm protected resource to be partitioned into a set of protection spaces. The BIG-IP system can redirect a request to a fallback host when all members of the targeted pool are unavailable, or if a selected pool member is unavailable (for example, the pool member is disabled, marked as down, or has exceeded Fallback Host None its connection limit). When one or more pool members are unavailable, the BIG- IP system can redirect the HTTP request to the specified fallback host, with the HTTP reply Status Code 302 Found. Indicates the HTTP error codes from server responses that should trigger a Fallback on redirection to the fallback host. If you are specifying more than one code, None Error Codes separate the codes with a blank space, such as 500 501 502. You can also specify a range of error codes, as in this example: 505-515. Indicates the name of an HTTP request header that the BIG-IP system removes from the client request. Only one header can be removed per pool, unless you Request are using LTM Policies or iRules. None Header Erase Note: The Header Erase option works only on headers in HTTP requests from clients to servers.

The Request Header Insert is a string that the system inserts as a header in an Request HTTP request. If the header exists already, the system does not replace it. For None Header Insert multiple header insertions, both LTM Policies and iRules support inserting multiple headers into a request Response Can be any headers within an HTTP response that the system is to allow. If you Headers None are specifying more than one header, separate the headers with a blank space. Allowed For example, if you enter the string Content-Type Set-Cookie Location, the system then allows the headers Content-Type, Set-Cookie, and Location. BIG-IP 15.0.0 and later The behavior in each mode depends on whether the client sends chunked or unchunked requests and replaces the previous options of Preserve and Selective. The options are defined in Response Chunking. Note: The Sustain option is the default, starting with the release of BIG-IP 15.0.0, replacing the earlier release versions' use of Preserve.

BIG-IP prior to 15.0.0 Preserve - The chunked transfer encoding method modifies the body of an HTTP message and transfers it as a series of chunks. The Request Chunking setting indicates how the BIG-IP system handles HTTP content that uses chunked encoding in the client request. The behavior in each mode depends on whether the client sends chunked or unchunked requests.

For content that is chunked:

Preserve: The BIG-IP system processes the chunked content and sends the request to the server unchanged. Selective: The BIG-IP system unchunks the HTTP content, processes the Sustain data, re-adds the chunk headers, and then sends the chunked request or Request response to the server. Note that for chunked content, this mode is the Chunking same as the Rechunk mode. Unchunk: For chunked content, specifies that the system unchunks the response, processes the HTTP content, and passes the response on as unchunked. The Keep-Alive value for the Connection header is not supported, so the system sets the value of the header to close. If the response is unchunked, the system processes the HTTP content and passes the response on untouched. Rechunk: The BIG-IP system unchunks the HTTP content, processes the data, re-adds the chunk headers, and then sends the chunked request or response to the server.

For content that is unchunked:

Preserve: The BIG-IP system processes the HTTP content and sends the request to the server unchanged. Selective: The BIG-IP system processes the HTTP content and sends the request to the server unchanged. Rechunk: The BIG-IP system processes the HTTP content, adds the transfer encoding and chunk headers to the response, and then sends the chunked request to the server. BIG-IP 15.0.0 and later Specifies that the system preserves request or response chunking unless there is a command to modify the body. If the request or response is chunked, unchunk the HTTP content, process the data, and re-add chunking headers on egress. Chunk extensions will be lost. When the response is chunked, it can be rechunked on egress to the client. Note: The Sustain option is the default, starting with the release of BIG-IP 15.0.0, replacing the earlier release versions' use of Selective.

Unchunk: For chunked content, specifies that the system unchunks the response, processes the HTTP content, and passes the response on as unchunked. The Keep-Alive value for the Connection header is not supported, so the system sets the value of the header to close. If the response is unchunked, the system processes the HTTP content and passes the response on untouched.

Rechunk: Specifies that the system unchunks the request or response, processes the HTTP content, re-adds the chunk trailer headers, and then passes on the request or response as chunked. Any chunk extensions are lost. If the request or response is unchunked, the system adds transfer encoding and chunking headers on egress.

Sustain: Specifies that the system preserves request or response chunking unless there is a command to modify the body. If the request or response is chunked, unchunk the HTTP content, process the data, and re-add chunking headers on egress. Chunk extensions will be lost. When the response is chunked, it can be rechunked on egress to the client.

BIG-IP prior to 15.0.0 Selective -The chunked transfer encoding modifies the body of an HTTP Sustain message and transfers it as a series of chunks. The Response Chunking Response setting specifies how the BIG-IP system handles HTTP content that is chunked Chunking by the server. The behavior in each mode depends on whether the server sends chunked or unchunked responses.

For content that is chunked:

Preserve: The BIG-IP system processes the chunked content and sends the response to the client unchanged. Selective: The BIG-IP system unchunks the HTTP content, processes the data, re-adds the chunk headers, and then sends the chunked request or response to the client. Note that for chunked content, this mode is the same as the Rechunk mode. Unchunk: The BIG-IP system removes the HTTP transfer encoding headers, removes the chunk headers, processes the HTTP content, and then sends the unchunked response to the client. The system closes the connection once it sends all data Rechunk: The BIG-IP system unchunks the HTTP content, processes the data, re-adds the chunk headers, and then sends the chunked request or response to the client.

For content that is unchunked:

Preserve: The BIG-IP system processes the HTTP content and sends the response to the client unchanged. Selective: The BIG-IP system processes the HTTP content and sends the response to the client unchanged. Unchunk: The BIG-IP system processes the HTTP content and sends the response to the client unchanged. Rechunk: The BIG-IP system processes the HTTP content, adds the transfer encoding and chunk headers to the response, and then sends the chunked response to the client.

When enabled the BIG-IP system performs HTTP header transformations for the purpose of allowing HTTP/1.0 connections to be transformed into HTTP/1.1 requests on the server side of the connection. This allows those connections to remain open for reuse when they would not otherwise be. When the OneConnect OneConnect Transformations setting is enabled in the HTTP profile, the BIG-IP Enabled Transformations system transforms Connection: close headers in HTTP/1.0 client-side requests to X-Cnection: close headers on the server side. This allows the BIG-IP system to make client requests containing the Connection: close header such as HTTP /1.0 requests, eligible for connection reuse. This setting is applicable only when you configure a OneConnect profile for a virtual server. A client request may be redirected from the HTTPS protocol to the HTTP protocol, which is a non-secure channel. If you want to ensure that the request remains on a secure channel, you can configure the BIG-IP system to rewrite the redirection so that it is redirected back to the HTTPS protocol. To enable the BIG-IP system to rewrite HTTP redirections, you use the Rewrite Redirections setting to specify the way that you want the system to handle URIs during the rewrite. The following options are available: Redirect None None: Specifies that the system does not rewrite the URI in any HTTP Rewrite redirect responses. All: Specifies that the system rewrites the URI in all HTTP redirect responses. Matching: Specifies that the system rewrites the URI in any HTTP redirect responses that match the request URI. Nodes: Specifies that if the URI contains a node IP address instead of a host name, the system changes it to the virtual server address. You can configure the HTTP profile so that the BIG-IP system encrypts HTTP cookies before sending them to the client system. The system can encrypt BIG- IP persistence cookies, as well as cookies that are embed in the response from the server. You can also configure the BIG-IP system to encrypt cookies to keep information private if the cookie contains sensitive information about the web application. When cookie encryption is enabled, the BIG-IP system Encrypt None extracts the unencrypted cookie from the server response, encrypts it using a Cookies 192-bit AES cipher, and then encodes it using the Base64 encoding scheme. The BIG-IP LTM system then embeds the encrypted cookie into the HTTP response to the client. On subsequent requests, when the client presents the encrypted cookie to the BIG-IP system, the system removes the cookie, decodes it using the Base64 encoding scheme, and decrypts it. The BIG-IP system then re-embeds the decrypted cookie in the HTTP request to the server. Cookie Encryption None Enter the passphrase for cookie encryption. Passphrase Confirm Cookie Re-enter the passphrase that you specified in the Cookie Encryption Encryption None Passphrase box. Passphrase When using connection pooling, which allows clients to make use of existing server-side connections, you can insert the X-Forwarded For header with the Insert X- Disabled client IP address into a request. When you configure the BIG-IP system to Forwarded-For insert this header, the target server can identify the request as coming from a client other than the client that initiated the connection. LWS Maximum Specifies the maximum column width for any given line, when inserting an 80 Columns HTTP header in an HTTP request. Specifies the linear white space (LWS) separator that the system inserts when LWS Separator \r\n a header exceeds the maximum width you specify in the LWS Maximum Columns setting. Specifies the number of HTTP requests that the system accepts on a per- Maximum 0 connection basis. By default, the system does not limit the number of requests Requests per connection. Send Proxy Via Specifies whether to Remove, Preserve, or Append Via headers included in a Header in Preserve client request to an origin web server. Request Send Proxy Via Specifies whether to Remove, Preserve, or Append Via headers included in an Header in Preserve origin web server response to a client. Response Enables or disables trusting the client IP, and statistics from the client IP address, based on the request's X-Forwarded-For (XFF) headers, if they exist.

Note: This option has an effect only when you use either AVR or ASM L7 DoS profile (ASM required). For AVR, the Accept XFF option allows the BIG-IP Accept XFF Disabled system to trust and take into consideration IP addresses from the X-Forwarded- For header for statistics purposes. For an L7 DoS profile, the Accept XFF option allows the BIG-IP system to take action based on IP addresses from the X-Forwarded-For header that match, for example, an Access List.

Specifies alternative XFF headers that are trusted, instead of the default X- Forwarded-For header; this does not specify the name of the inserted XFF XFF Alternative None header. If you are specifying more than one alternative XFF header, separate Names the alternative XFF headers with a blank space, such as client1 proxyserver 10.10.10.10. Specifies the string used as the server name in traffic generated by the BIG-IP Server Agent BigIP system. If this is set to an empty string, then no Server header is inserted in Name BIG-IP responses.

Enforcement

Note: Settings and Values within Enforcement change and/or made available based on the Proxy Mode chosen. The enforcement section is only available in BIG-IP 11.5.0 and later.

Setting Default Description BIG-IP LTM performs basic RFC compliance checks as described in the latest RFC for the HTTP protocol. If a client request fails these checks, then the Enforce connection is reset. RFC Disabled If you set Enforce RFC Compliance to Enabled for an HTTP profile that is Compliance assigned to a virtual server, the BIG-IP system attempts to reject non-RFC- compliant HTTP traffic. Note: This setting is available in BIG-IP 15.0.0 and later. Proxy Mode (Reverse): Disabled Allow Proxy Mode Specifies the pass-through behavior when a redirect lacking the trailing Truncated (Explicit): carriage-return and line feed pair at the end of the headers is parsed. By Redirect Disabled default, the system silently drops the invalid HTTP. Proxy Mode (Transparent): Enabled Proxy Mode (Reverse): 32,768 bytes Specifies the maximum size in bytes the system allows for all HTTP request Maximum Proxy Mode or response headers combined. (For an HTTP request, it includes the request Header (Explicit): line.) If the combined headers length in bytes in an HTTP request or response Size 32,768 bytes exceeds this value, the system stops parsing the headers and resets the TCP Proxy Mode connection. (Transparent): 16384 bytes Oversize Specifies the pass-through behavior when the Maximum Header Size value is Client Reject exceeded by the client. This setting is only available when the Proxy Mode is Headers set to Transparent. Oversize Specifies the pass-through behavior when the Maximum Header Size value is Server Reject exceeded by the server. This setting is only available when the Proxy Mode is Headers set to Transparent. Proxy Mode (Reverse): 64 headers Maximum Proxy Mode Header (Explicit): 64 Specifies the maximum number of headers the system supports. Count headers Proxy Mode (Transparent): 32 headers Excess Specifies the pass-through behavior when Maximum Header Count value is Client Pass Through exceeded by the client. This setting is only available when the Proxy Mode is Headers set to Transparent. Excess Specifies the pass-through behavior when Maximum Header Count value is Server Pass Through exceeded by the server. This setting is only available when the Proxy Mode is Headers set to Transparent. Enables or disables HTTP/1.1 pipelining. If Pass Through is selected, then the HTTP filter will switch to pass through mode and be disabled if pipelined data Pipeline is seen. By default, the profile allows pipelining and clients can make HTTP Allow Action requests even when prior requests have not received a response. In order for this to succeed, however, destination servers must include support for pipelining. Unknown Specifies the behavior (allow, reject, or pass through) when an unknown Allow Method HTTP method is parsed. CONNECT DELETE GET HEAD LOCK Optimizes the behavior of a known HTTP method, specified in the Enabled Known OPTIONS Methods list. If you delete a known method from the Enabled Methods list, the Method POST BIG-IP system applies the Unknown Method setting to manage that traffic. PROPFIND PUT TRACE UNLOCK

Explicit Proxy (Available when Proxy Mode is set to Explicit)

Setting Default Description DNS Specifies the DNS Resolver to use for DNS inquiries handled by the virtual servers Resolver None associated with this profile. Specifies the relative order of IPv4 and IPv6 DNS resolution for requested URIs:

Disabled: IPv4 lookup occurs before IPv6 IPv6 Disabled Enabled: IPv6 lookup occurs before IPv4

Note: This setting is available in BIG-IP 13.1.0 and later.

Route 0 Specifies the route domain that is used for outbound connect requests. Domain Tunnel http- Specifies the tunnel that is used for outbound connect requests, enabling other Name tunnel virtual servers to receive connections initiated by the proxy service. Specifies the name of hosts that should not be proxied. Host None Names Note: This setting does not work for HTTPS requests because they are encrypted.

Specifies the behavior of the proxy service when handling outbound requests. The available options are the following:

Allow: Indicates that outbound requests are delivered directly, regardless of Default the presence of listening virtual servers. Connect Deny Deny: Indicates that outbound requests are delivered only if another virtual Handling server is listening on the tunnel for the requested outbound connection. With this setting, virtual servers are required, and the system processes the outbound traffic before it leaves the device.

Connection Specifies the message that appears when a connection failure occurs. You can Failed None include TCL expressions. Message DNS Lookup Specifies the message that appears when a DNS lookup failure occurs. You can None Failed include TCL expressions. Message Bad Specifies the message that appears when a bad request occurs. You can include Request None TCL expressions. Message Bad Specifies the message that appears when a bad response occurs. You can include Response None TCL expressions. Message sFlow

Setting Default Description Specifies the maximum interval in seconds between two pollings. The default value, Polling Default 'Default' represents the value set on the System :: sFlow :: Global Settings :: http :: Interval Properties screen. The initial default is 10 seconds. Specifies the ratio of packets observed to the samples generated. For example, a sampling rate of 2000 specifies that the system randomly generates 1 sample for every Sampling Default 2000 packets observed. The default is Default, which represents the value set on the Rate System :: sFlow :: Global Settings :: http :: Properties screen. The initial default is 1024 packets.

HTTP Strict Transport Security

Note: The HTTP Strict Transport Security (HSTS) settings are only available in BIG-IP 12.0.0 and later. The Preload option is only available starting in BIG-IP 13.0.0 or later.

Setting Default Description Mode Disabled When selected (enabled), enables the HSTS settings. Specifies the maximum length of time, in seconds, that HSTS functionality Maximum 16070400 requests that clients only use HTTPS to connect to the current host and any Age sec subdomains of the current host's domain name. A value of 0 re-enables plaintext HTTP access. Include When selected (enabled), applies the HSTS policy to the HSTS host and its Enabled Subdomains subdomains. When selected (enabled), adds the HSTS host and its subdomains to the Preload Disabled browser's HSTS preload list of sites that are considered HTTPS only. The default is disabled.

Recommendations

When you configure profiles for HTTP traffic, consider the following:

Assess the needs of each HTTP virtual server individually and choose the HTTP profile when you need the virtual server to operate in full L7 inspection mode.

Supplemental Information

K4707: Choosing appropriate profiles for HTTP traffic K74767112: Overview of the TCP profile (15.x) K29377715: Overview of the TCP profile (14.x) K10711911: Overview of the TCP profile (13.x) K70025261: Overview of the TCP profile (12.x) K13924148: Overview of the TCP profile (11.x) K14903: Overview of the Web Acceleration profile K14784: Configuring cookie encryption within the HTTP profile K4816: Using the X-Forwarded-For HTTP header to preserve the original client IP address for traffic translated by a SNAT

Applies to: Product: BIG-IP, BIG-IP AFM, BIG-IP APM, BIG-IP ASM, BIG-IP LTM, BIG-IP AAM 16.X.X, 15.X.X, 14.X.X, 13.X.X, 12.X.X, 11.X.X