No-Vary-Search

No-Vary-Search Google LLC

d@domenic.me

Google LLC

jbroman@chromium.org

Applications HyperText Transfer Protocol http caching A proposed HTTP header field for changing how URL search parameters impact caching. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the HTTP Working Group mailing list (), which is archived at . Working Group information can be found at . Source for this draft and an issue tracker can be found at .

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document also adopts some conventions and notation typical in WHATWG and W3C usage, especially as it relates to algorithms. See .

HTTP header field definition The No-Vary-Search HTTP header field is a structured field whose value must be a dictionary (). It has the following authoring conformance requirements:

If present, the key-order entry's value must be a boolean ().
If present, the params entry's value must be either a boolean () or an inner list ().
If present, the except entry's value must be an inner list ().
The except entry must only be present if the params entry is also present, and the params entry's value is the boolean value true.

The dictionary may contain entries whose keys are not one of key-order, params, and except, but their meaning is not defined by this specification. Implementations of this specification will ignore such entries (but future documents may assign meaning to such entries).

Data model A URL search variance consists of the following:

no-vary params: either the special value wildcard or a list of strings
vary params: either the special value wildcard or a list of strings
vary on key order: a boolean

The default URL search variance is a URL search variance whose no-vary params is an empty list, vary params is wildcard, and vary on key order is true. The obtain a URL search variance algorithm () ensures that all URL search variances obey the following constraints:

vary params is a list if and only if the no-vary params is wildcard; and
no-vary params is a list if and only if the vary params is wildcard.

Parsing

Parse a URL search variance To parse a URL search variance given value:

If value is null, then return the default URL search variance.
Let result be a new URL search variance.
Set result's vary on key order to true.
If value["key-order"] exists:
1. If value["key-order"] is not a boolean, then return the default URL search variance.
2. Set result's vary on key order to the boolean negation of value["key-order"].
If value["params"] exists:
1. If value["params"] is a boolean:
  1. If value["params"] is true, then:
    1. Set result's no-vary params to wildcard.
    2. Set result's vary params to the empty list.
  2. Otherwise:
    1. Set result's no-vary params to the empty list.
    2. Set result's vary params to wildcard.
2. Otherwise, if value["params"] is an array:
  1. If any item in value["params"] is not a string, then return the default URL search variance.
  2. Set result's no-vary params to the result of applying parse a key () to each item in value["params"].
  3. Set result's vary params to wildcard.
3. Otherwise, return the default URL search variance.
If value["except"] exists:
1. If value["params"] is not true, then return the default URL search variance.
2. If value["except"] is not an array, then return the default URL search variance.
3. If any item in value["except"] is not a string, then return the default URL search variance.
4. Set result's vary params to the result of applying parse a key () to each item in value["except"].
Return result.

Obtain a URL search variance To obtain a URL search variance given a response response:

Let fieldValue be the result of getting a structured field value given `No-Vary-Search` and "dictionary" from response's header list.
Return the result of parsing a URL search variance () given fieldValue.

Examples The following illustrates how various inputs are parsed, in terms of their impacting on the resulting no-vary params and vary params:

Input	Result
`No-Vary-Search: params`	no-vary params: wildcard vary params: (empty list)
`No-Vary-Search: params=("a")`	no-vary params: « "`a`" » vary params: wildcard
`No-Vary-Search: params, except=("x")`	no-vary params: wildcard vary params: « "`x`" »

The following inputs are all invalid and will cause the default URL search variance to be returned:

No-Vary-Search: unknown-key
No-Vary-Search: key-order="not a boolean"
No-Vary-Search: params="not a boolean or inner list"
No-Vary-Search: params=(not-a-string)
No-Vary-Search: params=("a"), except=("x")
No-Vary-Search: params=(), except=()
No-Vary-Search: params=?0, except=("x")
No-Vary-Search: params, except=(not-a-string)
No-Vary-Search: params, except="not an inner list"
No-Vary-Search: params, except=?1
No-Vary-Search: except=("x")
No-Vary-Search: except=()

The following inputs are valid, but somewhat unconventional. They are shown alongside their more conventional form.

Input	Conventional form
`No-Vary-Search: params=?1`	`No-Vary-Search: params`
`No-Vary-Search: key-order=?1`	`No-Vary-Search: key-order`
`No-Vary-Search: params, key-order, except=("x")`	`No-Vary-Search: key-order, params, except=("x")`
`No-Vary-Search: params=?0`	(omit the header)
`No-Vary-Search: params=()`	(omit the header)
`No-Vary-Search: key-order=?0`	(omit the header)

Parse a key To parse a key given an ASCII string keyString:

Let keyBytes be the isomorphic encoding of keyString.
Replace any 0x2B (+) in keyBytes with 0x20 (SP).
Let keyBytesDecoded be the percent-decoding of keyBytes.
Let keyStringDecoded be the UTF-8 decoding without BOM of keyBytesDecoded.
Return keyStringDecoded.

Examples The parse a key algorithm allows encoding non-ASCII key strings in the ASCII structured header format, similar to how the application/x-www-form-urlencoded format allows encoding an entire entry list of keys and values in ASCII URL format. For example, will result in a URL search variance whose vary params are « "é 気" ». As explained in a later example, the canonicalization process during equivalence testing means this will treat as equivalent URL strings such as:

https://example.com/?é 気=1
https://example.com/?é+気=2
https://example.com/?%C3%A9%20気=3
https://example.com/?%C3%A9+%E6%B0%97=4

and so on, since they all are parsed to having the same key "é 気".

Comparing Two URLs urlA and urlB are equivalent modulo search variance given a URL search variance searchVariance if the following algorithm returns true:

If the scheme, username, password, host, port, or path of urlA and urlB differ, then return false.
If searchVariance is equivalent to the default URL search variance, then:
1. If urlA's query equals urlB's query, then return true.
2. Return false.
In this case, even URL pairs that might appear the same after running the application/x-www-form-urlencoded parser on their queries, such as https://example.com/a and https://example.com/a?, or https://example.com/foo?a=b&&&c and https://example.com/foo?a=b&c=, will be treated as inequivalent.
Let searchParamsA and searchParamsB be empty lists.
If wrlA's query is not null, then set searchParamsA to the result of running the application/x-www-form-urlencoded parser given the isomorphic encoding of urlA's query.
If wrlB's query is not null, then set searchParamsB to the result of running the application/x-www-form-urlencoded parser given the isomorphic encoding of urlB's query.
If searchVariance's no-vary params is a list, then:
1. Set searchParamsA to a list containing those items pair in searchParamsA where searchVariance's no-vary params does not contain pair[0].
2. Set searchParamsB to a list containing those items pair in searchParamsB where searchVariance's no-vary params does not contain pair[0].
Otherwise, if searchVariance's vary params is a list, then:
1. Set searchParamsA to a list containing those items pair in searchParamsA where searchVariance's vary params contains pair[0].
2. Set searchParamsB to a list containing those items pair in searchParamsB where searchVariance's vary params contains pair[0].
If searchVariance's vary on key order is false, then:
1. Let keyLessThan be an algorithm taking as inputs two pairs (keyA, valueA) and (keyB, valueB), which returns whether keyA is code unit less than keyB.
2. Set searchParamsA to the result of sorting searchParamsA in ascending order with keyLessThan.
3. Set searchParamsB to the result of sorting searchParamsB in ascending order with keyLessThan.
If searchParamsA's size is not equal to searchParamsB's size, then return false.
Let i be 0.
While i < searchParamsA's size:
1. If searchParamsA[i][0] does not equal searchParamsB[i][0], then return false.
2. If searchParamsA[i][1] does not equal searchParamsB[i][1], then return false.
3. Set i to i + 1.
Return true.

Examples Due to how the application/x-www-form-urlencoded parser canonicalizes query strings, there are some cases where query strings which do not appear obviously equivalent, will end up being treated as equivalent after parsing. So, for example, given any non-default value for No-Vary-Search, such as No-Vary-Search: key-order, we will have the following equivalences:

https://example.com https://example.com/?: A null query is parsed the same as an empty string
https://example.com/?a=x https://example.com/?%61=%78: Parsing performs percent-decoding
https://example.com/?a=é https://example.com/?a=%C3%A9: Parsing performs percent-decoding
https://example.com/?a=%f6 https://example.com/?a=%ef%bf%bd: Both values are parsed as U+FFFD (�)
https://example.com/?a=x&&&& https://example.com/?a=x: Parsing splits on & and discards empty strings
https://example.com/?a= https://example.com/?a: Both parse as having an empty string value for a
https://example.com/?a=%20 https://example.com/?a=+ https://example.com/?a= &: + and %20 are both parsed as U+0020 SPACE

Security Considerations The main risk to be aware of is the impact of mismatched URLs. In particular, this could cause the user to see a response that was originally fetched from a URL different from the one displayed when they hovered a link, or the URL displayed in the URL bar. However, since the impact is limited to query parameters, this does not cross the relevant security boundary, which is the origin . (Or perhaps just the host, from the perspective of web browser security UI. ) Indeed, we have already given origins complete control over how they present the (URL, reponse body) pair, including on the client side via technology such as history.replaceState() or service workers.

Privacy Considerations This proposal is adjacent to the highly-privacy-relevant space of navigational tracking, which often uses query parameters to pass along user identifiers. However, we believe this proposal itself does not have privacy impacts. It does not interfere with existing navigational tracking mitigations, or any known future ones being contemplated. Indeed, if a page were to encode user identifiers in its URL, the only ability this proposal gives is to reduce such user tracking by preventing server processing of such user IDs (since the server is bypassed in favor of the cache).

IANA Considerations TODO IANA

References Normative References Fetch Living Standard Apple Inc. n.d. WHATWG Structured Field Values for HTTP This document describes a set of data types and associated algorithms that are intended to make it easier and safer to define and handle HTTP header and trailer fields, known as "Structured Fields", "Structured Headers", or "Structured Trailers". It is intended for use by specifications of new HTTP fields that wish to use a common syntax that is more restrictive than traditional HTTP field values. Encoding Living Standard Apple Inc. n.d. WHATWG Infra Living Standard Apple Inc. Google LLC n.d. WHATWG URL Living Standard Apple Inc. n.d. WHATWG Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References HTML Living Standard Apple Inc. n.d. WHATWG Navigational-Tracking Mitigations Brave Software, Inc. Google LLC n.d. W3C Privacy CG

Acknowledgments TODO acknowledge.