Package com.squareup.okhttp

Class HttpUrl

  • public final class HttpUrl
extends Object
    A uniform resource locator (URL) with a scheme of either http or https. Use this class to compose and decompose Internet addresses. For example, this code will compose and print a URL for Google search: 
       

   HttpUrl url = new HttpUrl.Builder()
       .scheme("https")
       .host("www.google.com")
       .addPathSegment("search")
       .addQueryParameter("q", "polar bears")
       .build();
   System.out.println(url);
    which prints: 
       

     https://www.google.com/search?q=polar%20bears
    As another example, this code prints the human-readable query parameters of a Twitter search: 
       

   HttpUrl url = HttpUrl.parse("https://twitter.com/search?q=cute%20%23puppies&f=images");
   for (int i = 0, size = url.querySize(); i < size; i++) {
     System.out.println(url.queryParameterName(i) + ": " + url.queryParameterValue(i));
   }
    which prints: 
       

   q: cute #puppies
   f: images
    In addition to composing URLs from their component parts and decomposing URLs into their component parts, this class implements relative URL resolution: what address you'd reach by clicking a relative link on a specified page. For example: 
       

   HttpUrl base = HttpUrl.parse("https://www.youtube.com/user/WatchTheDaily/videos");
   HttpUrl link = base.resolve("../../watch?v=cbP2N1BQdYc");
   System.out.println(link);
    which prints: 
       

   https://www.youtube.com/watch?v=cbP2N1BQdYc

    What's in a URL?

    A URL has several components.

    Scheme

    Sometimes referred to as protocol, A URL's scheme describes what mechanism should be used to retrieve the resource. Although URLs have many schemes (mailto, file, ftp), this class only supports http and https. Use java.net.URI for URLs with arbitrary schemes.

    Username and Password

    Username and password are either present, or the empty string "" if absent. This class offers no mechanism to differentiate empty from absent. Neither of these components are popular in practice. Typically HTTP applications use other mechanisms for user identification and authentication.

    Host

    The host identifies the webserver that serves the URL's resource. It is either a hostname like square.com or localhost, an IPv4 address like 192.168.0.1, or an IPv6 address like ::1.

    Usually a webserver is reachable with multiple identifiers: its IP addresses, registered domain names, and even localhost when connecting from the server itself. Each of a webserver's names is a distinct URL and they are not interchangeable. For example, even if http://square.github.io/dagger and http://google.github.io/dagger are served by the same IP address, the two URLs identify different resources.

    Port

    The port used to connect to the webserver. By default this is 80 for HTTP and 443 for HTTPS. This class never returns -1 for the port: if no port is explicitly specified in the URL then the scheme's default is used.

    Path

    The path identifies a specific resource on the host. Paths have a hierarchical structure like "/square/okhttp/issues/1486". Each path segment is prefixed with "/". This class offers methods to compose and decompose paths by segment. If a path's last segment is the empty string, then the path ends with "/". This class always builds non-empty paths: if the path is omitted it defaults to "/", which is a path whose only segment is the empty string.

    Query

    The query is optional: it can be null, empty, or non-empty. For many HTTP URLs the query string is subdivided into a collection of name-value parameters. This class offers methods to set the query as the single string, or as individual name-value parameters. With name-value parameters the values are optional and names may be repeated.

    Fragment

    The fragment is optional: it can be null, empty, or non-empty. Unlike host, port, path, and query the fragment is not sent to the webserver: it's private to the client.

    Encoding

    Each component must be encoded before it is embedded in the complete URL. As we saw above, the string cute #puppies is encoded as cute%20%23puppies when used as a query parameter value.

    Percent encoding

    Percent encoding replaces a character (like 🍩) with its UTF-8 hex bytes (like %F0%9F%8D%A9). This approach works for whitespace characters, control characters, non-ASCII characters, and characters that already have another meaning in a particular context.

    Percent encoding is used in every URL component except for the hostname. But the set of characters that need to be encoded is different for each component. For example, the path component must escape all of its ? characters, otherwise it could be interpreted as the start of the URL's query. But within the query and fragment components, the ? character doesn't delimit anything and doesn't need to be escaped. 

       

   HttpUrl url = HttpUrl.parse("http://who-let-the-dogs.out").newBuilder()
       .addPathSegment("_Who?_")
       .query("_Who?_")
       .fragment("_Who?_")
       .build();
   System.out.println(url);
    This prints: 
       

   http://who-let-the-dogs.out/_Who%3F_?_Who?_#_Who?_
    When parsing URLs that lack percent encoding where it is required, this class will percent encode the offending characters.

    IDNA Mapping and Punycode encoding

    Hostnames have different requirements and use a different encoding scheme. It consists of IDNA mapping and Punycode encoding.

    In order to avoid confusion and discourage phishing attacks, IDNA Mapping transforms names to avoid confusing characters. This includes basic case folding: transforming shouting SQUARE.COM into cool and casual square.com. It also handles more exotic characters. For example, the Unicode trademark sign (™) could be confused for the letters "TM" in http://ho™mail.com. To mitigate this, the single character (™) maps to the string (tm). There is similar policy for all of the 1.1 million Unicode code points. Note that some code points such as "🍩" are not mapped and cannot be used in a hostname.

    Punycode converts a Unicode string to an ASCII string to make international domain names work everywhere. For example, "σ" encodes as "xn--4xa". The encoded string is not human readable, but can be used with classes like InetAddress to establish connections.

    Why another URL model?

    Java includes both java.net.URL and java.net.URI. We offer a new URL model to address problems that the others don't.

    Different URLs should be different

    Although they have different content, java.net.URL considers the following two URLs equal, and the equals() method between them returns true:
    • http://square.github.io/
    • http://google.github.io/
    This is because those two hosts share the same IP address. This is an old, bad design decision that makes java.net.URL unusable for many things. It shouldn't be used as a Map key or in a Set. Doing so is both inefficient because equality may require a DNS lookup, and incorrect because unequal URLs may be equal because of how they are hosted.

    Equal URLs should be equal

    These two URLs are semantically identical, but java.net.URI disagrees:
    • http://host:80/
    • http://host
    Both the unnecessary port specification (:80) and the absent trailing slash (/) cause URI to bucket the two URLs separately. This harms URI's usefulness in collections. Any application that stores information-per-URL will need to either canonicalize manually, or suffer unnecessary redundancy for such URLs.

    Because they don't attempt canonical form, these classes are surprisingly difficult to use securely. Suppose you're building a webservice that checks that incoming paths are prefixed "/static/images/" before serving the corresponding assets from the filesystem. 

       

   String attack = "http://example.com/static/images/../../../../../etc/passwd";
   System.out.println(new URL(attack).getPath());
   System.out.println(new URI(attack).getPath());
   System.out.println(HttpUrl.parse(attack).path());
    By canonicalizing the input paths, they are complicit in directory traversal attacks. Code that checks only the path prefix may suffer! 
       

    /static/images/../../../../../etc/passwd
    /static/images/../../../../../etc/passwd
    /etc/passwd

    If it works on the web, it should work in your application

    The java.net.URI class is strict around what URLs it accepts. It rejects URLs like "http://example.com/abc|def" because the '|' character is unsupported. This class is more forgiving: it will automatically percent-encode the '|', yielding "http://example.com/abc%7Cdef". This kind behavior is consistent with web browsers. HttpUrl prefers consistency with major web browsers over consistency with obsolete specifications.

    Paths and Queries should decompose

    Neither of the built-in URL models offer direct access to path segments or query parameters. Manually using StringBuilder to assemble these components is cumbersome: do '+' characters get silently replaced with spaces? If a query parameter contains a '&', does that get escaped? By offering methods to read and write individual query parameters directly, application developers are saved from the hassles of encoding and decoding.

    Plus a modern API

    The URL (JDK1.0) and URI (Java 1.4) classes predate builders and instead use telescoping constructors. For example, there's no API to compose a URI with a custom port without also providing a query and fragment.

    Instances of HttpUrl are well-formed and always have a scheme, host, and path. With java.net.URL it's possible to create an awkward URL like http:/ with scheme and path but no hostname. Building APIs that consume such malformed values is difficult!

    This class has a modern API. It avoids punitive checked exceptions: parse() returns null if the input is an invalid URL. You can even be explicit about whether each component has been encoded already.

    • Method Detail

      • uri

        public URI uri()
        Attempt to convert this URL to a java.net.URI. This method throws an unchecked IllegalStateException if the URL it holds isn't valid by URI's overly-stringent standard. For example, URI rejects paths containing the '[' character. Consult that class for the exact rules of what URLs are permitted.

      • scheme

        public String scheme()
        Returns either "http" or "https".

      • isHttps

        public boolean isHttps()

      • encodedUsername

        public String encodedUsername()
        Returns the username, or an empty string if none is set.

      • username

        public String username()

      • encodedPassword

        public String encodedPassword()
        Returns the password, or an empty string if none is set.

      • password

        public String password()
        Returns the decoded password, or an empty string if none is present.

      • host

        public String host()
        Returns the host address suitable for use with InetAddress.getAllByName(String). May be:
        • A regular host name, like android.com.
        • An IPv4 address, like 127.0.0.1.
        • An IPv6 address, like ::1. Note that there are no square braces.
        • An encoded IDN, like xn--n3h.net.

      • port

        public int port()
        Returns the explicitly-specified port if one was provided, or the default port for this URL's scheme. For example, this returns 8443 for https://square.com:8443/ and 443 for https://square.com/. The result is in [1..65535].

      • defaultPort

        public static int defaultPort​(String scheme)
        Returns 80 if scheme.equals("http"), 443 if scheme.equals("https") and -1 otherwise.

      • pathSize

        public int pathSize()

      • encodedPath

        public String encodedPath()
        Returns the entire path of this URL, encoded for use in HTTP resource resolution. The returned path is always nonempty and is prefixed with /.

      • encodedPathSegments

        public List<String> encodedPathSegments()

      • pathSegments

        public List<String> pathSegments()

      • encodedQuery

        public String encodedQuery()
        Returns the query of this URL, encoded for use in HTTP resource resolution. The returned string may be null (for URLs with no query), empty (for URLs with an empty query) or non-empty (all other URLs).

      • query

        public String query()

      • querySize

        public int querySize()

      • queryParameter

        public String queryParameter​(String name)
        Returns the first query parameter named name decoded using UTF-8, or null if there is no such query parameter.

      • queryParameterNames

        public Set<String> queryParameterNames()

      • queryParameterValues

        public List<String> queryParameterValues​(String name)

      • queryParameterName

        public String queryParameterName​(int index)

      • queryParameterValue

        public String queryParameterValue​(int index)

      • encodedFragment

        public String encodedFragment()

      • fragment

        public String fragment()

      • resolve

        public HttpUrl resolve​(String link)
        Returns the URL that would be retrieved by following link from this URL.

      • parse

        public static HttpUrl parse​(String url)
        Returns a new HttpUrl representing url if it is a well-formed HTTP or HTTPS URL, or null if it isn't.

      • get

        public static HttpUrl get​(URL url)
        Returns an HttpUrl for url if its protocol is http or https, or null if it has any other protocol.

      • equals

        public boolean equals​(Object o)
        Description copied from class: Object
        Compares this instance with the specified object and indicates if they are equal. In order to be equal, o must represent the same object as this instance using a class-specific comparison. The general contract is that this comparison should be reflexive, symmetric, and transitive. Also, no object reference other than null is equal to null.

        The default implementation returns true only if this == o. See Writing a correct equals method if you intend implementing your own equals method.

        The general contract for the equals and Object.hashCode() methods is that if equals returns true for any two objects, then hashCode() must return the same value for these objects. This means that subclasses of Object usually override either both methods or neither of them.

        Overrides:
        equals in class Object
        Parameters:
        o - the object to compare this instance with.
        Returns:
        true if the specified object is equal to this Object; false otherwise.
        See Also:
        Object.hashCode()

      • hashCode

        public int hashCode()
        Description copied from class: Object
        Returns an integer hash code for this object. By contract, any two objects for which Object.equals(java.lang.Object) returns true must return the same hash code value. This means that subclasses of Object usually override both methods or neither method.

        Note that hash values must not change over time unless information used in equals comparisons also changes.

        See Writing a correct hashCode method if you intend implementing your own hashCode method.

        Overrides:
        hashCode in class Object
        Returns:
        this object's hash code.
        See Also:
        Object.equals(java.lang.Object)

      • toString

        public String toString()
        Description copied from class: Object
        Returns a string containing a concise, human-readable description of this object. Subclasses are encouraged to override this method and provide an implementation that takes into account the object's type and data. The default implementation is equivalent to the following expression: 
           getClass().getName() + '@' + Integer.toHexString(hashCode())

        See Writing a useful toString method if you intend implementing your own toString method.

        Overrides:
        toString in class Object
        Returns:
        a printable representation of this object.