3 ��yiGC�@s ddlmZddlZddlZddlZddlmZmZddlm Z m Z ddlmZddlm Z mZmZddlmZdd lmZdd lmZddlmZddlmZd ddgZeje�Zd6Zd7Zej d/e�Z!d0d1�Z"ej#e"e!�ej#e"e!�d2�Z$e e d2�Z%Gd3d �d e�Z&Gd4d�de&�Z'd5d�Z(dS)8�)�absolute_importN�)�HTTPHeaderDict�RecentlyUsedContainer)�HTTPConnectionPool�HTTPSConnectionPool)�port_by_scheme)�LocationValueError� MaxRetryError�ProxySchemeUnknown)�six)�urljoin)�RequestMethods)� parse_url)�Retry�PoolManager�ProxyManager�proxy_from_url�key_file� cert_file� cert_reqs�ca_certs�ssl_version�ca_cert_dir�ssl_context� key_scheme�key_host�key_port�key_timeout�key_retries� key_strict� key_block�key_source_address�key_key_file� key_cert_file� key_cert_reqs�key_ca_certs�key_ssl_version�key_ca_cert_dir�key_ssl_context�key_maxsize�key_headers� key__proxy�key__proxy_headers�key_socket_options�key__socks_options�key_assert_hostname�key_assert_fingerprint�key_server_hostname�PoolKeycCs�|j�}|dj�|d<|dj�|d<x4d D],}||kr.||dk r.t||j��||<q.W|jd�}|dk r|t|�|d<x&t|j��D]}|j|�|d|<q�Wx|j D]}||kr�d||<q�W|f|�S) a� Create a pool key out of a request context dictionary. According to RFC 3986, both the scheme and host are case-insensitive. Therefore, this function normalizes both before constructing the pool key for an HTTPS request. If you wish to change this behaviour, provide alternate callables to ``key_fn_by_scheme``. :param key_class: The class to use when constructing the key. This should be a namedtuple with the ``scheme`` and ``host`` keys at a minimum. :type key_class: namedtuple :param request_context: A dictionary-like object that contain the context for a request. :type request_context: dict :return: A namedtuple that can be used as a connection pool key. :rtype: PoolKey �scheme�host�headers�_proxy_headers�_socks_optionsNZsocket_optionsZkey_)r6r7r8) �copy�lower� frozenset�items�get�tuple�list�keys�pop�_fields)Z key_class�request_context�context�keyZsocket_optsZfield�rF�!/usr/lib/python3.6/poolmanager.py�_default_key_normalizer;s rH)�http�httpsc@sxeZdZdZdZddd�Zdd�Zdd �Zdd d�Zdd �Z ddd�Z dd�Zddd�Zd dd�Z dd�Zd!dd�ZdS)"ra$ Allows for arbitrary requests while transparently keeping track of necessary connection pools for you. :param num_pools: Number of connection pools to cache before discarding the least recently used pool. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param \**connection_pool_kw: Additional parameters are used to create fresh :class:`urllib3.connectionpool.ConnectionPool` instances. Example:: >>> manager = PoolManager(num_pools=2) >>> r = manager.request('GET', 'http://google.com/') >>> r = manager.request('GET', 'http://google.com/mail') >>> r = manager.request('GET', 'http://yahoo.com/') >>> len(manager.pools) 2 N� cKs8tj||�||_t|dd�d�|_t|_tj�|_dS)NcSs|j�S)N)�close)�prFrFrG��sz&PoolManager.__init__..)Zdispose_func)r�__init__�connection_pool_kwr�pools�pool_classes_by_scheme�key_fn_by_schemer9)�self� num_poolsr6rPrFrFrGrO�szPoolManager.__init__cCs|S)NrF)rTrFrFrG� __enter__�szPoolManager.__enter__cCs|j�dS)NF)�clear)rT�exc_typeZexc_valZexc_tbrFrFrG�__exit__�szPoolManager.__exit__cCsf|j|}|dkr|jj�}xdD]}|j|d�q"W|dkrXxtD]}|j|d�qDW|||f|�S)a� Create a new :class:`ConnectionPool` based on host, port, scheme, and any additional pool keyword arguments. If ``request_context`` is provided, it is provided as keyword arguments to the pool class used. This method is used to actually create the connection pools handed out by :meth:`connection_from_url` and companion methods. It is intended to be overridden for customization. Nr4r5�portrI)r4r5rZ)rRrPr9rA�SSL_KEYWORDS)rTr4r5rZrCZpool_clsrE�kwrFrFrG� _new_pool�s zPoolManager._new_poolcCs|jj�dS)z� Empty our store of pools and direct them all to close. This will not affect in-flight connections, but they will not be re-used after completion. N)rQrW)rTrFrFrGrW�szPoolManager.clearrIcCsT|std��|j|�}|pd|d<|s:tj|dj�d�}||d<||d<|j|�S)a� Get a :class:`ConnectionPool` based on the host, port, and scheme. If ``port`` isn't given, it will be derived from the ``scheme`` using ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is provided, it is merged with the instance's ``connection_pool_kw`` variable and used to create the new connection pool, if one is needed. zNo host specified.rIr4�PrZr5)r �_merge_pool_kwargsrr=r:�connection_from_context)rTr5rZr4�pool_kwargsrCrFrFrG�connection_from_host�s z PoolManager.connection_from_hostcCs,|dj�}|j|}||�}|j||d�S)z� Get a :class:`ConnectionPool` based on the request context. ``request_context`` must at least contain the ``scheme`` key and its value must be a key in ``key_fn_by_scheme`` instance variable. r4)rC)r:rS�connection_from_pool_key)rTrCr4Zpool_key_constructor�pool_keyrFrFrGr`�s z#PoolManager.connection_from_contextc Cs`|jj�N|jj|�}|r|S|d}|d}|d}|j||||d�}||j|<WdQRX|S)z� Get a :class:`ConnectionPool` based on the provided pool key. ``pool_key`` should be a namedtuple that only contains immutable objects. At a minimum it must have the ``scheme``, ``host``, and ``port`` fields. r4r5rZ)rCN)rQ�lockr=r])rTrdrCZpoolr4r5rZrFrFrGrc�s z$PoolManager.connection_from_pool_keycCs t|�}|j|j|j|j|d�S)a� Similar to :func:`urllib3.connectionpool.connection_from_url`. If ``pool_kwargs`` is not provided and a new pool needs to be constructed, ``self.connection_pool_kw`` is used to initialize the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs`` is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided ``pool_kwargs`` are not used. )rZr4ra)rrbr5rZr4)rT�urlra�urFrFrG�connection_from_urlszPoolManager.connection_from_urlcCsZ|jj�}|rVxF|j�D]:\}}|dkrJy ||=WqRtk rFYqRXq|||<qW|S)a Merge a dictionary of override values for self.connection_pool_kw. This does not modify self.connection_pool_kw and returns a new dict. Any keys in the override dictionary with a value of ``None`` are removed from the merged dictionary. N)rPr9r<�KeyError)rT�overrideZbase_pool_kwargsrE�valuerFrFrGr_s zPoolManager._merge_pool_kwargsTcKs�t|�}|j|j|j|jd�}d|d<d|d<d|krD|jj�|d<|jdk rj|jdkrj|j||f|�}n|j||j f|�}|o�|j �}|s�|St||�}|jdkr�d }d|d <t |d�j�|d<|jd�} t| t�s�tj| |d�} | jo�|j|��r %s)rrbr5rZr4r6r9�proxy�urlopenZrequest_uriZget_redirect_locationr ZstatusrZ_prepare_for_method_changer=� isinstancerZfrom_intZremove_headers_on_redirectZis_same_hostr?rZiterkeysr:rAZ incrementr Zraise_on_redirect�log�info)rT�methodrfrlr\rgZconnrnZredirect_locationrmr6�headerrFrFrGrp/sH zPoolManager.urlopen)rKN)N)NrIN)N)N)T)�__name__� __module__�__qualname__�__doc__rorOrVrYr]rWrbr`rcrhr_rprFrFrFrGr{s csHeZdZdZd�fdd� Zd�fdd� Zdd d �Zd�fdd � Z�ZS)rax Behaves just like :class:`PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs. :param proxy_url: The URL of the proxy to be used. :param proxy_headers: A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication. Example: >>> proxy = urllib3.ProxyManager('http://localhost:3128/') >>> r1 = proxy.request('GET', 'http://google.com/') >>> r2 = proxy.request('GET', 'http://httpbin.org/') >>> len(proxy.pools) 1 >>> r3 = proxy.request('GET', 'https://httpbin.org/') >>> r4 = proxy.request('GET', 'https://twitter.com/') >>> len(proxy.pools) 3 rKNcs�t|t�rd|j|j|jf}t|�}|jsFtj|jd�}|j|d�}|jdkrZt |j��||_ |pfi|_|j |d<|j|d<tt |�j||f|�dS) Nz %s://%s:%ir^)rZrIrJ�_proxyr7)rIrJ)rqrr4r5rZrrr=�_replacerro� proxy_headers�superrrO)rT� proxy_urlrUr6r|rProrZ)� __class__rFrGrO�s zProxyManager.__init__rIcsD|dkr tt|�j||||d�Stt|�j|jj|jj|jj|d�S)NrJ)ra)r}rrbror5rZr4)rTr5rZr4ra)rrFrGrb�s z!ProxyManager.connection_from_hostcCs0ddi}t|�j}|r||d<|r,|j|�|S)z� Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user. ZAcceptz*/*ZHost)r�netloc�update)rTrfr6Zheaders_r�rFrFrG�_set_proxy_headers�s zProxyManager._set_proxy_headersTcsNt|�}|jdkr0|jd|j�}|j||�|d<tt|�j||fd|i|��S)z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.rIr6rl)rr4r=r6r�r}rrp)rTrtrfrlr\rgr6)rrFrGrp�s zProxyManager.urlopen)rKNN)NrIN)N)T) rvrwrxryrOrbr�rp� __classcell__rFrF)rrGrps cKstfd|i|��S)Nr~)r)rfr\rFrFrGr�s)rrrrrrr)rrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2))Z __future__r�collections� functoolsZlogging�_collectionsrrZconnectionpoolrrr� exceptionsr r rZpackagesrZpackages.six.moves.urllib.parser ZrequestrZutil.urlrZ util.retryr�__all__Z getLoggerrvrrr[Z_key_fields� namedtupler3rH�partialrSrRrrrrFrFrFrG�sd 6 vW