3 �Pf/l�@s�dZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddlZddlmZddl mZddlmZddlmZddlmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!dd l"m#Z#dd l$m%Z%ddl&m'Z'm(Z(m)Z)m*Z*dfZ+ej,�Z-ddd�Z.ej/�dk�r0dd�Z0dd�Zdd�Z1dd�Z2dgdd�Z3dd�Z4dd �Z5d!d"�Z6d#d$�Z7d%d&�Z8dhd'd(�Z9d)d*�Z:d+d,�Z;d-d.�Zd3d4�Z?d5d6�Z@eAdi�ZBd9d:�ZCd;d<�ZDd=d>�ZEd?d@�ZFdAdB�ZGdCdD�ZHejIdEdF��ZJdGdH�ZKdjdIdJ�ZLdKdL�ZMdkdNdO�ZNdPdQ�ZOdRdS�ZPdTjQdU�ZReRdVZSeRdWZTdXdY�ZUdZd[�ZVd\d]�ZWejXd^�ZYejXd_�ZZd`da�Z[dbdc�Z\ddde�Z]dS)lz� requests.utils ~~~~~~~~~~~~~~ This module provides utility functions that are used within Requests that are also useful for external consumption. �N�)�__version__)�certs)�to_native_string)�parse_http_list)�quote�urlparse�bytes�str�OrderedDict�unquote� getproxies�proxy_bypass� urlunparse� basestring� integer_types�is_py3�proxy_bypass_environment�getproxies_environment)�cookiejar_from_dict)�CaseInsensitiveDict)� InvalidURL� InvalidHeader�FileModeWarning�UnrewindableBodyError�.netrc�_netrc�Pi�)ZhttpZhttpsZWindowsc Cs�trddl}nddl}y2|j|jd�}|j|d�d}|j|d�d}Wntk r\dSX|sj|rndS|jd�}xX|D]P}|dkr�d|kr�d S|jdd �}|jdd�}|jd d�}t j ||t j�r~d Sq~WdS)Nrz;Software\Microsoft\Windows\CurrentVersion\Internet SettingsZProxyEnableZ ProxyOverrideF�;z�.Tz\.�*z.*�?)r�winreg�_winreg�OpenKey�HKEY_CURRENT_USERZQueryValueEx�OSError�split�replace�re�match�I)�hostr"ZinternetSettingsZproxyEnableZ proxyOverrideZtest�r-�/usr/lib/python3.6/utils.py�proxy_bypass_registry.s2 r/cCst�rt|�St|�SdS)z�Return True, if the host should be bypassed. Checks proxy settings gathered from the environment, if specified, or the registry. N)rrr/)r,r-r-r.rOsrcCst|d�r|j�}|S)z/Returns an internal sequence dictionary update.�items)�hasattrr0)�dr-r-r.�dict_to_sequence[s r3cCs2d}d}t|d�rt|�}nbt|d�r.|j}nPt|d�r~y|j�}Wntjk rZYn$Xtj|�j}d|jkr~t j dt�t|d��ry|j�}Wn$t tfk r�|dk r�|}Yn\Xt|d�o�|dk�ry&|jdd �|j�}|j|p�d�Wnt tfk �rd}YnX|dk�r$d}td||�S) Nr�__len__�len�fileno�ba%Requests has determined the content-length for this request using the binary size of the file: however, the file has been opened in text mode (i.e. without the 'b' flag in the mode). This may lead to an incorrect content-length. In Requests 3.0, support will be removed for files in text mode.�tell�seek�)r1r5r6�io�UnsupportedOperation�os�fstat�st_size�mode�warnings�warnrr8r&�IOErrorr9�max)�oZtotal_lengthZcurrent_positionr6r-r-r.� super_lends@ rFFcCsy�ddlm}m}d}xJtD]B}ytjjdj|��}Wntk rJdSXtjj|�r|}PqW|dkrndSt |�}d}t |t�r�|jd�}|j j|�d} y6||�j| �} | r�| dr�dnd}| || dfSWn|tfk r�|r�YnXWnttfk �rYnXdS) z;Returns the Requests tuple auth for a given url from netrc.r)�netrc�NetrcParseErrorNz~/{0}�:�asciirr:)rGrH�NETRC_FILESr=�path� expanduser�format�KeyError�existsr� isinstancer �decode�netlocr'ZauthenticatorsrC�ImportError�AttributeError)�urlZraise_errorsrGrHZ netrc_path�f�locZriZsplitstrr,rZlogin_ir-r-r.�get_netrc_auth�s8 rYcCsBt|dd�}|r>t|t�r>|ddkr>|ddkr>tjj|�SdS)z0Tries to guess the filename of the given object.�nameNr��)�getattrrQrr=rL�basename)�objrZr-r-r.�guess_filename�sracCs.|dkrdSt|ttttf�r&td��t|�S)a�Take an object and test to see if it can be represented as a dictionary. Unless it can not be represented as such, return an OrderedDict, e.g., :: >>> from_key_val_list([('key', 'val')]) OrderedDict([('key', 'val')]) >>> from_key_val_list('string') ValueError: need more than 1 value to unpack >>> from_key_val_list({'key': 'val'}) OrderedDict([('key', 'val')]) :rtype: OrderedDict Nz+cannot encode objects that are not 2-tuples)rQr r �bool�int� ValueErrorr)�valuer-r-r.�from_key_val_list�s rfcCsB|dkrdSt|ttttf�r&td��t|tj�r:|j�}t |�S)a�Take an object and test to see if it can be represented as a dictionary. If it can be, return a list of tuples, e.g., :: >>> to_key_val_list([('key', 'val')]) [('key', 'val')] >>> to_key_val_list({'key': 'val'}) [('key', 'val')] >>> to_key_val_list('string') ValueError: cannot encode objects that are not 2-tuples. :rtype: list Nz+cannot encode objects that are not 2-tuples) rQr r rbrcrd�collections�Mappingr0�list)rer-r-r.�to_key_val_list�srjcCs\g}xRt|�D]F}|dd�|dd�ko4dknrJt|dd��}|j|�qW|S)aParse lists as described by RFC 2068 Section 2. In particular, parse comma-separated lists where the elements of the list may include quoted-strings. A quoted-string could contain a comma. A non-quoted string could have quotes in the middle. Quotes are removed automatically after parsing. It basically works like :func:`parse_set_header` just that items may appear multiple times and case sensitivity is preserved. The return value is a standard :class:`list`: >>> parse_list_header('token, "quoted value"') ['token', 'quoted value'] To create a header from the :class:`list` again, use the :func:`dump_header` function. :param value: a string with a list header. :return: :class:`list` :rtype: list Nr�"r]r])�_parse_list_header�unquote_header_value�append)re�result�itemr-r-r.�parse_list_headers(rqcCs|i}xrt|�D]f}d|kr$d||<q|jdd�\}}|dd�|dd�koVdknrlt|dd��}|||<qW|S)a^Parse lists of key, value pairs as described by RFC 2068 Section 2 and convert them into a python dict: >>> d = parse_dict_header('foo="is a fish", bar="as well"') >>> type(d) is dict True >>> sorted(d.items()) [('bar', 'as well'), ('foo', 'is a fish')] If there is no value for a key it will be `None`: >>> parse_dict_header('key_without_value') {'key_without_value': None} To create a header from the :class:`dict` again, use the :func:`dump_header` function. :param value: a string with a dict header. :return: :class:`dict` :rtype: dict �=Nrrkr]r])rlr'rm)rerorprZr-r-r.�parse_dict_header1s(rscCs^|rZ|d|d kodknrZ|dd �}|sF|dd�dkrZ|jdd�jdd�S|S)z�Unquotes a header value. (Reversal of :func:`quote_header_value`). This does not use the real unquoting but what browsers are actually using for quoting. :param value: the header value to unquote. :rtype: str rrrkNr:z\\�\z\"r]r])r()reZis_filenamer-r-r.rmTs $rmcCs"i}x|D]}|j||j<q W|S)z�Returns a key/value dictionary from a CookieJar. :param cj: CookieJar object to extract cookies from. :rtype: dict )rerZ)�cj�cookie_dictZcookier-r-r.�dict_from_cookiejarms rwcCs t||�S)z�Returns a CookieJar from a key/value dictionary. :param cj: CookieJar to insert cookies into. :param cookie_dict: Dict of key/values to insert into CookieJar. :rtype: CookieJar )r)rurvr-r-r.�add_dict_to_cookiejar|srxcCsTtjdt�tjdtjd�}tjdtjd�}tjd�}|j|�|j|�|j|�S)zlReturns encodings from given content string. :param content: bytestring to extract encodings from. z�In requests 3.0, get_encodings_from_content will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)z!])�flagsz+]z$^<\?xml.*?encoding=["\']*(.+?)["\'>])rArB�DeprecationWarningr)�compiler+�findall)�contentZ charset_reZ pragma_reZxml_rer-r-r.�get_encodings_from_content�s r~cCsF|jd�}|sdStj|�\}}d|kr6|djd�Sd|krBdSdS)z}Returns encodings from given HTTP Header Dict. :param headers: dictionary to extract encoding from. :rtype: str zcontent-typeN�charsetz'"�textz ISO-8859-1)�get�cgiZparse_header�strip)�headersZcontent_type�paramsr-r-r.�get_encoding_from_headers�s r�ccsr|jdkr"x|D] }|VqWdStj|j�dd�}x |D]}|j|�}|r:|Vq:W|jddd�}|rn|VdS)zStream decodes a iterator.Nr()�errors�T)�final)�encoding�codecs�getincrementaldecoderrR)�iterator�rrp�decoder�chunk�rvr-r-r.�stream_decode_response_unicode�s r�ccsLd}|dks|dkrt|�}x*|t|�krF||||�V||7}qWdS)z Iterate over slices of a string.rN)r5)�stringZslice_length�posr-r-r.�iter_slices�sr�cCsvtjdt�g}t|j�}|rJyt|j|�Stk rH|j|�YnXyt|j|dd�St k rp|jSXdS)z�Returns the requested content back in unicode. :param r: Response object to get unicode content from. Tried: 1. charset from content-type 2. fall back and replace all unicode characters :rtype: str z�In requests 3.0, get_unicode_from_response will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)r()r�N) rArBrzr�r�r r}�UnicodeErrorrn� TypeError)r�Ztried_encodingsr�r-r-r.�get_unicode_from_response�s r�Z4ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzz0123456789-._~c Cs�|jd�}x�tdt|��D]�}||dd�}t|�dkr�|j�r�ytt|d��}Wn tk rttd|��YnX|tkr�|||dd�||<q�d||||<qd||||<qWdj |�S) z�Un-escape any percent-escape sequences in a URI that are unreserved characters. This leaves all reserved, illegal and non-ASCII bytes encoded. :rtype: str �%rrr:�z%Invalid percent-escape sequence: '%s'N�) r'�ranger5�isalnum�chrrcrdr�UNRESERVED_SET�join)�uri�parts�i�h�cr-r-r.�unquote_unreserved�s r�cCs:d}d}ytt|�|d�Stk r4t||d�SXdS)z�Re-quote the given URI. This function passes the given URI through an unquote/quote cycle to ensure that it is fully and consistently quoted. :rtype: str z!#$%&'()*+,/:;=?@[]~z!#$&'()*+,/:;=?@[]~)ZsafeN)rr�r)r�Zsafe_with_percentZsafe_without_percentr-r-r.�requote_uri sr�cCsltjdtj|��d}|jd�\}}tjdtjtt|��d}tjdtj|��d|@}||@||@kS)z�This function allows you to check if an IP belongs to a network subnet Example: returns True if ip = 192.168.1.1 and net = 192.168.1.0/24 returns False if ip = 192.168.1.1 and net = 192.168.100.0/24 :rtype: bool z=Lr�/)�struct�unpack�socket� inet_atonr'�dotted_netmaskrc)�ipZnetZipaddrZnetaddr�bitsZnetmaskZnetworkr-r-r.�address_in_network#s r�cCs&ddd|>dA}tjtjd|��S)z�Converts mask from /xx format to xxx.xxx.xxx.xxx Example: if mask is 24 function returns 255.255.255.0 :rtype: str l��r� z>I)r�Z inet_ntoar��pack)�maskr�r-r-r.r�2sr�cCs*ytj|�Wntjk r$dSXdS)z :rtype: bool FT)r�r��error)Z string_ipr-r-r.�is_ipv4_address=s r�cCs�|jd�dkr�yt|jd�d�}Wntk r8dSX|dksJ|dkrNdSytj|jd�d�Wq�tjk r|dSXndSdS)zV Very simple check of the cidr format in no_proxy variable. :rtype: bool r�rFr�rT)�countrcr'rdr�r�r�)Zstring_networkr�r-r-r.� is_valid_cidrHsr�ccsT|dk }|r"tjj|�}|tj|<z dVWd|rN|dkrDtj|=n |tj|<XdS)z�Set the environment variable 'env_name' to 'value' Save previous value, yield, and then restore the previous value stored in the environment variable 'env_name'. If 'value' is None, do nothingN)r=�environr�)Zenv_namereZ value_changedZ old_valuer-r-r.�set_environ`s r�c Csdd�}|}|dkr|d�}t|�j}|r�dd�|jdd�jd �D�}|jd �d}t|�r�xb|D](}t|�r~t||�r�dSqb||krbdSqbWn0x.|D]&}|j|�s�|jd �dj|�r�dSq�Wtd|��2yt |�}Wnt tjfk r�d }YnXWdQRX|�rdSd S)zL Returns whether we should bypass proxies or not. :rtype: bool cSstjj|�ptjj|j��S)N)r=r�r��upper)�kr-r-r.�|sz'should_bypass_proxies..N�no_proxycss|]}|r|VqdS)Nr-)�.0r,r-r-r.� �sz(should_bypass_proxies..� r��,�:rTF) rrSr(r'r�r�r��endswithr�rr�r�Zgaierror) rVr�Z get_proxyZno_proxy_argrSr�Zproxy_ipr,Zbypassr-r-r.�should_bypass_proxiesvs4 r�cCst||d�riSt�SdS)zA Return a dict of environment proxies. :rtype: dict )r�N)r�r )rVr�r-r-r.�get_environ_proxies�sr�cCsv|pi}t|�}|jdkr.|j|j|jd��S|jd|j|jd|jdg}d}x|D]}||krX||}PqXW|S)z�Select a proxy for the url, if applicable. :param url: The url being for the request :param proxies: A dictionary of schemes or schemes and hosts to proxy URLs N�allz://zall://)rZhostnamer��scheme)rVZproxiesZurlpartsZ proxy_keys�proxyZ proxy_keyr-r-r.�select_proxy�s r��python-requestscCsd|tfS)zO Return a string representing the default user agent. :rtype: str z%s/%s)r)rZr-r-r.�default_user_agent�sr�cCstt�djd�ddd��S)z9 :rtype: requests.structures.CaseInsensitiveDict z, �gzip�deflatez*/*z keep-alive)z User-AgentzAccept-EncodingZAcceptZ Connection)r�r�)rr�r�r-r-r-r.�default_headers�s r�c Cs�g}d}x�tjd|�D]�}y|jdd�\}}Wntk rL|d}}YnXd|jd�i}xP|jd�D]B}y|jd�\}}Wntk r�PYnX|j|�||j|�<qhW|j|�qW|S) z�Return a dict of parsed link headers proxies. i.e. Link: ; rel=front; type="image/jpeg",; rel=back;type="image/jpeg" :rtype: list z '"z, * '"rr)r)r'rdr�rn) reZlinksZ replace_chars�valrVr��linkZparam�keyr-r-r.�parse_header_links�s r��rJr:�cCs�|dd�}|tjtjfkr dS|dd�tjkr6dS|dd�tjtjfkrRdS|jt�}|dkrhd S|dkr�|ddd�tkr�d S|ddd�tkr�dS|dkr�|dd�t kr�d S|dd�t kr�dSdS)z :rtype: str N�zutf-32r�z utf-8-sigr:zutf-16rzutf-8z utf-16-berz utf-16-lez utf-32-bez utf-32-le) r��BOM_UTF32_LE�BOM_UTF32_BE�BOM_UTF8�BOM_UTF16_LE�BOM_UTF16_BEr��_null�_null2�_null3)�dataZsampleZ nullcountr-r-r.�guess_json_utfs* r�cCs8t||�\}}}}}}|s$||}}t||||||f�S)z�Given a URL that may or may not have a scheme, prepend the given scheme. Does not replace a present scheme with the one provided as an argument. :rtype: str )rr)rVZ new_schemer�rSrLr��query�fragmentr-r-r.�prepend_scheme_if_needed1s r�cCsBt|�}yt|j�t|j�f}Wnttfk r<d}YnX|S)z{Given a url with authentication components, extract them into a tuple of username,password. :rtype: (str,str) r�)r�r�)rrZusernameZpasswordrUr�)rVZparsedZauthr-r-r.�get_auth_from_urlBs r�s^\S[^\r\n]*$|^$z^\S[^\r\n]*$|^$cCsf|\}}t|t�rt}nt}y|j|�s4td|��Wn*tk r`td||t|�f��YnXdS)z�Verifies that header value is a string which doesn't contain leading whitespace or return characters. This prevents unintended header injection. :param header: tuple, in the format (name, value). z7Invalid return character or leading space in header: %sz>Value for header {%s: %s} must be of type str or bytes, not %sN)rQr �_CLEAN_HEADER_REGEX_BYTE�_CLEAN_HEADER_REGEX_STRr*rr��type)�headerrZreZpatr-r-r.�check_header_validityWs r�cCsFt|�\}}}}}}|s"||}}|jdd�d}t|||||df�S)zW Given a url remove the fragment and the authentication part. :rtype: str �@rr�r])r�rsplitr)rVr�rSrLr�r�r�r-r-r.� urldefragauthls r�cCs`t|jdd�}|dk rTt|jt�rTy||j�Wq\ttfk rPtd��Yq\Xntd��dS)zfMove file pointer back to its recorded starting position so it can be read again on redirect. r9Nz;An error occurred when rewinding request body for redirect.z+Unable to rewind request body for redirect.)r^ZbodyrQZ_body_positionrrCr&r)Zprepared_requestZ body_seekr-r-r.�rewind_body}sr�)rr)F)FzBABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~)N)r�)^�__doc__r�r�rg� contextlibr;r=�platformr)r�r�rArr�rZ_internal_utilsr�compatrrlrrr r rrr rrrrrrrZcookiesrZ structuresr� exceptionsrrrrrK�whereZDEFAULT_CA_BUNDLE_PATHZ DEFAULT_PORTS�systemr/r3rFrYrarfrjrqrsrmrwrxr~r�r�r�r�� frozensetr�r�r�r�r�r�r��contextmanagerr�r�r�r�r�r�r��encoder�r�r�r�r�r�r{r�r�r�r�r�r-r-r-r.� s�@ ! = 3 # %9 "