Current File : //usr/lib/python2.7/site-packages/salt/utils/http.pyo
�
���^c . @@ s� d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l Z d d l Z d d l
Z
d d l Z y* d d l m Z d d l m Z e Z Wn� e k
rYy* d d l m Z d d l m Z e Z WnU e k
rUy* d d l m Z d d l m Z e Z Wn e k
rQe Z n Xn Xn Xd d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l! Z d d l" Z d d l# Z d d l$ j% j& Z' d d l( m) Z* d d l+ m, Z, d d l- m. Z. d d l/ m0 Z0 d d l1 Z d d l2 Z d d l3 j4 j0 j5 j6 j7 Z8 d d
l9 m: Z: d d l; m< Z< m= Z= d d l; m> Z? d d l@ ZA d d lB ZA d d
lC mD ZD y d d lE ZA e ZF Wn e k
r�e ZF n Xy d d lG ZG e ZH Wn e k
r,e ZH n Xy d d lI ZI e ZJ Wn e k
r\e ZJ n Xy d d lK ZK e ZL Wn e k
r�e ZL n Xe jM eN � ZO d jP e jQ jR � ZS d � ZT e. d � d eU eU eU eU eU eU eU eU eU e d e e e eU eU d e eU e eU e eU eU e eU d d eU eU eU eU eU eU eU eU e eU eU e eS eU e d �, � ZV eU d � ZW eU eU eU eU d � ZX d � ZY d � ZZ d � Z[ d � Z\ d S( u�
Utils for making various web calls. Primarily designed for REST, SOAP, webhooks
and the like, but also useful for basic HTTP testing.
.. versionadded:: 2015.5.0
i ( t absolute_importt unicode_literalst print_functionN( t CertificateError( t match_hostname( t ElementTree( t compile_template( t jinja_filter( t six( t URLError( t
splitqueryt urlparse( t urlencode( t
HTTPClientu Salt/{0}c C@ s� t j d t | � | � | d k r> t j | t j � } n� | d k rz t j | � } t j d | � } | j
� } n` | d k r� t d � � nE | d k r� t d � � n* | d k r� t d
� � n | d k r� n t j d t | � � | S(
u�
Decompress returned HTTP content depending on the specified encoding.
Currently supports identity/none, deflate, and gzip, which should
cover 99%+ of the content on the internet.
u7 Decompressing %s byte content with compression type: %su deflateu gzipt fileobju sdchu+ SDCH compression is not currently supportedu bru- Brotli compression is not currently supportedu compressu* LZW compression is not currently supportedu identityu$ Content size after decompression: %s( t logt tracet lent zlibt
decompresst MAX_WBITSt iot BytesIOt gzipt GzipFilet readt
ValueError( t codingt pgctntt buft f( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyt __decompressContente s" u
http_queryu GETu autou lwpu minioniP c- c @ so i }. | d k r� | d k rH t j j t j j t j j d � � } q� | d k r~ t j j t j j t j j d � � } q� i } n | s� | j
d d � } n t j d | � }/ |/ s� t j
j j � n | d k r&t t k rd |. d <t j |. d � |. St j d � }0 |0 j t j � n | rAt j j | | � n | }1 | d k rbt | � } n |! d k r�| j
d t � }! n |" d k r�| j
d
d � }" n | d k r�t | | | | | � } n t |1 |+ � }2 t j d |2 � � t j d | � � d
k r�t j t j! � r�t" | t# � r�| j$ � }3 t" |+ t% � r�x; | D]0 }4 x' |+ D] }5 |4 |5 k rYd |3 |4 <qYqYWqLWn t j& d t' j( |3 � � q�t j& d t' j( | � � n | d k rt | | | | | � }6 t" |6 t# � r�|6 } q|6 j) � } n | d k ri } n | d k r0g } n | d k rft j j | j
d t j j* � d � } n | d k r�t j j | j
d t j j* � d � } n | t k rFt+ rFt j j, | � rt j
j- j. | d � � }7 t/ j0 |7 � }8 Wd QXt" |8 t# � rC| j1 |8 � qCqFt j
j- j. | d � � }7 t/ j2 d |7 � Wd QXn xS | D]K }9 |9 j3 d � }: t4 |: � d k rzqMn |: d j5 � | |: d j5 � <qMW|
s�| r�| r�| | f }
q�n |* t6 k r�d j7 |* � }* n |* | d <| d k rAt8 j9 � }; |
|; _: |; j; j1 | � t j& d |; j; � |; j< }< |! |; _= n | d k rVd }< n d }< | d k r�| d k r�t j> j? j@ jA jB | � }< n t j> j? j@ jA jC | � }< t j j, | � s�|< jD � n |< j0 � n | t k r| d k r�i S| } t |. d <n | d k r�i }= |& t k rSt8 jE d d! k rFt |= d" <qSt |= d# <n |" d k r�t" |" t? jF � r�t j jG |" � r�|" |= d
<q�q�t" |" t% � r�t j jG |" d � r�t j jG |" d � r�|" |= d
<q�q�t j d$ |" � n |; jH � | d% | d& | |= �}> |> jI � |& t k rV|( d' j7 |> jJ � � |' |> jK � i |> d( 6S|) t k rwi |> d( 6|> jK d) 6St j d* t |> jL |+ � � |> jJ }? |> j; }@ |> jK }A |> j< }B |> jK }C t" |C t? jM � s�|C jN |> jO p�d+ � }C n |C |. d) <nN | d k r tP jQ |1 | � }D tP jR tP jS |< � g }E | jT d, � r�
|D jU � }F tP jV d � |E d <tW stt jX d- � q�
|! t k r�t jX d. � q�
d |F k r�|F j3 d � \ }F } n d/ } tY jY tY jZ tY j[ � }G |G j\ |F t] | � f � t^ j_ |G d0 | d1 t^ j` �}H y ta |H jb � |F � Wn- tc k
rO }I d2 t' j( |I � f |. d <|. SX|" d k r�
d }J t" |" t? jF � r� t j jG |" � r� |" }J q� n[ t" |" t% � r� t j jG |" d � r� t j jG |" d � r� |" }J q� n t j d$ |" � d Std t^ d3 � r*
t^ je jf |J � }K |E jg tP jV d4 |K � � q�
i |D jU � d5 6| d6 6|J d d7 6}L t4 |J � d k rr
|J d |L d8 <n t j> j? j@ jh ji |L � |E d <q�
n tP jj |E � }M x" | D] }9 |D jk |9 | |9 � q�
W� f d9 � |D _l y |M jm |D � }> Wn$ tn k
r }I i t? jM |I � d: 6SX|& t k s/ |) t k rD i |> d( 6|> jK d) 6S|> jo }? t# |> jp � � }@ |> jq � }A d; |@ k r� tr js |@ d; � \ }N }O |N jT d< � r� d= |O k r� t" |A t? jM � r� |A jN |O d= � }A q� n t? jt r t" |A tu � r |A jN d+ � }A n |A |. d) <n:i }= |" d k r� t" |" t? jF � rT t j jG |" � r� |" |= d> <q� q� t" |" t% � r� t j jG |" d � r� t j jG |" d � r� |" d |= d> <|" d |= d? <q� q� t j d$ |" � n t" | t# � r� tv | � } n |! r� | |= d@ <n | j
dA t j jw dA � }P | j
dB t j jw dB � }Q | j
dC t j jw dC � }R d }S | j
dD d � }T |T r
t j
jx jy |T � }T n | j
dE d � }U | j
dF d � }V |V r�
t j
jx jy |V � }V n | j
dG d � }W |W r�
t j
jx jy |W � }W n | j
dH g � }X tz |1 � j{ |X k r'd }T d }U n |T r�|U r�t| t k r^dI |. d <t j |. d � |. St j} j~ j dJ � t j
j� j� t j� j� j� � }S n t j
j� j� t j� j� j� � }S dK |S j� k }Y |= j1 i � dL 6| dM 6| dN 6| dO 6| d) 6|! dP 6t dQ 6|' dR 6|( dS 6|Q dT 6|R dU 6|T dD 6|U dE 6|V dF 6|W dG 6|, dV 6t dW 6� t j
j� jN |= dX t �}= y4 |Y rvt� dY |P � n t� � }Z |Z j� |1 |= � }> Wn| t j} j� k
r�}I |I jo |. dZ <t? jM |I � |. d <|. StY j� k
r}I |
t k r�d |. dZ <n t? jM |I � |. d <|. SX|& t k s)|) t k r>i |> d( 6|> j� d) 6S|> jo }? |> j; }@ |> j� }A d; |@ k r�tr js |@ d; � \ }N }O |N jT d< � r�d= |O k r�t" |A t? jM � r�|A jN |O d= � }A q�n t? jt r�t" |A tu � r�|A jN d+ � }A n |A |. d) <d[ |@ k rB| d k rBt� |@ d[ � }B x$ |B D] }4 |< j� |4 � q(Wn d }B t" |@ t% � r�i }[ xD |@ D]< }9 |9 j3 d � }: d j |: d � j5 � |[ |: d j5 � <qdW|[ }@ n t j d\ |? � t j& d] |@ � t j& d^ |< � |@ j
d_ d` � }\ | d k r
t� |\ |A � }A n y t j& da |A � Wn# t� k
rF}I t j& db |I � n X|# d k r�t j
j- j. |# dc � � }] |] j� |A � Wd QXn |$ d k r�t j jG |$ � r�t j
j- j. |$ dc � � }^ |^ j� |@ � Wd QXn | d k r�|< jD � n | t k ryt+ rydd |@ k ryt j
j- j. | d � �O }7 |@ j
dd d � }8 |8 d k r]t/ j2 i |8 de 6|7 � n t/ j2 d |7 � Wd QXqyn |
t k r�|? |. dZ <n | t k r�|@ |. dM <n | t k rR| df k r |@ j
dg dh � }_ di |_ k r�di } q dj |_ k r�dj } q dk |_ k rdk } q dl } n dp }` | |` k r`dm j7 t' j( |` � � |. d <t j |. d � |. S| dj k r�t j
j� j� |A � |. dn <n� | di k r�g |. dn <t� j� |A � }a xh |a D] }4 |. dn jg t� j� |4 � � q�Wn= | dk k rt j
j� jN t j
j� j� |A � � |. dn <n t } |% rRt j
j- j. |% dc � � }b |b j� |A � Wd QXqRn | t k rk|A |. do <n |. S(q u6
Query a resource, and decode the return data
u masteru minionu backendu tornadouO https?://((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)($|/)u requestsu_ http.query has been set to use requests, but the requests library does not seem to be installedu erroru
verify_sslu certu! Requesting URL %s using %s methodu Using backend: %su POSTu
XXXXXXXXXXu Request POST Data: %su cachediru cookies.txtu cookies.session.pu rbNu wbu u :i i i u {0} http.query()u
User-agentu Request Headers: %su urllib2u mozillau testu 0u prefetchu streamuA The client-side certificate path that was passed is not valid: %st paramst datau HTTP/1.0 {0} MESSAGEu handleu bodyu" Final URL location of Response: %su utf-8u httpsug match_hostname() not available, SSL hostname checking not available. THIS CONNECTION MAY NOT BE SECURE!u] SSL certificate verification has been explicitly disabled. THIS CONNECTION MAY NOT BE SECURE!i� t ca_certst cert_reqsu3 The certificate was invalid. Error returned was: %su
SSLContextt contextu hostu portu cert_fileu key_filec @ s � S( N( ( ( t method( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyt <lambda>� s u Erroru Content-Typeu text/u charsetu client_certu
client_keyu ca_certsu
http_max_bodyu http_connect_timeoutu http_request_timeoutu
proxy_hostu
proxy_portu proxy_usernameu proxy_passwordu no_proxyuy proxy_host and proxy_port has been set. This requires pycurl and tornado, but the libraries does not seem to be installedu+ tornado.curl_httpclient.CurlAsyncHTTPClientu
max_body_sizeu methodu headersu
auth_usernameu
auth_passwordu
validate_certu allow_nonstandard_methodsu streaming_callbacku header_callbacku connect_timeoutu request_timeoutu raise_erroru decompress_responset to_strt
max_body_sizeu statusu
Set-Cookieu Response Status Code: %su Response Headers: %su Response Cookies: %su Content-Encodingu identityu Response Text: %suf Cannot Trace Log Response Text: %s. This may be due to incompatibilities between requests and logging.u wu
set-cookieu Cookieu autou content-typeu application/jsonu xmlu jsonu yamlu plainu: Invalid decode_type specified. Valid decode types are: {0}u dictu text( u jsonu xmlu yamlu plain(� t Nonet saltt configt
master_configt ost patht joint syspathst
CONFIG_DIRt
minion_configt gett ret matcht utilst networkt refresh_dnst HAS_REQUESTSt FalseR t errort loggingt getLoggert setLevelt WARNINGt tornadot httputilt
url_concatt
get_ca_bundlet Truet _rendert sanitize_urlt debugt isEnabledFort TRACEt
isinstancet dictt copyt listR t pprintt pformatt
splitlinest CACHE_DIRt HAS_MSGPACKt isfilet filest fopent msgpackt loadt updatet dumpt splitR t stript USERAGENTt formatt requestst Sessiont autht headerst cookiest verifyt extR t movest http_cookiejart MozillaCookieJart LWPCookieJart savet __version__t string_typest existst requestt raise_for_statust status_codet contentt urlt text_typet decodet encodingt urllib_requestt Requestt HTTPHandlert HTTPCookieProcessort
startswitht get_hostt HTTPSHandlert HAS_MATCHHOSTNAMEt warningt sockett AF_INETt SOCK_STREAMt connectt intt sslt wrap_sockett
CERT_REQUIREDR t getpeercertR t hasattrt
SSLContextt load_cert_chaint appendt http_clientt HTTPSConnectiont build_openert
add_headert
get_methodt openR t codet infoR t cgit parse_headert PY3t bytest
_urlencodet DEFAULT_MINION_OPTSt stringutilsR' R t hostnamet HAS_CURL_HTTPCLIENTt
httpclientt AsyncHTTPClientt configuret argst get_function_argspect curl_httpclientt CurlAsyncHTTPClientt
initializet simple_httpclientt SimpleAsyncHTTPClientR! R
t fetcht HTTPErrort gaierrort bodyt parse_cookie_headert
set_cookieR t UnicodeEncodeErrort writet jsont loadst ETt
fromstringt xmlt to_dictt yamlt safe_load(c Rq R% R R! t data_filet header_dictt header_listt header_filet usernamet passwordR` Rs t decode_typet statusRa t textRb t
cookie_jart
cookie_formatt persist_sessiont session_cookie_jart data_rendert
data_renderert
header_rendert header_renderert
template_dictt testt test_urlt nodet portt optst backendt ca_bundlet
verify_sslt certt text_outt headers_outt
decode_outt streamt streaming_callbackt header_callbackt handlet agentt hide_fieldst raise_errort kwargst retR5 t requests_logt url_fullt log_urlt log_datat itemt fieldt
header_tplt fh_t session_cookiest headert compst sesst sess_cookiest
req_kwargst resultt result_status_codet result_headerst result_textt result_cookiesR� Rm t handlersR� t sockt sockwrapt exct
cert_chainR$ t cert_kwargst openert res_content_typet
res_paramst max_bodyt connect_timeoutt timeoutt client_argspect
proxy_hostt
proxy_portt proxy_usernamet proxy_passwordt no_proxyt supports_max_body_sizet download_clientt result_headers_dictR t toft hoft content_typet
valid_decodest itemst dof( ( R% s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyt query� s� 2 ! !
!
* *
"
,
,
%
,
+
! +
c C@ s- t t d � r t j S| d
k r+ i } n | j d d
� } | d
k r_ t j j | � r_ | S| j d i t j j
g d 6� } xV | j d g � D]B } x9 d D]1 } t j j | | � } t j j | � r� | Sq� Wq� Wd } x$ | D] } t j j | � r� | Sq� Wt j j
j � r)t r)t j � Sd
S( us
Return the location of the ca bundle file. See the following article:
http://tinyurl.com/k7rx42a
u __return_value__u ca_bundleu
file_rootsu baseu
cacert.pemu
ca-bundle.crtu" /etc/ssl/certs/ca-certificates.crtu /etc/pki/tls/certs/ca-bundle.crtu& /etc/pki/tls/certs/ca-bundle.trust.crtu /etc/ssl/certs/ca-bundle.crtu& /var/lib/ca-certificates/ca-bundle.pemu /etc/ssl/cert.pemN( u
cacert.pemu
ca-bundle.crt( u" /etc/ssl/certs/ca-certificates.crtu /etc/pki/tls/certs/ca-bundle.crtu& /etc/pki/tls/certs/ca-bundle.trust.crtu /etc/ssl/certs/ca-bundle.crtu& /var/lib/ca-certificates/ca-bundle.pemu /etc/ssl/cert.pem( R� RC t __return_value__R) R3 R- R. Rl R* R0 t SRV_ROOT_DIRR/ R6 t platformt
is_windowst HAS_CERTIFIt certifit where( R� t opts_bundlet
file_rootst salt_rootR. t cert_patht locations( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyRC � s2 "
c C@ s# | d k r i } n | d k r0 t | � } n | d k rM t j d � d S| d k rn | j d d � } n t j d | | � t | d t d t d t d t d
| �| d k r t | t
j � r� | g } n t | t � s� t j d � d Sd } x� | D]� } t
j j | � r t j d
| | � y@ t j j j | d � �" } d j | | j � f � } Wd QXWq�t k
r�} t j d | | � q�Xq q W| r t j d | � y? t j j j | d � �! } | j d � | j | � Wd QXWq t k
r} t j d | | � q Xq n d S( u.
Attempt to update the CA bundle file from a URL
If not specified, the local location on disk (``target``) will be
auto-detected, if possible. If it is not found, then a new location on disk
will be created and updated.
The default ``source`` is:
http://curl.haxx.se/ca/cacert.pem
This is based on the information at:
http://curl.haxx.se/docs/caextract.html
A string or list of strings representing files to be appended to the end of
the CA bundle file may also be passed through as ``merge_files``.
u/ Unable to detect location to write CA bundle toNu
ca_bundle_urlu! http://curl.haxx.se/ca/cacert.pemu Attempting to download %s to %sR� Rs Ra R� R� uI A value was passed as merge_files which was not either a string or a listu u# Queueing up %s to be appended to %su ru
u. Reading from %s caused the following error: %su Appending merge_files to %su au, Writing to %s caused the following error: %s( R) RC R R; R3 RG R RD R: RJ R Rk RM R- R. Rl R* R6 RT RU R/ R t IOErrorR� ( t targett sourceR� t merge_filest
merge_contentt cert_filet fcfR� t tfp( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyt update_ca_bundle� s\
%
c
C@ s. | r� | d
k r i } n | s6 | j d d � } n t j j | i � } | j d � } | j d � } t | | | | | | � } t j j j | � r� | j � } n t
j | � j d � r� t
j | � j d � r� t
j | � j
d d � d } n | St j j j | d � � } | j � SWd
QXd
S( u
Render a template
u rendereru
jinja|yamlu renderer_blacklistu renderer_whitelistu #!u #!/u
i u rN( R) R3 R* t loadert renderR R6 t stringiot is_readableR R Rr Ry RZ RT RU (
t templateR" t rendererR� R� t rendt blacklistt whitelistR� R� ( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyRE D s 1"c C@ s� d } g } xr | j d � D]a } | j � } d
| k rp d | k rp x4 | j d
� D] } | j | � qV Wq | j | � q Wg } i } t } x� | D]� } | j d � } | d j � | k r,| d | k r� | j | � i } n t | � d
k rd j | d
� | | d <qut | | d <q� | t k rT| j | � i } t } n d j | d
� | | d <t } q� W| r�| j | � n d }
g } xO| D]G} d } d }
x; t | � D]- } | | k r�q�n | } | j
| � }
q�Wd | k r$t j j
j j j | d � | d <n x' |
D] } | | k r+d | | <q+q+W| d d k rkd | d <n | d d k r�i | d <n | d d k r�d | d <n d | k r�| d =n | j t j j
j j j d | d |
| � � q�W| S( u�
Parse the "Set-cookie" header, and return a list of cookies.
This function is here because Tornado's HTTPClient doesn't handle cookies.
u expiresu pathu domainu versionu httponlyu secureu commentu max-ageu ;u ,u =i i u portu port_specifiedu domain_specifiedu domain_initial_dotu path_specifiedu discardu comment_urlu restu t namet value( u expiresu pathu domainu versionu httponlyu secureu commentu max-age( u versionu portu port_specifiedu domainu domain_specifiedu domain_initial_dotu pathu path_specifiedu secureu expiresu discardu commentu comment_urlu restN( RZ R[ R� R: t lowerR R/ RD R) RM t popR* Rd R Re Rf t http2timet Cookie( R� t attribst morselsR� t partRb t cookiet value_sett morselt partst reqdR� R* R+ t req( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyR� Z s�
&
2c C@ s� t | t � r� t | � } | d } t | � d k rD | d 7} n xx | d D]l } d } xS | D]K } | j d � } | r� | j d � } t | | � } qb t | | � } qb W| | 7} qO W| j d � St j | � Sd S( u4
Make sure no secret fields show up in logs
i i u ?u &N(
RJ RM R
R R) RZ t _sanitize_url_componentst rstripR Rr ( Rq R� t url_compsR� t pairt url_tmpR� t
comps_list( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyRF � s
c C@ s� t | � d k r d S| d j d j | � � rc d j | � } | j | d � | t | | � Sd j | d � } | j | d � | t | | � Sd S( uC
Recursive function to sanitize each component of the url.
i u u {0}=u {0}=XXXXXXXXXX&u {0}&N( R Ry R] t removeR9 ( t comp_listR� R� ( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyR9 � s (] t __doc__t
__future__R R R R� R< R- RN R~ R R R R4 R� R R RD R| t ImportErrort backports.ssl_match_hostnamet salt.ext.ssl_match_hostnameR: t salt.configR* t salt.loadert
salt.syspathst salt.utils.argst salt.utils.datat salt.utils.filest salt.utils.jsont salt.utils.networkt salt.utils.platformt salt.utils.stringutilst salt.utils.yamlt salt.versiont salt.utils.xmlutilR6 t xmlutilR� t salt._compatR R� t
salt.templateR t salt.utils.decorators.jinjaR t salt.extR t salt.ext.six.moves.http_clientt! salt.ext.six.moves.http_cookiejart! salt.ext.six.moves.urllib.requestRd Re t urllibRm Ru t salt.ext.six.moves.urllib.errorR t salt.ext.six.moves.urllib.parseR
R R R� t tornado.httputilR@ t tornado.simple_httpclientt tornado.httpclientR
t tornado.curl_httpclientR� R^ R9 RV RR R R R= t __name__R R] t versionRj R\ R R) R RC R RE R� RF R9 ( ( ( s3 /usr/lib/python2.7/site-packages/salt/utils/http.pyt <module> s
� �
3 R _
Mini Shell