Current File : //lib/python2.7/site-packages/salt/utils/aws.pyo
�
���^c @@ sJ d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l m Z d d l Z d d l
Z
d d l Z d d l Z
d d l Z d d l Z d d l m Z d d l Z
d d l j j Z d d l m Z y d d l Z e Z Wn e k
r e Z n Xd d l m Z m! Z! m" Z" d d l# m$ Z$ m% Z% e j& e' � Z( d Z) d
Z* d d d
d d d g Z+ d Z, d Z- d Z. d a/ d a0 d a1 d a2 d a3 i Z4 d � Z5 d � Z6 d � Z7 d d � Z9 e* d d d d d d d d d � Z: d � Z; d � Z< d d d d e e d d d d d d � Z= d � Z> d d d! � Z? d S(" u�
Connection library for AWS
.. versionadded:: 2015.5.0
This is a base library used by a number of AWS services.
:depends: requests
i ( t absolute_importt print_functiont unicode_literalsN( t datetime( t six( t ElementTree( t mapt ranget zip( t urlencodet urlparseu us-east-1u
2014-10-01u RequestLimitExceededu InsufficientInstanceCapacityu
InternalErroru Unavailableu InsufficientAddressCapacityu$ InsufficientReservedInstanceCapacitygffffff@i u use-instance-role-credentialsu c C@ s! t j t j d d | � � d S( uz
backoff an exponential amount of time to throttle requests
during "API Rate Exceeded" failures as suggested by the AWS documentation here:
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/query-api-troubleshooting.html
and also here:
https://docs.aws.amazon.com/general/latest/gr/api-retries.html
Failure to implement this approach results in a failure rate of >30% when using salt-cloud with
"--parallel" when creating 50 or more instances with a fixed delay of 2 seconds.
A failure rate of >10% is observed when using the salt-api with an asynchronous client
specified (runner_async).
i i N( t timet sleept randomt uniform( t attempts( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyt sleep_exponential_backoffB s c C@ s d } | d t k s& | d t k r�t d k ri t j � } | j d � } | t k ri t t t f Sn y9 t j d d i d d 6d t
�} | j � | j } Wn5 t j
j t j
j f k
r� | d | d d f SXy9 t j d j | � d i d d 6d t
�} | j � Wn5 t j
j t j
j f k
rJ| d | d d f SX| j � } | d
a | d a | d a | d
a t t t f } n | d | d d f } | j d � d k r| j � } | j d d � t j d | j d � � t | d | j d � d d �} n | S( uB
Return the credentials for AWS signing. This could be just the id and key
specified in the provider configuration, or if the id or key is set to the
literal string 'use-instance-role-credentials' creds will pull the instance
role credentials from the meta data, cache them, and provide them instead.
u idu keyu u %Y-%m-%dT%H:%M:%SZuA http://169.254.169.254/latest/meta-data/iam/security-credentials/t proxiesu httpt timeoutuD http://169.254.169.254/latest/meta-data/iam/security-credentials/{0}u AccessKeyIdu SecretAccessKeyu Tokenu
Expirationu role_arnu Assuming the role: %st role_arnt locationu us-east-1( N( t
IROLE_CODEt __Expiration__R t utcnowt strftimet __AccessKeyId__t __SecretAccessKey__t __Token__t requestst gett AWS_METADATA_TIMEOUTt raise_for_statust textt
exceptionst HTTPErrort ConnectionErrort formatt jsont Nonet copyt popt logt infot
assumed_creds( t providert ret_credentialst timenowt timestampt resultt rolet datat provider_shadow( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyt credsQ sF
$c C@ sL t j � } | j d � } t | � \ } } } | j � }
| |
d <d |
d <d |
d <d j | � |
d <| |
d <t |
j � � } t t t |
j
| � � � } t t t | | � � � }
d
j | j
d � | j
d � |
j
d � � } t j | | t j � } t j | j � � } | j � |
d <| d
k rH| |
d <n |
S( u�
Sign a query against AWS services using Signature Version 2 Signing
Process. This is documented at:
http://docs.aws.amazon.com/general/latest/gr/signature-version-2.html
u %Y-%m-%dT%H:%M:%SZu AWSAccessKeyIdu 2u SignatureVersionu
HmacSHA256u SignatureMethodu {0}u Timestampu Versionu
{0}
{1}
/
{2}u utf-8u Signatureu u
SecurityToken( R R R R4 R' R$ t sortedt keyst listR R R R t encodet hmact newt hashlibt sha256t binasciit
b2a_base64t digestt strip( t methodt endpointt paramsR, t aws_api_versionR. R/ t
access_key_idt secret_access_keyt tokent params_with_headersR6 t valuest querystringt canonicalt hashedt sig( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyt sig2� s,
c C@ s� t j d � } t j t j � j � � } x> t j � D]0 \ } } | d | d k r7 t j | � q7 q7 W| t k r� t | } | d | d | d f Sd } | j
d t j j
i d d 6� d
� d
d !} t d d
d i | d 6d d 6| d 6| d 6d d 6d d 6d | d d d d d | d d d | d d! �\ }
} d" |
d# <t j d | d$ |
d d d% t �} | j d& k r�t j d' | j � n | j � | j � }
|
d( d) d* } | t | <| d | d | d f S(+ Nu [^a-z0-9A-Z+=,.@-]u
Expirationix u AccessKeyIdu SecretAccessKeyu SessionTokenu
2011-06-15u u root_diri i? u GETu sts.amazonaws.comRC u Versionu
AssumeRoleu Actionu RoleSessionNameu RoleArnud {"Version":"2012-10-17","Statement":[{"Sid":"Stmt1", "Effect":"Allow","Action":"*","Resource":"*"}]}u Policyu 3600u DurationSecondsRD R2 t uriu /t prov_dictt productu stsR t
requesturlu https://sts.amazonaws.com/u application/jsonu Acceptt headerst verifyi� u AssumeRole response: %su AssumeRoleResponseu AssumeRoleResultu Credentials( t ret compileR t mktimeR R t timetuplet __AssumeCache__t itemst deletet subt saltt configt get_idR& t sig4R t requestt Truet status_codeR) R* t contentR R% ( RP R R t valid_session_name_ret nowt keyR4 t ct versiont session_nameRS RR R0 t respR2 ( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyR+ � sL
0
u ec2u /c
% C@ s t j � }
| d k r0 t | � \ } } } n t | | d | �\ } } } | d k rf t � } n | d k r{ t } n | j � } | d k r� | | d <n t | j � � } t
t | j | � � } t
t
t | | � � � j d d � } |
j d � } |
j d � } i } t |
t � r3|
j � } n | sQt j j j | � } n | | d <| | d
<| | d <g } g } | d k r�| | d
<n xb t | j � d t j j �D]B } | j � } | j d j | | | j � � � | j | � q�Wd j | � d } d j | � } d } d j | | | | | | f � } d j | | | d f � } d j | | | t j j j | � f � }! t | | | | � }" t j |" |! j! d � t" j# � j$ � }# d j | | | | |# � }$ |$ | d <d j | | � } | | f S( u_
Sign a query against AWS services using Signature Version 4 Signing
Process. This is documented at:
http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
R u s3u ssmu Versionu +u %20u %Y%m%dT%H%M%SZu %Y%m%du
X-Amz-dateu hostu x-amz-content-sha256u u X-Amz-security-tokenRg u {0}:{1}u
u ;u AWS4-HMAC-SHA256u /u aws4_requestu utf-8u8 {0} Credential={1}/{2}, SignedHeaders={3}, Signature={4}u
Authorizationu {0}?{1}N( u s3u ssm(% R R R& R4 R+ t get_region_from_metadatat DEFAULT_LOCATIONR' R5 R6 R7 R R R R t replaceR t
isinstancet dictR] t utilst hashutilst
sha256_digestR t text_typet lowert appendR$ R@ t joint _sig_keyR9 R: R8 R; R< t hexdigest(% RA RB RC RP RD R RQ RO RR R2 RS R t payload_hashR. RE RF RG RH R6 RI RJ t amzdatet datestampt new_headerst a_canonical_headerst a_signed_headerst headert lower_headert canonical_headerst signed_headerst algorithmt canonical_requestt credential_scopet string_to_signt signing_keyt signaturet authorization_header( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyR` � s�
'
% #
c C@ s% t j | | j d � t j � j � S( u�
Key derivation functions. See:
http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
u utf-8( R9 R: R8 R; R< R? ( Rg t msg( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyt _signZ s c C@ se t d | j d � | � } | rC t | | � } t | | � } n t | | � } t | d � } | S( u�
Get a signature key. See:
http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
u AWS4u utf-8u aws4_request( R� R8 ( Rg t
date_stampt
regionNamet serviceNamet kDatet kRegiont kServicet kSigning( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyRx c s u 2c
C@ s� | d$ k r i } n | d$ k r* i } n | j d d$ | f � } | j d i � } | d$ k r� | j | d i � j | i � }
|
r� t t |
j � � � d } | j | | � } q� n | j | i � j | i � }
|
j d d � } | st | |
� } n | d$ k r�| sI|
j d d j | | | � � } d j | � } q�t | � j } | d
k r�d j | � } t j | � | t
k r�i | d 6| f Si | d 6Sn t j d
| � d } |
j d |
j d j | � t � � } |
j d d$ � sY| j | i � j d i � j d i � |
d <| j | i � j d i � j d i � |
d <n |
d k r�t
| | | |
| | | d | �\ } } i } n t | | | |
| � } i } d } x�| t k r@t j d | � t j d | � yP t j | d | d | �} t j d | j � t j d | j � | j � PWq�t j j k
r<} t j | j j � } t j | � } | j d i � j d i � j d d
� } | t k r�| r�| t k r�| d 7} t j d | j j | | | � t | � q�n t j d | j j | | � | t
k r1i | d 6| f Si | d 6SXq�Wt j d | j j | | � | t
k rzi | d 6| f Si | d 6St j | j � } | d } | t
k r�| } n | r<t j d% k r�t! | j" � � } n t! | � } xM t# d | � D]9 } | | j$ j% d# � } | d | k r�| | } q�q�Wn g } x$ | D] } | j& t j | � � qIW| t
k r| | f S| S(& u�
Perform a query against AWS services using Signature Version 2 Signing
Process. This is documented at:
http://docs.aws.amazon.com/general/latest/gr/signature-version-2.html
Regions and endpoints are documented at:
http://docs.aws.amazon.com/general/latest/gr/rande.html
Default ``product`` is ``ec2``. Valid ``product`` names are:
.. code-block: yaml
- autoscaling (Auto Scaling)
- cloudformation (CloudFormation)
- ec2 (Elastic Compute Cloud)
- elasticache (ElastiCache)
- elasticbeanstalk (Elastic BeanStalk)
- elasticloadbalancing (Elastic Load Balancing)
- elasticmapreduce (Elastic MapReduce)
- iam (Identity and Access Management)
- importexport (Import/Export)
- monitoring (CloudWatch)
- rds (Relational Database Service)
- simpledb (SimpleDB)
- sns (Simple Notification Service)
- sqs (Simple Queue Service)
u functionu providersi i u service_urlu
amazonaws.comu endpointu {0}.{1}.{2}u https://{0}/u ur Could not find a valid endpoint in the requesturl: {0}. Looking for something like https://some.aws.endpoint/?argsu erroru Using AWS endpoint: %su GETu aws_api_versionu {0}_api_versionu idu ec2u keyu 4RR u AWS Request: %su AWS Request Parameters: %sRS RC u AWS Response Status Code: %su AWS Response Text: %su Errorsu Erroru CodeuF AWS Response Status Code and Error: [%s %s] %s; Attempts remaining: %su. AWS Response Status Code and Error: [%s %s] %si i u }N( i i (' R& R R7 R6 t get_locationR$ R
t netlocR) t errorRb t debugt DEFAULT_AWS_API_VERSIONR` RN t AWS_MAX_RETRIESt traceR Rc R R R! R" t ETt
fromstringt responseRd t xmlt to_dictt AWS_RETRY_CODESR t syst version_infot lent getchildrenR t tagt splitRv ( RC t setnameRR R t
return_urlt return_roott optsR, RB RQ t sigvert functiont providersRP t drivert service_urlt endpoint_errRA RD RS RH R R0 t exct rootR2 t err_codeRZ t children_lent itemt compst ret( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyt querys s� "
.1 '
*
c C@ s� t d k r t j d � d St d k r- t Sy& t j d d i d d 6d t �} Wn1 t j j k
r� t j d d t
�d a d SXy | j � d
} | a t SWn% t t
f k
r� t j d � d SXd S(
uh
Try to get region from instance identity document and cache it
.. versionadded:: 2015.5.6
u do-not-get-from-metadatauD Previously failed to get AWS region from metadata. Not trying again.u u@ http://169.254.169.254/latest/dynamic/instance-identity/documentR u httpR u0 Failed to get AWS region from instance metadata.t exc_infou regionu- Failed to decode JSON from instance metadata.N( t __Location__R) R� R& R R R R! t RequestExceptiont warningRb R% t
ValueErrort KeyError( R0 t region( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyRl s*
c C@ s | d k r i } n | j d � } | d k rN | d k rN | j d � } n | d k rf t � } n | d k r{ t } n | S( u�
Return the region to use, in this order:
opts['location']
provider['location']
get_region_from_metadata()
DEFAULT_LOCATION
u locationN( R& R Rl Rm ( R� R, R� ( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyR� F s (@ t __doc__t
__future__R R R R� R R= R R; R9 t loggingt salt.configR] RU R
t salt.extR t salt.utils.hashutilst salt.utils.xmlutilRq t xmlutilR� t salt._compatR R� R Rb t HAS_REQUESTSt ImportErrort Falset salt.ext.six.movesR R R t salt.ext.six.moves.urllib.parseR R
t getLoggert __name__R) Rm R� R� R R� R R R R R R� RY R R4 RN R& R+ R` R� Rx R� Rl R� ( ( ( s2 /usr/lib/python2.7/site-packages/salt/utils/aws.pyt <module>
sl
? ( 4l � '
Mini Shell