Current File : //lib/python2.7/site-packages/salt/transport/zeromq.pyc
�
���^c @@ s� d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l Z d d l Z d d l
Z
d d l Z d d l m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l! m" Z" d d l# m$ Z$ m% Z% d d l& m' Z' d d l m( Z( m) Z) m* Z* m+ Z+ m, Z, d d l- Z( d d l. Z( d d l/ Z( y d d l0 Z( e1 Z2 Wn e3 k
re4 Z2 n Xd d l5 Z5 d d l6 Z5 d d l7 Z5 y d d l8 m9 Z9 e1 Z: WnO e3 k
r�e4 Z: y d d
l; m< Z< Wn! e3 k
r�d d
l= m< Z< n Xn Xe j> e? � Z@ eA eA d � ZB d e jC jD jE f d
� � YZF d e jC jG jH jI e jC jD jJ f d � � YZK d e jC jG jH jL e jC jM jN f d � � YZO d � ZP d e jC jM jQ f d � � YZR d e jC jS f d � � YZT d eU f d � � YZV d eU f d � � YZW d S( u
Zeromq transport classes
i ( t absolute_importt print_functiont unicode_literalsN( t randint( t six( t SaltReqTimeoutErrort
SaltException( t ipaddress( t zmqt ZMQDefaultLoopt install_zmqt ZMQ_VERSION_INFOt LIBZMQ_VERSION_INFO( t RSA( t
PKCS1_OAEPc C@ s` d d l m } d j d | | � d | � } | s: | r\t d k r2t d k r2| r� | r� d
j d | | � d | d | | � d | � } qY| r� | r� d
j d | | � d | | � d | � } qY| rY| rYt j | � j d k r� d n | d � } d j d | d | d | | � d | � } qYq\t j d � t j d � t j d � n | S( uR
Return the ZeroMQ URI to connect the Minion to the Master.
It supports different source IP / port, given the ZeroMQ syntax:
// Connecting using a IP address and bind to an IP address
rc = zmq_connect(socket, "tcp://192.168.1.17:5555;192.168.1.1:5555"); assert (rc == 0);
Source: http://api.zeromq.org/4-1:zmq-tcp
i ( t
ip_bracketu tcp://{master_ip}:{master_port}t master_ipt master_porti i i i u9 tcp://{source_ip}:{source_port};{master_ip}:{master_port}t source_ipt source_portu- tcp://{source_ip}:0;{master_ip}:{master_port}u 0.0.0.0u ::u6 tcp://{ip_any}:{source_port};{master_ip}:{master_port}t ip_anyuA Unable to connect to the Master using a specific source IP / portu9 Consider upgrading to pyzmq >= 16.0.1 and libzmq >= 4.1.6uV Specific source IP / port for connecting to master returner port: configuraion ignored( i i i ( i i i (
t salt.utils.zeromqR t formatR R R t
ip_addresst versiont logt warning( R R R R R t
master_uriR ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _get_master_uriF s,
*
t AsyncZeroMQReqChannelc B@ s� e Z d Z e j � Z d � Z d � Z e d � � Z d � Z
d � Z d � Z d � Z
e d � � Z d � Z e j j d d
d d � � Z e j j d
d e d
� � Z e j j d
d d � � Z e j j d
d e d � � Z RS( uZ
Encapsulate sending routines to ZeroMQ.
ZMQ Channels default to 'crypt=aes'
c K@ sB | j d � } | d k r1 t � t j � } n | | j k rV t j � | j | <n | j | } | j | | � } | j | � } | d k rt j
d | � t j | � } | j
| | � | | _ | | | <d | _ t j � | _ t j d t | � | t j � � n/ | j � | j d 7_ Wd QXt j
d | � | S( uA
Only create one instance of channel per __key()
u io_loopu- Initializing new AsyncZeroMQReqChannel for %si uC Inserted key into loop_instance_map id %s for key %s and process %sNu% Re-using AsyncZeroMQReqChannel for %s( t gett NoneR
R t currentt instance_mapt weakreft WeakValueDictionaryt _AsyncZeroMQReqChannel__keyR t debugt objectt __new__t __singleton_init__t
_instance_keyt _refcountt threadingt RLockt _refcount_lockt tracet idt ost getpid( t clst optst kwargst io_loopt loop_instance_mapt keyt obj( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR' y s.
c C@ s� | j } | j | t j | j | � � } | | t | � <x� | j D]� } | d k r\ qD n | d k r� t | | t | j d | j | j f d i | j
d 6�� qD n t | | t j | j | | � � qD W| S( Nu _io_loopu _refcountu _refcount_locku message_clientt argsR4 u io_loop( u _io_loopu _refcountu _refcount_lock( t __class__R' t copyt deepcopyR3 R/ t __dict__t setattrt AsyncReqMessageClientPoolR t _io_loop( t selft memoR2 t resultR7 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt __deepcopy__� s ! 'c K@ s9 | d | d | j d | j d � � | j d d � f S( Nu pki_diru idu
master_uriu cryptu aes( R ( R2 R3 R4 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt __key� s c K@ s d S( N( ( RA R3 R4 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt __init__� s c K@ s t | � | _ d | _ | j d d � | _ d | k rM | d | j d <n | j d � | _ | j d k r� t � t j � | _ n | j d k r� t
j j | j d | j �| _ n t
j d | j � t | j d | j | j f d
i | j d 6�| _ t | _ d S( Nu zeromqu cryptu aesu
master_uriu io_loopu clearR5 uC Connecting the Minion to the Master URI (for the return server): %sR9 R4 ( t dictR3 t ttypeR t cryptR@ R R
R R t saltt AsyncAutht authR R% R R? t message_clientt Falset _closing( RA R3 R4 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR( � s $ c C@ s� | j r
d S| j d k rU | j � | j d 8_ Wd QXt j d | j j � d St j d | j j � t | _ t | d � r� | j j
� n | j | j j k r� | j j | j } | j
| k r� | | j
=n | s� | j j | j =q� n d S( u�
Since the message_client creates sockets and assigns them to the IOLoop we have to
specifically destroy them, since we aren't the only ones with references to the FDs
Ni u2 This is not the last %s instance. Not closing yet.u Closing %s instanceu message_client( RO R* R- R R% R: t __name__t Truet hasattrRM t closeR@ R! R) ( RA R6 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRS � s&
c C@ s\ | j � d | _ Wd QXy | j � Wn. t j k
rW } | j t j k rX � qX n Xd S( Ni ( R- R* RS t sockett errort errnot EBADF( RA t exc( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt __del__� s
c C@ sx d | j k r | j d Sd | j k rh t | j d | j d d | j j d � d | j j d � �St d � � d S( Nu
master_uriu master_ipu master_portR u source_ipR u source_ret_portu5 ReqChannel: missing master_uri/master_ip in self.opts( R3 R R R ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR s
c C@ s i | j d 6| d 6S( Nu encu load( RI ( RA t load( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt
_package_load s
i i< c c@ s] | j j s | j j � Vn | j j | j | j j j | � � d | d | �V} | j j � } d | k r� | j j � V| j j | j | j j j | � � d | d | �V} n t r� | j
| d t j � } n" t
j | � } | j | d � } t j j | j | � } | j | | � }
t j rGt j j j |
� }
n t j j |
� � d S( Nt timeoutt triesu key( RL t
authenticatedt authenticateRM t sendR[ t crypticlet dumpst get_keyst HAS_M2t private_decryptR
t pkcs1_oaep_paddingR t newt decryptRJ RI t CrypticleR3 t loadsR t PY3t transportt framet decode_embedded_strst tornadot gent Return( RA RZ t dictkeyR] R\ t retR7 t aest ciphert pcryptt data( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt! crypted_transfer_decode_dictentry s.
c #@ s� t j j � � � � � f d � � } � j j sD � j j � Vn y | � V} Wn/ t j j k
r� � j j � V| � V} n Xt j j | � � d S( u�
Send a load across the wire, with encryption
In case of authentication errors, try to renegotiate authentication
and retry the method.
Indeed, we can fail too early in case of a master restart during a
minion state execution call
:param dict load: A load to send across the wire
:param int tries: The number of times to make before failure
:param int timeout: The number of seconds on a response before failing
c 3@ s� � j j � j � j j j � � � d � d � �V} | rX � j j j | � � } n t j r� � r� t j
j j | � } n t
j j | � � d S( NR\ R] ( RM R` R[ RL Ra Rb Rj R Rk RJ Rl Rm Rn Ro Rp Rq ( Rw ( RZ t rawRA R\ R] ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _do_transferI s
N(
Ro Rp t coroutineRL R^ R_ RJ RI t AuthenticationErrorRq ( RA RZ R] R\ Ry Rz Rs ( ( RZ Ry RA R\ R] s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _crypted_transfer: s ' c c@ s> | j j | j | � d | d | �V} t j j | � � d S( u
Send a load across the wire in cleartext
:param dict load: A load to send across the wire
:param int tries: The number of times to make before failure
:param int timeout: The number of seconds on a response before failing
R\ R] N( RM R` R[ Ro Rp Rq ( RA RZ R] R\ Rs ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _uncrypted_transferf s
c c@ sf | j d k r. | j | d | d | �V} n" | j | d | d | d | �V} t j j | � � d S( u^
Send a request, return a future which will complete when we send the message
u clearR] R\ Ry N( RI R~ R} Ro Rp Rq ( RA RZ R] R\ Ry Rs ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR` w s "N( RP t
__module__t __doc__R" t WeakKeyDictionaryR! R' RD t classmethodR$ RF R( RS RY t propertyR R[ Ro Rp R{ R Rx RN R} R~ R` ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR o s& # " + t AsyncZeroMQPubChannelc B@ s� e Z d Z d � Z d � Z d � Z d � Z e j j d � � Z
e d � � Z e j j d � � Z
e d � � Z d � Z RS(
uw
A transport channel backed by ZeroMQ for a Salt Publisher to use to
publish commands to connected minions
c K@ s� | | _ d | _ | j d � | _ | j d k rL t � t j � | _ n t j t
j j j
| j d � � j � | _ t
j j | j d | j �| _ t
j j | j � | _ t j � | _ | j j t j � | _ | j d rY| j j t j d � | j j d � d k r.| j j t j d � qo| j j t j t
j j j
| j � � n | j j t j d
� | j j t j t
j j j
| j d � � t t d � r!| j j t j | j d � | j j t j! | j d
� | j j t j" | j d � | j j t j# | j d � n | j d } | j d r�t$ | j d | j d | j d � } t% j& d | j d | j d | j d | � n t% j& d | � | j j t j' | � t t d � r t% j& d | j d | j d � | j j t j( | j d � n | j d t) k s2d | j d k rZt t d � rZ| j j t j* d � n t+ r�| j d r�t, | j � | _- | j- j. | j � n d S( Nu zeromqu io_loopu idR5 u
zmq_filteringt broadcastu __roleu syndict syndict u
TCP_KEEPALIVEu
tcp_keepaliveu tcp_keepalive_idleu tcp_keepalive_cntu tcp_keepalive_intvlu
recon_defaultu recon_randomizeu recon_maxu? Generated random reconnect delay between '%sms' and '%sms' (%s)u# Setting zmq_reconnect_ivl to '%sms'u RECONNECT_IVL_MAXu' Setting zmq_reconnect_ivl_max to '%sms'u ipv6u :u master_ipu IPV4ONLYi u zmq_monitor(/ R3 RH R R5 R R
R R t hashlibt sha1RJ t utilst stringutilst to_bytest hexdigestt hexidRI RK RL t payloadt Serialt serialR t Contextt contextRT t SUBt _sockett
setsockoptt SUBSCRIBEt IDENTITYRR t
TCP_KEEPALIVEt TCP_KEEPALIVE_IDLEt TCP_KEEPALIVE_CNTt TCP_KEEPALIVE_INTVLR R R% t
RECONNECT_IVLt RECONNECT_IVL_MAXRQ t IPV4ONLYt HAS_ZMQ_MONITORt ZeroMQSocketMonitort _monitort
start_io_loop( RA R3 R4 t recon_delay( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRF � sf .!
,
5c C@ s� t | d � r7 | j d k r7 | j j � d | _ n t | d � r� t d k r� | j j j | j j � | j j j d � q� | j j d � n" t | d � r� | j
j d � n t | d � r� | j j t
k r� | j j � n d S( Nu _monitoru _streami i i u _socketu context( i i i ( RR R� R t stopR t _streamR5 t remove_handlerRT RS R� R� t closedRN t term( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRS � s
!c C@ s9 t j j j d d j | j j � d d �| j � d S( Nu SodiumuE Calling {0}.destroy() is deprecated. Please call {0}.close() instead.t
stackleveli ( RJ R� t versionst
warn_untilR R: RP RS ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt destroy� s c C@ s | j � d S( N( RS ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRY � s c c@ sZ | j j s | j j � Vn | j j d | _ t j d | j � | j j | j � d S( Nu publish_portuC Connecting the Minion to the Master publish port, using the URI: %s(
RL R^ R_ t credst publish_portR R% t
master_pubR� t connect( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s
c C@ s; t | j d | j d | j j d � d | j j d � �S( u0
Return the master publish port
u master_ipR u source_ipR u source_publish_port( R R3 R� R ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� s
c c@ s& t | � } | d k r1 | j j | d � } n� | d k r� t j j j | d � } | j j d � d k r� | d | j f k s� | j j d � d k r� | d
k r� t
j d | � t j
j d � � n | j j | d � } n t d j t | � � � � | j | � V} t j
j | � � d S( u�
Take the zmq messages, decrypt/decode them into a payload
:param list messages: A list of messages to be decoded
i i i u __roleu syndicu broadcastu( Publish received for not this minion: %suA Invalid number of messages ({0}) in zeromq pubmessage from masterN( u broadcastu syndic( t lenR� Rj RJ R� R� t to_strR3 R R� R R% Ro Rp Rq R t ExceptionR t _decode_payload( RA t messagest messages_lenR� t message_targetRs ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _decode_messages
s -$ c C@ s= t | d � s6 t j j j | j d | j �| _ n | j S( uI
Return the current zmqstream, creating one if necessary
u _streamR5 ( RR R t eventloopt zmqstreamt ZMQStreamR� R5 R� ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt stream% s 'c @ sJ � d k r � j j d � St j j � � f d � � } � j j | � S( u�
Register a callback for received messages (that we didn't initiate)
:param func callback: A function which should be called when data is received
c 3@ s- � j | � V} | d k r) � | � n d S( N( R� R ( R� R� ( t callbackRA ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt
wrap_callback7 s N( R R� t on_recvRo Rp R{ ( RA R� R� ( ( R� RA s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� . s ( RP R R� RF RS R� RY Ro Rp R{ R� R� R� R� R� R� ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s Q
t ZeroMQReqServerChannelc B@ se e Z d � Z d � Z d � Z d � Z d � Z d � Z e j j
d � � Z d � Z d � Z
RS( c C@ s& t j j j j | | � t | _ d S( N( RJ Rl t servert ReqServerChannelRF RN RO ( RA R3 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRF B s c C@ s | j � t j j j d � t j | j d � | _ d j | j � | _
| j j t j � | _
| j d t k r� t t d � r� | j
j t j d � n | j
j t j | j j d d � � | j � | j j t j � | _ | j j d d
� d k r$d j | j j d
d � � | _ n( d j t j j | j d d � � | _ t j d � | j
j | j
� | j j | j � x� t r| j
j s�| j j r�Pn y t j t j | j
| j � Wq�t j! k
r�} | j" t" j# k r�q�n | � q�t$ t% f k
rPq�Xq�Wd S( uA
Multiprocessing target for the zmq queue device
u MWorkerQueueu worker_threadsu tcp://{interface}:{ret_port}u ipv6u IPV4ONLYi u zmq_backlogi� u ipc_modeu u tcpu tcp://127.0.0.1:{0}u tcp_master_workersi� u ipc://{0}u sock_diru workers.ipcu* Setting up the master communication serverN(& t& _ZeroMQReqServerChannel__setup_signalsRJ R� t processt appendproctitleR R� R3 R� R t uriRT t ROUTERt clientsRQ RR R� R� t BACKLOGR t _start_zmq_monitort DEALERt workerst w_uriR0 t patht joinR t infot bindR� t devicet QUEUEt ZMQErrorRV t EINTRt KeyboardInterruptt
SystemExit( RA RX ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt
zmq_deviceF s:
"%
"
c C@ su | j r
d St j d t j � � t | _ t | d d � d k r] | j j � d | _ n t | d d � d k r� | j
j � d | _
n t | d � r� | j j
t k r� | j j � n t | d � r� | j j
t k r� | j j � n t | d � r| j j � n t | d � r@| j j
t k r@| j j � n t | d � rq| j j
t k rq| j j � n d S(
u4
Cleanly shutdown the router socket
Nu$ MWorkerQueue under PID %s is closingu _monitoru
_w_monitoru clientsu workersu streamu _socketu context( RO R R� R0 R1 RQ t getattrR R� R� t
_w_monitorRR R� R� RN RS R� R� R� R� R� ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRS p s(
!!!!c C@ s0 t j j j j j | | � | j | j � d S( u�
Pre-fork we need to create the zmq router device
:param func process_manager: An instance of salt.utils.process.ProcessManager
N( RJ Rl t mixinsRL t AESReqServerMixint pre_forkt add_processR� ( RA t process_manager( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s c C@ sn t rj | j d rj t j d � d d l } t | j � | _ | j d | j j � j
� t j d � n d S( uM
Starts ZMQ monitor for debugging purposes.
:return:
u zmq_monitoru Starting ZMQ monitori Nt targetu$ ZMQ monitor has been started started( R� R3 R R% R+ R� R� R� t Threadt
start_pollt start( RA R+ ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s
c C@ s- | | _ | | _ t j d � | _ | j j t j � | _ | j � | j j
d d � d k r� d j | j j
d d � � | _ n( d j t
j j | j d d
� � | _ t j d | j � | j j | j � t j j j j j | | | � t j j j | j d | j �| _ | j j | j � d
S( ud
After forking we need to create all of the local sockets to listen to the
router
:param func payload_handler: A function to called to handle incoming payloads as
they are picked up off the wire
:param IOLoop io_loop: An instance of a Tornado IOLoop, to handle event scheduling
i u ipc_modeu u tcpu tcp://127.0.0.1:{0}u tcp_master_workersi� u ipc://{0}u sock_diru workers.ipcu Worker binding to socket %sR5 N( t payload_handlerR5 R R� R� RT t REPR� R� R3 R R R� R0 R� R� R R� R� RJ Rl R� RL R� t post_forkR� R� R� R� t on_recv_streamt handle_message( RA R� R5 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s
" $c
c@ sv y) | j j | d � } | j | � } Wny t k
r� } t | � j } | d k rf t j d � n t j d | | � | j | j j
d � � t j j
� � n Xt | t � s� t | j d � t � rt j d | | j d � � | j | j j
d � � t j j
� � n yg | d j d d
� } t d � | k r{t j d | � | j | j j
d
� � t j j
� � n WnR t k
r�t j d | � | j | j j
d j | � � � t j j
� � n X| d d k r:| j d i � j d � d k r:| j | j j
| j | d � � � t j j
� � n y | j | � V\ } } WnB t k
r�} | j d � t j d d t �t j j
� � n X| j d d � } | d k r�| j | j j
| � � n� | d k r| j | j j
| j j
| � � � n\ | d k rF| j | j j
| j | | d | d � � � n t j d | � | j d � t j j
� � d S( u
Handle incoming messages from underlying TCP streams
:stream ZMQStream stream: A ZeroMQ stream.
See http://zeromq.github.io/pyzmq/api/generated/zmq.eventloop.zmqstream.html
:param dict payload: A payload to process
i u AuthenticationErroru� Minion failed to auth to master. Since the payload is encrypted, it is not known which minion failed to authenticate. It is likely that this is a transient failure due to the master rotating its public key.u Bad load from minion: %s: %su bad loadu loadu@ payload and load must be a dict. Payload was: %s and load was %su payload and load must be a dictu idu u u+ Payload contains an id with a null byte: %su! bad load: id contains a null byteu" Payload contains non-string id: %su bad load: id {0} is not a stringu encu clearu cmdu _authu& Some exception handling minion payloadu- Some exception handling a payload from miniont exc_infou funu sendu
send_clearu send_privateu keyu tgtu Unknown req_fun %su& Server-side exception handling payloadN( R� Rj R� R� t typeRP R R% RU R` Rb Ro Rp Rq t
isinstanceRG R t strt TypeErrorR t _authR� RQ Ra t _encrypt_private(
RA R� R� RX t exc_typet id_Rs t req_optst et req_fun( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s\
)
"1&
(
c C@ s0 t j t j | j � t j t j | j � d S( N( t signalt SIGINTt _handle_signalst SIGTERM( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt __setup_signals s c C@ s� d j | j j � } | t j k r1 | d 7} n | t j k rM | d 7} n | d 7} t j | � | j � t j
t j j
j � d S( Nu {0} received a u SIGINTu SIGTERMu . Exiting( R R: RP R� R� R� R R% RS t syst exitRJ t defaultst exitcodest EX_OK( RA t signumt sigframet msg( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� s
( RP R RF R� RS R� R� R� Ro Rp R{ R� R� R� ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� ? s * M c C@ s� t t d � r� | r� d | k r; | j t j | d � n d | k ra | j t j | d � n d | k r� | j t j | d � n d | k r� | j t j | d � q� n d S( u�
Ensure that TCP keepalives are set as specified in "opts".
Warning: Failure to set TCP keepalives on the salt-master can result in
not detecting the loss of a minion when the connection is lost or when
it's host has been terminated without first closing the socket.
Salt's Presence System depends on this connection status to know if a minion
is "present".
Warning: Failure to set TCP keepalives on minions can result in frequent or
unexpected disconnects!
u
TCP_KEEPALIVEu
tcp_keepaliveu tcp_keepalive_idleu tcp_keepalive_cntu tcp_keepalive_intvlN( RR R R� R� R� R� R� ( t
zmq_socketR3 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _set_tcp_keepalive s
t ZeroMQPubServerChannelc B@ sn e Z d Z e j � Z d � Z d � Z d d � Z d d � Z
e d � � Z d � Z
d � Z d � Z RS(
uD
Encapsulate synchronous operations for a publisher channel
c C@ s@ | | _ t j j | j � | _ t j j j | j � | _ d S( N( R3 RJ R� R� R� R� t minionst CkMinionst ckminions( RA R3 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRF G s c C@ s t j j d � S( Ni ( Ro Rp t sleep( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� L s c
C@ s t j j j | j j � | rH t j j j | � t j j j | � n t
j d � } | j t
j
� } t | | j � y& | j t
j | j j d d � � WnU t k
r� | j t
j | j j d d � � | j t
j | j j d d � � n X| j d t k r/t t
d � r/| j t
j d � n | j t
j | j j d d � � | j t
j d � d j | j � } | j t
j � } | j t
j d � | j j d
d � d k r�d
j | j j d d � � } n% d j t j j | j d d � � } t j j j! | � t j" d | � | j# | � t j" d | � t j j$ j% d � � | j# | � Wd QXy+x$t r�y�t j& d | � | j' � } t j& d t( | � � t j) j* | � } t+ j, r�t j- j. j/ | � } n | d } t j0 d � | j d r)d | k r�x� | d D]y }
t j0 d | � t j j1 j2 t3 j4 t j j1 j2 |
� � j5 � � } | j6 | d t
j7 �| j6 | � t j0 d � qW| j j d � r&t j0 d! � | j6 d" d t
j7 �| j6 | � t j0 d# � q&qSt j0 d$ | � | j6 d% d t
j7 �| j6 | � t j0 d& � n* t j0 d' | � | j6 | � t j0 d( � Wqkt
j8 k
r�} | j9 t9 j: k r�qkn | � qkXqkWWn t; k
r�t j0 d) � n X| j< t= k r�| j> � n | j< t= k r�| j> � n | j< t= k r| j? � n d S(* uK
Bind to the interface specified in the configuration file
i u pub_hwmi� u ipv6u IPV4ONLYi u zmq_backlogi����u tcp://{interface}:{publish_port}u ipc_modeu u tcpu tcp://127.0.0.1:{0}u tcp_master_publish_pulli� u ipc://{0}u sock_diru publish_pull.ipcu! Starting the Salt Publisher on %su Starting the Salt Puller on %si Nu* Publish daemon getting data from puller %su( Publish daemon received payload. size=%du payloadu% Accepted unpacked package from pulleru
zmq_filteringu topic_lstu' Sending filtered data over publisher %st flagsu Filtered data has been sentu
order_mastersu Sending filtered data to syndicR� u% Filtered data has been sent to syndicu* Sending broadcasted data over publisher %sR� u Broadcasted data has been sentu- Sending ZMQ-unfiltered data over publisher %su Unfiltered data has been sentu5 Publish daemon caught Keyboard interupt, tearing down(@ RJ R� R� R� R: RP R t setupt! set_multiprocessing_logging_queuet setup_multiprocessing_loggingR R� RT t PUBR R3 R� t HWMR t AttributeErrort SNDHWMt RCVHWMRQ RR R� R� t LINGERR t PULLR0 R� R� t zeromqt check_ipc_path_max_lenR� R� t filest set_umaskR% t recvR� R� t unpackageR Rk Rl Rm Rn R. R� R� R� R� R� R` t SNDMORER� RV R� R� R� RN RS R� (
RA t log_queueR� t pub_sockt pub_urit pull_sockt pull_urit packaget unpacked_packageR� t topict htopicRX ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _publish_daemonO s� &
"&""
3
c C@ s | j | j d | �d S( u.
Do anything necessary pre-fork. Since this is on the master side this will
primarily be used to create IPC channels and create our daemon process to
do the actual publishing
:param func process_manager: A ProcessManager, from salt.utils.process.ProcessManager
R4 N( R� R% ( RA R� R4 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s c C@ s& y | j j SWn t k
r! n Xd S( u�
This thread's zmq publisher socket. This socket is stored on the class
so that multiple instantiations in the same thread will re-use a single
zmq socket.
N( t
_sock_datat sockR ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR � s
c C@ s� | j r | j � n t j j � } | j t j � | j _ | j j t j
d � | j j d d � d k r� d j
| j j d d � � } n% d j
t j j | j d d
� � } t j d | � | j j | � | j j S( u�
Create and connect this thread's zmq socket. If a publisher socket
already exists "pub_close" is called before creating and connecting a
new socket.
i����u ipc_modeu u tcpu tcp://127.0.0.1:{0}u tcp_master_publish_pulli� u ipc://{0}u sock_diru publish_pull.ipcu Connecting to pub server: %s( R t pub_closeR R� t instanceRT t PUSHR& R' R� R R3 R R R0 R� R� R R% R� ( RA t ctxR ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt pub_connect� s
c C@ s9 t | j d � r5 | j j j � t | j d � n d S( un
Disconnect an existing publisher socket and remove it from the local
thread's cache.
u sockN( RR R& R' RS t delattr( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR( � s c C@ s� i d d 6} t j j | j t j j j d d j � } | j | � | d <| j d r� t j
j | j d d � } t j
d � t j j | | d � | d <n i | j j | � d
6} | d d k r� | d
| d <n d d d g } | j d rO| d | k rO| j j | d
d | d �} | d } t j
d | � | | d <n | j j | � } t j
d | j d d � t | � � | j s�| j � n | j j | � t j
d � d S( u�
Publish "load" to minions. This send the load to the publisher daemon
process with does the actual sending to minions.
:param dict load: A load to be sent across the wire to minions
u aesu encu secretu loadu sign_pub_messagesu pki_diru
master.pemu Signing data packetu sigu payloadu tgt_typeu listu tgtu topic_lstu pcreu globu
zmq_filteringt tgt_typeu minionsu Publish Side Match: %su1 Sending payload to publish daemon. jid=%s size=%du jidu Sent payload to publish daemon.N( RJ RI Ri R3 t mastert SMastert secretst valueRb R0 R� R� R R% t sign_messageR� R t
check_minionsR R R� R R, R` ( RA RZ R� Ra t master_pem_patht int_payloadt
match_targetst _rest match_ids( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt publish� s2
,
N( RP R R� R+ t localR& RF R� R R% R� R� R R, R( R: ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR @ s n
R? c B@ sA e Z d Z d d d � Z d � Z d � Z d � Z d � Z RS( ul
Wrapper class of AsyncReqMessageClientPool to avoid blocking waiting while writing data to socket.
c C@ s2 t t | � j t | d | d | �t | _ d S( NR9 R4 ( t superR? RF t AsyncReqMessageClientRN RO ( RA R3 R9 R4 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRF ! s %c C@ sA | j r
d St | _ x | j D] } | j � q Wg | _ d S( N( RO RQ t message_clientsRS ( RA RM ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRS % s c O@ s, t | j d d � �} | d j | | � S( NR7 c S@ s
t | j � S( N( R� t
send_queue( t x( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt <lambda>/ s i ( t sortedR> R` ( RA R9 R4 R> ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR` . s c C@ s9 t j j j d d j | j j � d d �| j � d S( Nu SodiumuE Calling {0}.destroy() is deprecated. Please call {0}.close() instead.R� i ( RJ R� R� R� R R: RP RS ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� 2 s c C@ s | j � d S( N( RS ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRY = s N( RP R R� R RF RS R` R� RY ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR? s R= c B@ s� e Z d Z d d d � Z d � Z d � Z d � Z d � Z e j
j d � � Z d � Z
d � Z d d
d d e d � Z RS(
u�
This class wraps the underlying zeromq REQ socket and gives a future-based
interface to sending and recieving messages. This works around the primary
limitation of serialized send/recv on the underlying socket by queueing the
message sends in this class. In the future if we decide to attempt to multiplex
we can manage a pool of REQ/REP sockets-- but for now we'll just do them in serial
i c C@ s� | | _ | | _ | | _ | d k r; t � t j � n | | _ t j j
| j � | _ t j
� | _ | j � g | _ i | _ i | _ t | _ d S( u�
Create an asynchronous message client
:param dict opts: The salt opts dictionary
:param str addr: The interface IP address to bind to
:param int linger: The number of seconds to linger on a ZMQ socket. See
http://api.zeromq.org/2-1:zmq-setsockopt [ZMQ_LINGER]
:param IOLoop io_loop: A Tornado IOLoop event scheduler [tornado.ioloop.IOLoop]
N( R3 t addrt lingerR R
R R R5 RJ R� R� R� R R� R� t _init_socketR? t send_future_mapt send_timeout_mapRN RO ( RA R3 RC RD R5 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRF J s
c C@ s� | j r
d St | _ t | d � r� | j d k r� t d k r� | j j r_ | j j j � n | j j j | j j � d | j _ | j j � n | j j � d | _ d | _ n | j
j t k r� | j
j
� n d S( Nu streami i i ( i i i ( RO RQ RR R� R R RT RS R5 R� R� R� RN R� ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRS k s
c C@ s9 t j j j d d j | j j � d d �| j � d S( Nu SodiumuE Calling {0}.destroy() is deprecated. Please call {0}.close() instead.R� i ( RJ R� R� R� R R: RP RS ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� � s c C@ s | j � d S( N( RS ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRY � s c C@ sM t | d � r8 | j j � | j j � | ` | ` n | j j t j � | _ t t d � rx | j j t j d � n t | j | j
� | j j d � r� t t d � r� | j j t j
d � q� t t d � r� | j j t j d � q� n | j | j _ t j d | j � | j j | j � t j j j | j d
| j �| _ d S( Nu streamu RECONNECT_IVL_MAXi� u tcp://[u IPV6i u IPV4ONLYi u Trying to connect to: %sR5 ( RR R� RS RT R� R t REQR� R� R R3 RC t
startswitht IPV6R� RD R R% R� R� R� R� R5 ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRE � s&
c #@ s x� t � j � d k r� � j d } � j j | d � � � d k rV � j d =q n � � f d � } � j j | � � j j | � y � V} Wn: t k
r� } t j
d | � � j � � j d =q n X� j d =� j j | d � � j
| � q Wd S( Ni c @ s6 � j � s2 � j j | d � } � j | � n d S( Ni ( t doneR� Rj t
set_result( R Rw ( t futureRA ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt mark_future� s u Re-init ZMQ socket: %s( R� R? RF R R R� R� R` R� R R% RE t popt remove_message_timeout( RA t messageRN Rs t err( ( RM RA s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt _internal_send_recv� s&
c C@ sK | | j k r d S| j j | d � } | d k rG | j j | � n d S( N( RG RO R R5 t remove_timeout( RA RQ R\ ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRP � s
c C@ s� | j j | d � } | d k r� | j | =| j | j k r� | j d 7_ t j d | j | j � | j | d | j d | j d | �q� | j
t d � � n d S( u�
Handle a message timeout by removing it from the sending queue
and informing the caller
:raises: SaltReqTimeoutError
i u& SaltReqTimeoutError, retrying. (%s/%s)R\ R] RM u Message timed outN( RF RO R RG t attemptsR] R R% R` R\ t
set_exceptionR ( RA RQ RM ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt timeout_message� s
i c @ s | d k rK t j j � } | | _ d | _ | | _ � j j | � } n � d k ry � � f d � } | j | � n | � j
| <� j j d � t
k r� d } n | d k r� � j j | � j | � } | � j | <n t � j � d k r � j j � j � n � j j | � | S( uY
Return a future which will be completed when the message has a response
i c @ s# | j � } � j j � | � d S( N( RC R5 t add_callback( RM t response( R� RA ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt
handle_future� s u detect_modei N( R Ro t
concurrentt FutureR] RU R\ R� Rb t add_done_callbackRF R3 R RQ R5 t
call_laterRW RG R� R? t spawn_callbackRS t append( RA RQ R\ R] RM R� Ry RZ t send_timeout( ( R� RA s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR` � s&
N( RP R R� R RF RS R� RY RE Ro Rp R{ RS RP RW RN R` ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR= B s ! R� c B@ sJ e Z d Z d � Z d � Z d � Z e d � � Z d � Z d � Z
RS( c C@ s( | | _ | j j � | _ d | _ d S( u�
Create ZMQ monitor sockets
More information:
http://api.zeromq.org/4-0:zmq-socket-monitor
N( R� t get_monitor_sockett _monitor_socketR t _monitor_stream( RA RT ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRF s c C@ sE t j d � t j j j | j d | �| _ | j j | j � d S( Nu Event monitor start!R5 (
R R. R R� R� R� Rc Rd R� t monitor_callback( RA R5 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� s
!c C@ sv t j d � yE x> | j d k rP | j j � rP | j j � } | j | � q WWn t t j j
f k
rq n Xd S( Nu Event monitor start!( R R. Rc R t pollt recv_multipartRe R R RU t ContextTerminated( RA R ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� s
!c C@ sj t j d k rc i } x? t t � D]1 } | j d � r" t t | � } | | | <q" q" W| t _ n t j S( Nu EVENT_( R� t _ZeroMQSocketMonitor__EVENT_MAPR t dirR RI R� ( RA t event_mapt nameR2 ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRk ' s c C@ s^ t j j j | � } | j | d | d <t j d | � | d t j k rZ | j � n d S( Nu eventu descriptionu ZeroMQ event: %s( R R� t monitort parse_monitor_messageRk R R% t EVENT_MONITOR_STOPPEDR� ( RA R t evt( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyRe 2 s
c C@ sk | j d k r d S| j j � d | _ d | _ | j d k rZ | j j � d | _ n t j d � d S( Nu Event monitor done!( R� R t disable_monitorRc Rd RS R R. ( RA ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� 9 s
N( RP R R Ri RF R� R� R� Rk Re R� ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyR� s (X R� t
__future__R R R R0 R� R; RV R� RT R� t loggingR" R+ t randomR t salt.authRJ t
salt.cryptt salt.log.setupt salt.utils.eventt salt.utils.filest salt.utils.minionst salt.utils.processt salt.utils.stringutilst salt.utils.verifyR t salt.utils.versionst salt.payloadt salt.transport.clientt salt.transport.servert salt.transport.mixins.autht salt.extR t salt.exceptionsR R t salt._compatR R R R
R R t zmq.errort zmq.eventloop.ioloopt zmq.eventloop.zmqstreamt zmq.utils.monitorRQ R� t ImportErrorRN Ro t tornado.gent tornado.concurrentt M2CryptoR
Rd t Cryptodome.CipherR t
Crypto.Ciphert getLoggerRP R R R Rl t clientt
ReqChannelR R� RL t AESPubClientMixint AsyncPubChannelR� R� R� R� R� R t PubServerChannelR t MessageClientPoolR? R& R= R� ( ( ( s9 /usr/lib/python2.7/site-packages/salt/transport/zeromq.pyt <module> s� (
& � .� � � %�
Mini Shell