Current File : //lib/python2.7/site-packages/salt/fileserver/__init__.pyo
�
���^c @@ s� d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l Z d d l Z
d d l Z
d d l Z
d d l Z
d d l Z
d d l Z
d d l m Z d d l m Z y d d l m Z Wn! e k
rd d l m Z n Xd d l m Z e j e � Z d � Z! d � Z" d d
� Z# d � Z$ d � Z% d
� Z& d � Z' d � Z( d � Z) d � Z* e+ d d � Z, d e- f d � � YZ. d e- f d � � YZ/ d S( u=
File server pluggable modules and generic backend functions
i ( t absolute_importt print_functiont unicode_literalsN( t get_function_argspec( t ensure_unicode_args( t Sequence( t sixc C@ s� t j j | � s d SyH t j j | � r; t j | � n" t j j | � r] t j | � n Wn, t t f k
r� } t j
d | | � n Xd S( u)
Unlock a FS file/dir based lock
Nu Error removing lockfile %s: %s( t ost patht existst isdirt rmdirt isfilet unlinkt OSErrort IOErrort logt trace( t w_lockt exc( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt
_unlock_cache' s c C@ sX y t j | � Wn, t k
r? } | j t j k r; � n t SXt j d | � t Sd S( Nu Lockfile %s created( R t mkdirR t errnot EEXISTt FalseR R t True( R R ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt _lock_cache6 s c C@ sG t j j | � s t St j j | � sX t j d � t j j | � sX t | � t Sn d } | rw t j � | } n d } t j | � j
} x� t rBt j d � t j j | � s� t St j | � j
} | | k r | d 7} | d k rt | � t Sn | } | r� t j � | k r?t d j
| | | � � � q?q� q� Wt S( u�
If the write lock is there, check to see if the file is actually being
written. If there is no change in the file size after a short sleep,
remove the lock and move forward.
i i i u) Timeout({0}s) for {1} (lock: {2}) elapsedN( R R R R t timet sleepR R t Nonet statt st_sizeR t
ValueErrort format( t lk_fnt destt wait_timeoutt timeoutt s_countt s_sizet size( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt wait_lockB s:
c
C@ sO t } t } t j j | � } t | | d � t j j | � rV t | � rV t } n�d } x�| d k r&y�t j j
| � r� t | | d � n t j j
| � r
t j | � } t t
j
� � } t | j � }
|
| k r t j d | | |
� d } q#| |
} n | j d d � d } | d k r?t j d
� n d | k ob| j d d � k n r�t j j j | d � �Z } t j d | | j d d � | � t j j j | j | � j | g � � t t f SWd
QXn t | � r�t } Pn Wq_ t k
r"t
j d � | d 7} q_ q_ Xq_ W| d k rBt } t } n d
| | f S( u�
Checks the cache file to see if there is a new enough file list cache, and
returns the match (if found, along with booleans used by the fileserver
backend to determine if the cache needs to be refreshed/written).
i i< i i i uU Cache file modified time is in the future, ignoring. file=%s mtime=%s current_time=%su fileserver_list_cache_timei i u. The file list_cache was created in the future!u rbu7 Returning file list from cache: age=%s cache_time=%s %sNg�������?i
i, i� ( R R t saltt payloadt SerialR) R R R R R R t intR t st_mtimeR t debugt gett warningt utilst filest fopent datat decodet loadt ExceptionR R (
t optst formt
list_cacheR t
refresh_cachet
save_cachet serialt attemptt
cache_statt current_timet
file_mtimet aget fp_( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt check_file_list_cachep sR
(:
c C@ sg t j j | � } t j j j | d � �7 } | j | j | � � t | � t j
d | � Wd QXd S( u�
Checks the cache file to see if there is a new enough file list cache, and
returns the match (if found, along with booleans used by the fileserver
backend to determine if the cache needs to be refreshed/written).
u w+bu Lockfile %s removedN( R* R+ R, R2 R3 R4 t writet dumpsR R R ( R9 R5 R; R R> RD ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt write_file_list_cache� s
c C@ s� t j j | � s d Syc t j j j | d � �E } t j d | � t j
j | � } t j j j
| j | � � SWd QXWn t t f k
r� n Xd S( uG
Returns cached env names, if present. Otherwise returns None.
u rbu Returning env cache data from %sN( R R R R R* R2 R3 R4 R R R+ R, R5 R6 R7 R R ( R9 t env_cacheRD R> ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt check_env_cache� s &c
C@ s� i } x� t j | � D]� \ } } x� | D]� } x� t j j j | � D]� \ } } } x | D]w } yD t j j | | � }
t | |
� r� w[ n t j j |
� | |
<Wq[ t
t f k
r� t j
d |
� q[ q[ Xq[ WqE Wq) Wq W| S( u.
Generate a dict of filename -> mtime
u, Failed to get mtime on %s, dangling symlink?( R t iteritemsR* R2 R t os_walkR t joint is_file_ignoredt getmtimeR R R t info( R9 t path_mapt file_mapt saltenvt path_listR t directoryt _t filenamest itemt file_path( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt generate_mtime_map� s
%
c C@ sT t | � t | � k r t Sx1 t j | � D] \ } } | | | k r, t Sq, Wt S( u>
Is there a change to the mtime map? return a boolean
( t sortedR R RK R ( t map1t map2t filenamet mtime( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt diff_mtime_map� s c C@ sU xNt j | � D]=} t j j | | � } x t j j j | � D]\ } } } | r� | r� t j � t j j | � d k rA t j | � qA qA n x� | D]� } t j j | | � } t j j
| | � } y | j d d � \ }
} } Wn$ t k
rt
j d | � q� n X| |
d | �}
|
d d k r� t j | � q� q� WqA Wq Wd S( u�
Remove unused cache items assuming the cache directory follows a directory
convention:
cache_base -> saltenv -> relpath
i< u .i uD Found invalid hash file [%s] when attempting to reap cache directoryRS u pathu N( R t listdirR RM R* R2 RL R t getctimeR t relpatht rsplitR R R1 R
( t
cache_baset find_funcRS t env_baset roott dirsR3 t file_RY t
file_rel_pathR^ RV t hash_typet ret( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt reap_fileserver_cache_dir� s( %"
c C@ s� | d rH x; | d D], } t j | | � r t j d | � t Sq Wn | d r� x; | d D], } t j | | � r] t j d | � t Sq] Wn t S( u�
If file_ignore_regex or file_ignore_glob were given in config,
compare the given file path against all of them and return True
on the first match.
u file_ignore_regexu- File matching file_ignore_regex. Skipping: %su file_ignore_globu, File matching file_ignore_glob. Skipping: %s( t ret searchR R/ R t fnmatchR ( R9 t fnamet regext glob( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyRN s
u updatec C@ sN d j | | � } | r. | d j | � 7} n t j | � | d | d | � S( u�
Function to allow non-fileserver functions to clear update locks
clear_func
A function reference. This function will be run (with the ``remote``
param as an argument) to clear the lock, and must return a 2-tuple of
lists, one containing messages describing successfully cleared locks,
and one containing messages describing errors encountered.
role
What type of lock is being cleared (gitfs, git_pillar, etc.). Used
solely for logging purposes.
remote
Optional string which should be used in ``func`` to pattern match so
that a subset of remotes can be targeted.
lock_type : update
Which type of lock to clear
Returns the return data from ``clear_func``.
u! Clearing {0} lock for {1} remotesu
matching {0}t remotet lock_type( R! R R/ ( t
clear_funct roleRu Rv t msg( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt
clear_lock9 s
t
Fileserverc B@ s% e Z d Z d � Z d d � Z d � Z d � Z d d � Z d d d � Z d d d � Z
d d � Z d d � Z d e
d
� Z d d � Z d d � Z d
� Z d � Z d d � Z d � Z d � Z d � Z d � Z d � Z e d � � Z e d � � Z e d � � Z e d � � Z RS( u�
Create a fileserver wrapper object that wraps the fileserver functions and
iterates over them to execute the desired function within the scope of the
desired fileserver backend.
c C@ s) | | _ t j j | | d � | _ d S( Nu fileserver_backend( R9 R* t loadert
fileservert servers( t selfR9 ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt __init__] s c C@ s� | s | j d } nQ t | t � sg y | j d � } Wqg t k
rc t j | � j d � } qg Xn t | t � r� t | � } n g } t | t � s� | S| j j � } y t
d � | D� � } Wn t k
r� n� X| rc| j d } xn | D]f } d j | d � | k r(| j | d � q� d j | d d !� | k r� | j | d d !� q� q� W| Sx` | D]X } d j | � | k r�| j
| � qjd j | d � | k rj| j
| d � qjqjW| S( u)
Return the backend list
u fileserver_backendu ,c s@ s | ] } | j d � Vq d S( u -N( t
startswith( t .0t x( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pys <genexpr>~ s u {0}.envsi i����( R9 t
isinstancet listt splitt AttributeErrorR t text_typeR R~ t keyst allR! t removet append( R t backRm t server_funcst
subtract_onlyt sub( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt backendsa s>
c C@ s | j S( u&
Simplify master opts
( R9 ( R R7 ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt master_opts� s c C@ se x^ | j j � D]M \ } } y- d | j k rH | j d j | j � n Wq t k
r\ q Xq Wd S( Nu __opts__( R~ t itemst __globals__t updateR9 R� ( R t namet func( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt update_opts� s
c C@ s� | j | � } g } g } x{ | D]s } d j | � } | | j k r" t j d | � | j | � } | r| | j | � q� | j d j | � � q" q" W| | f S( u�
Clear the cache of all of the fileserver backends that support the
clear_cache function or the named backend(s) only.
u {0}.clear_cacheu Clearing %s fileserver cacheu1 The {0} fileserver cache was successfully cleared( R� R! R~ R R/ t extendR� ( R R� t clearedt errorst fsbt fstrt failed( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt clear_cache� s
c
C@ s� | j | � } g } g } x� | D]� } d j | � } | | j k r" d j | � } | r� t | t j � s� | j d j | � � q" q� | d j | � 7} n t j | � | j | d | � \ } } | j | � | j | � q" q" W| | f S( u�
``remote`` can either be a dictionary containing repo configuration
information, or a pattern. If the latter, then remotes for which the URL
matches the pattern will be locked.
u {0}.locku# Setting update lock for {0} remotesu$ Badly formatted remote pattern '{0}'u
matching {0}Ru (
R� R! R~ R� R t string_typesR� R R/ R� (
R R� Ru t lockedR� R� R� Ry t goodt bad( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt lock� s&
c C@ s� | j | � } g } g } xk | D]c } d j | � } | | j k r" t | j | | d | �\ } } | j | � | j | � q" q" W| | f S( ui
Clear the update lock for the enabled fileserver backends
back
Only clear the update lock for the specified backend(s). The
default is to clear the lock for all enabled backends
remote
If specified, then any remotes which contain the passed string will
have their lock cleared.
u {0}.clear_lockRu ( R� R! R~ Rz R� ( R R� Ru R� R� R� R� R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyRz � s
c C@ sc | j | � } xM | D]E } d j | � } | | j k r t j d | � | j | � q q Wd S( um
Update all of the enabled fileserver backends which support the update
function, or
u
{0}.updateu Updating %s fileserver cacheN( R� R! R~ R R/ ( R R� R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyR� � s
c C@ s_ | j | � } i } xC | D]; } d j | � } | | j k r | j | � | | <q q W| S( u�
Return the update intervals for all of the enabled fileserver backends
which support variable update intervals.
u {0}.update_intervals( R� R! R~ ( R R� Rm R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt update_intervals� s
c C@ s� | j | � } t � } | r' i } n x� | D]� } d j | � } d t | j | � j k r | j d d k r i t d 6n i } | r� | j | | � | | <q. | j | j | | � � q. W| r� | St | � S( uO
Return the environments for the named backend or all backends
u {0}.envsu ignore_cacheu __roleu minion(
R� t setR! t _argspecR~ t argsR9 R R� R� ( R R� t sourcesRm R� R� t kwargs( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt envs s
c C@ s2 | d k r i } n | j d d � | j | � S( uS
Return environments for all backends for requests from fileclient
u cmdN( R t popR� ( R R7 ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt file_envs s c C@ sS | j | � } x= | D]5 } d j | � } | | j k r | j | � q q Wd S( uX
Initialize the backend, only do so if the fs supports an init function
u {0}.initN( R� R! R~ ( R R� R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt init% s
c C@ sI | j d � } | s' i d d 6d d 6S| j d d � } | j | | � S( uL
Convenience function for calls made using the RemoteClient
u pathu u relu saltenvu base( R0 t find_file( R R7 R t tgt_env( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt
_find_file/ s
c C@ sI | j d � } | s' i d d 6d d 6S| j d d � } | j | | � S( uK
Convenience function for calls made using the LocalClient
u pathu u relu saltenvu base( R0 R� ( R R7 R R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt file_find: s
c C@ s� t j j j | � } t j j j | � } | j | � } i } i d d 6d d 6} t j j | � ri | Sd | k ry | St j j j | � r� t j j j
| � } n� d | k r(| j d � } | d } | d j d � } xG | D]< } d | k r� q� n | j d d � } | d | | d <q� Wn d
| k rD| j d
� n d | k rb| j d � } n t
| t j � s�t j | � } n xe | D]] }
d j |
� } | | j k r�| j | | | | � } | j d � r�|
| d
<| Sq�q�W| S( u{
Find the path and return the fnd structure, this structure is passed
to other backend interfaces.
u u pathu relu ../u ?i i u &u =u envu saltenvu
{0}.find_fileu back( R* R2 t stringutilst
to_unicodeR� R R t isabst urlt
is_escapedt unescapeR� R� R� R R� R� R! R~ R0 ( R R RS R� R� t fndt hcompst compst compR� R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyR� E sF
c C@ s� i d d 6d d 6} d | k r0 | j d � n d | k sT d | k sT d | k rX | St | d t j � s� t j | d � | d <n | j | d | d � } | j d � s� | Sd j | d � } | | j k r� | j | | | � S| S(
u,
Serve up a chunk of a file
u u datau destu envu pathu locu saltenvu backu {0}.serve_file( R� R� R R� R� R� R0 R! R~ ( R R7 Rm R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt
serve_fileu s
$c C@ s� d | k r | j d � n d | k s4 d | k r8 d St | d t j � sh t j | d � | d <n | j t j j j
| d � | d � } | j d � s� d
S| j d d � } d j | d � } | | j
k r� | j
| | | � | f Sd S( u;
Common code for hashing and stating files
u envu pathu saltenvu u backu statu
{0}.file_hashN( u N( u N( u N( R� R R� R R� R� R� R* R2 R� R� R0 R! R~ ( R R7 R� t stat_resultR� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt __file_hash_and_stat� s
c C@ s4 y | j | � d SWn t t f k
r/ d SXd S( u1
Return the hash of a given file
i u N( t _Fileserver__file_hash_and_statt
IndexErrort TypeError( R R7 ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt file_hash� s c C@ s0 y | j | � SWn t t f k
r+ d SXd S( uA
Return the hash and stat result of a given file
u N( u N( R� R� R� R ( R R7 ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt file_hash_and_stat� s c C@ s< d | k r | j d � n | j d g � } | d k rt | t � s� y, g | j d � D] } | j � ^ q\ } Wq� t k
r� g t j | � j d � D] } | j � ^ q� } q� Xn xE t
| � D]4 \ } } t | t j � s� t j | � | | <q� q� Wn i } | j | j d d � � } t
j j | j d d � } y t
j | � } WnF t k
r�}
|
j t j k r�t j d � i St j d |
� n Xx�| D]�} t j d d
| � } y" t
j t
j j | | � � }
Wn) t k
r
}
t j d | |
� q�n Xx#|
D]} y | j d d
� \ } } Wn t k
rMqn X| d k r`qn= | | k s�| d k r�| | k r�t j d | | � qn y# t
j t
j j | | | � � WnA t k
r}
|
j t j k r0t j d |
j |
j � q0qX| j | g � j | � t j d | | � qWq�W| S( u4
Deletes the file_lists cache files
u envu saltenvu ,u fsbackendu cachediru
file_listsu No file list caches founduM Failed to get list of saltenvs for which the master has cached file lists: %su fs$u u4 Failed to find file list caches for saltenv '%s': %su .i u pu, Skipping %s file list cache for saltenv '%s'u Failed to remove %s: %su+ Removed %s file list cache for saltenv '%s'N(! R� R0 R R� R� R� t stripR� R R� t enumerateR� R� R R RM R9 Ra R R t ENOENTR R/ t errorRo R� Rd R R� R^ t strerrort
setdefaultR� ( R R7 RS R� t idxt valRm R� t
list_cachedirt file_list_backendsR R� t back_virtualnamet cache_filest
cache_filet
cache_saltenvt extension( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt clear_file_list_cache� sr ,
8
"
# c C@ s" d | k r | j d � n t � } d | k r5 g St | d t j � se t j | d � | d <n x^ | j | j d d � � D]A } d j | � } | | j k r� | j
| j | | � � q� q� W| j d d � j d � } | d k rg | D] } | j
| � r� | ^ q� } n t | � S( uF
Return a list of files from the dominant environment
u envu saltenvu fsbackendu
{0}.file_listu prefixu u /N( R� R� R� R R� R� R� R R! R~ R� R0 R� R� R[ ( R R7 Rm R� R� t prefixt f( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt file_list s "! +c C@ s d | k r | j d � n t � } d | k r5 g St | d t j � se t j | d � | d <n xR | j d � D]A } d j | � } | | j k ru | j
| j | | � � qu qu W| j d d � j d � } | d k r g | D] } | j
| � r� | ^ q� } n t | � S( u=
List all emptydirs in the given environment
u envu saltenvu {0}.file_list_emptydirsu prefixu u /N( R� R� R� R R� R� R� R R! R~ R� R0 R� R� R[ ( R R7 Rm R� R� R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt file_list_emptydirs s ! +c C@ s" d | k r | j d � n t � } d | k r5 g St | d t j � se t j | d � | d <n x^ | j | j d d � � D]A } d j | � } | | j k r� | j
| j | | � � q� q� W| j d d � j d � } | d k rg | D] } | j
| � r� | ^ q� } n t | � S( u?
List all directories in the given environment
u envu saltenvu fsbackendu {0}.dir_listu prefixu u /N( R� R� R� R R� R� R� R R! R~ R� R0 R� R� R[ ( R R7 Rm R� R� R� R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt dir_list5 s "! +c C@ s- d | k r | j d � n i } d | k r2 i St | d t j � sb t j | d � | d <n xW | j | j d d � � D]: } d j | � } | | j k r~ | j | | � } q~ q~ W| j d d � j
d � } | d k r)t g t j | � D]' \ } } | j
| � r� | | f ^ q� � } n | S( u;
Return a list of symlinked files and dirs
u envu saltenvu fsbackendu {0}.symlink_listu prefixu u /N( R� R� R R� R� R� R R! R~ R0 R� t dictRK R� ( R R7 Rm R� t symlstrR� R� t y( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt symlink_listN s " CN( t __name__t
__module__t __doc__R� R R� R� R� R� R� Rz R� R� R R� R� R� R� R� R� R� R� R� R� R� R R� R� R� R� ( ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyR{ W s2 2
0 Mt FSChanc B@ s2 e Z d Z d � Z d d e d � Z d � Z RS( u}
A class that mimics the transport channels allowing for local access to
to the fileserver class class structure
c K@ s� | | _ | | _ t | j � | _ | j j � | j j d d � d k r{ d | j k r� | j j � t | j d <q� n
| j j � i i d 6i d 6| _ d S( Nu file_clientu remoteu localu __fs_updateu master_topsu ext_nodes( R9 R� R{ t fsR� R0 R� R t cmd_stub( R R9 R� ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyR� o s
c C@ s� d | k r t j d | � i S| d j d � } | | j k rM | j | St | j | � ss t j d | � i St | j | � | � S( uU
Emulate the channel send method, the tries and timeout are not used
u cmdu Malformed request, no cmd: %su _u" Malformed request, invalid cmd: %s( R R� t lstripR� t hasattrR� t getattr( R R7 t triesR% t rawt cmd( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt send} s c C@ s d S( N( ( R ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt close� s N( R� R� R� R� R R R� R� ( ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyR� j s (0 R� t
__future__R R R R Rq t loggingR Ro t sysR t salt.loaderR* t salt.utils.datat salt.utils.filest salt.utils.patht salt.utils.urlt salt.utils.versionst salt.utils.argsR R� t salt.utils.decoratorsR t collections.abcR t ImportErrort collectionst salt.extR t getLoggerR� R R R R) RE RH RJ RZ R` Rn RN R Rz t objectR{ R� ( ( ( s< /usr/lib/python2.7/site-packages/salt/fileserver/__init__.pyt <module> sJ
. @
& � �
Mini Shell