Current File : //usr/lib/python2.7/site-packages/salt/utils/files.pyc
�
���^c @@ sx d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l Z d d l Z d d l
Z
d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z m Z m Z d d l m Z d d l m Z d d l m Z d d l m Z y d d l! Z! e" Z# Wn e$ k
rae% Z# n Xe j& e' � Z( dB Z) dC Z* dD e* Z+ d Z, i d d 6d d 6d d 6d d 6d d 6d d 6Z- e. g e j/ e- � D] \ Z0 Z1 e1 e0 f ^ q�� Z2 d � Z3 d � Z4 d � Z5 d! � Z6 d d d" � Z7 d# � Z8 d d$ � Z: e j; d d% d& d d' � � Z< d( � Z= e j; d) � � Z> d* � Z? e j; d+ � � Z@ e j; d, � � ZA e" d e" d d- � ZB d. � ZC d/ � ZD e d0 � d1 � � ZE e% d2 � ZF d3 � ZG d d4 � ZH e d5 � d6 d7 � � ZI e d8 � d9 � � ZJ d: � ZK e d; � d< � � ZL d= � ZM d> � ZN d? � ZO d@ � ZP dA � ZQ d S(E u"
Functions for working with files
i ( t absolute_importt unicode_literalst print_functionN( t CommandExecutionErrort
FileLockErrort MinionError( t jinja_filter( t six( t range( t quoteu u fileu httpu httpsu ftpu swiftu s3u saltu __salt.tmp.i� u sha512i` u sha384i@ u sha256i8 u sha224i( u sha1i u md5c C@ s& y t | � Wn t k
r! n Xd S( u
Remove temporary files
N( t rm_rft Exception( t tmp( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt __clean_tmp: s
c
C@ s` | j � } x% d D] } | j d | � r d Sq Wx% d D] } | j d | � r; | Sq; Wd S( uH
Guess an archive type (tar, zip, or rar) by its file extension
u taru tar.gzu tgzu tar.bz2u tbz2u tbzu tar.xzu txzu tar.lzmau tlzu .u zipu rar(
u taru tar.gzu tgzu tar.bz2u tbz2u tbzu tar.xzu txzu tar.lzmau tlz( u zipu rarN( t lowert endswitht None( t namet ending( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt guess_archive_typeD s
c O@ sm d | k r d | d <n | j d t � } t j | | � \ } } | t k rY | | f St j | � ~ | S( u
Helper function which does exactly what ``tempfile.mkstemp()`` does but
accepts another argument, ``close_fd``, which, by default, is true and closes
the fd before returning the file path. Something commonly done throughout
Salt's code.
u prefixu __salt.tmp.u close_fd( t popt Truet tempfilet mkstempt Falset ost close( t argst kwargst close_fdt fd_t f_path( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyR U s
c
C@ s� x� t j j j | � D]� \ } } } | j | d � j t j � } t j j | | � } t j j | � sz t j
| � n xK | D]C } t j j | | | � } t j j | | � } t j | | � q� Wq Wd S( u�
Recursively copy the source directory to the destination,
leaving files with the source does not explicitly overwrite.
(identical to cp -r on a unix machine)
u N(
t saltt utilst patht os_walkt replacet lstripR t sept joint existst makedirst shutilt copyfile(
t sourcet destt roott _t filest path_from_sourcet target_directoryR t file_path_from_sourcet target_path( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt recursive_copyg s %
c
C@ s� t j j | � s* t d j | � � � n t j j t j j | � � s` t d j | � � � n t j j | � } t j j t j j | � � } t d | d | � } t
j | | � d } | r� t j j | d � } n | d k s� | d k r | r t j j
| � r t | | � q n | d k s=| d k r=| r=n d } t j j j � sy t j | � } Wqt k
r{qXn y t
j | | � Wn t k
r�t | � � n X| d k r�t j | | j | j � t j | | j � n t j j j d � } | r�t }
y t j j! j" � }
Wn t# t$ f k
r?n X|
d
k r�t% t j& d � �, } | | g } t' j( | d | d
| �Wd QXq�n t j j | � r�t | � n d S( uj
Copy files from a source to a destination in an atomic way, and if
specified cache the file.
u( [Errno 2] No such file or directory: {0}t prefixt diru u file_backupu minionu bothu masteru
restoreconu Enforcingu wt stdoutt stderrN() R R" t isfilet IOErrort formatt isdirt dirnamet basenamet abspathR R* R+ R' R( t
backup_minionR R R! t platformt
is_windowst statt OSErrort moveR R
t chownt st_uidt st_gidt chmodt st_modet whichR t modulest selinuxt
getenforcet ImportErrorR t fopent devnullt
subprocesst call(
R, R- t backup_modet cachedirt bnamet dnamet tgtt bkroott fstatt rcont policyt dev_nullt cmd( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyR+ y sZ
%c C@ s� y t j | | � Wn� t k
r� } | j t j k r> � n y t j | � WnC t k
r� } | j t j k r� t d j | | j � � � q� n Xt j | | � n Xd S( u�
On Windows, os.rename() will fail with a WindowsError exception if a file
exists at the destination path. This function checks for this error and if
found, it deletes the destination path first.
u Error: Unable to remove {0}: {1}N(
R t renameRE t errnot EEXISTt removet ENOENTR R< t strerror( t srct dstt exc( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyR` � s c C@ s� | d k r- t | t j � r3 | f } q3 n d } | j | k rF d S| j t j k rp t d j | � � � nK | j t j k r� t d j | � � � n! t d j | j | | j � � � d S( u�
Common code for raising exceptions when reading a file fails
The ignore argument can be an iterable of integer error codes (or a single
integer error code) that should be ignored.
Nu {0} does not existu" Permission denied reading from {0}u+ Error {0} encountered reading from {1}: {2}( (
R t
isinstanceR t
integer_typesRa Rd R R< t EACCESRe ( Rh R" t ignore( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt process_read_exception� s i g�������?c
#@ s% t | t j � s! t d � � n | d k r: | d } n � d k rU t j � � n t } t � f d � } z�yDt j j | � r� t j j
| � r� | d j | � � n t j t j
Bt j B} x� t j � � | k r�y t j | | � } Wnm t t f k
rc} | j t j k r@| d j | j | | j � � n t j d | | � t j | � q� Xt j | d � � Wd QXt j d | � t } d VPq� W| d
j | | � � Wn< t k
r�� n) t k
r�} | d j | | � � n XWd | r t j | � t j d | | � n Xd S(
uJ
Obtain a write lock. If one exists, wait for it to release first
u path must be a stringu .wc @ s t | d � �� d S( u'
Raise a FileLockError
t
time_startN( R ( t msgt race( Rn ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt _raise_error� s u$ lock_fn {0} exists and is not a fileu2 Error {0} encountered obtaining file lock {1}: {2}u( Lock file %s exists, sleeping %f secondsu wNu Write lock %s obtaineduF Timeout of {0} seconds exceeded waiting for lock_fn {1} to be releasedu. Error encountered obtaining file lock {0}: {1}u Write lock for %s (%s) released( Ri R t string_typesR R t timeR R R" R( R: R< t O_CREATt O_EXCLt O_WRONLYt openR; RE Ra Rb Re t logt tracet sleept fdopenR R Rc (
R" t lock_fnt timeoutRz Rn t
obtained_lockRq t
open_flagst fh_Rh ( ( Rn s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt wait_lock� sX
%
c C@ s t j d � } t j | � | S( u#
Returns the current umask
i ( R t umask( t ret( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt get_umask; s
c c@ sS | d k s t j j j � r& d Vn) z t j | � } d VWd t j | � Xd S( uM
Temporarily set the umask and restore once the contextmanager exits
N( R R R! RB RC R R� ( t maskt orig_mask( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt set_umaskD s c O@ s� t j rP y0 | d d
k r8 t d j | d � � � n WqP t k
rL qP Xn d } t j r} t j j j
� r} d | k s� | j d t � rot
| � d k r t | � } d | d k rl| d j d d � | d <d | d k r� | d c d 7<q� qlq| j d � rbd | d k rl| d j d d � | d <d | d k r_| d c d 7<q_qlqd | d <n� t j rd
| k rt } t
| � d k r�t | � } d | d k r�t } q�n | j d d � r�d | d k r�t } q�n | st | d
<qn t j r8| r8| j d d � r8d | d <n t | | � } t � r�y
t j } Wn t k
rvd } n Xt j | j � t j � } t j | j � t j | | B� n | S( u�
Wrapper around open() built-in to set CLOEXEC on the fd.
This flag specifies that the file descriptor should be closed when an exec
function is invoked;
When a file descriptor is allocated (as with open or dup), this bit is
initially cleared on the new file descriptor, meaning that descriptor will
survive into the new program after exec.
NB! We still have small race condition between open and fcntl.
i i i u& {0} is not a permitted file descriptoru binaryu bu tu modeu rbu encodingu newlineu ( i i i N( R t PY3t TypeErrorR< t
IndexErrorR t PY2R R! RB RC R R t lent listR$ t getR t __salt_system_encoding__Rw t is_fcntl_availablet fcntlt
FD_CLOEXECt AttributeErrort filenot F_GETFDt F_SETFD( R R t binaryt f_handleR� t old_flags( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyRQ T sX
'
#
#c o@ s� | d | d } } d } t | | | � �� } zu t d t � r� t j } | r� t g | D] } | | d k ^ q[ � r� t j } n t j | j � | � n | VWd t d t � r� t j | j � t j � n XWd QXd S( u;
Shortcut for fopen with lock and context manager.
i i u wat check_sunosN(
RQ R� R R� t LOCK_SHt anyt LOCK_EXt flockR� t LOCK_UN( R R t filenamet writingR� t lock_typet write( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt flopen� s / c
o@ s, | j d d � } | j d d � } | j d d � } t | | � �� } | d } t j | � } t t d � r� | j | k s� | j | k r� g | | f D] } | d k r� | ^ q� r� t j | | | � q� n | d k r t j | j
� } | | k r t j | | j
| A| B� q n | VWd QXd S( u�
Shortcut for fopen with extra uid, gid, and mode options.
Supported optional Keyword Arguments:
mode
Explicit mode to set. Mode is anything os.chmod would accept
as input for mode. Works only on unix/unix-like systems.
uid
The uid to set, if not set, or it is None or -1 no changes are
made. Same applies if the path is already owned by this uid.
Must be int. Works only on unix/unix-like systems.
gid
The gid to set, if not set, or it is None or -1 no changes are
made. Same applies if the path is already owned by this gid.
Must be int. Works only on unix/unix-like systems.
u uidi����u gidu modei u chownN( R R RQ R RD t hasattrRH RI RG t S_IMODERK RJ (
R R t uidt gidt modeR� R" t d_statt it mode_part( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt fpopen� s
+ !c c@ s� | d k r t � } n y t j | � } Wn0 t j k
r] } | d k rY | | � n d SX| r� t j | � } | j d k r� | j | j f } | | k r� d S| j | � q� n g g } }
xR | D]J } t j j
| | � } t j j | � r| j | � q� |
j | � q� W| r0| | |
f Vn xg | D]_ } t j j
| | � }
| skt j j
|
� r7x( t |
| | | | � D] } | Vq�Wq7q7W| s�| | |
f Vn d S( u�
A clone of the python os.walk function with some checks for recursive
symlinks. Unlike os.walk this follows symlinks by default.
Ni ( R t setR t listdirt errorRD t st_inot st_devt addR" R' R= t appendt islinkt safe_walk( t topt topdownt onerrort followlinkst _seent namest errt statust nodet dirst nondirsR t full_patht new_patht x( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyR� � s<
c C@ s/ y t j | � Wn t t f k
r* n Xd S( u
Safely remove a file
N( R Rc R; RE ( RY ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt safe_rm s c C@ s� d � } t j j | � s. t j j | � r> t j | � nU t j j j � r� y t j j j
| � } Wq� t k
r| q� Xn t j
| d | �d S( uj
Platform-independent recursive delete. Includes code from
http://stackoverflow.com/a/2656405
c S@ sO t j j j � rH t j | t j � rH t j | t j � | | � n � d S( u/
Error handler for `shutil.rmtree`.
If the error is due to an access error (read only file)
it attempts to add write permission and then retries.
If the error is for another reason it re-raises the error.
Usage : `shutil.rmtree(path, onerror=onerror)`
N(
R R! RB RC R t accesst W_OKRJ RD t S_IWUSR( t funcR" t exc_info( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt _onerror s (
R� N( R R" R� R= Rc R R! RB RC t stringutilst
to_unicodeR� R* t rmtree( R" R� ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyR
s %
u is_emptyc C@ s3 y t j | � j d k SWn t k
r. t SXd S( u
Is a file empty?
i N( R RD t st_sizeRE R ( R� ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt is_empty< s
c C@ s | r t j j j � r t St S( u�
Simple function to check if the ``fcntl`` module is available or not.
If ``check_sunos`` is passed as ``True`` an additional check to see if host is
SunOS is also made. For additional information see: http://goo.gl/159FF8
( R R! RB t is_sunosR t HAS_FCNTL( R� ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyR� H s c C@ sb d � } t | t j � sF t j d | t j | d � j d d � � St j d | | d t j �S( u�
Input the basename of a file, without the directory tree, and returns a safe name to use
i.e. only the required characters are converted by urllib.quote
If the input is a PY2 String, output a PY2 String. If input is Unicode output Unicode.
For consistency all platforms are treated the same. Hard coded to utf8 as its ascii compatible
windows is \ / : * ? " < > | posix is /
.. versionadded:: 2017.7.2
:codeauthor: Damon Atkins <https://github.com/damon-atkins>
c S@ s t | j d � d d �S( Ni t safeu ( R t group( t re_obj( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt _replace` s u [\\:/*?"<>|]u utf8u asciiu backslashreplacet flags( Ri R t text_typet ret subt encodet UNICODE( t
file_basenameR� ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt safe_filename_leafT s c C@ s� | s t j } n t | t j � rI t | t j � rI | j d � } n t j j | � \ } } | j g | j | � D] } t
| � ^ qw � } | r� | j | | g � } n | S( u3
Input the full path and filename, splits on directory separator and calls safe_filename_leaf for
each part of the path. dir_sep allows coder to force a directory separate to a particular character
.. versionadded:: 2017.7.2
:codeauthor: Damon Atkins <https://github.com/damon-atkins>
u ascii( R R& Ri R R� R� R" t
splitdriveR' t rsplitR� ( t file_path_namet dir_sept driveR" t file_section( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt
safe_filepathk s %1u is_text_filei c @ s t j r d � n t � d j � f d � t d d � D� � d } y | j | � } WnQ t k
r� y+ t | d � � } | j | � } Wd QXWq� t k
r� t SXn Xd | k r� t S| s� t
Sy | j d
� t
SWn t k
r� n X| j
d | � } t t | � � t | � d k S( u
Uses heuristics to guess whether the given file is text or binary,
by reading a single block of bytes from the file.
If more than 30% of the chars in the block are non-text, or there
are NUL (' ') bytes in the block, assume this is a binary file.
c S@ s
t | f � S( N( t bytes( R� ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt <lambda>� s t c 3@ s | ] } � | � Vq d S( N( ( t .0R� ( t int2byte( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pys <genexpr>� s i i s
u rbNt u utf-8g333333�?( R R� t chrR' R t readR� RQ R; R R t decodet UnicodeDecodeErrort translateR t floatR� ( t fp_t blocksizet text_characterst blockt fp2_t nontext( ( R� s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt is_text� s, %
u is_bin_filec C@ s� t j j | � s t Syr t | d � �] } yA | j d � } t j rX | j t � } n t
j j j
| � SWn t k
r� t SXWd QXWn t j k
r� t SXd S( u�
Detects if the file is a binary, returns bool. Returns True if the file is
a bin, False if the file is not and None if the file is not available.
u rbi N( R R" R: R RQ R� R R� R� R� R R! R� t is_binaryR� R R� ( R" R� t data( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyR� � s
c C@ sC y t j | � Wn+ t k
r> } | j t j k r? � q? n Xd S( uS
Runs os.remove(path) and suppresses the OSError if the file doesn't exist
N( R Rc RE Ra Rd ( R" Rh ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyRc � s
u
list_filesc C@ s� t � } | j | � xw t | � D]i \ } } } x* | D]" } | j t j j | | � � q9 Wx* | D]" } | j t j j | | � � qf Wq# Wt | � S( uS
Return a list of all files found under directory (and its subdirectories)
( R� R� R� R R" R' R� ( t directoryR� R. R� R0 R ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt
list_files� s
$c C@ s1 y t | � d SWn t t f k
r, d SXd S( uj
Convert the st_mode value from a stat(2) call (as returned from os.stat())
to an octal mode.
i����u N( t octR� R� ( R� ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt st_mode_to_octal� s c C@ sz | d k r d St | t j � s4 t j | � } n t j rR | j d d � } n | j d � j d � j d � j d � S( u�
Return a mode value, normalized to a string and containing a leading zero
if it does not have one.
Allow "keep" as a valid mode (used by file state/module to preserve mode
from the Salt fileserver in file states).
u 0ou 0u "u 'i N(
R Ri R Rr R� R� R$ t stripR% t zfill( R� ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt normalize_mode� s c C@ s� i d d 6d d 6d d 6d d 6d d
6} t j | � } t j d | � } | s_ t d � � n t | j d � � } d
| j | j d � d � } | | S( u/
Convert human-readable units to bytes
i u Ki u Mi u Gi u Ti u Pu ^(\d+)([KMGTP])?$uF Size must be all digits, with an optional unit type (K, M, G, T, or P)i i ( R R� R� t matcht
ValueErrort intR� R� ( t
human_sizet size_exp_mapt human_size_strR t size_numt unit_multiplier( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt human_size_to_bytes� s ) c C@ s� t j j | � \ } } t j j j � r? | j d d � } n
| d } t j j j � sm t j | � } n t j
t t j � d � � d } t j j j � r� t j
d � } n t j
d � } d j | d | | d � } t j j | | d
j | | � � } t j j t j j | � � s>t j t j j | � � n t j | | � t j j j � s�t j | | j | j � t j | | j � n d S( u%
Backup a file on the minion
u :u _i i@B i����u %a_%b_%d_%H-%M-%S_%Yu %a_%b_%d_%H:%M:%S_%Yu
{0}{1}_{2}i����u {0}_{1}N( R R" t splitR R! RB RC R$ RD R R� R Rs t strftimeR< R' R= R> R) R* R+ RG RH RI RJ RK ( R" RZ RX RW t src_dirR[ t msecst stampt bkpath( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyRA s(
# c C@ s� d � } d � } d � } d � } t j j | � sE t d � � n y+ t | d � � } | j d � } Wd QXWn t j k
r� t d � � n X| | � } | r� | S| | � r� d
S| | � r� t S| | � r� d St d � � d S(
uo
Detect a file's encoding using the following:
- Check for Byte Order Marks (BOM)
- Check for UTF-8 Markers
- Check System Encoding
- Check for ascii
Args:
path (str): The path to the file to check
Returns:
str: The encoding of the file
Raises:
CommandExecutionError: If the encoding cannot be detected
c S@ s; y | j d � t j d � Wn t k
r2 t SXt Sd S( Nu ASCIIu Found ASCII( R� Rx t debugR� R R ( t _data( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt check_ascii7 s
c S@ sC d t j j j t j � f d t j j j t j � f d t j j j t j � f d t j j j t j � f d t j j j t j � f d t j j j d � f d t j j j d � f d t j j j d � f d t j j j d
� f d t j j j d � f g
} x7 | D]/ \ } } | j
| � r t j d | � | Sq Wt
S(
Nu UTF-32-BEu UTF-32-LEu UTF-16-BEu UTF-16-LEu UTF-8u UTF-7u +/v8-u +/v8u +/v9u +/v+u +/v/u Found BOM for %s( R R! R� t to_bytest codecst BOM_UTF32_BEt BOM_UTF32_LEt BOM_UTF16_BEt BOM_UTF16_LEt BOM_UTF8t
startswithRx R R ( R t bomst _encodingt bom( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt check_bomA s c S@ ss y | j d � } Wn t k
r' t SXt j rk x7 | D], } d t | � k o[ d k n r8 t Sq8 Wn t Sd S( Nu UTF-8i � i�� ( R� R� R R R� t ordR ( R t decodedt char( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt check_utf8_markersW s
" c S@ s. y | j t � Wn t k
r% t SXt Sd S( N( R� R� R� R R ( R ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt check_system_encodingd s
u
Not a fileu rbi Nu Failed to open fileu UTF-8u ASCIIu Could not detect file encoding( R R" R: R RQ R� R� R� ( R" R R R# R$ R� R� t encoding( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt get_encoding% s*
( u u file( u httpu httpsu ftpu swiftu s3( u saltu file(R t __doc__t
__future__R R R R t
contextlibRa t loggingR R� R* RD RS R Rs t salt.utils.pathR t salt.utils.platformt salt.utils.stringutilst salt.modules.selinuxt salt.exceptionsR R R t salt.utils.decorators.jinjaR t salt.extR t salt.ext.six.movesR t salt.ext.six.moves.urllib.parseR R� R R� RP R t getLoggert __name__Rx t LOCAL_PROTOSt
REMOTE_PROTOSt VALID_PROTOSt TEMPFILE_PREFIXt HASHESt dictt iteritemsR� t yt
HASHES_REVMAPR
R R R5 R+ R` R Rm t contextmanagerR� R� R� RQ R� R� R� R� R
R� R� R� R� R� R� Rc R� R� R R
RA R& ( ( ( s4 /usr/lib/python2.7/site-packages/salt/utils/files.pyt <module> s�
4
? L J-2
! &
Mini Shell