Current File : //lib/python2.7/site-packages/salt/pillar/file_tree.pyc
�
���^c @@ s d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l Z d d l
Z d d l Z d d l Z d d l
Z d d l Z d d l Z d d l Z d d l m Z e j e � Z d � Z d � Z e d d d e d � Z d e e e d d d e d � Z d � Z d S(
uo
The ``file_tree`` external pillar allows values from all files in a directory
tree to be imported as Pillar data.
.. note::
This is an external pillar and is subject to the :ref:`rules and
constraints <external-pillars>` governing external pillars.
.. versionadded:: 2015.5.0
In this pillar, data is organized by either Minion ID or Nodegroup name. To
setup pillar data for a specific Minion, place it in
``<root_dir>/hosts/<minion_id>``. To setup pillar data for an entire
Nodegroup, place it in ``<root_dir>/nodegroups/<node_group>`` where
``<node_group>`` is the Nodegroup's name.
Example ``file_tree`` Pillar
============================
Master Configuration
--------------------
.. code-block:: yaml
ext_pillar:
- file_tree:
root_dir: /srv/ext_pillar
follow_dir_links: False
keep_newline: True
The ``root_dir`` parameter is required and points to the directory where files
for each host are stored. The ``follow_dir_links`` parameter is optional and
defaults to False. If ``follow_dir_links`` is set to True, this external pillar
will follow symbolic links to other directories.
.. warning::
Be careful when using ``follow_dir_links``, as a recursive symlink chain
will result in unexpected results.
.. versionchanged:: 2018.3.0
If ``root_dir`` is a relative path, it will be treated as relative to the
:conf_master:`pillar_roots` of the environment specified by
:conf_minion:`pillarenv`. If an environment specifies multiple
roots, this module will search for files relative to all of them, in order,
merging the results.
If ``keep_newline`` is set to ``True``, then the pillar values for files ending
in newlines will keep that newline. The default behavior is to remove the
end-of-file newline. ``keep_newline`` should be turned on if the pillar data is
intended to be used to deploy a file using ``contents_pillar`` with a
:py:func:`file.managed <salt.states.file.managed>` state.
.. versionchanged:: 2015.8.4
The ``raw_data`` parameter has been renamed to ``keep_newline``. In earlier
releases, ``raw_data`` must be used. Also, this parameter can now be a list
of globs, allowing for more granular control over which pillar values keep
their end-of-file newline. The globs match paths relative to the
directories named for minion IDs and nodegroups underneath the ``root_dir``
(see the layout examples in the below sections).
.. code-block:: yaml
ext_pillar:
- file_tree:
root_dir: /path/to/root/directory
keep_newline:
- files/testdir/*
.. note::
In earlier releases, this documentation incorrectly stated that binary
files would not affected by the ``keep_newline`` configuration. However,
this module does not actually distinguish between binary and text files.
.. versionchanged:: 2017.7.0
Templating/rendering has been added. You can now specify a default render
pipeline and a black- and whitelist of (dis)allowed renderers.
``template`` must be set to ``True`` for templating to happen.
.. code-block:: yaml
ext_pillar:
- file_tree:
root_dir: /path/to/root/directory
render_default: jinja|yaml
renderer_blacklist:
- gpg
renderer_whitelist:
- jinja
- yaml
template: True
Assigning Pillar Data to Individual Hosts
-----------------------------------------
To configure pillar data for each host, this external pillar will recursively
iterate over ``root_dir``/hosts/``id`` (where ``id`` is a minion ID), and
compile pillar data with each subdirectory as a dictionary key and each file
as a value.
For example, the following ``root_dir`` tree:
.. code-block:: text
./hosts/
./hosts/test-host/
./hosts/test-host/files/
./hosts/test-host/files/testdir/
./hosts/test-host/files/testdir/file1.txt
./hosts/test-host/files/testdir/file2.txt
./hosts/test-host/files/another-testdir/
./hosts/test-host/files/another-testdir/symlink-to-file1.txt
will result in the following pillar tree for minion with ID ``test-host``:
.. code-block:: text
test-host:
----------
apache:
----------
config.d:
----------
00_important.conf:
<important_config important_setting="yes" />
20_bob_extra.conf:
<bob_specific_cfg has_freeze_ray="yes" />
corporate_app:
----------
settings:
----------
common_settings:
// This is the main settings file for the corporate
// internal web app
main_setting: probably
bob_settings:
role: bob
.. note::
The leaf data in the example shown is the contents of the pillar files.
i ( t absolute_importt print_functiont unicode_literalsN( t sixc C@ s t j d | j | j � d S( u.
Log salt.utils.path.os_walk() error.
u %s: %sN( t logt errort filenamet strerror( t err( ( s9 /usr/lib/python2.7/site-packages/salt/pillar/file_tree.pyt _on_walk_error� s c C@ s� t | t � r | St j j | | � } x[ | D]S } y t j | | � rO t SWq0 t k
r� t j | t j | � � r� t Sq0 Xq0 Wt
S( u`
Return a boolean stating whether or not a file's trailing newline should be
removed. To figure this out, first check if keep_newline is a boolean and
if so, return its opposite. Otherwise, iterate over keep_newline and check
if any of the patterns match the file path. If a match is found, return
False, otherwise return True.
( t
isinstancet boolt ost patht joint fnmatcht Falset TypeErrorR t text_typet True( t prefixt file_namet keep_newlinet full_patht pattern( ( s9 /usr/lib/python2.7/site-packages/salt/pillar/file_tree.pyt _check_newline� s
c C@ s� i } t j j t t � } t j j | � } x�t j j j | d t
d t d | �D]`\ }
} } | }
t j j |
� } t j j | | � } | | k rg } | } x2 | r� t j j
| � \ } } | j d | � q� Wx | r� |
| j d � }
q� Wn x | D] } i |
| <qWx�| D]�} t j j |
| � } t j j | � sft j d | � q#n d } y� t j j j | d � �r } | j t d � } x' | r�| | 7} | j t d � } q�W| j d � r�t | | | � r�| d
} n Wd QXWn/ t t f k
r-} t j d | | j � q#X| } | t
k rt j j d
t j j j | � d | d | d | d | � } n t j j j! | � r�| j" � |
| <q#| |
| <q#WqU W| S( u*
Construct pillar from file tree.
t topdownt onerrort followlinksi u! file_tree: %s: not a regular filet u rbu file_buffer_sizes
i����Nu file_tree: Error reading %s: %st templatet rendererst defaultt blacklistt whitelist(# t saltt loadert rendert __opts__t __salt__R R
t normpatht utilst os_walkR R t relpatht splitt insertt popR t isfileR R t filest fopent readt endswithR t IOErrort OSErrorR R t compile_template_strt stringutilst
to_unicodet stringiot is_readablet getvalue( t top_dirt follow_dir_linksR t render_defaultt renderer_blacklistt renderer_whitelistR t pillarR t norm_top_dirt dir_patht dir_namest
file_namest pillar_nodet
norm_dir_pathR t
path_partst headt tailt dir_nameR t file_patht contentst fhrt buft exct data( ( s9 /usr/lib/python2.7/site-packages/salt/pillar/file_tree.pyt _construct_pillar� sf
! c
C@ sl ~ | s t j d � i St j j | � st d }
|
d
k rS t j d � i St j d |
� t d j |
d
� } | d
k r� t j d |
� i Sg } x[ | D]S }
t j j t j j
|
| � � } | | k s� | | d k r� | j | � q� q� W| } n | g } i } xP | D]H } t | | | | | | | | | � } t
j j j | | d d �} q W| S( ul
Compile pillar data from the given ``root_dir`` specific to Nodegroup names
and Minion IDs.
If a Minion's ID is not found at ``<root_dir>/host/<minion_id>`` or if it
is not included in any Nodegroups named at
``<root_dir>/nodegroups/<node_group>``, no pillar data provided by this
pillar module will be available for that Minion.
.. versionchanged:: 2017.7.0
Templating/rendering has been added. You can now specify a default
render pipeline and a black- and whitelist of (dis)allowed renderers.
``template`` must be set to ``True`` for templating to happen.
.. code-block:: yaml
ext_pillar:
- file_tree:
root_dir: /path/to/root/directory
render_default: jinja|yaml
renderer_blacklist:
- gpg
renderer_whitelist:
- jinja
- yaml
template: True
:param minion_id:
The ID of the Minion whose pillar data is to be collected
:param pillar:
Unused by the ``file_tree`` pillar module
:param root_dir:
Filesystem directory used as the root for pillar data (e.g.
``/srv/ext_pillar``)
.. versionchanged:: 2018.3.0
If ``root_dir`` is a relative path, it will be treated as relative to the
:conf_master:`pillar_roots` of the environment specified by
:conf_minion:`pillarenv`. If an environment specifies multiple
roots, this module will search for files relative to all of them, in order,
merging the results.
:param follow_dir_links:
Follow symbolic links to directories while collecting pillar files.
Defaults to ``False``.
.. warning::
Care should be exercised when enabling this option as it will
follow links that point outside of ``root_dir``.
.. warning::
Symbolic links that lead to infinite recursion are not filtered.
:param debug:
Enable debug information at log level ``debug``. Defaults to
``False``. This option may be useful to help debug errors when setting
up the ``file_tree`` pillar module.
:param keep_newline:
Preserve the end-of-file newline in files. Defaults to ``False``.
This option may either be a boolean or a list of file globs (as defined
by the `Python fnmatch package
<https://docs.python.org/library/fnmatch.html>`_) for which end-of-file
newlines are to be kept.
``keep_newline`` should be turned on if the pillar data is intended to
be used to deploy a file using ``contents_pillar`` with a
:py:func:`file.managed <salt.states.file.managed>` state.
.. versionchanged:: 2015.8.4
The ``raw_data`` parameter has been renamed to ``keep_newline``. In
earlier releases, ``raw_data`` must be used. Also, this parameter
can now be a list of globs, allowing for more granular control over
which pillar values keep their end-of-file newline. The globs match
paths relative to the directories named for Minion IDs and
Nodegroup namess underneath the ``root_dir``.
.. code-block:: yaml
ext_pillar:
- file_tree:
root_dir: /srv/ext_pillar
keep_newline:
- apache/config.d/*
- corporate_app/settings/*
.. note::
In earlier releases, this documentation incorrectly stated that
binary files would not affected by the ``keep_newline``. However,
this module does not actually distinguish between binary and text
files.
:param render_default:
Override Salt's :conf_master:`default global renderer <renderer>` for
the ``file_tree`` pillar.
.. code-block:: yaml
render_default: jinja
:param renderer_blacklist:
Disallow renderers for pillar files.
.. code-block:: yaml
renderer_blacklist:
- json
:param renderer_whitelist:
Allow renderers for pillar files.
.. code-block:: yaml
renderer_whitelist:
- yaml
- jinja
:param template:
Enable templating of pillar files. Defaults to ``False``.
u file_tree: no root_dir specifiedu pillarenvu8 file_tree: root_dir is relative but pillarenv is not setu file_tree: pillarenv = %su pillar_rootsuS file_tree: root_dir is relative but no pillar_roots are specified for pillarenv %si����t strategyu recurseN( R R R R
t isabsR& t Nonet debugt getR( R t appendt _ext_pillarR# R) t
dictupdatet merge( t minion_idRA t root_dirR= RV R R> R? R@ R t pillarenvt env_rootst env_dirst env_roott env_dirt dirst
result_pillart roott
dir_pillar( ( s9 /usr/lib/python2.7/site-packages/salt/pillar/file_tree.pyt
ext_pillar sJ �
!
c
C@ s� t j d | � t j j | � s6 t j d | � i St | t t f � s_ t j d | � i Si } t j j | d � }
t j j
|
� r�t t j
d d � � d k r�t d } t j |
� } t | � d k r�x� | D]� }
t j j |
� r� |
| k r� t j j j t � } | j | |
d � } | d } | | k r�t j j |
t j |
� � } t j j j | t | | | | | | | � d d �} q�q� q� Wq�| t k r�t j d
| � q�n | t k r�t j d � n t j j | d | � } t j j
| � s0| t k r,t j d
| | � n | St j j | � sVt j d | � | St | | | | | | | � } t j j j | | d d �S( uO
Compile pillar data for a single root_dir for the specified minion ID
u file_tree: reading %su; file_tree: root_dir %s does not exist or is not a directoryus file_tree: keep_newline must be either True/False or a list of file globs. Skipping this ext_pillar for root_dir %su
nodegroupsi u compoundu minionsRS u recurseuE file_tree: no nodegroups found in file tree directory %s, skipping...u6 file_tree: no nodegroups found in master configurationu hostsuG file_tree: no pillar data for minion %s found in file tree directory %su, file_tree: %s exists, but is not a directory( ( R RV R R
t isdirR R
R t listR t existst lenR& RW t listdirR# R) t minionst CkMinionst
check_minionsR R RZ R[ RR R ( R\ R] R= RV R R> R? R@ R t
ngroup_pillart nodegroups_dirt master_ngroupst ext_pillar_dirst nodegroupt ckminionst _rest matcht
ngroup_dirt host_dirt host_pillar( ( s9 /usr/lib/python2.7/site-packages/salt/pillar/file_tree.pyRY � s� 0
( t __doc__t
__future__R R R R t loggingR t salt.loaderR# t salt.utils.dictupdatet salt.utils.filest salt.utils.minionst salt.utils.patht salt.utils.stringiot salt.utils.stringutilst
salt.templatet salt.extR t getLoggert __name__R R R R RU RR Rg RY ( ( ( s9 /usr/lib/python2.7/site-packages/salt/pillar/file_tree.pyt <module>� s< E �
Mini Shell