Current File : //lib/python2.7/site-packages/salt/pillar/consul_pillar.pyo
�
���^c @@ s d Z d d l m Z m Z m Z d d l Z d d l Z d d l m Z d d l m
Z d d l Z
d d l Z
y+ d d l Z e e d � s� d e _ n Wn e k
r� d Z n Xd Z e j e � Z d � Z d
� Z d � Z d � Z d
� Z d � Z d � Z d S( u0
Use Consul K/V as a Pillar source with values parsed as YAML
:depends: - python-consul
In order to use an consul server, a profile must be created in the master
configuration file:
.. code-block:: yaml
my_consul_config:
consul.host: 127.0.0.1
consul.port: 8500
consul.token: b6376760-a8bb-edd5-fcda-33bc13bfc556
consul.scheme: http
consul.consistency: default
consul.dc: dev
consul.verify: True
All parameters are optional.
The ``consul.token`` requires python-consul >= 0.4.7.
If you have a multi-datacenter Consul cluster you can map your ``pillarenv``s
to your data centers by providing a dictionary of mappings in ``consul.dc``
field:
.. code-block:: yaml
my_consul_config:
consul.dc:
dev: us-east-1
prod: us-west-1
In the example above we specifying static mapping between Pillar environments
and data centers: the data for ``dev`` and ``prod`` Pillar environments will
be fetched from ``us-east-1`` and ``us-west-1`` datacenter respectively.
In fact when ``consul.dc`` is set to dictionary keys are processed as regular
expressions (that can capture named parameters) and values are processed as
string templates as per PEP 3101.
.. code-block:: yaml
my_consul_config:
consul.dc:
^dev-.*$: dev-datacenter
^(?P<region>.*)-prod$: prod-datacenter-{region}
This example maps all Pillar environments starting with ``dev-`` to
``dev-datacenter`` whereas Pillar environment like ``eu-prod`` will be
mapped to ``prod-datacenter-eu``.
Before evaluation patterns are sorted by length in descending order.
If Pillar environment names correspond to data center names a single pattern
can be used:
.. code-block:: yaml
my_consul_config:
consul.dc:
^(?P<env>.*)$: '{env}'
After the profile is created, configure the external pillar system to use it.
Optionally, a root may be specified.
.. code-block:: yaml
ext_pillar:
- consul: my_consul_config
ext_pillar:
- consul: my_consul_config root=salt
Using these configuration profiles, multiple consul sources may also be used:
.. code-block:: yaml
ext_pillar:
- consul: my_consul_config
- consul: my_other_consul_config
Either the ``minion_id``, or the ``role``, or the ``environment`` grain may be used in the ``root``
path to expose minion-specific information stored in consul.
.. code-block:: yaml
ext_pillar:
- consul: my_consul_config root=salt/%(minion_id)s
- consul: my_consul_config root=salt/%(role)s
- consul: my_consul_config root=salt/%(environment)s
Minion-specific values may override shared values when the minion-specific root
appears after the shared root:
.. code-block:: yaml
ext_pillar:
- consul: my_consul_config root=salt-shared
- consul: my_other_consul_config root=salt-private/%(minion_id)s
If using the ``role`` or ``environment`` grain in the consul key path, be sure to define it using
`/etc/salt/grains`, or similar:
.. code-block:: yaml
role: my-minion-role
environment: dev
It's possible to lock down where the pillar values are shared through minion
targeting. Note that double quotes ``"`` are required around the target value
and cannot be used inside the matching statement. See the section on Compound
Matchers for more examples.
.. code-block:: yaml
ext_pillar:
- consul: my_consul_config root=salt target="L@salt.example.com and G@osarch:x86_64"
The data from Consul can be merged into a nested key in Pillar.
.. code-block:: yaml
ext_pillar:
- consul: my_consul_config pillar_root=consul_data
By default, keys containing YAML data will be deserialized before being merged into Pillar.
This behavior can be disabled by setting ``expand_keys`` to ``false``.
.. code-block:: yaml
ext_pillar:
- consul: my_consul_config expand_keys=false
i ( t absolute_importt print_functiont unicode_literalsN( t CommandExecutionError( t updateu __version__u 0.1u consulc C@ s t d k r t St S( u3
Only return if python-consul is installed
N( t consult Nonet __virtualname__t False( ( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyt __virtual__� s c C@ s� i } | } t j d � } | j | � } | r� | j d � | d <| j | j d � d � } t j j j t � } | j
| d d � } | d } t j d | � | | k r� i Sn t j d � }
|
j | � } | r | j d � | d
<| j | j d � d � } n
d | d
<t j d � } | j | � } | rj| j d � | d <| j | j d � d � } n
d | d <t j d
� } | j | � } | r�| j d � | d <| j | j d � d � } n
d | d <t j d t j � }
|
j | � } | r%t | d <| j | j d � d � } n
t | d <t t | d � } t d d d � } t d d d � } | d
c i | d 6| d 6| d 6;<y� t | | d
| d � } | d rNt j d | d
| d � i } | } | d j d � j d � } xZ t | � D]C \ } } | t | � d k r3| | | <qi | | <| | } qWn | } Wn2 t k
r�t j d | d | d
� i } n X| S( u#
Check consul for all data
u target="(.*?)"i u targeti u u compoundu minionsu Targeted minions: %ru
root=(\S*)u rootu pillar_root=(\S*)u pillar_rootu (?:profile=)?(\S+)u profileu expand_keys=Falseu expand_keysu
grains.getu roleu environmentu minion_idu* Merging consul path %s/ into pillar at %s/u /u$ No such key in consul profile %s: %sN( t ret compilet searcht groupt replacet saltt utilst minionst CkMinionst __opts__t
check_minionst logt debugR t
IGNORECASER t Truet get_connt __salt__t
fetch_treet rstript splitt enumeratet lent KeyErrort error( t minion_idt pillart conft optst tempt target_ret matcht checkert _resR t root_ret pillar_root_ret
profile_ret expand_keys_ret clientt rolet environmentt pillar_treet brancht keyst it k( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyt
ext_pillar� sz
c C@ s | j j | d t �S( u;
Query consul for all keys/values within base path
t recurse( t kvt getR ( R/ t path( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyt consul_fetch s c C@ s t | | � \ } } i } t j d � } t j d | � | d k rJ | Sx� t | � D]� } t j d | d d | d � } | d k r� t j d | | � t j d | j | � � n | j | � d k rW t | | j
d � | d
| � } t j d | � qW qW W| S(
u�
Grab data from consul, trim base path and remove any keys which
are folders. Take the remaining data and send it to be formatted
in such a way as to be used as pillar data.
u /$u Fetched items: %ru ^u /?u u Keyu key/path - %s: %su has_children? %ru /u Valueu Fetching subkeys for key: %rN( R< R
R R R R t reversedt subR t
pillar_formatR ( R/ R; t expand_keyst _t itemst rett has_childrent itemt key( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyR s ! "c C@ s� | d k r | S| r. t j j j | � } n | } | j � } i | | 6} | j � x | D] } i | | 6} q^ Wt | | � S( ui
Perform data formatting to be used as pillar data and
merge it with the current pillar data
N( R R R t yamlt safe_loadt popt reverset
dict_merge( RC R4 t valueR@ t pillar_valuet keyvaluet pilR6 ( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyR? " s
c C@ sB | j d i � } | j d i � } i } | j | � | j | � | j | � | rl | j | i � } n | } i } x; | D]3 } | j d � r | | | | j d � d <q q Wd | k r� | j d � p� d } t | d | � | d <n t r2t j d k r%| j d
� r%| j d
� n t j | � St d � � d S(
u5
Return a client object for accessing consul
u pillaru masteru consul.u .i u dcu pillarenvu baseu 0.4.7u targetuw (unable to import consul, module most likely not installed. Download python-consul module and be sure to import consul)N(
R: R t
startswithR t _resolve_datacenterR t __version__RI t ConsulR ( R% t profilet opts_pillart opts_mastert opts_mergedR$ t paramsRF t pillarenv( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyR < s,
"
c C@ s t j d | � y | j � } Wn" t k
rD t j d | � | SXt j d t | � � t j d | � t | d d � �} xe | D]] \ } } t j | | � } | r� t j d | � | j | j � � } t j d | � | Sq� Wt j d | d
j
d � | D� � � d S(
u�
If ``dc`` is a string - return it as is.
If it's a dict then sort it in descending order by key length and try
to use keys as RegEx patterns to match against ``pillarenv``.
The value for matched pattern should be a string (that can use
``str.format`` syntax togetehr with captured variables from pattern)
pointing to targe data center to use.
If none patterns matched return ``None`` which meanse us datacenter of
conencted Consul agent.
u( Resolving Consul datacenter based on: %su Using pre-defined DC: '%s'u3 Selecting DC based on pillarenv using %d pattern(s)u Pillarenv set to '%s'RF c S@ s t | d � | d f S( Ni ( R ( t m( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyt <lambda>~ s u Matched pattern: '%s'u Resolved datacenter: '%s'u3 None of following patterns matched pillarenv=%s: %su , c s@ s | ] } t | � Vq d S( N( t repr( t .0t x( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pys <genexpr>� s N( R R RB t AttributeErrorR t sortedR
R( t formatt groupdictt join( t dcRY t mappingst sorted_mappingst patternt targetR( t result( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyRQ d s&
( t __doc__t
__future__R R R t loggingR
t salt.exceptionsR t salt.utils.dictupdateR RK t salt.utils.minionsR t salt.utils.yamlR t hasattrRR t ImportErrorR R t getLoggert __name__R R R7 R< R R? R RQ ( ( ( s= /usr/lib/python2.7/site-packages/salt/pillar/consul_pillar.pyt <module>� s,
W (
Mini Shell