Current File : //usr/lib/python2.7/site-packages/salt/pillar/pepa.pyc
�
���^c @@ s# d Z d d l m Z m Z m Z d Z d Z d Z d Z d d l Z d d l
Z
d d l Z d d l Z d d l
Z
d d l Z d d l m Z d d l m Z d d l Z d d l Z y d d l Z e Z Wn e k
r� e Z n Xd Z e d
k rd d l Z e j � Z e j! d d d
�e j! d d d d d d �e j! d d d d d d �e j! d d d d �e j! d d d d �e j! d d d d d d �e j! d! d" d d d d# �e j! d$ d% d d d d& �e j! d' d d( d d) �e j! d* d+ d d, �e j! d- d. d d/ �e j" � Z# e j$ Z% e# j& rNe j' Z% n d Z( e# j) s�y d d l* Z* e* j+ d0 � Z( Wq�e k
r�e j, d1 � Z( q�Xn e j, d1 � Z( e j- � Z. e. j/ e% � e. j0 e( � e j1 d2 � Z e j/ e% � e j2 e. � n e j1 e � Z i i d3 d4 6d5 6d6 d7 6e d8 6Z3 d9 � Z4 d: � Z5 e e d; � Z6 d< � Z7 e d
k r e j8 j9 e# j: � s�e j; d= e# j: � e j< d> � n e j= j> j? e# j: � �# Z@ e3 jA e j= jB jC e@ � � Wd QXd ZD xW g e3 d? D]$ ZE eF eG eH eE jI � � � � ^ q�D] ZJ eJ d@ k r1Pn eD d> 7ZD qWi ZK dA e3 k r^e3 dA ZK n e# jL r�eK jA e j= jB jC e# jL � � n i ZM dB e3 k r�e3 dB ZM n e# jN r�eM jA e j= jB jC e# jN � � n e# j7 r�e e3 d8 <n e# jO rfd d l Z d d lP ZP e# jQ ZQ e# jR ZR eQ d k r7e dC � ZQ n eR d k rReP jP � ZR n e jS dD � i eQ dE 6eR dF 6dG dH 6ZT e jU e# jV dI eT � ZW eW jX s�eY dJ jZ eW j[ � � � n eW j\ � Z] e] dK d dL Z^ e jS dM � i e^ dN 6dO dP 6Z_ e j` e# jV dQ e# ja dR e_ �ZW eW j\ � j` dK i g � d Zb e# ja eb k rVeY dS � � n eb e# ja ZK n e Zc e Zd dT e3 d? eD d@ k r�e3 d? eD d@ dT Zc n dU e3 d? eD d@ k r�e3 d? eD d@ dU Zd n e6 e# ja eM e3 d? eD d@ dV e3 d? eD d@ dW ec ed � Zb e3 d8 r<e7 eb e3 d? eD d@ dV � n z� e j= jB je jf Zg dX � e j= jB je _f dY � Zh e# j) s�yb d d li Zi d d lj Zi d d lk Zi el ei jm e j= jB jn eb � ei jo jp � ei jq jr � � � Wqe k
r�eh eb � qXn
eh eb � Wd eg e j= jB je _f Xn d S(Z u�
Pepa
====
Configuration templating for SaltStack using Hierarchical substitution and Jinja.
Configuring Pepa
================
.. code-block:: yaml
extension_modules: /srv/salt/ext
ext_pillar:
- pepa:
resource: host # Name of resource directory and sub-key in pillars
sequence: # Sequence used for hierarchical substitution
- hostname: # Name of key
name: input # Alias used for template directory
base_only: True # Only use templates from Base environment, i.e. no staging
- default:
- environment:
- location..region:
name: region
- location..country:
name: country
- location..datacenter:
name: datacenter
- roles:
- osfinger:
name: os
- hostname:
name: override
base_only: True
subkey: True # Create a sub-key in pillars, named after the resource in this case [host]
subkey_only: True # Only create a sub-key, and leave the top level untouched
pepa_roots: # Base directory for each environment
base: /srv/pepa/base # Path for base environment
dev: /srv/pepa/base # Associate dev with base
qa: /srv/pepa/qa
prod: /srv/pepa/prod
# Use a different delimiter for nested dictionaries, defaults to '..' since some keys may use '.' in the name
#pepa_delimiter: ..
# Supply Grains for Pepa, this should **ONLY** be used for testing or validation
#pepa_grains:
# environment: dev
# Supply Pillar for Pepa, this should **ONLY** be used for testing or validation
#pepa_pillars:
# saltversion: 0.17.4
# Enable debug for Pepa, and keep Salt on warning
#log_level: debug
#log_granular_levels:
# salt: warning
# salt.loaded.ext.pillar.pepa: debug
Pepa can also be used in Master-less SaltStack setup.
Command line
============
.. code-block:: bash
usage: pepa.py [-h] [-c CONFIG] [-d] [-g GRAINS] [-p PILLAR] [-n] [-v]
hostname
positional arguments:
hostname Hostname
optional arguments:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
Configuration file
-d, --debug Print debug info
-g GRAINS, --grains GRAINS
Input Grains as YAML
-p PILLAR, --pillar PILLAR
Input Pillar as YAML
-n, --no-color No color output
-v, --validate Validate output
Templates
=========
Templates is configuration for a host or software, that can use information from Grains or Pillars. These can then be used for hierarchically substitution.
**Example File:** host/input/test_example_com.yaml
.. code-block:: yaml
location..region: emea
location..country: nl
location..datacenter: foobar
environment: dev
roles:
- salt.master
network..gateway: 10.0.0.254
network..interfaces..eth0..hwaddr: 00:20:26:a1:12:12
network..interfaces..eth0..dhcp: False
network..interfaces..eth0..ipv4: 10.0.0.3
network..interfaces..eth0..netmask: 255.255.255.0
network..interfaces..eth0..fqdn: {{ hostname }}
cobbler..profile: fedora-19-x86_64
As you see in this example you can use Jinja directly inside the template.
**Example File:** host/region/amer.yaml
.. code-block:: yaml
network..dns..servers:
- 10.0.0.1
- 10.0.0.2
time..ntp..servers:
- ntp1.amer.example.com
- ntp2.amer.example.com
- ntp3.amer.example.com
time..timezone: America/Chihuahua
yum..mirror: yum.amer.example.com
Each template is named after the value of the key using lowercase and all extended characters are replaced with underscore.
**Example:**
osfinger: Fedora-19
**Would become:**
fedora_19.yaml
Nested dictionaries
===================
In order to create nested dictionaries as output you can use double dot **".."** as a delimiter. You can change this using "pepa_delimiter" we choose double dot since single dot is already used by key names in some modules, and using ":" requires quoting in the YAML.
**Example:**
.. code-block:: yaml
network..dns..servers:
- 10.0.0.1
- 10.0.0.2
network..dns..options:
- timeout:2
- attempts:1
- ndots:1
network..dns..search:
- example.com
**Would become:**
.. code-block:: yaml
network:
dns:
servers:
- 10.0.0.1
- 10.0.0.2
options:
- timeout:2
- attempts:1
- ndots:1
search:
- example.com
Operators
=========
Operators can be used to merge/unset a list/hash or set the key as immutable, so it can't be changed.
=========== ================================================
Operator Description
=========== ================================================
merge() Merge list or hash
unset() Unset key
immutable() Set the key as immutable, so it can't be changed
imerge() Set immutable and merge
iunset() Set immutable and unset
=========== ================================================
**Example:**
.. code-block:: yaml
network..dns..search..merge():
- foobar.com
- dummy.nl
owner..immutable(): Operations
host..printers..unset():
Validation
==========
Since it's very hard to test Jinja as is, the best approach is to run all the permutations of input and validate the output, i.e. Unit Testing.
To facilitate this in Pepa we use YAML, Jinja and Cerberus <https://github.com/nicolaiarocci/cerberus>.
Schema
======
So this is a validation schema for network configuration, as you see it can be customized with Jinja just as Pepa templates.
This was designed to be run as a build job in Jenkins or similar tool. You can provide Grains/Pillar input using either the config file or command line arguments.
**File Example: host/validation/network.yaml**
.. code-block:: jinja
network..dns..search:
type: list
allowed:
- example.com
network..dns..options:
type: list
allowed: ['timeout:2', 'attempts:1', 'ndots:1']
network..dns..servers:
type: list
schema:
regex: ^([0-9]{1,3}\.){3}[0-9]{1,3}$
network..gateway:
type: string
regex: ^([0-9]{1,3}\.){3}[0-9]{1,3}$
{% if network.interfaces is defined %}
{% for interface in network.interfaces %}
network..interfaces..{{ interface }}..dhcp:
type: boolean
network..interfaces..{{ interface }}..fqdn:
type: string
regex: ^([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)+[a-zA-Z]{2,6}$
network..interfaces..{{ interface }}..hwaddr:
type: string
regex: ^([0-9a-f]{1,2}\:){5}[0-9a-f]{1,2}$
network..interfaces..{{ interface }}..ipv4:
type: string
regex: ^([0-9]{1,3}\.){3}[0-9]{1,3}$
network..interfaces..{{ interface }}..netmask:
type: string
regex: ^([0-9]{1,3}\.){3}[0-9]{1,3}$
{% endfor %}
{% endif %}
Links
=====
For more examples and information see <https://github.com/mickep76/pepa>.
i ( t absolute_importt print_functiont unicode_literalsu/ Michael Persson <michael.ake.persson@gmail.com>u" Copyright (c) 2013 Michael Perssonu Apache License, Version 2.0u 0.6.6N( t six( t inputu __main__u hostnamet helpu Hostnameu -cu --configt defaultu /etc/salt/masteru Configuration fileu -du --debugt actionu
store_trueu Print debug infou -gu --grainsu Input Grains as YAMLu -pu --pillaru Input Pillar as YAMLu -nu
--no-coloru No color outputu -vu
--validateu Validate outputu -qu --query-apiu# Query Saltstack REST API for Grainsu --urlu https://salt:8000u URL for SaltStack REST APIu -uu
--usernameu Username for SaltStack REST APIu -Pu
--passwordu Password for SaltStack REST APIuI [%(log_color)s%(levelname)-8s%(reset)s] %(log_color)s%(message)s%(reset)su [%(levelname)-8s] %(message)su pythonConfigu /srv/saltu baseu
pepa_rootsu ..u pepa_delimiteru
pepa_validatec C@ s t s
t St S( u6
Only return if all the modules are available
( t HAS_REQUESTSt Falset True( ( ( s4 /usr/lib/python2.7/site-packages/salt/pillar/pepa.pyt __virtual__] s c C@ s� i } x� t j | � D]v \ } } | } | j t d � } xN t | d � D]= \ } } | t | � k rv | | | <qK | j | i � } qK Wq W| S( u#
Convert key/value to tree
u pepa_delimiteri ( R t iteritemst splitt __opts__t enumeratet lent
setdefault( t datat treet flatkeyt valuet tt keyst it key( ( s4 /usr/lib/python2.7/site-packages/salt/pillar/pepa.pyt key_value_to_treeg s
c C@ sx t d } i } d | d <| | d <d | k rA | d | d <n' d t k r^ t d | d <n
d | d <| } g | d <i } x�g | D] }
t t j |
� � ^ q� D]\\ } } | | k r� t j d | � q� n d }
t | t � rd | k r| d }
n | }
d } | rJd | k rJ| d rJt
j j | d | |
� } n t
j j | | d | |
� } g } t | | t
� r�| | } n- | | s�t j d
| � q� n
| | g } xC| D];} d } d } t
j j | t j d d | j � � d
� } t
j j | � rt j d | � t j j j | � � } t j | j � � } Wd QX| d j | � yT t | � } t j � | d <| j � | d <| j | � } t j j j | � } Wq.t j k
r�} t j! d | | � q.t j j j" k
r} t j! d | | � q.Xn t j d | � q�| d k r�x�| D]�} | j# t d d � } d } d } t$ | � d k r�| j% d � d k r�| d j& t d � } | d } n | | k r�t j d | � qA| | k r�t j d | � qA| d k s
| d k r | d k r0t j' d | | | � n! t j' d | | | � t( | | <| | k rpt j! d | � q�t | | t) | | � � s�t j! d | � q�t | | t � r�| | j* | | � q�t | | t
� r�| | j+ | | � q�t j! d | � qA| d! k s!| d" k rs| d! k r@t j' d# | � n t j' d$ | � t( | | <| | k r�| | =q�qA| d% k r�t j' d& | | | � t( | | <| | | | <qA| d k r�t j! d' | | � qAt j' d( | | | � | | | | <qAWq�q�Wq� Wt | � } i } | r2| j � | | <n% | rQ| } | j � | | <n | } t d) rt| j � | d* <n | S(+ u!
Evaluate Pepa templates
u
pepa_rootsu defaultu hostnameu environmentu baseu pepa_templatesu Category is not defined: %su nameu base_onlyu Category has no value set: %su \Wu _u .yamlu Loading template: %sNu grainsu pillaru% Failed to parse JINJA template: %s
%su' Failed to parse YAML in template: %s
%su Template doesn't exist: %su pepa_delimiteri u ()i u, Key %s is immutable, changes are not allowedu merge()u imerge()u Merge key %s: %su" Set immutable and merge key %s: %su! Cant't merge key %s doesn't existu& Can't merge different types for key %su3 Unsupported type need to be list or dict for key %su unset()u iunset()u Unset key %su Set immutable and unset key %su immutable()u' Set immutable and substitute key %s: %su( Unsupported operator %s, skipping key %su Substitute key %s: %su
pepa_validateu pepa_keys(, R t
__grains__t nextR R t logt warningt Nonet
isinstancet dictt ost patht joint listt ret subt lowert isfilet infot saltt utilst filest fopent jinja2t Templatet readt appendR t copyt rendert yamlt safe_loadt UndefinedErrort errort YAMLErrort rsplitR t rfindt rstript debugR
t typet updatet extend( t minion_idt pillart resourcet sequencet subkeyt subkey_onlyt rootst inpt outputt immutablet st categR* t aliast templdirt entriest entryt
results_jinjat resultst fnt fhrt templateR t errR t skeyt rkeyt operatorR t pillar_data( ( s4 /usr/lib/python2.7/site-packages/salt/pillar/pepa.pyt
ext_pillarw s�
5
.
'
c C@ s� y d d l } Wn t k
r1 t j d � d SXt d } t j j | d | d � } i } g } x� t j | d � D]� } t j d | � t
j j j
| � � } t j | j � � } Wd QX| }
t j � |
d <t j � |
d
<t
j j j | j |
� � } | j | � | j | � qx W| j � } | j | d | � s~x6 t j | j � D] \ }
} t j d |
| � qXWn | | d
<| | d <d S( u!
Validate Pepa templates
i Nu3 You need module cerberus in order to use validationu
pepa_rootsu baseu validateu /*.yamlu Loading schema: %su grainsu pillaru pepa_keysu Validation failed for key %s: %su pepa_schema_keysu pepa_schemas( t cerberust ImportErrorR t criticalR R" R# R$ t globR* R+ R, R- R. R/ R0 R1 R R3 t
__pillar__R5 R6 R4 R? R2 t Validatort validateR R t errorsR ( RI RC R\ RG t valdirt all_schemast pepa_schemasRS RT RU R t schemat valt ekeyR8 ( ( s4 /usr/lib/python2.7/site-packages/salt/pillar/pepa.pyRb � s2
u$ Configuration file doesn't exist: %si u
ext_pillaru pepau pepa_grainsu pepa_pillaru
Username: u Authenticate REST APIu usernameu passwordu pamu eauthu /loginu1 Failed to authenticate to SaltStack REST API: {0}u returnu tokenu Request Grains from REST APIu X-Auth-Tokenu application/jsonu Acceptu /minions/t headersu, Failed to get Grains from SaltStack REST APIu subkeyu subkey_onlyu resourceu sequencec C@ s t S( N( R
( t xt y( ( s4 /usr/lib/python2.7/site-packages/salt/pillar/pepa.pyt <lambda>s s c C@ s) t t j j j | d d d t �� d S( Nt indenti t default_flow_style( t printR+ R, R5 t safe_dumpR ( t result( ( s4 /usr/lib/python2.7/site-packages/salt/pillar/pepa.pyt
_print_resultu s (s t __doc__t
__future__R R R t
__author__t
__copyright__t __license__t __version__R_ R/ t loggingR" R& t syst salt.extR t salt.ext.six.movesR t salt.utils.filesR+ t salt.utils.yamlt requestsR
R R] R R R t __name__t argparset ArgumentParsert parsert add_argumentt
parse_argst argst WARNINGt LOG_LEVELR= t DEBUGt formattert no_colort colorlogt ColoredFormattert Formattert
StreamHandlert streamt setLevelt setFormattert getLoggert
addHandlerR R R R[ Rb R# R) t configR^ t exitR, R- R. t fh_R? R5 R6 t loct eR t iterR% R t nameR t grainsR` RB t query_apit getpasst usernamet passwordR* t autht postt urlt requestt okt RuntimeErrort formatt textt jsont responset tokenRj t gett hostnameRr t ex_subkeyt ex_subkey_onlyt SafeOrderedDumpert ignore_aliasest orig_ignoreRs t pygmentst pygments.lexerst pygments.formattersRp t highlightRq t lexerst YamlLexert
formatterst TerminalFormatter( ( ( s4 /usr/lib/python2.7/site-packages/salt/pillar/pepa.pyt <module> s
� % "<
"
"
#
Mini Shell