Current File : //usr/lib/python2.7/site-packages/salt/runners/queue.pyc
�
���^c @@ s� d Z d d l m Z m Z m Z d d l Z d d l m Z d d l m
Z
m Z d d l m
Z
d d � Z d d � Z d d
� Z d d � Z d d � Z d
d e d � Z d
d e d � Z d d d � Z d d d d d � Z d
d d d � Z d S( u�
General management and processing of queues.
============================================
This runner facilitates interacting with various queue backends such as the
included sqlite3 queue or the planned AWS SQS and Redis queues
The queue functions such as `insert`, `delete`, and `pop` can be used for
typical management of the queue.
The `process_queue` function pops the requested number of items from the queue
and creates a Salt Event that can then be processed by a Reactor. The
`process_queue` function can be called manually, or can be configured to run on
a schedule with the Salt Scheduler or regular system cron. It is also possible
to use the peer system to allow a minion to call the runner.
This runner, as well as the Queues system, is not api stable at this time.
There are many things that could potentially be done with queues within Salt.
For the time being the focus will be on queueing infrastructure actions on
specific minions. The queues generally will be populated with minion IDs. When
the `process_queue` runner function is called events are created on the Salt
Event bus that indicate the queue and a list of one or more minion IDs. The
reactor is set up to match on event tags for a specific queue and then take
infrastructure actions on those minion IDs. These actions might be to delete
the minion's key from the master, use salt-cloud to destroy the vm, or some
other custom action.
Queued runners
==============
Using the Salt Queues, references to the commandline arguments of other runners
can be saved to be processed later. The queue runners require a queue backend
that can store json data (default: :mod:`pgjsonb <salt.queues.pgjsonb_queue>`).
Once the queue is setup, the `runner_queue` will need to be configured.
.. code-block:: yaml
runner_queue:
queue: runners
backend: pgjsonb
.. note:: only the queue is required, this defaults to using pgjsonb
Once this is set, then the following can be added to the scheduler on the
master and it will run the specified amount of commands per time period.
.. code-block:: yaml
schedule:
runner queue:
schedule:
function: queue.process_runner
minutes: 1
kwargs:
quantity: 2
The above configuration will pop 2 runner jobs off the runner queue, and then
run them. And it will do this every minute, unless there are any jobs that are
still running from the last time the process_runner task was executed.
i ( t absolute_importt print_functiont unicode_literalsN( t six( t get_eventt tagify( t SaltInvocationErroru sqlitec C@ sb t j j t � } d j | � } | | k rE t d j | � � � n | | d | d | � } | S( uV
Add an item or items to a queue
CLI Example:
.. code-block:: bash
salt-run queue.insert myqueue myitem
salt-run queue.insert myqueue "['item1', 'item2', 'item3']"
salt-run queue.insert myqueue myitem backend=sqlite
salt-run queue.insert myqueue "['item1', 'item2', 'item3']" backend=sqlite
u
{0}.insertu Function "{0}" is not availablet itemst queue( t saltt loadert queuest __opts__t formatR ( R R t backendt queue_funcst cmdt ret( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt insertK s
c C@ sb t j j t � } d j | � } | | k rE t d j | � � � n | | d | d | � } | S( u
Delete an item or items from a queue
CLI Example:
.. code-block:: bash
salt-run queue.delete myqueue myitem
salt-run queue.delete myqueue myitem backend=sqlite
salt-run queue.delete myqueue "['item1', 'item2', 'item3']"
u
{0}.deleteu Function "{0}" is not availableR R ( R R
R R R
R ( R R R R R R ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt delete` s c C@ sV t j j t � } d j | � } | | k rE t d j | � � � n | | � } | S( u�
Return a list of Salt Queues on the backend
CLI Example:
.. code-block:: bash
salt-run queue.list_queues
salt-run queue.list_queues backend=sqlite
u {0}.list_queuesu Function "{0}" is not available( R R
R R R
R ( R R R R ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt list_queuest s
c C@ s\ t j j t � } d j | � } | | k rE t d j | � � � n | | d | � } | S( u�
Provide the number of items in a queue
CLI Example:
.. code-block:: bash
salt-run queue.list_length myqueue
salt-run queue.list_length myqueue backend=sqlite
u {0}.list_lengthu Function "{0}" is not availableR ( R R
R R R
R ( R R R R R ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt list_length� s c C@ s\ t j j t � } d j | � } | | k rE t d j | � � � n | | d | � } | S( u�
List contents of a queue
CLI Example:
.. code-block:: bash
salt-run queue.list_items myqueue
salt-run queue.list_items myqueue backend=sqlite
u {0}.list_itemsu Function "{0}" is not availableR ( R R
R R R
R ( R R R R R ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt
list_items� s i c C@ sh t j j t � } d j | � } | | k rE t d j | � � � n | | d | d | d | � } | S( u9
Pop one or more or all items from a queue
CLI Example:
.. code-block:: bash
salt-run queue.pop myqueue
salt-run queue.pop myqueue 6
salt-run queue.pop myqueue all
salt-run queue.pop myqueue 6 backend=sqlite
salt-run queue.pop myqueue all backend=sqlite
u {0}.popu Function "{0}" is not availablet quantityR t is_runner( R R
R R R
R ( R R R R R R R ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt pop� s c C@ s� t d t d t d d t d t �} y% t d | d | d | d | � } Wn: t k
r� } d
j | � } t j i | d 6d � t SXi | d
6| d 6| d 6} | j | t | d g d d �� | S( u4
Pop items off a queue and create an event on the Salt event bus to be
processed by a Reactor.
CLI Example:
.. code-block:: bash
salt-run queue.process_queue myqueue
salt-run queue.process_queue myqueue 6
salt-run queue.process_queue myqueue all backend=sqlite
u masteru sock_diru transportt optst listenR R R R u {0}u errorsu progressu itemsu backendu queueu processt prefix( R R t FalseR R R
t
__jid_event__t
fire_eventR ( R R R R t eventR t exct error_txtt data( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt
process_queue� s"
%
"c C@ si | d k r* t j d i � j d � } n | d k rW t j d i � j d d � } n i | d 6| d 6S( u4
Get consistent opts for the queued runners
u runner_queueu queueu backendu pgjsonbN( t NoneR t get( R R ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt __get_queue_opts� s !
c C@ s� | d k r g } n$ t | t j � r9 | j d � } n | d k rN i } n t d | d | � } i | d 6| d 6| d 6} t d | | � S( uZ
Insert a reference to a runner into the queue so that it can be run later.
fun
The runner function that is going to be run
args
list or comma-seperated string of args to send to fun
kwargs
dictionary of keyword arguments to send to fun
queue
queue to insert the runner reference into
backend
backend that to use for the queue
CLI Example:
.. code-block:: bash
salt-run queue.insert_runner test.stdout_print
salt-run queue.insert_runner event.send test_insert_runner kwargs='{"data": {"foo": "bar"}}'
u ,R R u funu argsu kwargsR N( R% t
isinstanceR t string_typest splitR' R ( t funt argst kwargsR R t queue_kwargsR# ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt
insert_runner� s c C@ sc t d | d | � } t d | d t | � } x/ | d D]# } t | d | d | d � q8 Wd S(
uC
Process queued runners
quantity
number of runners to process
queue
queue to insert the runner reference into
backend
backend that to use for the queue
CLI Example:
.. code-block:: bash
salt-run queue.process_runner
salt-run queue.process_runner 5
R R R R u itemsu funu argsu kwargsN( R' R$ t Truet __salt__( R R R R. R# t job( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt process_runner s ( t __doc__t
__future__R R R t salt.loaderR t salt.extR t salt.utils.eventR R t salt.exceptionsR R R R R R R R R$ R% R' R/ R3 ( ( ( s6 /usr/lib/python2.7/site-packages/salt/runners/queue.pyt <module>? s # &
Mini Shell