Current File : //usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyc
�
���^c @@ s� d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l m Z d d l m
Z
d d l Z d d l Z
d d l Z
d d l Z
d d l Z
d d l Z
d d l m Z d d l j j Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d l Z d d l! Z d d l" Z d d l# m$ Z$ m% Z% m& Z& e j' j( j) � e j' j* j+ � Z* e j, e- � Z. d � Z/ d
Z0 d Z1 d e2 f d
� � YZ3 d e f d � � YZ4 d e5 f d � � YZ6 d e
j7 j8 f d � � YZ9 d e9 f d � � YZ: d e9 f d � � YZ; d e; f d � � YZ< d e; f d � � YZ= d e; f d � � YZ> d e; f d � � YZ? d e; f d! � � YZ@ d" � ZA d S(# u�
A non-blocking REST API for Salt
================================
.. py:currentmodule:: salt.netapi.rest_tornado.saltnado
:depends: - tornado Python module
:configuration: All authentication is done through Salt's :ref:`external auth
<acl-eauth>` system which requires additional configuration not described
here.
In order to run rest_tornado with the salt-master
add the following to the Salt master config file.
.. code-block:: yaml
rest_tornado:
# can be any port
port: 8000
# address to bind to (defaults to 0.0.0.0)
address: 0.0.0.0
# socket backlog
backlog: 128
ssl_crt: /etc/pki/api/certs/server.crt
# no need to specify ssl_key if cert and key
# are in one single file
ssl_key: /etc/pki/api/certs/server.key
debug: False
disable_ssl: False
webhook_disable_auth: False
cors_origin: null
.. _rest_tornado-auth:
Authentication
--------------
Authentication is performed by passing a session token with each request.
Tokens are generated via the :py:class:`SaltAuthHandler` URL.
The token may be sent in one of two ways:
* Include a custom header named :mailheader:`X-Auth-Token`.
* Sent via a cookie. This option is a convenience for HTTP clients that
automatically handle cookie support (such as browsers).
.. seealso:: You can bypass the session handling via the :py:class:`RunSaltAPIHandler` URL.
CORS
----
rest_tornado supports Cross-site HTTP requests out of the box. It is by default
deactivated and controlled by the `cors_origin` config key.
You can allow all origins by settings `cors_origin` to `*`.
You can allow only one origin with this configuration:
.. code-block:: yaml
rest_tornado:
cors_origin: http://salt.yourcompany.com
You can also be more specific and select only a few allowed origins by using
a list. For example:
.. code-block:: yaml
rest_tornado:
cors_origin:
- http://salt.yourcompany.com
- http://salt-preprod.yourcampany.com
The format for origin are full URL, with both scheme and port if not standard.
In this case, rest_tornado will check if the Origin header is in the allowed
list if it's the case allow the origin. Else it will returns nothing,
effectively preventing the origin to make request.
For reference, CORS is a mechanism used by browser to allow (or disallow)
requests made from browser from a different origin than salt-api. It's
complementary to Authentication and mandatory only if you plan to use
a salt client developed as a Javascript browser application.
Usage
-----
Commands are sent to a running Salt master via this module by sending HTTP
requests to the URLs detailed below.
.. admonition:: Content negotiation
This REST interface is flexible in what data formats it will accept as well
as what formats it will return (e.g., JSON, YAML, x-www-form-urlencoded).
* Specify the format of data in the request body by including the
:mailheader:`Content-Type` header.
* Specify the desired data format for the response body with the
:mailheader:`Accept` header.
Data sent in :http:method:`post` and :http:method:`put` requests must be in
the format of a list of lowstate dictionaries. This allows multiple commands to
be executed in a single HTTP request.
.. glossary::
lowstate
A dictionary containing various keys that instruct Salt which command
to run, where that command lives, any parameters for that command, any
authentication credentials, what returner to use, etc.
Salt uses the lowstate data format internally in many places to pass
command data between functions. Salt also uses lowstate for the
:ref:`LocalClient() <python-api>` Python API interface.
The following example (in JSON format) causes Salt to execute two commands::
[{
"client": "local",
"tgt": "*",
"fun": "test.fib",
"arg": ["10"]
},
{
"client": "runner",
"fun": "jobs.lookup_jid",
"jid": "20130603122505459265"
}]
Multiple commands in a Salt API request will be executed in serial and makes
no gaurantees that all commands will run. Meaning that if test.fib (from the
example above) had an exception, the API would still execute "jobs.lookup_jid".
Responses to these lowstates are an in-order list of dicts containing the
return data, a yaml response could look like::
- ms-1: true
ms-2: true
- ms-1: foo
ms-2: bar
In the event of an exception while executing a command the return for that lowstate
will be a string, for example if no minions matched the first lowstate we would get
a return like::
- No minions matched the target. No command was sent, no jid was assigned.
- ms-1: true
ms-2: true
.. admonition:: x-www-form-urlencoded
Sending JSON or YAML in the request body is simple and most flexible,
however sending data in urlencoded format is also supported with the
caveats below. It is the default format for HTML forms, many JavaScript
libraries, and the :command:`curl` command.
For example, the equivalent to running ``salt '*' test.ping`` is sending
``fun=test.ping&arg&client=local&tgt=*`` in the HTTP request body.
Caveats:
* Only a single command may be sent per HTTP request.
* Repeating the ``arg`` parameter multiple times will cause those
parameters to be combined into a single list.
Note, some popular frameworks and languages (notably jQuery, PHP, and
Ruby on Rails) will automatically append empty brackets onto repeated
parameters. E.g., ``arg=one``, ``arg=two`` will be sent as ``arg[]=one``,
``arg[]=two``. This is not supported; send JSON or YAML instead.
.. |req_token| replace:: a session token from :py:class:`~SaltAuthHandler`.
.. |req_accept| replace:: the desired response format.
.. |req_ct| replace:: the format of the request body.
.. |res_ct| replace:: the format of the response body; depends on the
:mailheader:`Accept` request header.
.. |200| replace:: success
.. |400| replace:: bad request
.. |401| replace:: authentication required
.. |406| replace:: requested Content-Type not available
.. |500| replace:: internal server error
i ( t absolute_importt print_functiont unicode_literalsN( t copy( t defaultdict( t Future( t tagify( t AuthenticationErrort AuthorizationErrort EauthAuthenticationErrorc K@ s t j j j | d t | �S( u�
Invoke salt.utils.json.dumps using the alternate json module loaded using
salt.utils.json.import_json(). This ensures that we properly encode any
strings in the object before we perform the serialization.
t _json_module( t saltt utilst jsont dumps( t objt kwargs( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _json_dumps� s u X-Auth-Tokenu
session_idt TimeoutExceptionc B@ s e Z RS( ( t __name__t
__module__( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR � s t Anyc B@ s e Z d Z d � Z d � Z RS( uF
Future that wraps other futures to "block" until one is done
c C@ s8 t t | � j � x | D] } | j | j � q Wd S( N( t superR t __init__t add_done_callbackt
done_callback( t selft futurest future( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR s
c C@ s | j � s | j | � n d S( N( t donet
set_result( R R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR
s ( R R t __doc__R R ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR s t
EventListenerc B@ sh e Z d Z d � Z d � Z e d � � Z e d � � Z d e j d d d � Z
d � Z d � Z RS(
u�
Class responsible for listening to the salt master event bus and updating
futures. This is the core of what makes this asynchronous, this allows us to do
non-blocking work in the main processes and "wait" for an event to happen
c
C@ s� | | _ | | _ t j j j d | d | d d | d t d t j j j
� �| _ t t � | _
t t � | _ i | _ | j j | j � d S( Nu masteru sock_diru transportt optst listent io_loop( t mod_optsR! R R t eventt get_eventt Truet tornadot ioloopt IOLoopt currentR t listt tag_mapt request_mapt timeout_mapt set_event_handlert _handle_event_socket_recv( R R$ R! ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR s
c C@ s� | | j k r d Sxm | j | D]^ \ } } } | j | | | � | | j k r! t j j j � j | j | � | j | =q! q! W| j | =d S( ue
Remove all futures that were waiting for request `request` since it is done waiting
N( R. t _timeout_futureR/ R( R) R* R+ t remove_timeout( R t requestt tagt matcherR ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt clean_by_request. s c C@ s4 | d k s | d k r' t d � � n | j | � S( Nu mtag or tag can not be None( t Nonet TypeErrort
startswith( t mtagR5 ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt prefix_matcher> s c C@ s1 | d k s | d k r' t d � � n | | k S( Nu mtag or tag can not be None( R8 R9 ( R; R5 ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt
exact_matcherD s u c @ s� | j r& t � } | j t � � | St � } � d k rZ � f d � } | j | � n | j | | f j | � | j | j | | | f � | r� t j
j j � j
| | j | | | � } | | j | <n | S( u^
Get an event (asynchronous of course) return a future that will get it later
c @ s t j j j � j � | � d S( N( R( R) R* R+ t add_callback( R ( t callback( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt
handle_future] s N( t _finishedR t
set_exceptionR R8 R R- t appendR. R( R) R* R+ t
call_laterR2 R/ ( R R4 R5 R6 R? t timeoutR R@ t timeout_future( ( R? sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR& J s *c C@ s� | | f | j k r d S| j � sR | j t � � | j | | f j | � n t | j | | f � d k r� | j | | f =n d S( u+
Timeout a specific future
Ni ( R- R RB R t removet len( R R5 R6 R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR2 j s c C@ s. | j j | | j j � \ } } xt j | j � D]� \ \ } } } y | | | � } Wn* t k
r� t j d d t �t
} n X| s� q4 n x� | D]� } | j � r� q� n | j i | d 6| d 6� | j | | f j
| � | | j k r� t j j j � j | j | � | j | =q� q� Wq4 Wd S( u=
Callback for events on the event sub socket
u Failed to run a matcher.t exc_infou datau tagN( R% t unpackt serialt sixt iteritemsR- t Exceptiont logt errorR' t FalseR R RG R/ R( R) R* R+ R3 ( R t rawR; t dataR5 R6 R t
is_matchedR ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR1 v s" !%
N(
R R R R R7 t staticmethodR< R= t __func__R8 R& R2 R1 ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR s t BaseSaltAPIHandlerc B@ s� e Z d e f d e j j j f f Z d � Z d � Z e
d � � Z d � Z d � Z
d � Z d � Z d � Z d
� Z d � Z d � Z d
� Z d � Z d � Z RS( u application/jsonu application/x-yamlc C@ sP d | k s$ | j d � | j k rL | j d � | j d � | j � t St S( u?
Verify that the client is in fact one we have
u clienti� u4 400 Invalid Client: Client not found in salt clients( t gett saltclientst
set_statust writet finishRQ R' ( R t low( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _verify_client� s $
c C@ s� t | j d � sC t j d � t | j j | j j � | j _ n t | d � s� t j j
d | j j � } i | j d 6| j d 6t j j
d | j j � j d 6d d 6| _ n t | d
� s� t j j j | j j � | _ n d S( uC
Initialize the handler before requests are called
u event_listeneru init a listeneru saltclientst moptsu localu local_asyncR! u runneru runner_asyncu ckminionsN( t hasattrt applicationRO t debugR R$ R! t event_listenerR t clientt get_local_clientt
run_job_asynct runnert RunnerClientt cmd_asyncR8 RY R t minionst CkMinionst ckminions( R t local_client( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt
initialize� s
c C@ s1 t | j j k r | j j t S| j t � Sd S( u0
The token used for the request
N( t AUTH_TOKEN_HEADERR4 t headerst
get_cookiet AUTH_COOKIE_NAME( R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt token� s c C@ s% | j o$ t | j j j | j � � S( u7
Boolean whether the request is auth'd
( Rs t boolRa t autht get_tok( R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _verify_auth� s c @ s� � j j j d d � } g | j d � D] } t j | � d ^ q( } � f d � } | | � \ } } | s~ � j d � n | � _ | � _ t j � � _
t � _ � j
� � _ d S( u�
Run before get/posts etc. Pre-flight checks:
- verify that we can speak back to them (compatible accept header)
u Acceptu */*u ,i c @ sK xD | D]< } x3 � j D]( \ } } t j | | � r | | f Sq Wq Wd S( N( NN( t
ct_out_mapt fnmatchR8 ( t parsed_accept_headert media_ranget content_typet dumper( R ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt find_acceptable_content_type� s
i� N( R4 Rp RX t splitt cgit parse_headert
send_errorR| R} t timet startR' t connectedt
_get_lowstatet lowstate( R t
accept_headert hRz R~ R| R} ( ( R sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt prepare� s / c C@ s | j j j | � d S( u#
timeout a session
N( Ra Rc R7 ( R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt timeout_futures� s c C@ s | j � | ` d S( u:
When the job has been done, lets cleanup
N( R� RY ( R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt on_finish� s
c C@ s | j � d S( u;
If the client disconnects, lets close out
N( R\ ( R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt on_connection_close� s c C@ s | j d | j � | j | � S( uB
Serlialize the output based on the Accept header
u Content-Type( t
set_headerR| R} ( R RS ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt serialize s c C@ s] i } xP | j j D]B } | j | � } t | � d k rK | d | | <q | | | <q W| S( u�
function to get the data from the urlencoded forms
ignore the data passed in and just get the args from wherever they are
i i ( R4 t argumentst
get_argumentsRH ( R t _RS t keyt val( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _form_loader
s c C@ s� i | j d 6t j j j d 6t j j j d 6t j j j d 6t j j j d 6} y= t j | j j
d � \ } } | | t j j
| � � SWn; t k
r� | j d � n t k
r� | j d � n Xd S(
uL
Deserialize the data based on request content type headers
u! application/x-www-form-urlencodedu application/jsonu application/x-yamlu text/yamlu
text/plainu Content-Typei� i� N( R� R R R
t loadst yamlt safe_loadR� R� R4 Rp R( t escapet
native_strt KeyErrorR� t
ValueError( R RS t ct_in_mapt valuet
parameters( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt deserialize s
c C@ s� | j j s d S| j | j j � } t | � | _ | rn d | k rn t | d t � rn | d g | d <n t | t � s� | g } n | } | S( uA
Format the incoming data into a lowstate object
Nu arg( R4 t bodyR� R t request_payloadt
isinstanceR, ( R RS R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� . s & c C@ sc | j j } | j d � r_ | j j j d � } t | | d � } | r_ | j d | � q_ n d S( u*
Set default CORS headers
u cors_originu Originu Access-Control-Allow-OriginN( Ra R$ RX R4 Rp t _check_cors_originR� ( R R$ t origint allowed_origin( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt set_default_headersA s c O@ sx | j j j d � } | j d � } | j d d j | � � | j d d � | j d d � | j d � | j � d S(
u<
Return CORS headers for preflight requests
u Access-Control-Request-Headersu ,u Access-Control-Allow-Headersu Access-Control-Expose-Headersu X-Auth-Tokenu Access-Control-Allow-Methodsu OPTIONS, GET, POSTi� N( R4 Rp RX R R� t joinRZ R\ ( R t argsR t request_headerst allowed_headers( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt optionsO s
( R R R R R R� t safe_dumpRx R^ Rn t propertyRs Rw R� R� R� R� R� R� R� R� R� R� ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyRW � s t SaltAuthHandlerc B@ s e Z d Z d � Z d � Z RS( u$
Handler for login requests
c C@ sK | j d � | j d d � i d d 6d d 6} | j | j | � � d S( u�
All logins are done over post, this is a parked endpoint
.. http:get:: /login
:status 401: |401|
:status 406: |406|
**Example request:**
.. code-block:: bash
curl -i localhost:8000/login
.. code-block:: text
GET /login HTTP/1.1
Host: localhost:8000
Accept: application/json
**Example response:**
.. code-block:: text
HTTP/1.1 401 Unauthorized
Content-Type: application/json
Content-Length: 58
{"status": "401 Unauthorized", "return": "Please log in"}
i� u WWW-Authenticateu Sessionu 401 Unauthorizedu statusu
Please log inu returnN( RZ R� R[ R� ( R t ret( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyRX j s
c
C@ s| yW t | j t � s& | j d � d Si | j d d 6| j d d 6| j d d 6} Wn t k
rx | j d � d SX| j j j | � } d | k r� | j d � d Sy� | j j d | d } | j | d g � } | j
| j d
g � � d | k r�| d r�t | d � } t g | j � D]$ } | j
d � r+| j d � ^ q+� } x/ | | @D] } | j
| d
j | � � qcWn t t t | � � � } Wnn t k
r�| j d � d St t f k
rt j d | j d � | j d � d t �| j d � d SXi i | d d 6| d d 6| d d 6| d d 6| d d 6| d 6g d 6} | j | j | � � d S( u
:ref:`Authenticate <rest_tornado-auth>` against Salt's eauth system
.. http:post:: /login
:reqheader X-Auth-Token: |req_token|
:reqheader Accept: |req_accept|
:reqheader Content-Type: |req_ct|
:form eauth: the eauth backend configured for the user
:form username: username
:form password: password
:status 200: |200|
:status 400: |400|
:status 401: |401|
:status 406: |406|
:status 500: |500|
**Example request:**
.. code-block:: bash
curl -si localhost:8000/login \
-H "Accept: application/json" \
-d username='saltuser' \
-d password='saltpass' \
-d eauth='pam'
.. code-block:: text
POST / HTTP/1.1
Host: localhost:8000
Content-Length: 42
Content-Type: application/x-www-form-urlencoded
Accept: application/json
username=saltuser&password=saltpass&eauth=pam
**Example response:**
.. code-block:: text
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 206
X-Auth-Token: 6d1b722e
Set-Cookie: session_id=6d1b722e; expires=Sat, 17 Nov 2012 03:23:52 GMT; Path=/
{"return": {
"token": "6d1b722e",
"start": 1363805943.776223,
"expire": 1363849143.776224,
"user": "saltuser",
"eauth": "pam",
"perms": [
"grains.*",
"status.*",
"sys.*",
"test.*"
]
}}
i� Nu usernameu passwordu eauthu tokeni� u
external_authu nameu *u groupsu %u {0}%uH Configuration for external_auth malformed for eauth '%s', and user '%s'.RI i� u expireu startu useru permsu return( R� R� t dictR� R� Ra Ru t mk_tokenR! RX t extendt sett keyst endswitht rstript formatt sortedR, t AttributeErrort
IndexErrorRO Rb R' R[ R� (
R t credsRs t eautht permst user_groupst it eauth_groupst groupR� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt post� sT @
=!
( R R R RX R� ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� f s (t SaltAPIHandlerc B@ s� e Z d Z d � Z e j j d � � Z e j j d � � Z
e j j d � � Z e j j d � � Z e j j d � � Z
e j j d � � Z e j j d � � Z d � Z RS(
u'
Main API handler for base "/"
c C@ s= i t | j j � � d 6d d 6} | j | j | � � d S( u�
An endpoint to determine salt-api capabilities
.. http:get:: /
:reqheader Accept: |req_accept|
:status 200: |200|
:status 401: |401|
:status 406: |406|
**Example request:**
.. code-block:: bash
curl -i localhost:8000
.. code-block:: text
GET / HTTP/1.1
Host: localhost:8000
Accept: application/json
**Example response:**
.. code-block:: text
HTTP/1.1 200 OK
Content-Type: application/json
Content-Legnth: 83
{"clients": ["local", "local_async", "runner", "runner_async"], "return": "Welcome"}
u clientsu Welcomeu returnN( R, RY R� R[ R� ( R R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyRX s "
c C@ s+ | j � s | j d � d S| j � d S( u
Send one or more Salt commands (lowstates) in the request body
.. http:post:: /
:reqheader X-Auth-Token: |req_token|
:reqheader Accept: |req_accept|
:reqheader Content-Type: |req_ct|
:resheader Content-Type: |res_ct|
:status 200: |200|
:status 401: |401|
:status 406: |406|
:term:`lowstate` data describing Salt commands must be sent in the
request body.
**Example request:**
.. code-block:: bash
curl -si https://localhost:8000 \
-H "Accept: application/x-yaml" \
-H "X-Auth-Token: d40d1e1e" \
-d client=local \
-d tgt='*' \
-d fun='test.ping' \
-d arg
.. code-block:: text
POST / HTTP/1.1
Host: localhost:8000
Accept: application/x-yaml
X-Auth-Token: d40d1e1e
Content-Length: 36
Content-Type: application/x-www-form-urlencoded
fun=test.ping&arg&client=local&tgt=*
**Example response:**
Responses are an in-order list of the lowstate's return data. In the
event of an exception running a command the return will be a string
instead of a mapping.
.. code-block:: text
HTTP/1.1 200 OK
Content-Length: 200
Allow: GET, HEAD, POST
Content-Type: application/x-yaml
return:
- ms-0: true
ms-1: true
ms-2: true
ms-3: true
ms-4: true
.. admonition:: multiple commands
Note that if multiple :term:`lowstate` structures are sent, the Salt
API will execute them in serial, and will not stop execution upon failure
of a previous job. If you need to have commands executed in order and
stop on failure please use compound-command-execution.
u /loginN( Rw t redirectt disbatch( R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� < s H
c c@ s^ g } x*| j D] } | j | � s) d S| j d k rT d | k rT | j | d <n d | k p� d | k o� d | k o� d | k s� | j d � Pn y4 t | d j | d � � | � V} | j | � Wq t t t f k
r� | j d � Pq t
k
r.} | j d j | � � t j d
d t
�q Xq W| j | j i | d 6� � | j � d S(
uC
Disbatch all lowstates to the appropriate clients
Nu tokenu usernameu passwordu eauthu Failed to authenticateu
_disbatch_{0}u clientu0 Unexpected exception while handling request: {0}u, Unexpected exception while handling request:RI u return( R� R^ Rs R8 RC t getattrR� R R R RN RO RP R' R[ R� R\ ( R R� R] t chunk_rett ex( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� � s* $
#
c #@ s � j d d � s- t j j j � j j � n � d � d <t � j j
� d � j d d � � � � � � f d � } g } x: � D]2 } | | � \ } } | j | � | j | � q� W� j � � } � j
d | j d d � | j d i � � V} d | k rPx3 | D]+ } y | j d � Wq t k
r6q Xq Wt j j d � � n xT t t | d
� t � � � D]2 } | | � \ } } | j | � | j | � qqWd � | d
D� � t � }
|
j t � � j j d r t j j � j j d
� }
n t j j � j j d � } t j j j � j � j | d � d | d d � | � � f d � } i }
x�t r| | g � |
j � s�� |
g 7� n � f d � } t � � V} y1| | k r�| � t j j |
� � n7 | |
k r | � s| � t j j |
� � n won | j � } | | k rD| j | � n | d j d � r�x� | d d
D] } | � k rft! � | <qfqfWn^ | d d |
| d d <t � | d d <| � r�|
j � r�| � t j j |
� � n Wqot" k
r�qoXqoWd S( u0
Dispatch local client commands
u jidu tgtu tgt_typeu globc @ sr � j j j � d d j � d | � d t j �} � j j j � d d j � d | � d t j �} | | f S( NR5 u salt/job/{}/ret/{}u jidR6 u syndic/job/{}/ret/{}( Ra Rc R& R� R R= ( t miniont salt_evtt
syndic_evt( t chunkR ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt subscribe_minion� s u localu argsu kwargsuH No minions matched the target. No command was sent, no jid was assigned.u minionsc S@ s i | ] } t | � q S( ( RQ ( t .0t m( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pys
<dictcomp>� s u
order_mastersu syndic_waitu gather_job_timeoutc @ s t d � t j � � D� � S( u\
Check if there are any more minions we are waiting on returns from
c s@ s | ] } | t k Vq d S( N( RQ ( R� t x( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pys <genexpr>� s ( t anyRL t
itervalues( ( Rj ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt more_todo� s c @ s1 x* � D]" } | j � s | j d � q q Wd S( N( R R R8 ( R% ( t to_wait( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt cancel_inflight_futures s
u tagu /newu datau returnu idN( (# RX R8 R R t jidt gen_jidRa R! R� Rl t
check_minionsRC t _format_call_run_job_asyncRY R RN R( t gent ReturnR, R R' t sleepR) R* R+ t spawn_callbackt job_not_runningR R t resultRG R� RQ R ( R R� R� t eventsR� R� R� t f_callt pub_dataR t
min_wait_timet is_finishedR� R� R� t ft f_resultt minion_id( ( R� Rj R R� sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _disbatch_local� s~ ;+
/
'
c c@ s� | j d | d | g d | �V} t | d d g d � } t } xZt r�y� | j j j | d | d | j j d �} t | | g � V}
|
| k r� | j � s� | j
d
� n t j
j t � � n |
j � } Wnq t k
rI| st j
j t � � qJ| j d | d | g d | �V} t | d d g d � } t } qE n X| d
j d i � i k rlqE n | d
d | k r�t | | d
d <n t } qE Wd
S( un
Return a future which will complete once jid (passed in) is no longer
running on tgt
u localu saltutil.find_jobt tgt_typeu jidu retu jobR5 RE u gather_job_timeoutu datau returnu idN( RY R RQ R' Ra Rc R& R! R R R R8 R( R� R� R� R RX ( R R� t tgtR� Rj R� t
ping_pub_datat ping_tagt minion_runningR% R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� ( s@
c c@ sT | j | � } | j d | j d d � | j d i � � V} t j j | � � d S( u6
Disbatch local client_async commands
u local_asyncu argsu kwargsN( ( R� RY RX R( R� R� ( R R� R� R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _disbatch_local_asyncT s /c c@ s� | j d t � } | j d | � } | d d } yL | j j j | d | �V} | r^ | n | d d } t j j | � � Wn# t k
r� t j j d � � n Xd S(
u1
Disbatch runner client commands
u full_returnu runneru tagu /retR5 u datau returnu% Timeout waiting for runner to executeN(
t popRQ RY Ra Rc R& R( R� R� R ( R R� t full_returnR� R5 R% R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _disbatch_runner_ s
c C@ s) | j d | � } t j j | � � d S( u7
Disbatch runner client_async commands
u runnerN( RY R( R� R� ( R R� R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt _disbatch_runner_asyncp s c C@ sM t j j j t j j j | d t �} t j j
j � | j d i � d <| S( Nt is_class_methodu kwargsu io_loop(
R R R� t format_callRd t LocalClientt run_jobR' R( R) R* R+ RX ( R R� R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� z s "( R R R RX R( t webt asynchronousR� R� t coroutineR� R� R� R� R� R� R� ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� s &N$z,
t MinionSaltAPIHandlerc B@ s; e Z d Z e j j d d � � Z e j j d � � Z RS( u=
A convenience endpoint for minion related functions
c C@ sR | j � s | j d � d Si d d 6| p0 d d 6d d 6g | _ | j � d S( uH
A convenience URL for getting lists of minions or getting minion
details
.. http:get:: /minions/(mid)
:reqheader X-Auth-Token: |req_token|
:reqheader Accept: |req_accept|
:status 200: |200|
:status 401: |401|
:status 406: |406|
**Example request:**
.. code-block:: bash
curl -i localhost:8000/minions/ms-3
.. code-block:: text
GET /minions/ms-3 HTTP/1.1
Host: localhost:8000
Accept: application/x-yaml
**Example response:**
.. code-block:: text
HTTP/1.1 200 OK
Content-Length: 129005
Content-Type: application/x-yaml
return:
- ms-3:
grains.items:
...
u /loginNu localu clientu *u tgtu grains.itemsu fun( Rw R� R� R� ( R t mid( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyRX � s )
c C@ s� | j � s | j d � d Sxj | j D]_ } d | k rI d | d <q' n | j d � d k r' | j d � | j d � | j � d Sq' W| j � d S( ug
Start an execution command and immediately return the job id
.. http:post:: /minions
:reqheader X-Auth-Token: |req_token|
:reqheader Accept: |req_accept|
:reqheader Content-Type: |req_ct|
:resheader Content-Type: |res_ct|
:status 200: |200|
:status 401: |401|
:status 406: |406|
:term:`lowstate` data describing Salt commands must be sent in the
request body. The ``client`` option will be set to
:py:meth:`~salt.client.LocalClient.local_async`.
**Example request:**
.. code-block:: bash
curl -sSi localhost:8000/minions \
-H "Accept: application/x-yaml" \
-d tgt='*' \
-d fun='status.diskusage'
.. code-block:: text
POST /minions HTTP/1.1
Host: localhost:8000
Accept: application/x-yaml
Content-Length: 26
Content-Type: application/x-www-form-urlencoded
tgt=*&fun=status.diskusage
**Example response:**
.. code-block:: text
HTTP/1.1 202 Accepted
Content-Length: 86
Content-Type: application/x-yaml
return:
- jid: '20130603122505459265'
minions: [ms-4, ms-3, ms-2, ms-1, ms-0]
u /loginNu clientu local_asynci� u We don't serve your kind here( Rw R� R� RX RZ R[ R\ R� ( R R] ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� � s 5
N( R R R R( R� R� R8 RX R� ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� � s 3t JobsSaltAPIHandlerc B@ s& e Z d Z e j j d d � � Z RS( u3
A convenience endpoint for job cache data
c C@ so | j � s | j d � d S| rG i d d 6| d 6d d 6g | _ n i d d 6d d 6g | _ | j � d S( u�
A convenience URL for getting lists of previously run jobs or getting
the return from a single job
.. http:get:: /jobs/(jid)
List jobs or show a single job from the job cache.
:status 200: |200|
:status 401: |401|
:status 406: |406|
**Example request:**
.. code-block:: bash
curl -i localhost:8000/jobs
.. code-block:: text
GET /jobs HTTP/1.1
Host: localhost:8000
Accept: application/x-yaml
**Example response:**
.. code-block:: text
HTTP/1.1 200 OK
Content-Length: 165
Content-Type: application/x-yaml
return:
- '20121130104633606931':
Arguments:
- '3'
Function: test.fib
Start Time: 2012, Nov 30 10:46:33.606931
Target: jerry
Target-type: glob
**Example request:**
.. code-block:: bash
curl -i localhost:8000/jobs/20121130104633606931
.. code-block:: text
GET /jobs/20121130104633606931 HTTP/1.1
Host: localhost:8000
Accept: application/x-yaml
**Example response:**
.. code-block:: text
HTTP/1.1 200 OK
Content-Length: 73
Content-Type: application/x-yaml
info:
- Arguments:
- '3'
Function: test.fib
Minions:
- jerry
Start Time: 2012, Nov 30 10:46:33.606931
Target: '*'
Target-type: glob
User: saltdev
jid: '20121130104633606931'
return:
- jerry:
- - 0
- 1
- 1
- 2
- 6.9141387939453125e-06
u /loginNu
jobs.list_jobu funu jidu runneru clientu jobs.list_jobs( Rw R� R� R� ( R R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyRX s S
N( R R R R( R� R� R8 RX ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� s t RunSaltAPIHandlerc B@ s# e Z d Z e j j d � � Z RS( uB
Endpoint to run commands without normal session handling
c C@ s | j � d S( u$
Run commands bypassing the :ref:`normal session handling
<rest_cherrypy-auth>`
.. http:post:: /run
This entry point is primarily for "one-off" commands. Each request
must pass full Salt authentication credentials. Otherwise this URL
is identical to the :py:meth:`root URL (/) <LowDataAdapter.POST>`.
:term:`lowstate` data describing Salt commands must be sent in the
request body.
:status 200: |200|
:status 401: |401|
:status 406: |406|
**Example request:**
.. code-block:: bash
curl -sS localhost:8000/run \
-H 'Accept: application/x-yaml' \
-d client='local' \
-d tgt='*' \
-d fun='test.ping' \
-d username='saltdev' \
-d password='saltdev' \
-d eauth='pam'
.. code-block:: text
POST /run HTTP/1.1
Host: localhost:8000
Accept: application/x-yaml
Content-Length: 75
Content-Type: application/x-www-form-urlencoded
client=local&tgt=*&fun=test.ping&username=saltdev&password=saltdev&eauth=pam
**Example response:**
.. code-block:: text
HTTP/1.1 200 OK
Content-Length: 73
Content-Type: application/x-yaml
return:
- ms-0: true
ms-1: true
ms-2: true
ms-3: true
ms-4: true
N( R� ( R ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� r s 9( R R R R( R� R� R� ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� n s t EventsSaltAPIHandlerc B@ s# e Z d Z e j j d � � Z RS( uJ
Expose the Salt event bus
The event bus on the Salt master exposes a large variety of things, notably
when executions are started on the master and also when minions ultimately
return their results. This URL provides a real-time window into a running
Salt infrastructure.
.. seealso:: :ref:`events`
c c@ s� | j � s | j d � d S| j d d � | j d d � | j d d � | j d j d
� � | j � x� t r� yh | j j j | � V} | j d j | j
d d
� � � | j t d � j t | � � � | j � Wqp t
k
r� Pqp Xqp Wd S( u/
An HTTP stream of the Salt master event bus
This stream is formatted per the Server Sent Events (SSE) spec. Each
event is formatted as JSON.
.. http:get:: /events
:status 200: |200|
:status 401: |401|
:status 406: |406|
**Example request:**
.. code-block:: bash
curl -NsS localhost:8000/events
.. code-block:: text
GET /events HTTP/1.1
Host: localhost:8000
**Example response:**
.. code-block:: text
HTTP/1.1 200 OK
Connection: keep-alive
Cache-Control: no-cache
Content-Type: text/event-stream;charset=utf-8
retry: 400
data: {'tag': '', 'data': {'minions': ['ms-4', 'ms-3', 'ms-2', 'ms-1', 'ms-0']}}
data: {'tag': '20130802115730568475', 'data': {'jid': '20130802115730568475', 'return': True, 'retcode': 0, 'success': True, 'cmd': '_return', 'fun': 'test.ping', 'id': 'ms-1'}}
The event stream can be easily consumed via JavaScript:
.. code-block:: javascript
# Note, you must be authenticated!
var source = new EventSource('/events');
source.onopen = function() { console.debug('opening') };
source.onerror = function(e) { console.debug('error!', e) };
source.onmessage = function(e) { console.debug(e.data) };
Or using CORS:
.. code-block:: javascript
var source = new EventSource('/events', {withCredentials: true});
Some browser clients lack CORS support for the ``EventSource()`` API. Such
clients may instead pass the :mailheader:`X-Auth-Token` value as an URL
parameter:
.. code-block:: bash
curl -NsS localhost:8000/events/6d1b722e
It is also possible to consume the stream via the shell.
Records are separated by blank lines; the ``data:`` and ``tag:``
prefixes will need to be removed manually before attempting to
unserialize the JSON.
curl's ``-N`` flag turns off input buffering which is required to
process the stream incrementally.
Here is a basic example of printing each event as it comes in:
.. code-block:: bash
curl -NsS localhost:8000/events |\
while IFS= read -r line ; do
echo $line
done
Here is an example of using awk to filter events based on tag:
.. code-block:: bash
curl -NsS localhost:8000/events |\
awk '
BEGIN { RS=""; FS="\\n" }
$1 ~ /^tag: salt\/job\/[0-9]+\/new$/ { print $0 }
'
tag: salt/job/20140112010149808995/new
data: {"tag": "salt/job/20140112010149808995/new", "data": {"tgt_type": "glob", "jid": "20140112010149808995", "tgt": "jerry", "_stamp": "2014-01-12_01:01:49.809617", "user": "shouse", "arg": [], "fun": "test.ping", "minions": ["jerry"]}}
tag: 20140112010149808995
data: {"tag": "20140112010149808995", "data": {"fun_args": [], "jid": "20140112010149808995", "return": true, "retcode": 0, "success": true, "cmd": "_return", "_stamp": "2014-01-12_01:01:49.819316", "fun": "test.ping", "id": "jerry"}}
u /loginNu Content-Typeu text/event-streamu
Cache-Controlu no-cacheu
Connectionu
keep-aliveu retry: {0}
i� u tag: {0}
u tagu u data: {0}
( Rw R� R� R[ R� t flushR' Ra Rc R& RX t strR R ( R R% ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyRX � s `
""
( R R R R( R� R� RX ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� � s
t WebhookSaltAPIHandlerc B@ s e Z d Z d d � Z RS( u�
A generic web hook entry point that fires an event on Salt's event bus
External services can POST data to this URL to trigger an event in Salt.
For example, Amazon SNS, Jenkins-CI or Travis-CI, or GitHub web hooks.
.. note:: Be mindful of security
Salt's Reactor can run any code. A Reactor SLS that responds to a hook
event is responsible for validating that the event came from a trusted
source and contains valid data.
**This is a generic interface and securing it is up to you!**
This URL requires authentication however not all external services can
be configured to authenticate. For this reason authentication can be
selectively disabled for this URL. Follow best practices -- always use
SSL, pass a secret key, configure the firewall to only allow traffic
from a known source, etc.
The event data is taken from the request body. The
:mailheader:`Content-Type` header is respected for the payload.
The event tag is prefixed with ``salt/netapi/hook`` and the URL path is
appended to the end. For example, a ``POST`` request sent to
``/hook/mycompany/myapp/mydata`` will produce a Salt event with the tag
``salt/netapi/hook/mycompany/myapp/mydata``.
The following is an example ``.travis.yml`` file to send notifications to
Salt of successful test runs:
.. code-block:: yaml
language: python
script: python -m unittest tests
after_success:
- 'curl -sS http://saltapi-url.example.com:8000/hook/travis/build/success -d branch="${TRAVIS_BRANCH}" -d commit="${TRAVIS_COMMIT}"'
.. seealso:: :ref:`events`, :ref:`reactor`
c C@ sF | j j j d � } | r: | j � r: | j d � d Sd } | rS | | 7} n t j j j d | j j d | j j d d | j j d t
�| _ i } xL | j j D]> } | j
| � } t | � d
k r� | d } n | | | <q� W| j j i | j d 6| d
6t | j j � d 6| � } | j | j i | d 6� � d S( u
Fire an event in Salt with a custom event tag and data
.. http:post:: /hook
:status 200: |200|
:status 401: |401|
:status 406: |406|
:status 413: request body is too large
**Example request:**
.. code-block:: bash
curl -sS localhost:8000/hook -d foo='Foo!' -d bar='Bar!'
.. code-block:: text
POST /hook HTTP/1.1
Host: localhost:8000
Content-Length: 16
Content-Type: application/x-www-form-urlencoded
foo=Foo&bar=Bar!
**Example response**:
.. code-block:: text
HTTP/1.1 200 OK
Content-Length: 14
Content-Type: application/json
{"success": true}
As a practical example, an internal continuous-integration build
server could send an HTTP POST request to the URL
``http://localhost:8000/hook/mycompany/build/success`` which contains
the result of a build and the SHA of the version that was built as
JSON. That would then produce the following event in Salt that could be
used to kick off a deployment via Salt's Reactor:
.. code-block:: text
Event fired at Fri Feb 14 17:40:11 2014
*************************
Tag: salt/netapi/hook/mycompany/build/success
Data:
{'_stamp': '2014-02-14_17:40:11.440996',
'headers': {
'X-My-Secret-Key': 'F0fAgoQjIT@W',
'Content-Length': '37',
'Content-Type': 'application/json',
'Host': 'localhost:8000',
'Remote-Addr': '127.0.0.1'},
'post': {'revision': 'aa22a3c4b2e7', 'result': True}}
Salt's Reactor could listen for the event:
.. code-block:: yaml
reactor:
- 'salt/netapi/hook/mycompany/build/*':
- /srv/reactor/react_ci_builds.sls
And finally deploy the new build:
.. code-block:: jinja
{% set secret_key = data.get('headers', {}).get('X-My-Secret-Key') %}
{% set build = data.get('post', {}) %}
{% if secret_key == 'F0fAgoQjIT@W' and build.result == True %}
deploy_my_app:
cmd.state.sls:
- tgt: 'application*'
- arg:
- myapp.deploy
- kwarg:
pillar:
revision: {{ revision }}
{% endif %}
u webhook_disable_authu /loginNu salt/netapi/hooku masteru sock_diru transportR! R" i i u postu getu headersu success( Ra R$ RX Rw R� R R R% R& R! RQ R4 t query_argumentsR� RH t
fire_eventR� R� Rp R[ R� ( R t
tag_suffixt disable_authR5 R� t argnameR� R� ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� W s2 T
N( R R R R8 R� ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR . s (c C@ sF t | t � r" | | k rB | Sn | d k r2 | S| | k rB | Sd S( u7
Check if an origin match cors allowed origins
u *N( R� R, ( R� t allowed_origins( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyR� � s (B R t
__future__R R R R� Ry t loggingR t collectionsR R� t tornado.escapeR( t tornado.httpservert tornado.ioloopt tornado.webt tornado.gent tornado.concurrentR t salt.ext.sixt extRL t salt.netapiR t salt.utils.argst salt.utils.eventt salt.utils.jsont salt.utils.minionst salt.utils.yamlt salt.utils.zeromqR t salt.clientt salt.runnert salt.autht salt.exceptionsR R R R t zeromqt install_zmqR
t import_jsont getLoggerR RO R Ro Rr RN R R t objectR R� t RequestHandlerRW R� R� R� R� R� R� R R� ( ( ( sE /usr/lib/python2.7/site-packages/salt/netapi/rest_tornado/saltnado.pyt <module>� sZ � ��� r�j@��
Mini Shell