Current File : //usr/lib/python2.7/site-packages/salt/engines/napalm_syslog.py
# -*- coding: utf-8 -*-
'''
NAPALM syslog engine
====================
.. versionadded:: 2017.7.0
An engine that takes syslog messages structured in
OpenConfig_ or IETF format
and fires Salt events.
.. _OpenConfig: http://www.openconfig.net/
As there can be many messages pushed into the event bus,
the user is able to filter based on the object structure.
Requirements
------------
- `napalm-logs`_
.. _`napalm-logs`: https://github.com/napalm-automation/napalm-logs
This engine transfers objects from the napalm-logs library
into the event bus. The top dictionary has the following keys:
- ``ip``
- ``host``
- ``timestamp``
- ``os``: the network OS identified
- ``model_name``: the OpenConfig or IETF model name
- ``error``: the error name (consult the documentation)
- ``message_details``: details extracted from the syslog message
- ``open_config``: the OpenConfig model
The napalm-logs transfers the messages via widely used transport
mechanisms such as: ZeroMQ (default), Kafka, etc.
The user can select the right transport using the ``transport``
option in the configuration.
:configuration: Example configuration
.. code-block:: yaml
engines:
- napalm_syslog:
transport: zmq
address: 1.2.3.4
port: 49018
:configuration: Configuration example, excluding messages from IOS-XR devices:
.. code-block:: yaml
engines:
- napalm_syslog:
transport: kafka
address: 1.2.3.4
port: 49018
os_blacklist:
- iosxr
Event example:
.. code-block:: json
{
"_stamp": "2017-05-26T10:03:18.653045",
"error": "BGP_PREFIX_THRESH_EXCEEDED",
"host": "vmx01",
"ip": "192.168.140.252",
"message_details": {
"date": "May 25",
"host": "vmx01",
"message": "192.168.140.254 (External AS 65001): Configured maximum prefix-limit threshold(22) exceeded for inet-unicast nlri: 28 (instance master)",
"pri": "28",
"processId": "2957",
"processName": "rpd",
"tag": "BGP_PREFIX_THRESH_EXCEEDED",
"time": "20:50:41"
},
"model_name": "openconfig_bgp",
"open_config": {
"bgp": {
"neighbors": {
"neighbor": {
"192.168.140.254": {
"afi_safis": {
"afi_safi": {
"inet": {
"afi_safi_name": "inet",
"ipv4_unicast": {
"prefix_limit": {
"state": {
"max_prefixes": 22
}
}
},
"state": {
"prefixes": {
"received": 28
}
}
}
}
},
"neighbor_address": "192.168.140.254",
"state": {
"peer_as": 65001
}
}
}
}
}
},
"os": "junos",
"timestamp": "1495741841"
}
To consume the events and eventually react and deploy a configuration changes
on the device(s) firing the event, one is able to identify the minion ID, using
one of the following alternatives, but not limited to:
- :mod:`Host grains <salt.grains.napalm.host>` to match the event tag
- :mod:`Host DNS grain <salt.grains.napalm.host_dns>` to match the IP address in the event data
- :mod:`Hostname grains <salt.grains.napalm.hostname>` to match the event tag
- :ref:`Define static grains <static-custom-grains>`
- :ref:`Write a grains module <writing-grains>`
- :ref:`Targeting minions using pillar data <targeting-pillar>` - The user can
configure certain information in the Pillar data and then use it to identify
minions
Master configuration example, to match the event and react:
.. code-block:: yaml
reactor:
- 'napalm/syslog/*/BGP_PREFIX_THRESH_EXCEEDED/*':
- salt://increase_prefix_limit_on_thresh_exceeded.sls
Which matches the events having the error code ``BGP_PREFIX_THRESH_EXCEEDED``
from any network operating system, from any host and reacts, executing the
``increase_prefix_limit_on_thresh_exceeded.sls`` reactor, found under
one of the :conf_master:`file_roots` paths.
Reactor example:
.. code-block:: yaml
increase_prefix_limit_on_thresh_exceeded:
local.net.load_template:
- tgt: "hostname:{{ data['host'] }}"
- tgt_type: grain
- kwarg:
template_name: salt://increase_prefix_limit.jinja
openconfig_structure: {{ data['open_config'] }}
The reactor in the example increases the BGP prefix limit
when triggered by an event as above. The minion is matched using the ``host``
field from the ``data`` (which is the body of the event), compared to the
:mod:`hostname grain <salt.grains.napalm.hostname>` field. When the event
occurs, the reactor will execute the
:mod:`net.load_template <salt.modules.napalm_network.load_template>` function,
sending as arguments the template ``salt://increase_prefix_limit.jinja`` defined
by the user in their environment and the complete OpenConfig object under
the variable name ``openconfig_structure``. Inside the Jinja template, the user
can process the object from ``openconfig_structure`` and define the bussiness
logic as required.
'''
from __future__ import absolute_import, print_function, unicode_literals
# Import python stdlib
import logging
# Import third party libraries
from salt.utils.zeromq import zmq
try:
# pylint: disable=W0611
import napalm_logs
import napalm_logs.utils
# pylint: enable=W0611
HAS_NAPALM_LOGS = True
except ImportError:
HAS_NAPALM_LOGS = False
# Import salt libs
import salt.utils.event as event
import salt.utils.network
import salt.utils.stringutils
# ----------------------------------------------------------------------------------------------------------------------
# module properties
# ----------------------------------------------------------------------------------------------------------------------
log = logging.getLogger(__name__)
__virtualname__ = 'napalm_syslog'
# ----------------------------------------------------------------------------------------------------------------------
# helpers
# ----------------------------------------------------------------------------------------------------------------------
def __virtual__():
'''
Load only if napalm-logs is installed.
'''
if not HAS_NAPALM_LOGS or not zmq:
return (False, 'napalm_syslog could not be loaded. \
Please install napalm-logs library amd ZeroMQ.')
return True
def _zmq(address, port, **kwargs):
context = zmq.Context()
socket = context.socket(zmq.SUB)
if salt.utils.network.is_ipv6(address):
socket.ipv6 = True
socket.connect('tcp://{addr}:{port}'.format(
addr=address,
port=port)
)
socket.setsockopt(zmq.SUBSCRIBE, b'')
return socket.recv
def _get_transport_recv(name='zmq',
address='0.0.0.0',
port=49017,
**kwargs):
if name not in TRANSPORT_FUN_MAP:
log.error('Invalid transport: %s. Falling back to ZeroMQ.', name)
name = 'zmq'
return TRANSPORT_FUN_MAP[name](address, port, **kwargs)
TRANSPORT_FUN_MAP = {
'zmq': _zmq,
'zeromq': _zmq
}
# ----------------------------------------------------------------------------------------------------------------------
# main
# ----------------------------------------------------------------------------------------------------------------------
def start(transport='zmq',
address='0.0.0.0',
port=49017,
auth_address='0.0.0.0',
auth_port=49018,
disable_security=False,
certificate=None,
os_whitelist=None,
os_blacklist=None,
error_whitelist=None,
error_blacklist=None,
host_whitelist=None,
host_blacklist=None):
'''
Listen to napalm-logs and publish events into the Salt event bus.
transport: ``zmq``
Choose the desired transport.
.. note::
Currently ``zmq`` is the only valid option.
address: ``0.0.0.0``
The address of the publisher, as configured on napalm-logs.
port: ``49017``
The port of the publisher, as configured on napalm-logs.
auth_address: ``0.0.0.0``
The address used for authentication
when security is not disabled.
auth_port: ``49018``
Port used for authentication.
disable_security: ``False``
Trust unencrypted messages.
Strongly discouraged in production.
certificate: ``None``
Absolute path to the SSL certificate.
os_whitelist: ``None``
List of operating systems allowed. By default everything is allowed.
os_blacklist: ``None``
List of operating system to be ignored. Nothing ignored by default.
error_whitelist: ``None``
List of errors allowed.
error_blacklist: ``None``
List of errors ignored.
host_whitelist: ``None``
List of hosts or IPs to be allowed.
host_blacklist: ``None``
List of hosts of IPs to be ignored.
'''
if not disable_security:
if not certificate:
log.critical('Please use a certificate, or disable the security.')
return
auth = napalm_logs.utils.ClientAuth(certificate,
address=auth_address,
port=auth_port)
transport_recv_fun = _get_transport_recv(name=transport,
address=address,
port=port)
if not transport_recv_fun:
log.critical('Unable to start the engine', exc_info=True)
return
master = False
if __opts__['__role'] == 'master':
master = True
while True:
log.debug('Waiting for napalm-logs to send anything...')
raw_object = transport_recv_fun()
log.debug('Received from napalm-logs:')
log.debug(raw_object)
if not disable_security:
dict_object = auth.decrypt(raw_object)
else:
dict_object = napalm_logs.utils.unserialize(raw_object)
try:
event_os = dict_object['os']
if os_blacklist or os_whitelist:
valid_os = salt.utils.stringutils.check_whitelist_blacklist(
event_os,
whitelist=os_whitelist,
blacklist=os_blacklist)
if not valid_os:
log.info('Ignoring NOS %s as per whitelist/blacklist', event_os)
continue
event_error = dict_object['error']
if error_blacklist or error_whitelist:
valid_error = salt.utils.stringutils.check_whitelist_blacklist(
event_error,
whitelist=error_whitelist,
blacklist=error_blacklist)
if not valid_error:
log.info('Ignoring error %s as per whitelist/blacklist', event_error)
continue
event_host = dict_object.get('host') or dict_object.get('ip')
if host_blacklist or host_whitelist:
valid_host = salt.utils.stringutils.check_whitelist_blacklist(
event_host,
whitelist=host_whitelist,
blacklist=host_blacklist)
if not valid_host:
log.info('Ignoring messages from %s as per whitelist/blacklist', event_host)
continue
tag = 'napalm/syslog/{os}/{error}/{host}'.format(
os=event_os,
error=event_error,
host=event_host
)
except KeyError as kerr:
log.warning('Missing keys from the napalm-logs object:', exc_info=True)
log.warning(dict_object)
continue # jump to the next object in the queue
log.debug('Sending event %s', tag)
log.debug(raw_object)
if master:
event.get_master_event(__opts__,
__opts__['sock_dir']
).fire_event(dict_object, tag)
else:
__salt__['event.send'](tag, dict_object)
Mini Shell