Current File : //lib/python2.7/site-packages/salt/states/boto_kinesis.pyc
�
���^c @@ s� d Z d d l m Z m Z m Z d d l Z e j e � Z d Z d � Z
e e e e e e e e d � Z
e e e e d � Z d � Z d S( u�
Manage Kinesis Streams
======================
.. versionadded:: 2017.7.0
Create and destroy Kinesis streams. Be aware that this interacts with Amazon's
services, and so may incur charges.
This module uses ``boto3``, which can be installed via package, or pip.
This module accepts explicit Kinesis credentials but can also utilize
IAM roles assigned to the instance through Instance Profiles. Dynamic
credentials are then automatically obtained from AWS API and no further
configuration is necessary. More information available `here
<http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html>`_.
If IAM roles are not used you need to specify them either in a pillar file or
in the minion's config file:
.. code-block:: yaml
keyid: GKTADJGHEIQSXMKKRBJ08H
key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
region: us-east-1
It's also possible to specify ``key``, ``keyid`` and ``region`` via a
profile, either passed in as a dict, or as a string to pull from
pillars or minion config:
.. code-block:: yaml
myprofile:
keyid: GKTADJGHEIQSXMKKRBJ08H
key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
region: us-east-1
.. code-block:: yaml
Ensure Kinesis stream does not exist:
boto_kinesis.absent:
- name: new_stream
- keyid: GKTADJGHEIQSXMKKRBJ08H
- key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
- region: us-east-1
Ensure Kinesis stream exists:
boto_kinesis.present:
- name: new_stream
- retention_hours: 168
- enhanced_monitoring: ['ALL']
- num_shards: 2
- keyid: GKTADJGHEIQSXMKKRBJ08H
- key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
- region: us-east-1
i ( t absolute_importt print_functiont unicode_literalsNu boto_kinesisc C@ s d t k r t St d f S( u1
Only load if boto_kinesis is available.
u boto_kinesis.existsuF The boto_kinesis module could not be loaded: boto libraries not found.( t __salt__t __virtualname__t False( ( ( s< /usr/lib/python2.7/site-packages/salt/states/boto_kinesis.pyt __virtual__E s c # C@ s9 i | d 6t d 6d d 6i d 6} g }
i } i } t d | | | | | � }
|
d t k r7t d r� d; | d <|
j d j | � � t | | | |
� | St d | | | | | | � } d
| k r
t | d <|
j d j | | d
� � t | | | |
� | S|
j d j | � � | | d <| | d
<n |
j d j | � � t d | | | | | � } d
| k r�t | d <|
j d j | | d
� � t | | | |
� | S| d d } | d; k rm| d } | | k } | sQt d rd; | d <|
j d j | | � � qj| | k rGt d | | | | | | � } n t d | | | | | | � } d
| k r�t | d <|
j d j | | d
� � t | | | |
� | S|
j d j | � � | | d <| | d <t d | | | | | � } d
| k r@t | d <|
j d j | | d
� � t | | | |
� | S| d d } q�|
j d j | | � � n |
j d j | � � | d; k r�| t k s�| d g k r�d d d d d d! d" g } n! | t k s�| d# k r�g } n | j d$ � d% d& } t | � } t | � } | j
| � } t | j | � � } t | j | � � } t
| � d% k rt d r�d; | d <|
j d' j | | � � qt d( | | | | | | � } d
| k r�t | d <|
j d) j | | d
� � t | | | |
� | S|
j d* j | | � � n t
| � d% k r�t d rZd; | d <|
j d+ j | | � � q�t d, | | | | | | � } d
| k r�t | d <|
j d- j | | d
� � t | | | |
� | S|
j d. j | | � � n t
| � d% k r7t
| � d% k r7|
j d/ j | t
| � d% k r*| n d# � � q�t d s�t
| � d% k rY| n d# | d0 <t
| � d% k r{| n d# | d0 <q�n |
j d1 j | � � t d2 | � \ } } } t
| d3 � } | d; k r | r | | k } | s�t d r!d; | d <|
j d4 j | | | � � qt j d5 | | � t }! x� |! r�t d6 | | | | | | | � }" d
|" k r�t | d <|
j d7 j | |" d
� � t | | | |
� | S|" d }! q=W|
j d8 j | | � � | | d
<| | d
<q"|
j d9 j | | � � n |
j d: j | | � � t | | | |
� | S(< u�
Ensure the kinesis stream is properly configured and scaled.
name (string)
Stream name
retention_hours (int)
Retain data for this many hours.
AWS allows minimum 24 hours, maximum 168 hours.
enhanced_monitoring (list of string)
Turn on enhanced monitoring for the specified shard-level metrics.
Pass in ['ALL'] or True for all metrics, [] or False for no metrics.
Turn on individual metrics by passing in a list: ['IncomingBytes', 'OutgoingBytes']
Note that if only some metrics are supplied, the remaining metrics will be turned off.
num_shards (int)
Reshard stream (if necessary) to this number of shards
!!!!! Resharding is expensive! Each split or merge can take up to 30 seconds,
and the reshard method balances the partition space evenly.
Resharding from N to N+1 can require 2N operations.
Resharding is much faster with powers of 2 (e.g. 2^N to 2^N+1) !!!!!
do_reshard (boolean)
If set to False, this script will NEVER reshard the stream,
regardless of other input. Useful for testing.
region (string)
Region to connect to.
key (string)
Secret key to be used.
keyid (string)
Access key to be used.
profile (dict)
A dict with region, key and keyid, or a pillar key (string)
that contains a dict with region, key and keyid.
u nameu resultu u commentu changesu boto_kinesis.existsu testu# Kinesis stream {0} would be createdu boto_kinesis.create_streamu erroru Failed to create stream {0}: {1}u' Kinesis stream {0} successfully createdu
num_shardsu! Kinesis stream {0} already existsu# boto_kinesis.get_stream_when_activeu2 Kinesis stream {0}: error getting description: {1}u StreamDescriptionu RetentionPeriodHoursu; Kinesis stream {0}: retention hours would be updated to {1}u- boto_kinesis.decrease_stream_retention_periodu- boto_kinesis.increase_stream_retention_periodu9 Kinesis stream {0}: failed to update retention hours: {1}u< Kinesis stream {0}: retention hours was successfully updatedu retention_hoursuN Kinesis stream {0}: retention hours did not require change, already set at {1}u5 Kinesis stream {0}: did not configure retention hoursu ALLu
IncomingBytesu OutgoingRecordsu IteratorAgeMillisecondsu IncomingRecordsu! ReadProvisionedThroughputExceededu" WriteProvisionedThroughputExceededu
OutgoingBytesu Noneu EnhancedMonitoringi u ShardLevelMetricsu< Kinesis stream {0}: would enable enhanced monitoring for {1}u' boto_kinesis.enable_enhanced_monitoringu= Kinesis stream {0}: failed to enable enhanced monitoring: {1}uO Kinesis stream {0}: enhanced monitoring was enabled for shard-level metrics {1}u= Kinesis stream {0}: would disable enhanced monitoring for {1}u( boto_kinesis.disable_enhanced_monitoringu> Kinesis stream {0}: failed to disable enhanced monitoring: {1}uP Kinesis stream {0}: enhanced monitoring was disabled for shard-level metrics {1}uR Kinesis stream {0}: enhanced monitoring did not require change, already set at {1}u enhanced_monitoringu9 Kinesis stream {0}: did not configure enhanced monitoringu! boto_kinesis.get_info_for_reshardu
OpenShardsu= Kinesis stream {0}: would be resharded from {1} to {2} shardsu? Resharding stream from %s to %s shards, this could take a whileu boto_kinesis.reshardu+ Encountered error while resharding {0}: {1}u8 Kinesis stream {0}: successfully resharded to {1} shardsuE Kinesis stream {0}: did not require resharding, remains at {1} shardsu: Kinesis stream {0}: did not reshard, remains at {1} shardsN( t TrueR R t __opts__t Nonet appendt formatt _add_changest gett sett intersectiont listt
differencet lent logt info(# t namet retention_hourst enhanced_monitoringt
num_shardst
do_reshardt regiont keyt keyidt profilet rett commentst changes_oldt changes_newt existst
is_createdt stream_responset stream_detailst old_retention_hourst retention_matchest retention_updatedt old_enhanced_monitoringt new_monitoring_sett old_monitoring_sett matching_metricst enable_metricst disable_metricst metrics_enabledt metrics_disabledt min_hash_keyt max_hash_keyt full_stream_detailst old_num_shardst num_shards_matchest continue_reshardt reshard_response( ( s< /usr/lib/python2.7/site-packages/salt/states/boto_kinesis.pyt presentN s� 2"
$ (
c C@ s0 i | d 6t d 6d d 6i d 6} t d | | | | | � } | d t k re d j | � | d <| St d r� d j | � | d <d | d <| St d
| | | | | � } d | k r� d j | | d � | d <t | d <nM d
j | � | d <| d j d d j | � � | d j d d j | � � | S( u�
Delete the kinesis stream, if it exists.
name (string)
Stream name
region (string)
Region to connect to.
key (string)
Secret key to be used.
keyid (string)
Access key to be used.
profile (dict)
A dict with region, key and keyid, or a pillar key (string)
that contains a dict with region, key and keyid.
u nameu resultu u commentu changesu boto_kinesis.existsu! Kinesis stream {0} does not existu testu# Kinesis stream {0} would be deletedu boto_kinesis.delete_streamu erroru Failed to delete stream {0}: {1}u Deleted stream {0}u oldu Stream {0} existsu newu Stream {0} deletedN( R R R R R R t
setdefault( R R R R R R R" t
is_deleted( ( s< /usr/lib/python2.7/site-packages/salt/states/boto_kinesis.pyt absent� s6 "
c C@ sE d j | � | d <| r* | | d d <n | rA | | d d <n d S( Nu ,
u commentu changesu oldu new( t join( R R R! R ( ( s< /usr/lib/python2.7/site-packages/salt/states/boto_kinesis.pyR � s
( t __doc__t
__future__R R R t loggingt getLoggert __name__R R R R R R8 R; R ( ( ( s< /usr/lib/python2.7/site-packages/salt/states/boto_kinesis.pyt <module>9 s&
� / 8
Mini Shell