Current File : //opt/alt/python37/lib/python3.7/site-packages/lvestats/snapshots/__pycache__/reader.cpython-37.pyc
B
��1b�= � @ s� d dl mZ d dl mZ d dlmZ d dlZd dlZd dlZd dlm Z d dl
m Z d dl Z
d dlmZmZ d dlmZ d d lmZ d d
lmZ d d lmZ d d lmZ d
ZdZ dZ dd� Z e
j j j!j"ddd��Z#dd� Z$G dd� de%�Z&dS )� )�print_function)�absolute_import)�mapN)�or_)� sessionmaker)�dbengine� uidconverter)�dateutil)�prepare_data_json)�lve_read_snapshot_parser)�Snapshot)�incidentz?Snapshots collected starting from %s to %s for lve id %d @ %s:
zDone..
Z localhostc C sv | j r| j \}}n\| jr(| j}|d }nFy(t�d�| j��}t�d�| j��}W n tk
rl td� dS X ||fS )Ng� !��?� z:please use [YY]YY-MM-DD[ HH:MM] format for --from and --to)NN) ZperiodZ timestampr Z
parse_date�joinZffrom�to�
ValueError�print)�opts�start�end� r �J/opt/alt/python37/lib/python3.7/site-packages/lvestats/snapshots/reader.py�_calculate_period s
r c
C s" t � }|�|�}|js0|js0|�� td� dS yt�| �}W n, tjk
rj } z t|� dS d }~X Y nX | � dt
�}|jr�t � |j|||�}|d kr�td|j|f � dS n|j}t
|�\}} |d kr�| d kr�dS t||| |||j|j�}
|j�r |
�� n |j�r|
�|j� n|
�� d S )Nz/One of -u --user or -i --id should be specified� � server_idzUser %s@%s not found)r �
parse_args�id�user�
print_helpr r Zmake_db_engineZMakeDbException�get�DEFAULT_SERVER_IDr Zusername_to_uidr �LVEReadSnaphot�outputZjson�list�statsZunit�run) �configZargv_�parserr �engine�er �uidr r Zlve_read_snapshotr r r �snapshot_reader_main/ sD
r+ c C s t | tj�r t�t�| ��S | S )z�
Convert local datetime to unix timestamp, or just passes
unix timestamp as output if specified.
:param o:
:return:
)�
isinstance�datetimer Z gm_datetime_to_unixtimestampZ local_to_gm)�or r r �_try_convert_to_timestamp\ s r/ c @ s� e Zd Zdd� Zdd� Zdd� Zedejfdd ��Z ed
d � �Z
d d
� Z dd� Z dd� Z
dd� Zdd� Zdd� Zdd� Zdd� Zed d � �Zd d � Zd S )!r! c C sL || _ || _|| _|| _t|�| _t|�| _|| _dddddddd�| _d S )
z�
:param start: datetime.datetime | int (unix timestamp)
:param end: datetime.datetime | int (unix timestamp)
:param uid:
:param server_id:
:param output_file: filename
:param do_json: boolean
�CPUzVirtual memoryZEPzPhysical memoryZNPROCZIOZIOPS)Z cpu_faultZ mem_faultZ mep_faultZ memphy_faultZ nproc_faultZio_faultZ
iops_faultN) r( �do_json� output_filer* r/ r r r � fault_names)�selfr( r r r* r r2 r1 r r r �__init__j s
zLVEReadSnaphot.__init__c
C sR |� t��tj| jktj| jkttj�| j| j �tj
�| j| j ���� tj�� � S )N)
Zqueryr
�filterr* r r �incident_start_timeZbetweenr r �incident_end_timeZorder_by�all)r4 �sessionr r r �
get_incidents� s
z LVEReadSnaphot.get_incidentsc
C sJ g }x@| � |�D ]2}|�|jt|j|jp*d�d|j| �|�d�� qW |S )Nr r )�fromr � incidents� snapshots�duration)r; �appendr7 �max� dump_timer8 Zsnapshot_count� get_duration)r4 r: �result�ir r r �stats_by_incident� s z LVEReadSnaphot.stats_by_incidentr c C s, t | j|�}tt | j| jp d�|�}|| S )Nr )rA r7 �minrB r8 )rE Zfrom_Zto_r r r rC � s zLVEReadSnaphot.get_durationc C s� d}d}xZ|t | �k rb| | }|j|k r2|d7 }q
|j|kr>P |d7 }|d7 }|t�|||�7 }q
W |dkrvdd|fS |||d fS d S )Nr r )�lenrB r7 r! rC )r= �pos�from_ts�to_ts�countr? rE r r r �get_incident_count� s
z!LVEReadSnaphot.get_incident_countc C s� | � |�}tt| jd���� }g }| j}d}x�|| jk r�t|| | j�}| �||||�\} }
}| dkrl|}q.t� |||�} t
| �dkr�| � d� |� ||| t
| �|
d�� |}q.W |S )N)r* r )r< r r= r> r? ) r; r �dictr* Z_get_file_listr r rG rM Zsnapshot_filterrH r@ ) r4 r: � time_unitr= Zsnapshot_filesrD rJ rI rK Zincident_countr? r> r r r �stats_by_time_unit� s*
z!LVEReadSnaphot.stats_by_time_unitc C s< | j | j|d�}| �� }|�t|�� |�d� |�� d S )N)r< r r$ �
)r r �open�writer
�flush)r4 r$ �data�outr r r �print_stats_json� s
z LVEReadSnaphot.print_stats_jsonc C s� | � � }|�dt�| j�t�| j�f � x||D ]t}|�d� |�dt�|d � � |�dt�|d � � |�d|d � |�d |d
� |�d |d � q0W |�� d S )
NzStats from %s to %s
z---
z
from: %s
r< z to: %s
r z incidents: %d
r= z snapshots: %d
r> z duration: %d sec.
r? )rR rS r � ts_to_isor r rT )r4 r$ rV �statr r r � print_stats� s
zLVEReadSnaphot.print_statsc
C s� y.t �|�}| j| j |k r(| j| | _d}W n: tk
rh } z d }|dk}|sXt|��W d d }~X Y nX t| jd�� }zD|r�| �|�}n | �||�}|� � | j
r�| � |� n
| � |� W d |�
� X d S )NF�auto)Zbind)r Z
parse_period2r r r r r( rF rP Z expunge_allr1 rW rZ �close)r4 Zstats_unit_strrO Zgroup_by_incident�exr: r$ r r r r$ � s(
zLVEReadSnaphot.statsc C s* t t| jd��}| �|�| j| j�� d S )N)r* )r rN r* �reportZ
get_snapshotsr r )r4 r> r r r r% s zLVEReadSnaphot.runc C s� t t| jd��}|�| j| j�}| �� }| jr>|�t |�� nX|�dt
� | j�t
� | j�| jf � x&|D ] }|�t
� |�� |�d� qjW |�t � |�
� d S )N)r* z6Snapshots timestamp list; from %s to %s for lve id %s
rQ )r rN r* Z get_ts_listr r rR r1 rS r
r rX �
REPORT_FOOTERrT )r4 r> Zsnapshots_listrV �tsr r r r# s
zLVEReadSnaphot.listc C s� | � � }| jr6|�td|i�� |�d� |�� d S |�tt�| j�t�| j �| j
| j f � x|D ]}| � ||� qfW |�t
� |�� d S )Nr> rQ )rR r1 rS r
rT �
REPORT_HEADERr rX r r r* r �format_snapshotr_ )r4 r> rV �
snapshot_datar r r r^ s
zLVEReadSnaphot.reportc C s. | j r(y t| j d�S tk
r& Y nX tjS )N�w)r2 rR �IOError�sys�stdout)r4 r r r rR & s zLVEReadSnaphot.openc C s| t � }�xJ| �� D �]<\}}d|krd|d }t � }||d< |d |d< ||d< |d |d< |||< q|d }|d dkr�t|d �dd��|d< |d dkr�t|d �|d< ||k�rH|d dkr�|| d dkr�|| d |d 7 < n|d || d< |d dk�rP|| d dk�r6|| d |d 7 < n|d || d< q|||< qW t� }x |D ]}|�|| � �q`W |S )a�
Aggregates process data by PID by summing CPU % and MEM for same PIDs
:param process_data: input data. Dictionary:
{ u'151048': {u'MEM': u'1', u'CMD': u'bash', u'PID': u'151048', u'CPU': u'0%'},
u'151047': {u'MEM': u'1', u'CMD': u'su cltest1', u'PID': u'151047', u'CPU': u'0%'},
u'153642': {u'MEM': u'1', u'CMD': u'./threads', u'PID': u'153640', u'CPU': u'0%'},
u'153641': {u'MEM': u'1', u'CMD': u'./threads', u'PID': u'153640', u'CPU': u'0%'},
u'153640': {u'MEM': u'1', u'CMD': u'./threads', u'PID': u'153640', u'CPU': u'5%'}
}
:return: Output data - List of dictionaries:
[
{u'MEM': u'1', u'CMD': u'bash', u'PID': u'151048', u'CPU': u'0%'},
{u'MEM': u'1', u'CMD': u'su cltest1', u'PID': u'151047', u'CPU': u'0%'},
{u'MEM': u'3', u'CMD': u'./threads', u'PID': u'153640', u'CPU': u'5%'},
]
�PID�CMD�MEMr0 zN/A�%� )rN �items�int�replacer# r@ )Z process_dataZ thread_dictZp_idZ proc_data�pidZprocess_data_newZout_datar r r �_process_data_aggregate/ s<
z&LVEReadSnaphot._process_data_aggregatec
s |� dt�|d �|d f � |� d� x4|d �� D ]$\}}|� d| j�||�|f � q8W |d r�|� d� t�d d
d g�}tt |j
|d �� |� |� � � |� d � t � }x8t|d
�
� �D ]$� x t� �� �D ]}|�|� q�W q�W tdg�}dg} d|k�r | �d� |�d� d|k�r:| �d� |�d� d |k�rX| �d � |�d � t�| �}
d | k�rvd|
jd < | �|d
�} x&| D ] � |
�
� fdd�|D �� �q�W |� t|
�� |� d� |d �r|� d� t�dddd d d
g�} tt | j
|d �� |� t| �� |� d� d S ) Nz>>> %s, UID %d
rB r* z
Faults:
Z snap_faultsz
* %s: %d
Zsnap_sqlz
SQL Queries:
ri ZTimez SQL-queryz
Processes:
Z snap_procrh rj z Memory (Mb)r0 zCPU (%)�lc s g | ]}� � |d ��qS )zN/A)r )�.0�k)rU r r �
<listcomp>� s z2LVEReadSnaphot.format_snapshot.<locals>.<listcomp>z
Z snap_httpzHttp requests:
ZPidZDomainz Http typeZPathz Http version)rS r rX rm r3 r � prettytableZ PrettyTabler# r Zadd_rowZ
get_string�set�values�keys�addr@ Zalignrq �str)
r4 rV rc rt �vZ sql_tableZfields�keyZ data_keysZ
table_columns�tableZsnap_proc_aggrZ
http_tabler )rU r rb m sN
z LVEReadSnaphot.format_snapshotN)�__name__�
__module__� __qualname__r5 r; rF � staticmethodrf �maxsizerC rM rP rW rZ r$ r% r# r^ rR rq rb r r r r r! i s >r! )N)'Z
__future__r r �builtinsr r- rf rv Z
sqlalchemyr Zsqlalchemy.ormr Z lvestats.lib.commons.decorators�lvestatsZ lvestats.libr r Zlvestats.lib.commonsr Zlvestats.lib.jsonhandlerr
Z/lvestats.lib.parsers.lve_read_snapshot_argparser Zlvestats.lib.snapshotr Zlvestats.orm.incidentr
ra r_ r r �libZcommonsZ
decoratorsZ
no_sigpiper+ r/ �objectr! r r r r �<module> s,
,
Mini Shell