Current File : //proc/227033/root/opt/alt/python37/lib/python3.7/site-packages/wmt/__pycache__/main.cpython-37.pyc
B
��wa�4 � @ s* d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlmZm Z d dl
m Z m Z d dl
mZ d dlmZ d dlmZmZmZmZmZ d dlmZmZmZmZ d dl m Z d d l m Z d d
l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z( d d l)m*Z*m+Z+ d d l,m-Z- d d
l.m/Z/ d dl0m1Z1 ed�Z2dd� Z3dd� Z4eG dd� d��Z5dd� Z6d3dd�Z7d4e d�dd �Z8d5d d �Z9d d � Z:d!d"� Z;d#d$� Z<d%d&� Z=d'd(� Z>d)d*� Z?d+d,� Z@d-d.� ZAeBd/k�r&eCe�D� �ZEe2�Fd0eE� ee%eE��F e d1e d2�e"� e�ejGe3� e�ejHe4� e �I� ZJeJ�KeA� � W dQ R X dS )6� N)�datetime� timedelta)�Optional�Iterable)� dataclass)�or_)� ScrapeResult� DomainAlerts�setup_database�
session_scope�cleanup_old_data)� get_domains� setup_logger�save_pid_and_lock� intersect)�init_sentry_client)�get_pkg_version)�PING_TIMEOUT_STATUS_CODE�
SENTRY_DNS�!ERROR_DOMAINS_PING_RETRY_INTERVAL� ERROR_DOMAINS_ALERT_INTERVAL�
WMT_LOCK_FILE�PING_CONNECTIONS�&LICENSE_EXPIRED_FAREWELL_LETTER_MARKER�LICENSE_CHECK_PAUSE)�Notifier�SupportedNotificationTypes)� ErrorReport)�cfg)�CloudlinuxLicenseLibZ wmt_scannerc C s t �� t�dtt �� �� d S )NzReloading config: %s)r �reload�logger�info�strZto_dict)�
sig_number�frame� r& �9/opt/alt/python37/lib/python3.7/site-packages/wmt/main.py� reload_conf- s r( c C s t �d� dS )zp
Shutdown to call finally block to
close all fds, remove lock and file
see: save_pid_and_lock()
r N)�sys�exit)r$ r% r& r& r' �shutdown2 s r+ c @ s6 e Zd ZU eed< dZee ed< dZee ed< dS )�ScrapeCoroResult�urlN�
response_code�response_time_ms) �__name__�
__module__� __qualname__r# �__annotations__r. r �intr/ r& r& r& r' r, ; s
r, c � s� t �� 4 I dH ��}|4 I dH �� t�� }yH|j| |d�4 I dH �"}t| |jtdt�� | �d�S Q I dH R X W nD tjj k
r� t| t
|d d�S t j j k
r� t| dd�S X W dQ I dH R X W dQ I dH R X dS )a
Main 'pinger'
1. Requests domains
- if domain responded - keep status code
- if no response for timeout - keep Timeout status code
- if unreachable (ConnectionError or so) - keep 523 status code
(same logic as go implementation)
N)�timeouti� )r. r/ i )r. )
�aiohttpZ
ClientSession�time�getr, Zstatusr4 �
concurrentZfutures� TimeoutErrorr Zclient_exceptionsZ ClientError)r- � ping_timeout� semaphore�session�startZrespr& r& r' �pingC s ( r? c c s< |dkrt � n|}x$|D ] }t�|�st|| |�V qW dS )z�
ping_timeout: specified in config timeout time (s) for request
semaphore: semaphore obj to handle asyncio tasks
ping_target_domains: mostly needed for re-pinging error domains
N)r
r Zis_domain_ignoredr? )r; r< �ping_target_domains�domains�domainr& r& r' � executors_ s
rC )�returnc � s@ t t| ||��}t|�dkr g S tjt |�|d�I d H \}}|S )Nr )r5 )�tuplerC �len�asyncio�wait)Zping_site_timeout�
ping_intervalr< r@ Ztasks�pinged�_r& r& r' � scrape_sitesl s
rL c
C s t � }i }g }t| ���}x`|D ]X}|�� }|�t|jd|j|jd�� |�|j� |jdkrj|j||j< q |�|j� q W |� t
�� t
j �
|��jtdd�dd� W dQ R X |dk r�|nt� } t | �| }
t| ��&}x |
D ]} |�t| dd�� q�W W dQ R X |S ) a
- obtains all scrape coro results from asyncio tasks
- saves ping results to ScrapeResult table
- updates 'is_resolved' field in DomainAlerts table, in case
error domain`s status code was changed to 200
- returns domains with non-200 status code code
T)�website� is_finishedr. r/ �� )� is_resolvedF)Zsynchronize_sessionN)rM rN )�setr �result�addr r- r. r/ �append�queryr �filterrM �in_�update�dictr
) �enginerJ r@ Zfinished_domainsZerrors_domainsZresolvedr= ZtaskrR rA Zunfinished_domainsZ
unfinishedr& r& r' �manage_ping_resultst s6
r[ c C sj t �� ttd� }t| ��F}|�tj��tj� t
|� � ��t tj
|ktjdk��}dd� |D �S Q R X dS )z�
- gets websites that must NOT be included in alert email:
less than ERROR_DOMAINS_ALERT_INTERVAL passed or is_resolved marker was not
changed from last alerting
)ZhoursFc S s g | ]
}|j �qS r& )rM )�.0�rowr& r& r' �
<listcomp>� s z%get_recent_alerts.<locals>.<listcomp>N)r �nowr r r rU r rM rV rW �list�keysr �
alert_timerP )rZ �
alert_domainsZrepeat_intervalr= �recently_alertedr& r& r' �get_recent_alerts� s
re c C sL t �dtt| �� ��� dd� | �� D �}ttjtj d|it
j d�� � | S )z�
prepares needed error report object with error domains
to be alerted and sends this mail
returns alerted domains
zAlerts will be sent for %sc
S s2 g | ]*\}}t |d �ttt|���t|�d��qS )z, )r- �codeZ count_errors)r �join�mapr# rQ rF )r\ rB Zcodesr& r& r' r^ � s zalert.<locals>.<listcomp>� error_report)� target_email�
from_email�report�notification_type)
r! r" r# r` ra �itemsr r rj rk r ZALERT�notify)Z domains_datari r& r& r' �alert� s rp c s� t | |�� � fdd�|�� D �}|sBt�dtt|�� ��t� dS t|� t �
� }t | ��x}|� t
��t
j��� }dd� |D �}xL|D ]D}||kr�|� t
��t
j|k��t|dd�� q�|�t
||d �� q�W W dQ R X dS )
aJ
- gets recently alerted domains (those that must not be alerted again)
and does not include them for alerting
- calls alerting for left domains
- updates DomainAlerts table:
if website was not alerted -> adds new record
if website was alerted before -> updates alert time and is_resolved marker
c s i | ]\}}|� kr||�qS r& r& )r\ �k�v)rd r& r' �
<dictcomp>� s z flush_alerts.<locals>.<dictcomp>zDAll domains "%s" were alerted or still not resolved in last %d hoursNc S s g | ]
}|j �qS r& )rM )r\ r] r& r& r' r^ � s z flush_alerts.<locals>.<listcomp>F)rb rP )rM rb )re rn r! r" r# r` ra r rp r r_ r rU r Z
with_entitiesrM �allrV rX rY rS )rZ rc Zdomains_to_alertr_ r= ZwebsitesZurlsrB r& )rd r' � flush_alerts� s&
ru c C s | rt j jrdS dS )NTF)r Zalert_notifications_enabled)�
error_domainsr& r& r' �should_be_repinged� s rw c C s$ t j�t�r t�d� t �t� d S )Nz CloudLinux license was updated)�os�path�existsr r! r" �remover& r& r& r' � cleanup_farewell_letter_marker� s
r| c C sX y4t j�t�s2t�d� ttjtj i t
j d�� � W n t
k
rR t�d� Y nX dS )z@
Sends farewell letter once (if it was not sent before)
z/Going to send last email about expired license!)rj rk rl rm z$Error while managing farewell letterN)rx ry rz r r! �warningr r rj rk r ZFAREWELLro � Exception� exceptionr& r& r& r' �manage_license_farewell� s
r� c
� s� t � � }tjjd }�zF�y t� t|� t�t�}ttjj ||�I dH }t
||�}t |��r2t �
dt|�� x�tjjd }|tkr�t � � | }t�ttt| d�d��I dH |tkr�ttjj |||�� d�I dH }t
|||�� �} t|| �}
P qpt| |�}
|} P qpW |
�r6t �
dtt|
�� ��� t||
� ng } W n tk
�rX t �d� Y nX W dxNtjjd }t � � | } tt|| d�d�} t�| �I dH | |k�r`P �q`W X | S ) aF
Scanner logic:
1. Scrapes domains and obtains ping results;
2. Manage ping results (e.g: saving to DB)
3. In case error domains found -> start re-pinging
Re-pinging:
- in min(ping_interval, 5 mins)
- flush alerts if needed
4. Sleep for ping_interval until next ping iteration
�< Nz8Those domains are unsuccessful: %s
Try to re-ping themr �
)r@ z1Domains with unsuccessful status code found: "%s"z Error during ping iteration!)r7 r rI r| r rG Z Semaphorer rL r; r[ rw r! r" r# r �sleep�min�maxra r r` ru r~ r )
�previously_erroredrZ r> �ping_interval_secondsr< Z ping_resultrv Zelapsed_for_pingZping_retry_resultZ retry_errorsrc �elapsed�
sleep_timer& r& r' �scrape_iteration sR
r� c � s� t � } g }d}xvt� �� r0d}t|| �I dH }q|d7 }|dkrJt�d� |dkr\t� d}tjjd }t |t
�}t � |�I dH qW dS )z�
Main loop for wmt_scanner_solo service
each 'while: True' iteration returns errored domains
(domains that responded with non-200 status code)
r N� z)Seems your CloudLinux license is expired!� r� )
r
r Zget_license_statusr� r! r} r� r rI r� r rG r� )rZ r� Zlicense_attemptr� r� r& r& r' � scrape_loopI s
r� �__main__zPID: %szweb-monitoring-toolzcl-web-monitoring-tool)N)N)N)LrG rx r) r6 �signalr7 �concurrent.futuresr9 r r �typingr r Z dataclassesr Z
sqlalchemyr Zwmt.dbr r r
r r Zwmt.common.utilsr
r r r Zclsentryr Zclsentry.utilsr Zwmt.common.constr r r r r r r r Zwmt.common.notificationr r Zwmt.common.reportr Z
wmt.commonr Z cllicenser r! r( r+ r, r? rC rL r[ re rp ru rw r| r� r� r� r0 r# �getpid�pidr" �SIGUSR1�SIGTERMZget_event_loopZloopZrun_until_completer& r& r& r' �<module> s\ (
*"D
Mini Shell