Current File : //opt/alt/python37/lib/python3.7/site-packages/clsummary/__pycache__/summary.cpython-37.pyc
B
�2Lb�� � @ s� d dl mZ d dl mZ d dlZd dlZd dlmZ d dlZd dlZd dl Z d dl
Z
d dl Z d dl Z d dl
Z
d dlZd dlZd dlZd dlZd dlmZmZ d dlmZmZ d dlmZ d dlmZ d d lm Z d d
l m Z d d l m Z d d l!m"Z" d dl#Z#d dl$Z%d dl&Z&d d
l'm(Z( d dl)m*Z* d dl+m,Z, d dl-m.Z.m/Z/m0Z0m1Z1m2Z2 d dl3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;m<Z< d dl=m>Z>m?Z?m@Z@ d dlAmBZB d dlCmDZDmEZEmFZF d dlGmHZHmIZI d dlJmKZKmLZL d dlMmNZN d dlOmPZP ddlQmRZR d dlSmTZTmUZUmVZVmWZWmXZXmYZYmZZZm[Z[ d d l\m]Z]m^Z^m_Z_m`Z`maZambZbmcZcmdZdmeZemfZfmgZgmhZhmiZimjZjmkZk d d llmmZm d d lnmoZompZpmqZq d d lrmsZs d d ltmuZv d d!lwmxZxmyZy d d"lzm{Z{ d d#l|m}Z} yTd d$l~mZm�Z�m�Z�m�Z�m�Z�m�Z� d d%l�m�Z� d d&l�m�Z� d d l�muZ� d d'l�m�Z� W n> e�k
�rN eeZ�eeZ�eeZ�eeZ�eeZeeZ�eeZ�dZ�dZ�dZ�Y nX d(Z�e ��d)�Z�d*Z�d+Z�d,Z�d-Z�d.Z�d/Z�d0Z�d1Z�G d2d3� d3e��Z�e�e�d4�d5d6�Z�G d7d8� d8e��Z�d9d:� Z�dS );� )�absolute_import)�divisionN)�range)� lru_cache�partial)�Counter� defaultdict)�groupby)� iteritems)�
attrgetter)� cpu_count)�getfqdn)�Path)�cpapi)�Feature)�
MySQLGovernor)�is_cl_solo_edition�is_cl_shared_edition�
CLEditions� is_ubuntu�is_secureboot_enabled) �get_rhn_systemid_value� run_command�ExternalProgramFailed�is_testing_enabled_repo�is_litespeed_running�get_cl_version�
get_virt_type�grep�is_nginx_running)�
find_wp_paths�get_wp_cache_plugin� get_wp_paths_with_enabled_module)�
SysCtlConf)�cagefs_statistics_config�db_governor_lib�clconfig_utils)�print_dictionary�replace_params)�LimitsDataStorage�DEFAULTS)� EMPTY_LIMITS)�LvpMap� )� parse_cloudlinux_summary_opts)�Dict�Optional�Callable�Union�List�Any�Tuple�AnyStr)�is_statistic_enabled� set_statistic_collection_enabled�$write_statistics_send_status_to_file�
SummaryStatus� is_python_selector_installed�is_ruby_selector_installed� is_nodejs_selector_installed�is_php_selector_installed�dummy_none_function� get_packages_with_lve_extensions�is_sending_process_running�$get_statistics_send_status_from_file� get_client_data_from_jwt_token�get_cl_plus_sender_status� is_active_cloudlinux_license)�get_rpm_packages_info)�get_cpu_metrics�get_memory_metrics� NotSupported)� ALL_MODULES)� PKG_VERSION)� CONFIG_PATH�_read_config_file)� is_email_notification_enabled)� get_hidepid_typing_from_mounts)�get_versions_statistics�get_native_version_safe�get_php_selector_usage�iter_server_applications�get_default_php_version�get_mode_of_php_selector)�CloudlinuxSelectorLib)�get_default_version)� PKG_RELEASEz /var/log/cloudlinux-summary.logzcloudlinux-summary�unknownZ installedZ
not_installedZnot_initializedZ not_selected�enabledZdisabledz-42c
@ sL e Zd ZdZddddgdgddiddiddig d� Zd dd�Zd
d d
�Zd S )�StatisticsDictz>
Special class to store all metrics before sending it
z-42i���g E�) �str�int�float�str_list�int_list�int_dictZ
float_dict�str_dict�
rpm_stat_listTc
C sR | j | }y
|� }W n8 tk
rL } z|r8t�d||� |}W dd}~X Y nX |S )a
Method for catching any exceptions while calling passed function,
logging them and return default result in case if exception is present or
return formatted result in case if exception is absent
:param log_exception: True - exception will been logged, False - exception won't been logged
This flag must be used if param `func` is function, which processes intermediate value, which can equal
to -42 (it means that exception was logged in previous moment) and
doesn't call external functions or doesn't process another values. In other cases
log_exception should be equal True, because in calling of external function or processing of another
value can be raised exception, which should be logged.
Example: functions `lambda: intermediate_value[0].property` or
[len(value) for value intermediate_value.values()] should be used with log_exception equal False,
because they only process intermediate value. But function:
def some_function(intermediate_value):
result = external_function()
return len(intermediate_value), result
should be used with log_exception equal True
z%s. Exception: "%s"N)�DEFAULT_RESULTS� Exception�
app_logger� exception)�self�func�type_of_result� log_message�
log_exceptionZdefault_result�result�err� ro �B/opt/alt/python37/lib/python3.7/site-packages/clsummary/summary.py�run_safe_or_log_errors� s
z%StatisticsDict.run_safe_or_log_errorsc C s | j ||||d�}|| |< dS )z�
Method which call `run_safe_or_log_errors` and save result from it
:param log_exception: True - exception will been logged, False - exception won't been logged
See method `run_safe_or_log_errors`
)rl N)rq )rh ri rj Zname_of_metricrk rl rm ro ro rp �
format_metric� s z StatisticsDict.format_metricN)T)T)�__name__�
__module__� __qualname__�__doc__rd rq rr ro ro ro rp r[ � s
&r[ )� site_dict�returnc C s t tdd� | d �� ��S )zB
Checks if wordpress site has at least one enabled module
c S s | d S )NrZ ro )�modulero ro rp �<lambda>� � z*site_has_enabled_modules.<locals>.<lambda>�modules)�any�map�values)rw ro ro rp �site_has_enabled_modules� s r� c @ s� e Zd ZdZdZe�dddge�e j
�e�e j �e�e j �g�Z
dZdZdZd d
� Zed d � �Zed
d� �Zedd� �Zedd� �Zedd� �Zedd� �Zdd� Zedd� �Zdd � Z d d � Z ed d � �Z ed!d"� �Z ed#d$� �Z d%d&� Z!d'd(� Z"ed)d*� �Z#d+d,� Z$ed-d.�d/d0��Z%d1d2� Z&ed3d4� �Z'ed5d6� �Z(ee)e*e*f e+d7�d8d9��Z,d:d;� Z-d<d=� Z.ee/e0d-d>�d?d@��Z1e)e0e/f e0e+d-dA�dBdC�Z2dDdE� Z3edFdG� �Z4dHdI� Z5dJdK� Z6edLdM� �Z7dNdO� Z8d-d.�dPdQ�Z9dRdS� Z:edTdU� �Z;dVdW� Z<dXdY� Z=dZd[� Z>ed\d]� �Z?d^d_� Z@d`da� ZAdbdc� ZBddde� ZCdfdg� ZDdhdi� ZEdjdk� ZFdldm� ZGdndo� ZHeeIeJdp�dqdr��ZKeLeJeM eJeM e)eIe/f f d.�dsdt�ZNeIeIe)eIeIf e)eIeOeJeIf f du�dvdw�ZPeIeIdx�dydz�ZQeeRd-d{�d|d}� ��ZSd~d� ZTd�d�� ZUed�d�� �ZVed�d�� �ZWed�d�� �ZXed�d�� �ZYed�d�� �ZZed�d�� �Z[d�d�� Z\ed�d�� �Z]ed�d�� �Z^d�d�� Z_ed�d�� �Z`ed�d�� �Zad�d�� Zbed�d�� �Zcd�d�� Zded�d�� �Zed�d�� Zfed�d�� �Zged�d�� �Zhed�d�� �Ziee+d.�d�d���Zjee+d.�d�d���Zkee/d.�d�d���ZleeId.�d�d���Zmed�d�� �Znd�d�� Zod�d�� Zpe0eqe/ d��d�d��Zrd�d�� Zsd�d�� Ztd�d�� Zuee)d.�d�d�Zvd�dĄ Zwd�dƄ ZxeLe/e/e/f d.�d�dȄZyeeLe/eIf d.�d�dʄ�Zzd�d̄ Z{d�d΄ Z|d�dЄ Z}d�d҄ Z~d�dԄ Zed�dք �Z�d�d Z�e�d�dڄ �Z�ed�d܄ �Z�ed�dބ �Z�d�d� Z�d�d� Z�d�d� Z�ed�d�d��Z�ed�d�d��Z�ed�d� �Z�ed�d� �Z�ed�d� �Z�ed�d� �Z�ed�d�� �Z�d�d�� Z�ed�d�� �Z�d-S )��CloudlinuxSummaryz /var/lve/dashboard_certificatez/etc/cl_plus/.disabled�python�ruby�nodejsz-https://stat-api.cloudlinux.com/api/clos-statz-https://stat-api.cloudlinux.com/api/rpm-statsz6https://repo.cloudlinux.com/static/cl-settings-v1.jsonc C s> t � | _d | _t� | _d | _d | _| �� | _d | _ t
� | _ d S )N) �dict�_opts�_security_tokenr[ �
statistics�_lvpmap�
_system_id�_is_process_not_limited�is_process_not_limited�packages_by_lenr# �sysctl)rh ro ro rp �__init__� s
zCloudlinuxSummary.__init__c C s, t tj�d��} t tj�d��}| o*| S )zm
Return true if process is running outside LVE or it's not running by
utility `nice`
�RUNNING_BY_NICE�RUNNING_IN_LVE)�bool�os�environ�get)Zis_running_by_niceZis_running_in_lvero ro rp r� s z)CloudlinuxSummary._is_process_not_limitedc C s | j dkrt� | _ | j S )z.
Load lvpmap only when needed
N)r� � _get_lvpmap)rh ro ro rp �lvpmap s
zCloudlinuxSummary.lvpmapc C s | j d krtd�| _ | j S )N� system_id)r� r )rh ro ro rp r� s
zCloudlinuxSummary.system_idc C s t � r
dS dS )NZubuntuZ rhel_like)r ro ro ro rp �
_get_platform# s z CloudlinuxSummary._get_platformc C s t � r
tS tS )N)r �ENABLED�DISABLEDro ro ro rp �_detect_secureboot' s z$CloudlinuxSummary._detect_securebootc s, t jt j � d�� fdd�td�D ��} | S )N� c 3 s | ]}t �� �V qd S )N)�randomZchoice)�.0�_)�range_for_random_choicero rp � <genexpr>. s z=CloudlinuxSummary._generate_security_token.<locals>.<genexpr>�@ )�stringZ
ascii_lettersZdigits�joinr )�security_tokenro )r� rp �_generate_security_token+ s z*CloudlinuxSummary._generate_security_tokenc
C s� i }| j d kr*d}t�|� | �d|i� d| j }t�|� | j| j d�}d }ytj| j|dd�}W nD tj k
r� } z$t
|�}t�|� | �d|i� W d d }~X Y nX |j s�d� |j
|j�}t�|� | �d|i� t�d� y2|�� d }t|d t��r|d �| �� � W n~ ttfk
�rL d }t�|� | �d|i� Y nT tk
�r� } z*t�d
t
|�|j� | �dd t i� W d d }~X Y n X t�d � |S )
NzSecurity token is emptyrm z!Getting statistics from server %s)r� r� �< )�params�timeoutz*Server answer is: HTTP code {}; Reason: {}z!Received response from the serverz Can't parse api response to jsonz=Invalid json response from server, field %s not found in "%s"z1Invalid response from server. See %s for details.z,SUCCESS: received statistics from the server)r� rf �error�_error_and_exit� SUMMARY_URL�infor� �requestsr� �RequestExceptionr\ �ok�format� status_code�reason�json�
isinstancer� �update� _get_smart_advice_statistics� TypeError�
ValueError�KeyError�text�LOG_FILE)rh Z stat_data�messager� �response�ero ro rp �_get_remote_data1 sP
z"CloudlinuxSummary._get_remote_datac C sL | j d k r| j S tj�| j�r*| �� | _ n | �� }| �|�r@|nd | _ | j S )N)r� r� �path�isfile�DASHBOARD_CERTIFICATE�_read_token_from_filer� �_write_token_to_file)rh �tokenro ro rp r� h s
z CloudlinuxSummary.security_tokenc
C sp y4t | jd��}|�|� W dQ R X t�| jd� dS ttfk
rj } zt�dt |�� dS d}~X Y nX dS )z�
Write security token to file and return success/fail status
:param token: generated security token
:return: T/F status
�wNi� Tz,Error while writing secure token to file: %sF)
�openr� �writer� �chmod�IOError�OSErrorrf r� r\ )rh r� �fr� ro ro rp r� t s z&CloudlinuxSummary._write_token_to_filec
C sb y&t | j��}|�� �� p d S Q R X W n6 ttfk
r\ } zt�dt|�� d S d }~X Y nX d S )Nz.Error while reading file with secure token: %s) r� r� �read�stripr� r� rf r� r\ )rh r� r� ro ro rp r� � s z'CloudlinuxSummary._read_token_from_filec C s t jt jdddd�S )z�
Detect old LVE limits integration presence according to
https://docs.cloudlinux.com/index.html?lve_limits_with_packages.html
:return: True/False - present/absent
ZCUSTOM_GETPACKAGE_SCRIPT�=F)� file_name�
param_nameZ separatorZ default_val)�detectZget_boolean_paramZCL_CONFIG_FILEro ro ro rp �_detect_old_lve_integration� s
z-CloudlinuxSummary._detect_old_lve_integrationc C s t j�d�S )zO
Detects presence/absence of lsapi
:return: True/False
z/usr/bin/switch_mod_lsapi)r� r� �existsro ro ro rp �_is_lsapi_present� s z#CloudlinuxSummary._is_lsapi_presentc C s� | dkrt � sPtS n<| dkr*t� r&tS tS | dkr>t� sPtS n| dkrPt� sPtS t| �}|dkrdtS | dkr�y|�� d r~tS tS t k
r� tS X n| dkr�|�
� r�tS tS t d� | ���dS ) zP
Get selector status for nodejs, python, ruby and php selectors
r� r� r� �phpN)r� r� Zselector_enabledzUnknown interpreter: {})
r; �
NOT_INSTALLEDr< r� r= r> rV Zget_selector_statusr� r� Zphp_selector_is_disabledr� r� )� interpreter�libro ro rp �_get_status_of_selector� s* z)CloudlinuxSummary._get_status_of_selectorc
C s� yt �|��� }|S t jk
rX } z&t�dt|�� | �dt|�i� W d d }~X Y n6 tt fk
r� } zt�dt|�� d S d }~X Y nX d S )Nz3Request exception while getting remote settings: %srm z'Error while parsing remote settings: %s)
r� r� r� r� rf r� r\ r� r� r� )rh Z settings_url�settingsr� ro ro rp �_get_remote_settings� s "z&CloudlinuxSummary._get_remote_settingsc
C s� | j �d�rdS ttj�d��}|r(dS | �| j�}|dkr@dS y"|d d }|d d ob| �|�S tk
r� } z&t �
d t |�� | � d
t |�i� W dd}~X Y nX dS ) z-
Return cl-statistics status
z--force-collectTZCL_TEST_SYSTEMNFz
cl-statisticsz
rollout-grouprZ z4Error occurred while trying to get rollout group: %srm )
r� r� r� r� r� r� � SETTINGS_URL�
_match_serverr� rf r� r\ r� )rh Zis_test_environmentr� Z
rollout_groupr� ro ro rp �_is_statistics_enabled� s z(CloudlinuxSummary._is_statistics_enabledc C s
t | d�S )N� )r] )� hash_serverro ro rp �
_to_number� s z CloudlinuxSummary._to_numberc C s@ | j d krdS t�| j �� ��� d d� }| �|�d| dkS )NT� � r )r� �hashlibZsha256�encodeZ hexdigestr� )rh Zurl_numr� ro ro rp r� � s
z CloudlinuxSummary._match_serverN)rx c C s* d} x | r$t � s$| d8 } t�d� qW dS )zC
Wait for running background process of cl-summary
�2 r- g�������?N)rA �time�sleep)Zretriesro ro rp � _wait_for_background_process� s z.CloudlinuxSummary._wait_for_background_processc C s� | j d s| j d r>t| j d � t�� dd�}t|d� d S | j d r�t� rRdnd}t�� |dd �}|�d
t� i� t|d� td � | j �d �r�| � � }| j
|t � d
� | j �d�r�| � � | j �d�r�| j
| j| jdd� t�d� n t| jd� d S | �� �s2tjdt�� d�}t|� | jddid d� | j �d��r�| j �d��s�t � �s�tjdt�� d�}t|� | jddid d� | j �d��r�| jd k�r�d}t�|� tjd t t�� d�}t|� | �d|i� | j �d ��rLt� �r�t�� ddd �}n<ttjt�� d d �� tj d dd d!� | � � t�� dd�}t|d� td � | j d �r�| j �r�tdd"��r�t�� ddd �}t|d� td � nttjt�� d d �� d S )#N�enable�disable�success)� timestamprm T�statusZ
collectingZready)r� r� rm Zsending_statusr z
get-remote)�datar7 z rpm-packagesz--sendF)�summary�url� save_statuszRPM statistics sentzkStatistics collection is disabled globally. Please, try again later or contact support if it happens again.)rm r� r� rm zYCollecting statistics is disabled globally. Use --force-collect to ignore global settings)�
error_codez--force-collectz_Statistics collection is disabled by admin. Run `cloudlinux-summary enable` and then try again.zXSending statistics is disabled by admin. Use --force-collect to ignore admin`s settings.z8Statistics was not sent, because security token is emptyz�We are not able to collect statistics because we are not able to make a security token. Check %s for details or contact support.z--async)rm r� r� z9/usr/sbin/cloudlinux-summary --send --json &> /dev/null &z /bin/bash)�shell�
executable)� acquire_lock) r� r8 r� r' rA r� rB �exitr� r� �_print_result_and_exitr7 �_get_rpm_packages_summary� _send_statistics_and_save_statusr� �RPM_PACKAGES_URLrf r� r� r: �FAILEDr9 r� r� r� r� Z IN_PROGRESS�
subprocess�runr� r� )rh r� r� rm � status_dictr� ro ro rp �&_actions_before_run_process_as_limited� s�
z8CloudlinuxSummary._actions_before_run_process_as_limitedc C sj t jdddd�}t �|��F}| �dt|�g� tjddt|�dg| ttj d t
� t j �d
�d �S Q R X d S )
z�
Run same command in lve and set environ RUNNING_IN_LVE=1
in order to check it in child process.
:return:
� r i @)Zls_cpuZls_cpusZ
ls_memory_phyz--lve-idz/bin/lve_suwrapperz-nz /usr/sbin/cloudlinux-summary�1)r� ZCPU_DATA)�envN)
� lvectllibZmake_liblve_settingsZ
temporary_lve�extendr\ r �callr� r� r� r� �dumpsZ CPUINFO_DATA)�argsr� Zlve_idro ro rp �_run_self_in_lvep s
z"CloudlinuxSummary._run_self_in_lvec C s$ t jddddg| ttjdd�d�S )z�
Run same command using utility `nice` and set environ RUNNING_BY_NICE=1
in order to check it in child process.
:return:
z
/usr/bin/nicez-nZ19z /usr/sbin/cloudlinux-summaryr )r� )r )r r r� r� r� )r ro ro rp �_run_self_by_nice� s z#CloudlinuxSummary._run_self_by_nice)�optsrx c s8 t � fdd�dD ��rdS t � fdd�dD ��r4dS dS )zK
Check that passed command should run as unlimited process
c 3 s | ]}� | V qd S )Nro )r� �option)r ro rp r� � s zECloudlinuxSummary._should_run_as_unlimited_process.<locals>.<genexpr>)z rpm-packagesr� z
get-remoter� r� Tc 3 s | ]}� | V qd S )Nro )r� r )r ro rp r� � s )z--sendz--asyncz--jsonz--force-collectF)r} )r ro )r rp � _should_run_as_unlimited_process� s
z2CloudlinuxSummary._should_run_as_unlimited_processc
C s | � |�| _| jr�| �� | �| j�r0td� n�tdd�rN| �|�}t|� nly| �|�}t|� W nT t j
k
r� } z4d| }t |� t �
t�}|j|dd� td� W d d }~X Y nX nH| �| j�r�d}t�� |d�}tj|| jd � t|d� td
� | �� d S ) Nr T)�skip_jwt_checkz$failed to run task in lve, error: %s)�exc_info���z|You shouldn't use env var "RUNNING_IN_LVE" or "RUNNING_BY_NICE" for run of any command except collection of main statistics.)r� rm )Zextrar- )� _parse_argsr� r� r r r� r r
r r Z
PyLveError�print�logging� getLoggerrs rg r� rf r� r' �run_get_summary)rh �argv�rcr� Z error_msg�logZerr_msgr� ro ro rp r � s2
zCloudlinuxSummary.runc C s� t � � }| �� t � � | }|| jd< | jd rJ| j�t| j|�ddd� | j�d�rv| j| j| j dd� t
� d � n t | jd� d S )
NZcl_summary_execution_timez--lve-idr\ Zcl_summary_max_mem_usedz,Can't get memory usage by cloudlinux-summaryz--sendT)r� r� r� zMain statistics sent)
r� � _get_summaryr� r� rr r �_get_max_memoryr� r� r� rf r� r' )rh Z
start_time� running_timero ro rp r � s"
z!CloudlinuxSummary.run_get_summary)r� �summary_resultrx c C s2 t j| d�}|dkr&||d< t j|d< t|� dS )z@
Save status of sending statistics to json file
)rm r� r� r� rm N)r: �SUCCESSr� r9 )r� r r ro ro rp � _save_status� s
z CloudlinuxSummary._save_status)r� r� r� rx c C sJ t t�� �}||d< | j||d�}||d�}t|d� |rF| �||� dS )zH
Send statistics data to server and save status to file
r� )r� )rm r� TN)r] r� �_send_statisticsr' r! )rh r� r� r� r� Zs_resultrm ro ro rp r� � s
z2CloudlinuxSummary._send_statistics_and_save_statusc
C s� |d }|dk rd S dddt | jd �ddd d
�tt�|���g}yt|d d �\}}}W n. tk
r� } zt� d
|� d S d }~X Y nX |dkr�t
� |�}yt dd� |d D ��S t
tfk
r� d S X t�d||� d S )Nr� r- z/usr/sbin/lveinfoz--jsonz--idz--lve-idz--show-columns�mPMemz--periodz{}mT)�return_full_outputz Unable to run lveinfo, error: %sr c S s g | ] }|d �qS )r# ro )r� �xro ro rp �
<listcomp> s z5CloudlinuxSummary._get_max_memory.<locals>.<listcomp>r� z-lveinfo failed with exit code: %i, output: %s)r\ r� r� r] �mathZceilr r rf �warningr� �loads�maxr� r� r� ) rh r Z time_minutes�cmdr �json_strr� r� Z parsed_dataro ro rp r s*
z!CloudlinuxSummary._get_max_memoryc C s� d}y�d| }t �|� t�d�}xbtd�D ]R}ytj|| dd�}W n6 tjk
rz } z|}t�d| � W dd}~X Y q,X P q,W |�|j d kr�t �d
� nd �
|j |j �}t �|� W n6 tjk
r� } zt |�}t �
|� W dd}~X Y nX |S ) z�
Sends statistics to server
:param data: Statistics data dict
:return: string - message for JSON 'result' key
r� z Sending statictics to server %sz*Unknown exception while sending statistics� r� )r� r� � N�� zSending statictics OKz*Server answer is: HTTP code {}; Reason: {})rf r� r� r� r Zpost�ConnectionErrorr� r� r� r� r� r\ r� )r� r� Z out_messager� Z expected_err�ir� rn ro ro rp r" & s0
z"CloudlinuxSummary._send_statisticsc C s& dt � � d�}| �� | j�|� |S )Nr- )�versionr� )r� � _prepare_cl_normal_statisticsr� r� )rh rm ro ro rp r N s
z CloudlinuxSummary._get_summaryc C s& dt � � d�}| �� | j�|� |S )Nr- )r2 r� )r� �'_fill_dict_with_rpm_packages_statisticsr� r� )rh rm ro ro rp r� V s
z+CloudlinuxSummary._get_rpm_packages_summaryc C s t �� t jS )z/
Get version of control panel
)r� ZgetCPZ
CP_VERSIONro ro ro rp �_get_panel_version^ s z$CloudlinuxSummary._get_panel_versionc C s` | j �tjdd�}|dk rR|| j d< | j �dd� ddd� | j �d d� dd
d � n
t| j d
< dS ) zN
Fill dict with statistics by statistics about MySQL governor
r\ z Can't get MySQL governor modeNZmysql_governor_modec S s
t � �� S )N)r Zget_governor_versionro ro ro rp rz v r{ zCCloudlinuxSummary._fill_mysql_governor_statistics.<locals>.<lambda>Zmysql_governor_versionz Can't get MySQL governor versionc S s t � �� d S )Nr )r Zget_governor_statusro ro ro rp rz | r{ Zmysql_governor_statusz Can't get MySQL governor status)r� rq r% Zget_gov_mode_operationrr r� )rh Zmysql_gov_modero ro rp � _fill_mysql_governor_statisticsh s"
z1CloudlinuxSummary._fill_mysql_governor_statisticsc C s~ | j �tjddd� | j �dd� ddd� | j �| jdd d
� | j �| jdd d � | j �td
dd� | j �dd� d
dd� dS )zM
Fill dict with statistics by statistics about control panel
r\ �control_panel_namez Can't get control panel namec S s dd� t �� �� D �S )Nc S s g | ]\}}|r|�qS ro ro )r� �nameZ is_supportedro ro rp r& � s zVCloudlinuxSummary._fill_control_panel_statistics.<locals>.<lambda>.<locals>.<listcomp>)r Zget_supported_cl_features�itemsro ro ro rp rz � r{ zBCloudlinuxSummary._fill_control_panel_statistics.<locals>.<lambda>r_ Zsupported_cl_featuresz8Can't get list of supported cl features by control panelZcontrol_panel_versionz Can't get control panel versionZcontrol_panel_apachez Can't get control panel apacher] Z
nginx_runningz Can't get control panel nginxc S s t t�� �S )N)�lenr �cpusersro ro ro rp rz � r{ Z users_totalzCan't get amount of usersN)r� rr r� � getCPNamer5 �_get_control_panel_apacher )rh ro ro rp � _fill_control_panel_statistics� s< z0CloudlinuxSummary._fill_control_panel_statisticsc C sP t � r(t�� rd}qLt�� r"d}qLd}n$| jd dkrHt�� rBdnd}nd}|S ) z�
Wrapper to retrieve control panel Apache version:
EA3 or EA4 for cPanel, native otherwise
:return: EA3|EA4|native
Z litespeedZ
openlitespeedZunknown_litespeedr7 ZcPanelZEA4ZEA3�native)r r� Zdetect_enterprise_litespeedZdetect_open_litespeedr� Zis_ea4)rh rm ro ro rp r= � s z+CloudlinuxSummary._get_control_panel_apachec C s8 t �� } | dkr| S t jtt jtdtdti}|�| d�S )z�
Wrapper to convert internal values from cagefs_statistics_config.get_cagefs_status function to values
for statistics
:return:
NZEnabledZDisabledZUnknown) r$ Zget_cagefs_statusZCAGEFS_STATUS_NOT_INSTALLEDr� Z CAGEFS_STATUS_NOT_INITIALIZED�NOT_INITIALIZEDr� r� r� )�
cagefs_statusZcagefs_status_mapro ro rp �_cagefs_status_wrapper� s z(CloudlinuxSummary._cagefs_status_wrapperc C s~ | j �| jddd� | j d ttgkr2d| j d< n| j �tjddd� | j �ttjd�ddd � | j �ttjd
�dd d � dS )
zF
Fill dict with statistics by statistics about CageFS
r\ rA zCan't get CageFS statusNZcagefs_user_modezCan't get CageFS user modeTZcagefs_enabled_quantityz/Can't get quantity of users with enabled CageFSFZcagefs_disabled_quantityz0Can't get quantity of users with disabled CageFS) r� rr rB r� r@ r$ Zget_cagefs_user_moder Z get_quantity)rh ro ro rp �_fill_cagefs_statistics� s,
z)CloudlinuxSummary._fill_cagefs_statisticsc s^ y t dd� | j�� D ��}W n tjk
r2 dS X t| jj�� �� t� fdd�|� � D ��S )zJ
Get amount of end-users which belong to active resellers
c s s | ]\}}|d kr|V qdS )r Nro )r� r� �lvpro ro rp r� � s zLCloudlinuxSummary._get_amount_of_endusers_under_resellers.<locals>.<genexpr>Nc 3 s | ]}|� kV qd S )Nro )r� Zlvp_id)�enabled_lvp_idro rp r� s )
r r� �
lve_lvp_pairsr rI �set�name_mapZid_list�sum�elements)rh � lvp_countro )rE rp �'_get_amount_of_endusers_under_resellers� s z9CloudlinuxSummary._get_amount_of_endusers_under_resellersc C s@ y t dd� | j�� D ��}W n tjk
r2 dS X t|�� �S )z/
Get total amount of end-users
c s s | ]\}}|d kr|V qdS )r Nro )r� r� rD ro ro rp r� s zBCloudlinuxSummary._get_total_amount_of_endusers.<locals>.<genexpr>N)r r� rF r rI rI r )rh rK ro ro rp � _get_total_amount_of_endusers s
z/CloudlinuxSummary._get_total_amount_of_endusersc C s( y t t�� �S tjk
r" Y nX dS )z)
Get amount of resellers
N)r: r � resellersrI ro ro ro rp �_get_amount_of_resellers s z*CloudlinuxSummary._get_amount_of_resellersc s� � j �tjjddd� � j �tjjddd� � j �tjjddd� � j �� fdd �dd
d � � j �� jdd d
� � j �� jddd� � j �� j ddd� � j �� fdd �ddd� � j d rԈ j �dd � ddd� n d� j d< d� j d< d� j d< dS )zP
Fill dict with statistics by varied statistics about resellers
r] Z reseller_limits_supported_kernelz8Can't detect status of support reseller limits by kernelZ'reseller_limits_supported_control_panelz?Can't detect status of support reseller limits by control panelZreseller_limits_enabledz1Can't detect status of support of reseller limitsc s � � � d S )Nr )�#get_users_and_resellers_with_faultsro )rh ro rp rz 6 r{ z>CloudlinuxSummary._fill_resellers_statistics.<locals>.<lambda>Zusers_with_faultsz6Can't get amount of users with faults for the past 24hZresellers_totalz#Can't get total amount of resellersZ(resellers_endusers_under_reseller_limitsz>Can't get amount of end-users which belong to active resellersZresellers_endusers_totalz#Can't get total amount of end-usersc s � � � d S )Nr- )rP ro )rh ro rp rz N r{ Zresellers_with_faultsz:Can't get amount of resellers with faults for the past 24hc S s t tt�� ��S )N)r: �listr Zlvp_listro ro ro rp rz U r{ Zresellers_activez$Can't get amount of active resellersN)
r� rr r ZlveZis_lve10Zis_panel_supportedZreseller_limit_supportedrO rL rM )rh ro )rh rp �_fill_resellers_statistics sb
z,CloudlinuxSummary._fill_resellers_statisticsc s | j �t| j� jt j� jt j�ddd� | j �t| j� jt j�ddd� | j �� fdd�dd d
� | j �� fd d�dd d
� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fdd�ddd� | j �t| j � jt j
�ddd� | j �t| j � jt j �ddd� dS ) zN
Fill dict with statistics by statistics about default limits
r] Zdefault_limit_speedz Can't get default speed limitr\ Z default_limit_cpu_origin_unitsz+Can't get cpu origin units of default limitc s � j t jS )N)�defaultsr* �ncpuro )�xml_cfg_providerro rp rz v r{ zCCloudlinuxSummary._fill_default_limits_statistics.<locals>.<lambda>Zdefault_limit_ncpuz Can't get default ncpu limitc s � j t jS )N)rS r* �ioro )rU ro rp rz | r{ Zdefault_limit_iozCan't get default io limitc s � j t jS )N)rS r* �nprocro )rU ro rp rz � r{ Zdefault_limit_nprocz Can't get default nproc limitc s � j t jS )N)rS r* �epro )rU ro rp rz � r{ Zdefault_limit_epzCan't get default ep limitc s � j t jS )N)rS r* �iopsro )rU ro rp rz � r{ Zdefault_limit_iopsz Can't get default iops limitZdefault_limit_vmem_mbz Can't get default vmem limitZdefault_limit_pmem_mbz Can't get default pmem limitN) r� rr r �_cpu_limit_to_percentsrS r* �cpurT �_get_cpu_limit_units�_mempages_to_mb�vmem�pmem)rh rU ro )rU rp � _fill_default_limits_statistics_ s`
z1CloudlinuxSummary._fill_default_limits_statisticsc sR | j �� fdd�ddd� | j �� fdd�ddd� | j �� fd d�dd
d � d S )
z�
Fill dict with statistics by other statistics about limits:
packages_total, users_total, amount users/packages with custom limits
c s
t � j�S )N)r: �packagesro )rU ro rp rz � r{ zACloudlinuxSummary._fill_other_limits_statistics.<locals>.<lambda>r] Zpackages_totalz"Can't get total amount of packagesc s t � �� �S )N)r: Z get_packages_with_custom_limitsro )rU ro rp rz � r{ Zpackages_with_custom_limitsz/Can't get amount of packages with custom limitsc s t � �� �S )N)r: Z get_users_with_custom_limitsro )rU ro rp rz � r{ Zusers_with_custom_limitsz,Can't get amount of users with custom limitsN)r� rr )rh rU ro )rU rp � _fill_other_limits_statistics� s
z/CloudlinuxSummary._fill_other_limits_statisticsc s� �x�t dd�D �]�}� j�t� j||�dd| ���dkr<P � jj�fdd�dd| d | d
d � � jj�fd d�d
d| d| d
d � � jj� �fdd�d
d| d| d
d � � jj� �fdd�dd| d| d
d � � jj�fdd�d
d| d| d
d � � jj�fdd�d
d| d| d
d � � jj�fd d�d
d | d | d
d � � jj�fd d�d
d | d!| d
d � � jj�fd"d�d
d#| d$| d
d � � jj� �fd%d�d
d&| d'| d
d � � jj� �fd(d�d
d)| d*| d
d � qW dS )+zV
Fill dict with statistics by statistics about top packages on server
r- r. r\ z!Can't get top %s package by usersNc s
� d j S )Nr- )r8 ro )�
top_resultro rp rz � r{ zACloudlinuxSummary._fill_top_packages_statistics.<locals>.<lambda>ztop_%s_package_namez1Can't get package name of top %s package by usersF)rl c s t � d �S )Nr )r] ro )rc ro rp rz � r{ r] ztop_%s_package_users_numz4Can't get amount of users in top %s package by usersc s$ � � �d jptj�d jp tj�S )Nr- )rZ �limitsr+ r[ rT ro )rh rc ro rp rz � s ztop_%s_package_limit_speedz0Can't get speed limit of top %s package by usersc s � � �d jptj�S )Nr- )r\ rd r+ r[ ro )rh rc ro rp rz � s z%top_%s_package_limit_cpu_origin_unitsz>Can't get cpu origin units of limit of top %s package by usersc s � d j p tjS )Nr- )rd r+ rT ro )rc ro rp rz � r{ ztop_%s_package_limit_ncpuz/Can't get ncpu limit of top %s package by usersc s � d j p tjS )Nr- )rd r+ rV ro )rc ro rp rz � r{ ztop_%s_package_limit_ioz-Can't get io limit of top %s package by usersc s � d j p tjS )Nr- )rd r+ rW ro )rc ro rp rz � r{ ztop_%s_package_limit_nprocz0Can't get nproc limit of top %s package by usersc s � d j p tjS )Nr- )rd r+ rX ro )rc ro rp rz r{ ztop_%s_package_limit_epz-Can't get ep limit of top %s package by usersc s � d j p tjS )Nr- )rd r+ rY ro )rc ro rp rz
r{ ztop_%s_package_limit_iopsz/Can't get iops limit of top %s package by usersc s � � �d jptj�S )Nr- )r] rd r+ r^ ro )rh rc ro rp rz r{ z top_%s_package_limit_vmem_mbz/Can't get vmem limit of top %s package by usersc s � � �d jptj�S )Nr- )r] rd r+ r_ ro )rh rc ro rp rz r{ z top_%s_package_limit_pmem_mbz/Can't get pmem limit of top %s package by users)r r� rq r �#_get_top_package_by_number_of_usersrr )rh rU r1 ro )rh rc rp � _fill_top_packages_statistics� s�
z/CloudlinuxSummary._fill_top_packages_statisticsc C s( t � }| �|� | �|� | �|� dS )zN
Fill dict with statistiscs by varied statistics about limits
N)r) r` rb rf )rh rU ro ro rp �_fill_limits_statistics$ s
z)CloudlinuxSummary._fill_limits_statisticsc sX | j �| jdd�� � dk �rJ| j j� fdd�ddddd � | j j� fd
d�dd d dd � | j j� fd
d�ddddd � | j j� fdd�ddddd � | j j� fdd�ddddd � | j j� fdd�ddddd � | j j� fdd�ddddd � | j j� fd d�d d d dd � | j j� fd d�d!d"d#dd � | j j� fd$d�dd%d&dd � n
t| j d< dS )'zI
Fill dict with statistics by statistics about mod_lsapi
r\ z Can't get raw mod_lsapi infoNc s � d d S )N�criur� ro ro )�raw_lsapi_inforo rp rz : r{ z:CloudlinuxSummary._fill_lsapi_statistics.<locals>.<lambda>Zlsapi_criu_service_statusz Can't get status of criu serviceF)rl c s � d d S )Nrh r2 ro ro )ri ro rp rz A r{ Zlsapi_criu_service_versionz!Can't get version of criu servicec s � d d S )N� lsapiConfZ
lsapi_criuro ro )ri ro rp rz H r{ Zlsapi_option_criuzCan't get state of criuc s � d d S )Nrj Zlsapi_with_connection_poolro ro )ri ro rp rz O r{ Z lsapi_option_connection_poolz,Can't get state of mod_lsapi connection poolc s � d S )NZ
libVersionro ro )ri ro rp rz V r{ Zlsapi_lib_versionz"Can't get version of mod_lsapi libc s � d S )NZ modStatusro ro )ri ro rp rz ] r{ Zlsapi_mod_statuszCan't get mod_lsapi statusc s � d S )NZ
modVersionro ro )ri ro rp rz d r{ Zlsapi_mod_versionzCan't get mod_lsapi versionc s � d S )NZ totalDomainro ro )ri ro rp rz k r{ r] Zlsapi_total_domain_countz5Can't get total amount of domains which use mod_lsapic s � d S )NZ
domainStatro ro )ri ro rp rz r r{ ra Zlsapi_domain_statz.Can't get statistics of domains with mod_lsapic s � d S )NZ controlPanelro ro )ri ro rp rz y r{ Zlsapi_apache_environmentz Can't get apache environment)r� rq �get_raw_lsapi_inforr r� )rh ro )ri rp �_fill_lsapi_statistics. s�
z(CloudlinuxSummary._fill_lsapi_statisticsc C s | j �| jddd� d S )Nrb Z
wmt_configz!Cant get status of wmt_api_config)r� rr �_get_wmt_api_config)rh ro ro rp � _fill_wmt_settings_statistics� s
z/CloudlinuxSummary._fill_wmt_settings_statisticsc
sp y| � � \�}�W nB tk
rT } z$t�dt|�� g g i �}�W d d }~X Y nX tt��t|� �}y| �|�\�� W n< tk
r� } z t�dt|�� i i �� W d d }~X Y nX |rЈ�dt |�i� | j
� �fdd�ddd� | j
� �fd d�d
d d � | j
� � fd
d�d
dd� | j
� �fdd�ddd� | j
� � fdd�ddd� | j
� �fdd�d
dd� d S )Nz+Error while getting WordPress statistics %sz3Error while getting WordPress plugins statistics %sZ cloudlinux_wpos_object_cachec s t � �S )N)r: ro )� all_wp_pathsro rp rz � r{ z7CloudlinuxSummary._fill_wp_statistics.<locals>.<lambda>r] Zwordpress_installed_totalz,Can't get total count of installed wordpressc s � S )Nro ro )�object_cache_pluginsro rp rz � r{ ra Z wordpress_object_cache_pluginsz&Can't get WP object cache plugins infoc s � S )Nro ro )�advanced_cache_pluginsro rp rz � r{ Z wordpress_advanced_cache_pluginsz(Can't get WP advanced cache plugins infoc s t � �� �S )N)rI r ro )rp ro rp rz � r{ Z$wordpress_object_cache_plugins_totalz,Can't get WP object cache total plugins infoc s t � �� �S )N)rI r ro )rq ro rp rz � r{ Z&wordpress_advanced_cache_plugins_totalz.Can't get WP advanced cache total plugins infoc s � S )Nro ro )�wp_php_versionsro rp rz � r{ Zwordpress_php_versionsz Can't get WP php version info) �_prepare_wp_statisticsre rf r� r\ rQ rG � _prepare_wp_plugins_statisticsr� r: r� rr )rh Z paths_with_wpos_object_cacher� Zwithout_wpos_object_cachero )rq ro rp rr rp �_fill_wp_statistics� sX "
z%CloudlinuxSummary._fill_wp_statisticsc C s | � d|�}| � d|�}||fS )zq
Prepares dict with plugin-counter pairs
e.g: {'w3-total-cache': 4, 'redis': 1, 'ls': 2}
z object-cachezadvanced-cache)�_get_plugin_info)rh �wp_pathsrp rq ro ro rp rt � s z0CloudlinuxSummary._prepare_wp_plugins_statistics)� plugin_typerw c sF t td � fdd�|D ���}tt�}x |D ]}|| d7 < q*W |S )Nc s g | ]}t |� ��qS ro )r! )r� r� )rx ro rp r& � s z6CloudlinuxSummary._get_plugin_info.<locals>.<listcomp>r- )rQ �filterr r] )rx rw rp Zobject_plugin_counter_pairsZpluginro )rx rp rv � s
z"CloudlinuxSummary._get_plugin_infoc
C s� g g }}t t�}t�� }x�|D ]�}yt�|�}W n tk
rH w Y nX t t�}x*|D ]"\}} tj� | �rX|| �
|� qXW xb|D ]Z} | � || |�}
|
d } |
d } |� | � |� |
d � | r�| dkr�|| t
| �7 < q�W q W |||fS )z�
Return Wordpress absolute paths, paths with install WPOS object cache module
and counter of php versions that are used by these Wordpresses.
rw � php_version�'wp_paths_with_enabled_wpos_object_cache� undefined)r r] r r; � userdomainsre rQ r� r� r� �append�_get_docroot_wp_infor r: )
rh ro Z$paths_with_enabled_wpos_object_cacheZ php_versionsZ panel_users�userZ domains_info�docroot_domains_map�domain�docroot� docroot_infoZdocroot_wp_pathsZdocroot_php_versionro ro rp rs � s,
z(CloudlinuxSummary._prepare_wp_statistics)r� r� r� rx c s� g g dd�}� �fdd�t � �D �}|s,|S � fdd�|D �|d< � fdd�t||�D �|d< x,�� D ] }| �|�}|dkrh||d < P qhW |S )
zM
Return data about docroot's WP paths and used php versions.
r| )rw r{ rz c s4 g | ],}t j�� |�� ks,t j�� |��kr|�qS ro )r� r� r� )r� Zwp_path)r� r� ro rp r&
s z:CloudlinuxSummary._get_docroot_wp_info.<locals>.<listcomp>c s g | ]}t � ��|��qS ro )r �joinpath)r� r� )r� ro rp r& s rw c s" g | ]}|d k rt � ��|��qS )N)r r� )r� r� )r� ro rp r& s r{ rz )r r" �_get_php_version_for_domain)rh r� r� r� r� rw r� rz ro )r� r� rp r � s
z&CloudlinuxSummary._get_docroot_wp_info)r� rx c
C s� t �� tjtjtjfkr dS | �� \}}}|�|i �}|�dd�}t �� tjkr\||kr\|S x:|�� D ].\}}||krf|dkrfd|� dd�� �} | S qfW t �� tjkr�d|� dd�� �}
|
S t �� tjkr�|�dd�} |d t
| � � � d
�} | p�dS |S ) z�
Return php version that is used for specified domain.
Determine whether of MultiPHP Manager or PHP Selector is used.
r| Zphp_version_idr? zalt-php�.r� zda-phpZ handler_typeN�-) r� r< r Z CPANEL_NAMEZDIRECTADMIN_NAMEZ
PLESK_NAME�_get_system_php_infor� r9 �replacer: r� )
rh r� Zsystem_default_php_version�domains_php_info�selector_php_infoZphp_infoZmulti_php_versionr2 �domainsZalt_php_versionZda_php_versionZhandlerrz ro ro rp r� ! s( z-CloudlinuxSummary._get_php_version_for_domain)�maxsizec C sX yt �� d } W n tk
r( d} Y nX t �� }t� }|dk rJ|d }ni }| ||fS )z�
Return info about:
- system_php_version (implemented only on cPanel)
- domains_php_info (php versions and handlers turned on in MultiPHP)
- selector_php_info (php version used by panel users via PHP Selector)
Zdefault_version_idN�domains_by_php_version)r Zget_system_php_infore Zget_domains_php_inforR )Zsystem_php_versionr� Zphp_selector_usager� ro ro rp r� H s
z&CloudlinuxSummary._get_system_php_infoc s0 | j �| jdd�� � dk �r"| j �� fdd�ddd� | j �� fd d�dd
d � | j �� fd d�dd
d� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fdd�ddd� | j �� fd d�dd d � | j �� fd d�dd d!� | j �� fd"d�d#d$d%� n
t| j d
< dS )&z/
Fill dict with SSA statistics
r\ zCan't get SSA statisticsNc s � d S )N�configro ro )� ssa_statsro rp rz m r{ z8CloudlinuxSummary._fill_ssa_statistics.<locals>.<lambda>rb Z
ssa_configzCan't get SSA configc s � d S )Nr2 ro ro )r� ro rp rz s r{ Z ssa_versionzCan't get version of SSAc s � d S )Nr� ro ro )r� ro rp rz y r{ Z
ssa_statuszCan't get state of SSAc s � d S )NZ agent_statusro ro )r� ro rp rz r{ Zssa_agent_statusz Can't get state of SSA agentc s � d d S )N� autotracingr� ro ro )r� ro rp rz � r{ Zautotracing_statusz"Can't get state of SSA Autotracingc s � d d S )Nr� Zdisabled_users_quantityro ro )r� ro rp rz � r{ r] Z#autotracing_disabled_users_quantityz1Can't get state of SSA Autotracing disabled usersc s � d d S )Nr� Z
rules_versionro ro )r� ro rp rz � r{ Zautotracing_rules_versionz'Can't get SSA Autotracing rules versionc s � d d S )Nr� Zurls_processedro ro )r� ro rp rz � r{ Zautotracing_urls_processedz.Can't get SSA Autotracing processed URLs countc s � d d S )Nr� Z
urls_selectedro ro )r� ro rp rz � r{ Zautotracing_urls_selectedz-Can't get SSA Autotracing selected URLs countc s � d d S )Nr� Zrejectsro ro )r� ro rp rz � r{ ra Zautotracing_rejectsz!Can't get SSA Autotracing rejects)r� rq �
get_ssa_statsrr r� )rh ro )r� rp �_fill_ssa_statistics` sp
z&CloudlinuxSummary._fill_ssa_statisticsc s@ | j �dd� dd�}|dk �r2| j �t| jd�ddd � | j jt| j|t�d
d d d
d� | j jt| j|t�d
ddd
d� | j �tddd� | j �t ddd� | j �t
d�j ddd� | j �t ddd� | j �t
dd�� � dkr�d| j d< d| j d< n<| j j� fd d�d dd d
d� | j j� fd d�d dd d
d� n
t| j d< dS )!zS
Fill dict with statistics by varied statistics about PHP selector
c S s t d�S )Nr� )rP ro ro ro rp rz � r{ zACloudlinuxSummary._fill_php_selector_statistics.<locals>.<lambda>r] z+Can't get statistics about PHP interpretersNr� r\ Zselector_php_statusz Can't get status of PHP selectorr_ Z selector_php_versions_installedz8Can't get list of versions of installed PHP interpretersF)rl Z selector_php_versions_enabledz6Can't get list of versions of enabled PHP interpretersZ selector_php_version_defaultz,Can't get default version of PHP interpreterZselector_php_version_nativez+Can't get native version of PHP interpreterZselector_php_enabled_uiz%Can't get state of UI of PHP selectorZselector_php_modez Can't get mode of PHP selectorz'Can't get summary usage of PHP selectorZ'selector_php_num_domains_by_interpreterZ%selector_php_num_users_by_interpreterc s dd� � d � � D �S )Nc S s i | ]\}}t |�|�qS ro )r: )r� �vr� ro ro rp �
<dictcomp>� s zUCloudlinuxSummary._fill_php_selector_statistics.<locals>.<lambda>.<locals>.<dictcomp>r� )r9 ro )�php_usage_summaryro rp rz � r{ ra zBCan't get amount of domains which use PHP selector per PHP versionc s dd� � d � � D �S )Nc S s i | ]\}}t |�|�qS ro )r: )r� r� r� ro ro rp r� � s zUCloudlinuxSummary._fill_php_selector_statistics.<locals>.<lambda>.<locals>.<dictcomp>Zusers_by_php_version)r9 ro )r� ro rp rz � r{ z@Can't get amount of users which use PHP selector per PHP version)r� rq rr r r� �"_get_list_versions_of_interperters� INSTALLEDr� rT rQ rV Zphp_selector_is_enabledrU rR r� )rh Zphp_interpretersro )r� rp � _fill_php_selector_statistics� s|
z/CloudlinuxSummary._fill_php_selector_statisticsc C s | dk s|dk rdS | | S )z�
Get average amount of applications per domain
:param total_apps: total amount of applications
:param amount_of_apps_per_domain: amount of applications per domain
r- Nro )�
total_appsZamount_of_apps_per_domainro ro rp � _get_average_apps_per_domain� s z.CloudlinuxSummary._get_average_apps_per_domainc C s | dk s|dk rdS | | S )z�
Get average amount of applications per user
:param total_apps: total amount of applications
:param amount_of_apps_per_user: amount of applications per user
r- Nro )r� Zamount_of_apps_per_userro ro rp �_get_average_apps_per_user s z,CloudlinuxSummary._get_average_apps_per_userc C s t dd� | D ��S )z
Get amount of running applications on server
:param apps: list of applications for something selector
c S s g | ]}|j r|j d kr|�qS )Zstarted)Z
app_status)r� �appro ro rp r& s z@CloudlinuxSummary._get_amount_of_runned_apps.<locals>.<listcomp>)r: )�appsro ro rp �_get_amount_of_runned_apps s z,CloudlinuxSummary._get_amount_of_runned_appsc C sB t � }x | D ]}||j d7 < q W |�d�p4dg}|d d S )z�
Get maximum amount of applications per domain
:param apps: list of applications for something selector
r- )r r r )r �doc_root� most_common)r� Zapps_per_domainr� � most_commonsro ro rp �_get_max_apps_per_domain" s
z*CloudlinuxSummary._get_max_apps_per_domainc C sB t � }x | D ]}||j d7 < q W |�d�p4dg}|d d S )z~
Get maximum amount of applications per user
:param apps: list of applications for something selector
r- )r r r )r r� r� )r� Z
apps_per_userr� r� ro ro rp �_get_max_apps_per_user3 s
z(CloudlinuxSummary._get_max_apps_per_userc C s* t � }x | D ]}||j d7 < q W |S )z�
Get Counter object which contains amount applications per version of interpreter
:param apps: list of applications for something selector
r- )r r2 )r� �apps_per_versionr� ro ro rp � _get_counter_apps_per_versionD s
z/CloudlinuxSummary._get_counter_apps_per_versionc C s& | � |�}|�d�pdg}|d d S )z�
Get maximum amount of applications per version of interpreter
:param apps: list of applications for something selector
r- )r r r )r� r� )rh r� r� r� ro ro rp �_get_max_apps_per_versionR s
z+CloudlinuxSummary._get_max_apps_per_versionc C s( t � }x| D ]}|�|j� q W t|�S )zz
Get amount of domains with applications
:param apps: list of applications for something selector
)rG �addr� r: )r� r� r� ro ro rp � _get_amount_of_domains_with_apps` s
z2CloudlinuxSummary._get_amount_of_domains_with_appsc C s( t � }x| D ]}|�|j� q W t|�S )zx
Get amount of users with applications
:param apps: list of applications for something selector
)rG r� r� r: )r� �usersr� ro ro rp � _get_amount_of_users_with_appsn s
z0CloudlinuxSummary._get_amount_of_users_with_appsc C s | � |�}t|�S )z�
Get amount of applications per each versoin of interpeters
:param apps: list of applications for something selector
)r� r� )rh r� r� ro ro rp �4_get_amount_of_apps_per_each_version_of_interpreters| s
zFCloudlinuxSummary._get_amount_of_apps_per_each_version_of_interpretersc s � fdd�t | �D �S )z�
Get list of versions of interpreters on server
:param interpreters_stats: dict with varied statistics about each version of interpeters
:param state: state of interpeters (installed, enabled)
c s g | ]\}}|� r|�qS ro ro )r� Zinterpreter_stats�stat)�statero rp r& � s zHCloudlinuxSummary._get_list_versions_of_interperters.<locals>.<listcomp>)r
)�interpreters_statsr� ro )r� rp r� � s z4CloudlinuxSummary._get_list_versions_of_interpertersc C s" t | �}|dk rt|�S t� S dS )zI
Get list of apllications on server for defined selector
N)rS rQ )r� Z iter_appsro ro rp �_get_list_of_applications� s z+CloudlinuxSummary._get_list_of_applicationsc s �x| j D �]
}| j�tt|�dd�|��}|dkrHt| jd| d < q
| j�t| j|�dd| d d�|�� | jjt| j |t
�dd| d d
�|�d d � | jjt| j |t �dd| d
d�|�d d � | j�t| j |�dd�|��� | jj� fdd�dd| d d�|�d d � | jd| d dk| jd| d < | jjt| j
� �dd| d d�|�d d � | j�tt|�dd�|��}|dk �r�|| jd| d < | j�t| j� �dd| d d�|�� | j�t| j� �dd| d d �|�� | j�t| j� �dd| d d �|�� | j�t| j� �dd| d d!�|�� | j�t| j| jd| d | jd| d �dd"�|��}|dk �r�|| jd| d# < | j�t| j| jd| d | jd| d �dd$�|��}|dk �r�|| jd| d% < | j�t| j� �d&d| d' d(�|�� q
W dS ))zc
Fill dict with statistics by varied statistics about ruby/nodejs/python selectors
r] z*Can't get statistics about {} interpretersNZ selector_Z_statusr\ z Can't get status of {} selectorr_ Z_versions_installedz7Can't get list of versions of installed {} interpretersF)rl Z_versions_enabledz5Can't get list of versions of enabled {} interpretersz!Can't get list of {} applicationsc s t � �S )N)r: ro )�interpreter_appsro rp rz � r{ z>CloudlinuxSummary._fill_selectors_statistics.<locals>.<lambda>Z_applications_amountz)Can't get total amount of {} applicationsr Z_usedZ_applications_runningz6Can't get amount of runned application for {} selectorz(Can't get default version of {} selectorZ_default_versionZ _max_applications_per_domainz8Can't get max applications per domain for {} interpreterZ_max_applications_per_userz6Can't get max applications per user for {} interpreterZ_num_users_with_appsz=Can't get amount of users with applications for {} interpeterZ_num_domains_with_appsz?Can't get amount of domains with applications for {} interpeterzFCan't get average amount of applications per domain for {} interpreterZ _average_applications_per_domainzDCan't get average amount of applications per user for {} interpreterZ _average_applications_per_userra Z _num_applications_by_interpreterzDCan't get amount of applications per each version of {} interpreters)� SELECTORSr� rq r rP r� r� rr r� r� r� r� r� r� rW r� r� r� r� r� r� r� )rh Zselectorr� Zdefault_version_of_selectorZaverage_apps_per_domainZaverage_apps_per_userro )r� rp �_fill_selectors_statistics� s�
$
z,CloudlinuxSummary._fill_selectors_statisticsc C s~ ddg} t | dd�\}}}|dkr*t|��t�|�}t� }|d |d< dd� |d D �}x tD ]}|�|t�|d
| < q^W |S ) zA
Get wizard status and list of installed modules
z/usr/sbin/cloudlinux-wizardr� T)r$ r �
wizard_statusc S s i | ]}|d |d �qS )r� r8 ro )r� ry ro ro rp r� ' s z<CloudlinuxSummary._get_wizard_statistics.<locals>.<dictcomp>r| �wizard_module_)r r r� r) r� rJ r� � NOT_SELECTED)r+ Zret_codeZstd_outZstd_errZ parsed_json�wizard_statisticsZparsed_modulesry ro ro rp �_get_wizard_statistics s
z(CloudlinuxSummary._get_wizard_statisticsc sl | j �| jdd��| j j�fdd�ddddd� x6tD ].� | j j� �fd d�dd
� d �� �dd� q6W d S )
zX
Fill dict with statistics by varied statistics about cloudlinux-wizard
r\ z,Can't get statistics about cloudlinux-wizardc s � d S )Nr� ro ro )r� ro rp rz : r{ z;CloudlinuxSummary._fill_wizard_statistics.<locals>.<lambda>r� z%Can't get status of cloudlinux-wizardF)rl c s �d� S )Nr� ro ro )ry r� ro rp rz B r{ r� z;Can't get statistics about module "{}" of cloudlinux-wizardN)r� rq r� rr rJ r� )rh ro )ry r� rp �_fill_wizard_statistics- s"
z)CloudlinuxSummary._fill_wizard_statisticsc C s0 t � } g }x | D ]}|t| | �� �7 }qW |S )zH
Returns list of implemented scripts in integration.ini
)rM rQ �keys)r� ZscriptsZsectionro ro rp �$_get_implemented_integration_scriptsI s
z6CloudlinuxSummary._get_implemented_integration_scriptsc C s2 dg d�}t j�t�s|S d|d< | �� |d< |S )zh
Checks integration script exists and if exists
get list of implemented scripts
F)�integration_scripts_used� integration_scripts_specifiedTr� r� )r� r� r� rL r� )rh rm ro ro rp �_get_integration_infoT s
z'CloudlinuxSummary._get_integration_infoc C s8 ddl } d}| �� }t|j�| }t|j�| }||fS )zG
Gets memory usage: total and used memory in megabytes
r Ni )�psutilZvirtual_memoryr^ �totalZused)r� Z bytes_in_mbZmemZ mem_totalZmem_usedro ro rp �_get_memory_used` s z"CloudlinuxSummary._get_memory_usedc C sN t �� } d}d}tdd�sFtj�|�rFt|��}|�� �� }W dQ R X | |fS )za
Gets kernel info release and module version (starting from 7h)
:return:
z/sys/module/kmodlve/versionNT)r ) �platform�releaser r� r� r� r� r� r� )�kernel_releaseZkmodlve_version_fileZkmodlve_versionr� ro ro rp �_get_kernel_infol s
z"CloudlinuxSummary._get_kernel_infoc C s
t t� �S )zp
Gets info about lve extensions usage
Calculates amount of packages with lve extensions
)r: r@ ro ro ro rp �#_get_lve_extensions_packages_amount{ s z5CloudlinuxSummary._get_lve_extensions_packages_amountc C s@ d} t j�| �sdS t| ��}|�� �� }W dQ R X tt|��S )zk
Reads /proc/sys/fs/datacycle/enable in order to check
datacycle enabled parameter
z /proc/sys/fs/datacycle/enableFN)r� r� r� r� r� r� r� r] )Zdatacycle_filer� r� ro ro rp �$_is_kernel_datacycle_enabled_in_file� s
z6CloudlinuxSummary._is_kernel_datacycle_enabled_in_filec C sF d\} }t j�| �sdS t| ��}|�� �� �d�}W dQ R X ||kS )zR
Checks if datacycle parameter was given
for current boot
)z
/proc/cmdlineZ datacycleF� N)r� r� r� r� r� r� �split)Z cmdline_filer� r� r� ro ro rp � _is_datacycle_param_was_passed� s
z0CloudlinuxSummary._is_datacycle_param_was_passedc C s0 t �� } d}x | D ]}|tt �|��7 }qW |S )z=
Returns general amount of domains on server
r )r r; r: r} )Z cpusers_listZ
domains_countr� ro ro rp �_get_total_domains_amount� s
z+CloudlinuxSummary._get_total_domains_amountc C sN t � } tt| �d���}tt| �d���}|r6|r6dS |r>dS |rFdS dS dS )zi
Returns is links traversal protection enabled on server
(symlinks or hardlinks)
z fs.protected_symlinks_createz fs.protected_hardlinks_create�allZ
symlinks_onlyZhardlinks_onlyZnoN)r# r� r] r� )r� Zsymlink_protection_enabledZhardlink_protection_enabledro ro rp �%_is_link_traversal_protection_enabled� s z7CloudlinuxSummary._is_link_traversal_protection_enabledc C s@ d} d}d}t j�d�s$t j�d�r(|S t j�d�r8|S | S d S )N�iso�cldeploy�qcow2z/etc/cl-convert-savedz/var/log/cldeployz /etc/cloudlinux-qcow2-install)r� r� r� )r� r� r� ro ro rp �_get_cl_installation_source� s z-CloudlinuxSummary._get_cl_installation_sourcec s� �� � � tdd�sv�jj�jddddd� �jjtddd dd� �jj�jd
d d dd� �jj� fd
d�d
dddd� �jjtd
dddd� �jjtddddd� �j��j d
dd� �j��j
d
dd� �jj� fdd�d
dddd� �� � ��jj�fd d�dd d dd� �jj�fd d�d d!d"dd� �� � ��jj�fd#d�d$d%d&dd� �jj�fd'd�d$d(d)dd� �jj�j
dd*d+dd� tdd��s��jj�fd,d�dd-d.dd� �jjtd
d/d0dd� �jjtd
d1d2dd� �jj�jd
d3d4dd� d S )5NT)r r] Z lve_extension_packages_amountz"Can't get lve extension usage infoF)rl Zcldiag_cron_check_enabledz&Can't get is cldiag cron check enabledr\ Z!link_traversal_protection_enabledz5Can't get link traversal protection enabled parameterc s � d S )Nr- ro ro )� kernel_inforo rp rz � r{ z;CloudlinuxSummary._fill_system_statistics.<locals>.<lambda>Zinstalled_kmod_lve_versionz)Can't get installed kmod-lve version infoZ
cpu_amountzCan't get cpu amountZtesting_repository_enabledz#Can't get testing repository statusZ base_distroz Can't detect platform for CLOSZsecureboot_statusz+Can't detect whether secure boot is enabledc s � d S )Nr ro ro )r� ro rp rz r{ r� z Can't get kernel release infoc s � d S )Nr� ro ro )�vendor_integration_inforo rp rz r{ r� z'Can't get integration_scripts_used infoc s � d S )Nr� ro ro )r� ro rp rz r{ r_ r� z-Can't get integrations scripts specified infoc s � d S )Nr ro ro )� memory_usagero rp rz $ r{ r^ Zmemory_total_mbzCan't get total memory infoc s � d S )Nr- ro ro )r� ro rp rz + r{ Zmemory_used_mbzCan't get used memory infoZ
domains_totalzCan't get domains amountc s � � � p� �� S )N)r� r� ro )rh ro rp rz ; s Z kernel_datacycle_usage_enabledz,Can't get kernel datacycle enabled parameterZ virt_typez!Can't get the virtualization typeZhostnamezCan't get the hostnameZ cloudlinux_installation_sourcez Can't get installation source)r� r r� rr r� rN r� r r r� r� r� r� r� r r
r� )rh ro )r� r� rh r� rp �_fill_system_statistics� s�
z)CloudlinuxSummary._fill_system_statisticsc sB � j �� fdd�ddd� � j �tddd� � j �tdd d
� d S ) Nc s
� j ptS )N)r� �UNKNOWN_RHN_IDro )rh ro rp rz ] r{ zKCloudlinuxSummary._fill_dict_with_rpm_packages_statistics.<locals>.<lambda>r\ r� zCan't get system IDZ
os_versionzCan't get version of OSrc ra z*Can't get info about client's rpm packages)r� rr r rF )rh ro )rh rp r4 [ s
z9CloudlinuxSummary._fill_dict_with_rpm_packages_statistics)�paramrx c C s t �| j�|��S )z�
Retrieve data from proc/mounts for param
:return: param_value - Optional[int],
if there is no value - None
)r& Z
str_to_intr� r� )rh r� ro ro rp �_get_proc_paramo s z!CloudlinuxSummary._get_proc_paramc C s� dg}t dd�s.|�dddddd d
d d d
g
� x8|D ]0}| jjt| j|�d|�dd�d�|�dd� q4W | jjtddddd� dS )z�
Filling stats about mounting
- mount params from parameters list
- separate hidepid getting, since it is more complicated than
other mounting params
zkernel.memcg_oom_disableT)r zfs.enforce_symlinksifownerzfs.symlinkown_gidz fs.protected_symlinks_allow_gidz fs.protected_hardlinks_allow_gidzfs.global_root_enablezfs.proc_can_see_other_uidzfs.proc_super_gidz fs.xfs.cap_res_quota_disablezubc.ubc_oom_disablezfs.process_symlinks_by_taskr] r� r� z
Can't get {0})rl ZhidepidzCan't get hidepid valueN) r r r� rr r r� r� r� rO )rh Z
parameters�pro ro rp � _fill_proc_params_statisticsw s4
z.CloudlinuxSummary._fill_proc_params_statisticsc sZ t dd�� | j�� fdd�ddd� | j�� fdd�dd d
� | j�tt� �dd d � d
S )zv
Fill statistics fields from jwt token.
- jwt token metrics (cl_plus existence and client_id)
F)Zcheck_expirationc s � d kr d S � � dd �S )N�cl_plus)r� ro )r� ro rp rz � s z=CloudlinuxSummary._fill_cln_jwt_information.<locals>.<lambda>r] r� z Can't get cl_plus informationc s � d kr d S � � dd �S )N� client_id)r� ro )r� ro rp rz � s r� zCan't get client_id valueZlicense_activez Can't get license status valueN)rC r� rr r rE )rh ro )r� rp �_fill_cln_jwt_information� s
z+CloudlinuxSummary._fill_cln_jwt_informationc s0 � j �� fdd�ddd� � j �tddd� d S )
za
Filling stats centralized management
- centralized management existence
c s t j�� j�S )N)r� r� r� �CL_PLUS_CM_DISABLED_PATHro )rh ro rp rz � r{ zKCloudlinuxSummary._fill_centralized_management_statistics.<locals>.<lambda>r] Z centralized_management_disabledz Can't check CM disabling statusr\ Z cl_plus_sender_service_statusz)Can't check cl plus sender service statusN)r� rr rD )rh ro )rh rp �'_fill_centralized_management_statistics� s
z9CloudlinuxSummary._fill_centralized_management_statisticsc
C s� t � } yFt� }x:|D ]2}x,|�� D ] \}}| �d|� �g ��|� q"W qW W n2 ttfk
r~ } zt�d|� W dd}~X Y nX | S )a�
Prepare list of dicts with CPU metrics
Method get_cpu_metrics returns data in following format:
[
{
"id": 0,
"model": "QEMU Virtual CPU version 2.5+"
},
{
"id": 0,
"model": "QEMU Virtual CPU version 2.5+"
}
]
This helper produces a dict, where each key - metric_name,
value - list of values for all CPUs
"cpu_model": [
"QEMU Virtual CPU version 2.5+",
"QEMU Virtual CPU version 2.5+"
],
"cpu_id": [
0,
0
]
Zcpu_z CPU metrics getting error: %sN) r� rG r9 �
setdefaultr~ r� rI rf r� )rm Z cpu_coresZcpu_core�metric�valueZexro ro rp �make_flat_cpu_metrics� s
$ z'CloudlinuxSummary.make_flat_cpu_metricsc C sl ddddg}| � � }x@|D ]8}|dkr*dnd}| j�t|j|d�||d|� d �� qW | j�td
d d � dS )
z�
Filling stats about hardware metrics, specifically:
CPU:
- cache
- frequency
- model
- id
RAM:
- ram
- swap
Zcpu_idZ cpu_cache_mbZ cpu_modelZcpu_frequency_mhzr` r_ Nz Can't parse z metric for all coresra ZmemoryzCan't parse memory metrics)r� r� rr r r� rH )rh Z exp_metricsZ cpu_metricsr� Z metric_typero ro rp �_fill_hardware_statistics� s
z+CloudlinuxSummary._fill_hardware_statisticsc C s6 | � � \| jd< | jd< | jd< tj�d�| jd< dS )am
Collect CageFS enter errors number:
1. "Act like CageFS is disabled (unable to create LVE).. %d"
2. "Act like CageFS is disabled (unable to enter into NameSpace).. %d"
3. "Act like CageFS is disabled (unable to acquire lock for user %s uid %d)"
4. File-marker /etc/cagefs/fail.on.error presense
:return None
Z(act_cagefs_disabled_unable_to_create_lveZ&act_cagefs_disabled_unable_to_enter_nsZ)act_cagefs_disabled_unable_to_acqure_lockz/etc/cagefs/fail.on.errorZ"act_cagefs_disabled_marker_presentN)�*_scan_log_for_act_cagefs_disabled_messagesr� r� r� r� )rh ro ro rp �$_fill_act_cagefs_disabled_statistics
s z6CloudlinuxSummary._fill_act_cagefs_disabled_statisticsc C s� y�| � � \}}|dkrv|�d�}ttddd|d��}t|�}ttddd|d��}t|�}ttddd|d��}t|�}n d}d}d}|||fS ttfk
r� dS X d S )
a�
Scan /var/log/messages for all needed "Act like CageFS is disabled ..." messages for yesterday
:return tuple of ints:
Number of "Act like CageFS is disabled (unable to create LVE).. " messages,
Number of "Act like CageFS is disabled (unable to enter into NameSpace).. " messages,
Number of "Act like CageFS is disabled (unable to acquire lock for user %s uid %d)" messages
r �
z2Act like CageFS is disabled (unable to create LVE)T)Z fixed_stringZmultiple_searchZdata_from_filez<Act like CageFS is disabled (unable to enter into NameSpace)z<Act like CageFS is disabled (unable to acquire lock for user)i���i���i���N)�_get_data_from_logr� rQ r r: r� r� )rh �
returncode�stdoutZ
lines_listZfound_lines_listZnum_unable_to_create_lveZnum_unable_to_enter_nsZnum_unable_to_acqure_lockro ro rp r� ,
s&
z<CloudlinuxSummary._scan_log_for_act_cagefs_disabled_messagesc C sf t � } | dkrd}n,tj�� tjdd� }|�d�}d|� d�}tj|ddd tjtj d
�}|j
|j fS ) z�
Scan /var/log/messages for all needed "Act like CageFS is disabled ..." messages for yesterday
:return: Tuple (ret code, std_out string)
)Zcl7Zcl7hZcl8za/usr/bin/journalctl --since yesterday --until today | /usr/bin/grep 'Act like CageFS is disabled'r- )Zdaysz%Y-%m-%dz /usr/share/cloudlinux/logs-at z< /var/log/messages | /bin/grep 'Act like CageFS is disabled'Tz /bin/bash)r� r� r� r� �stderr) r �datetimeZdateZtodayZ timedeltaZstrftimer r �PIPEZDEVNULLr� r� )Zos_typeZs_cmdZyesterday_dateZ date_to_scanr� ro ro rp r� L
s
z$CloudlinuxSummary._get_data_from_logc s� t dd�s � j�� jddd� t� jd< � j�ttjdd�d d
d � � j�� jdd d
� � j�� fdd�d dd� � j�� fdd�d dd� t d k r�t
d k r�d� t t
�� jd< n
d � jd< � � � d S )NT)r r] �old_way_of_integration_usedz*Can't detect old LVE integration mechanism�lve_utils_versionF)Z
verify_expr\ Z
cl_editionz Can't get CloudLinux editionZis_solo_meta_file_presentz$Can't detect solo meta file presencec s � j S )N)r� ro )rh ro rp rz z
r{ zACloudlinuxSummary._prepare_cl_normal_statistics.<locals>.<lambda>r� z$Can't get or generate security tokenc s
� j ptS )N)r� r� ro )rh ro rp rz �
r{ r� zCan't get system IDz{}-{}�lvemanager_version)
r r� rr r� �LVE_UTILS_PKG_VERSIONr r Zget_cl_edition�_is_solo_marker_exists�LVEMANAGER_PKG_VERSION�LVEMANAGER_PKG_RELEASEr� �_fill_dict_with_statistics)rh ro )rh rp r3 c
sD
z/CloudlinuxSummary._prepare_cl_normal_statisticsc s� d�d�d�d�d�t j���s$dS G dd� dt�� � �����fd d
�}�d �d �d i}d d
� t ���D �}|�� y x|D ]}|||� q~W W n< � k
r� Y n* tk
r� x|D ] }d||< q�W Y nX |� | j�< |� | j�< |� | j�< dS )z�
Filling stats lvemanager
- number of visits on X-Ray tab
- number of X-Ray installation
- number of LVEManager opens
z/var/log/cloudlinux/i�Q Zlve_number_of_opensZxray_number_of_installationZxray_number_of_visitsNc @ s e Zd ZdS )zECloudlinuxSummary._fill_lvemanager_statistics.<locals>.BreakOuterLoopN)rs rt ru ro ro ro rp �BreakOuterLoop�
s r� c
s t j �� �� }t�| d�}d}x�|D ]�}y�d�|�d�d d� �}t j �|d��� }|� | k rn|k r�n njd�|�� �d�dd � �}|dkr�| � d7 < q�|dkr�| � d7 < q�|d kr�| � d7 < nd
}W q& tt fk
r� w&Y q&X q&W |�r � �d S ) N�rFr� r� z%Y-%m-%d %H:%M:%S,%fzlvemanager-openedr- zxray-install-startedzxray-tab-clickedT)
r� Znowr� r� r� r� Zstrptimer� r� �AttributeError) Zstats�fileZcurrent_date_timestampr� Zbreak_outer_loop�lineZlog_timeZlog_date_timestampr� )r� �COLLECTING_PERIOD�
LOG_FOLDER�LVE_NUMBER_OF_OPENS�XRAY_NUMBER_OF_INSTALLS�XRAY_NUMBER_OF_VISITSro rp �fill_statistics�
s(
zFCloudlinuxSummary._fill_lvemanager_statistics.<locals>.fill_statisticsr c S s" g | ]}d |kr|� d�s|�qS )zlvemanager.logz.gz)�endswith)r� r8 ro ro rp r& �
s zACloudlinuxSummary._fill_lvemanager_statistics.<locals>.<listcomp>r )r� r� �isdirre �listdir�sortr� r� )rh r r� Z log_filesZlog_filer� ro )r� r� r� r� r� r rp �_fill_lvemanager_statistics�
s2
z-CloudlinuxSummary._fill_lvemanager_statisticsc s� � j �� fdd�ddd� � j �� fdd�ddd� t� j d < td k rbtd k rbd
�tt�� j d < n
d � j d < � j �� jd d
d� � �� d S )Nc s � j S )N)r� ro )rh ro rp rz �
r{ z7CloudlinuxSummary._prepare_statistics.<locals>.<lambda>r\ r� z$Can't get or generate security tokenc s
� j ptS )N)r� r� ro )rh ro rp rz �
r{ r� zCan't get system IDr� z{}-{}r� r] r� z*Can't detect old LVE integration mechanism)r� rr r� r� r� r� r� r� )rh ro )rh rp �_prepare_statistics�
s,
z%CloudlinuxSummary._prepare_statisticsc C s� t dd�sZ| �� | �� | �� | �� | �� | �� | �� | �� | � � | �
� | � � | � � | �
� | �� | �� | �� | �� | �� | �� | �� d S )NT)r )r r6 rC rR rg r� r� r� rl r� r� �_fill_wpos_statisticsr� r> r� r� r� rn r� r ru )rh ro ro rp r� �
s*
z,CloudlinuxSummary._fill_dict_with_statisticsc s� | j �| jdd�� t� t�rƈ �d�dkr�| j �� fdd�ddd � | j �� fd
d�dd d � | j �� fd d�dd
d � | j �� fdd�ddd � | j �� fdd�ddd � | j �� fdd�ddd � d S )Nr\ z!Can't get AccelerateWP statisticsrm r� c s � d d S )N�
enabled_sitesr� ro ro )�
wpos_statsro rp rz r{ z9CloudlinuxSummary._fill_wpos_statistics.<locals>.<lambda>r] Z sites_count_with_enabled_wposz*Can't get statistic of using AccelerateWP.c s � d d S )N�
allowed_usersr� ro ro )r
ro rp rz r{ Z users_count_with_allowed_wposc s � d d S )Nr � object_cachero ro )r
ro rp rz r{ Z%sites_count_with_enabled_object_cachec s � d d S )Nr r ro ro )r
ro rp rz & r{ Z%users_count_with_allowed_object_cachec s � d d S )Nr �site_optimizationro ro )r
ro rp rz , r{ Z*sites_count_with_enabled_site_optimizationc s � d d S )Nr r
ro ro )r
ro rp rz 2 r{ Z*users_count_with_allowed_site_optimization)r� rq �get_wpos_statsr� r� r� rr )rh ro )r
rp r sF
z'CloudlinuxSummary._fill_wpos_statisticsc
C sn dg| }yt |dd�\}}}W n. tk
rN } zt�d|� dS d}~X Y nX |dkr\|S t�d||� dS )z�
Run cloudlinux-statistics using subprocess and handle errors.
:type args: list[str]
:rtype: str or None
z /usr/sbin/cloudlinux-statisticsT)r$ z.Unable to run cloudlinux-statistics, error: %sNr z;cloudlinux-statistics failed with exit code: %i, output: %s)r r rf r( r� )r r+ r r, r� r� ro ro rp �_run_cloudlinux_statistics8 s
z,CloudlinuxSummary._run_cloudlinux_statisticsc
C s� | � ddddg�}|dkr dS y t�|�}|d }|d }W n8 tttfk
rt } zt�d t|�� dS d}~X Y nX t |�t |�fS )
zp
Get number of users and resellers with faults for the past 24h
:rtype: tuple[int, int]
z
--by-faultr} z--jsonz --period=1dN)NNrN r� zISomething really bad happened to cloudlinux-statistics, The reason is: %s)
r r� r) r� r� r� rf r( r\ r: )rh r, Z json_datarN r� r� ro ro rp rP L s
z5CloudlinuxSummary.get_users_and_resellers_with_faultsc C sV |dkr dS |� � }|�d�r"dS |�d�r0dS |�d�r>dS |�� rJdS d| S dS )zGet config cpu limit formatN�%�speedZmhzZghzZold_cpu_formatz unknown: %s)�lowerr �isdigit)�clsr[ Zunitro ro rp r\ a s
z&CloudlinuxSummary._get_cpu_limit_unitsc C s | dkr dS d| d S )z/Convert memory limit from mempages to megabytesNr. i ro )r� ro ro rp r] s s z!CloudlinuxSummary._mempages_to_mbc C s8 | dkr dS t j| |pdd�}|dkr*dS t|d d�S )z,Convert cpu and ncpu to percents of one coreNr )Zlncpug Y@r- )r Zconvert_to_kernel_format�round)r[ rT r ro ro rp rZ z s z(CloudlinuxSummary._cpu_limit_to_percentsc C s� | j dkr|ttdd� |j�� D ��td�d�}g }x6t|td�d�D ]"\}}tt|��}|�||f� qDW |j dd� || _ | j S )zC
Return list of tuples [users_in_package, package]
Nc s s | ]}|j jd k r|V qd S )N)�packager8 )r� r� ro ro rp r� � s z>CloudlinuxSummary.get_users_amount_per_plan.<locals>.<genexpr>r )�keyT)�reverse)
r� �sortedrQ r� r r r r: r~ r )rh rU Zusers_sorted_by_packager� r �groupZ num_usersro ro rp �get_users_amount_per_plan� s
z+CloudlinuxSummary.get_users_amount_per_planc C s, y| � |�|d S tk
r& d S X d S )Nr- )r �
IndexError)rh Z
number_of_toprU ro ro rp re � s z5CloudlinuxSummary._get_top_package_by_number_of_usersc C s" t |�\}}|s | �t|�� |S )z%
Parse CLI arguments
)r. r� r( )rh r r� r� ro ro rp r � s z CloudlinuxSummary._parse_argsr� r c C s8 | t � � |d�}|dk r ||d< t|d� t�|� dS )z?
Print data in default format for web and exit
)rm r� r� NZstatistic_enabledT)r� r' �sysr� )rm r� Z exit_coder7 r� ro ro rp r� � s
z(CloudlinuxSummary._print_result_and_exitr- c C s* | � dt�� i� t| d� t�|� dS )z�
Print error and exit
:param dict message: Dictionary with keys "result" as string and optional "context" as dict
r� TN)r� r� r' r r� )r� r� ro ro rp r� � s
z!CloudlinuxSummary._error_and_exitc C s6 t j�d�r2tjddgdtjtjd�} t�| j�S dS )zD
Return mod_lsapi info from switch_mod_lsapi script
z/usr/bin/switch_mod_lsapiz--statT)r� r� r� N) r� r� r� r r r� r� r) r� )r� ro ro rp rk � s
z$CloudlinuxSummary.get_raw_lsapi_infoc C sd d} t � r| d7 } tj�| �r`tj| dgdtjtjd�}t�|j ��
d�}dd� |� � D �}|S d S )
zd
Return wmt-api config
if is_solo_edition True, run wpt-api-solo is command
z-/usr/share/web-monitoring-tool/wmtbin/wmt-apiz-soloz --config-getT)r� r� r� r� c S s i | ]\}}t |�|�qS ro )r\ )r� �kr� ro ro rp r� � s z9CloudlinuxSummary._get_wmt_api_config.<locals>.<dictcomp>N) r r� r� r� r r r� r� r) r� r� r9 )Zcommandr� Zwmt_api_config_summaryro ro rp rm � s z%CloudlinuxSummary._get_wmt_api_configc C s t j�d�S )Nz /etc/cloudlinux-edition-solo)r� r� r� ro ro ro rp r� � s z(CloudlinuxSummary._is_solo_marker_existsc C s0 t j�d�r,tjddgddd�} t�| j�S dS )zL
Return statistics from cloudlinux-ssa-manager get-stat API
z /usr/sbin/cloudlinux-ssa-managerzget-statT)r� �capture_outputN)r� r� r� r r r� r) r� )r� ro ro rp r� � s
z CloudlinuxSummary.get_ssa_statsc C s: d} t j�| �r6tj| dgddd�}t�|j��d�S dS )zE
Return statistics from cl-smart-advice counters API
z/usr/sbin/cl-smart-adviceZcountersT)r� r r� N) r� r� r� r r r� r) r� r� )�utilr� ro ro rp �get_smart_advice_stats� s z(CloudlinuxSummary.get_smart_advice_statsc
C s~ t ddd�}tdd�sz| j�| jdd�}|dk rzyt |d |d d�S tk
rx } zt�d t|�|� |S d}~X Y nX |S )
z=
Construct dict with Smart Advice statistics
N)Zsmart_advice_totalZsmart_advice_appliedT)Zskip_marker_checkrb z!Can't get Smart Advice statisticsr� ZappliedzAUnexpected json response from server, field %s is missing in "%s") r� r r� rq r! r� rf rg r\ )rh rm Zsa_statsr� ro ro rp r� s&
z.CloudlinuxSummary._get_smart_advice_statisticsc C s4 d} t j�| �r0tj| dgddd�}t�|j�S dS )zE
Return statistics from cl-smart-advice counters API
z/usr/bin/clwpos-adminzget-statT)r� r N)r� r� r� r r r� r) r� )r r� ro ro rp r ) s z CloudlinuxSummary.get_wpos_stats)r� Nr N)r- )�rs rt ru r� r� � itertools�compressr Zis_panel_feature_supportedr ZPYTHON_SELECTORZ
RUBY_SELECTORZNODEJS_SELECTORr� r� r� r� r� � staticmethodr� �propertyr� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r r
r/ r4 r� r r r r] r6 r! r� r r" r r� r5 r6 r> r= rB rC rL rM rO rR r` rb rf rg rl rn ru rt r\ r3 rv r5 r rs r2 r r� r r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r4 r0 r� r� r� r� r� r� r� r� r� r3 r r r� r r rP � classmethodr\ r] rZ r re r r� r� rk rm r� r� r! r� r ro ro ro rp r� � s
7 z + (
- " DAj
T: (%
#'LR
v
- &! ,F -
r� c C s t � } | j�� | S )N)r, rH Z
link_xml_node)r� ro ro rp r� 8 s
r� )�Z
__future__r r r" r �builtinsr r� r� r r' r� r� r� r r� r� r� � functoolsr r � collectionsr r r Z future.utilsr
�operatorr Zmultiprocessingr Zsocketr
Zpathlibr r� Z cldetectlibr� r Zclcommonr Zclcommon.constr Z clcommon.libr Zclcommon.lib.cleditionr r r r r Zclcommon.utilsr r r r r r r r r Zclcommon.clwpos_libr r! r" Zclcommon.sysctlr# Zclconfigr$ r% r& Z cli_utilsr' r( Zcllimitslib_v2r) r* Z
clveconfigr+ Zlveapir, Zcl_summary_arg_parser. �typingr/ r0 r1 r2 r3 r4 r5 r6 Zclsummary.cl_summary_utilsr7 r8 r9 r: r; r< r= r> r? r@ rA rB rC rD rE Z!clsummary.rpm_packages_statisticsrF Z clsummary.hardware_statisticsrG rH rI Zclwizard.modulesrJ Z lve_utilsrK r� Zvendors_api.configrL rM Z cldiaglibrN Zcl_proc_hidepidrO Zclselect.clselectstatisticsrP rQ rR rS rT rU Zclselector.selectorlibrV Zclselect.clselectctlrW Z
lvemanagerr� rX r� � ImportErrorr� r rf r� r� r� r@ r� r� r� ZERRORr� r[ r� r� �objectr� r� ro ro ro rp �<module> s� , (D
R b
Mini Shell