%PDF-
%PDF-
Mini Shell
Mini Shell
�
N�si� � �� � d Z ddlZddlmZ ddlmZmZmZmZ ddl Z ddl m
Z
m Z m Z m
Z
ddlmZmZ e G d� d� � � � Zd ee fd
�Zd � Zd eeee f fd �ZdS )
zF
Represents storage where ssa data is collected, stored and extracted
� N)� dataclass)�List�Iterator�Tuple�Dict)�func�cast�distinct�text)�
session_scope�
RequestResultc �D � e Zd ZU eed< ee ed< eed< eed< dS )�
DomainData� domain_name�domain_total_reqs�is_a_wordpress_domain� urls_numberN)�__name__�
__module__� __qualname__�str�__annotations__r �int�bool� � �J/opt/cloudlinux/venv/lib64/python3.11/site-packages/ssa/modules/storage.pyr r sF � � � � � � ������C�y� � � � � � � ������r r �returnc # �z K � t | � � 5 }|� t j t j dt j � � t j t j � � t j t j
� � t j t t j
� � � � � � � t j t j dt j � � � � � t j t j dt j � � � � }t! j |d� �� � }|D ]^\ }}t% |� � }d}dgdz }|D ]*\ } }
} } }
| |t' |
� � < t ||
� � }�+t) ||| |�� � V � �_ ddd� � dS # 1 swxY w Y dS )z7
Iterates data from database domain-by-domain.
�%Hc � � | d S �Nr r ��items r �<lambda>z#iter_domains_data.<locals>.<lambda>/ s
� �VZ�[\�V]� r )�keyr � )r r r r N)r �queryr
�domainr �strftime�
created_at�Count�id�max� wordpress�countr
�path�group_by�order_by� itertools�groupby�tupler r )�engine�db�results_by_hour�results_by_hour_groupedr �group�domain_results_by_hourr �requests_number_by_hour�_�hour� requests_num� is_wordpress�urlss r �iter_domains_datarC s� � � � �
�v� � � !�"� �(�(� � � �M�$�
� 8� 9� 9� �J�}�'� (� (� �H�]�,� -� -� �J�x�
� 2�3�3� 4� 4�
�
� �(� � � �M�$�
� 8� 9� 9�
�
� �(� � � �M�$�
� 8� 9� 9�
�
� � #,�"3�O�I]�I]�"^�"^�"^� �"9� � � �K��%*�5�\�\� "��K� ()�c�B�h� #�=S�
5�
5�9��4� �|�T�5A�'��D� � �2� !�+�t� 4� 4� � � � '�"9�&2� '� � � �
�
�
�
� �#!� !� !� !� !� !� !� !� !� !� !� !���� !� !� !� !� !� !s �FF0�0F4�7F4c
# � K � t | � � 5 }d� |D � � }|� t j t j dt j � � t j t t j t j � � � � � d� � t j
t j � � � d� � t j t t j t j � � � � � d� � � � � t j |k � � � t% t j � |� � j � ddi�� � j � � � � � t j t j dt j � � � � � t j t j dt j � � � � }d }d
gd z d
gd z d
gd z } }}|D ]q\ }
} } }
}|r/||
k r)|t3 |||| � � � fV � d
gd z d
gd z d
gd z } }}| |t5 | � � < |
|t5 | � � < || t5 | � � < |
}�r|
t3 |
||| � � � fV � d d d � � d S # 1 swxY w Y d S )
z6
Iterates urls data from database url-by-url.
c �: � g | ]}|� d d� � ��S )�:z\:)�replace)�.0r1 s r �
<listcomp>z"iter_urls_data.<locals>.<listcomp>H s& � � L� L� L�$�T�\�\�#�u� 5� 5� L� L� Lr r �url_throttled_reqs�url_total_reqs�
url_slow_reqs�
literal_bindsT)�compile_kwargsNr r' )r1 rJ rK rL )r r( r
r1 r r* r+ �Sumr �hitting_limits�
sqlalchemy�Integer�labelr, r- �is_slow_request�filterr) r �in_�
expression�compile�stringr2 r3 �dictr )r7 r � all_pathsr8 �all_paths_escaped� urls_data�
previous_pathrJ rK rL r1 r? �url_throttled_req�
url_total_req� url_slow_reqs r �iter_urls_datarb C s � � � �
�v� � � 2
�"� L� L�)� L� L� L���H�H� � � �M�$�
� 8� 9� 9� �H�T� �,�j�.@�� �
�
��u�)�*�*� �J� � �
�
��e�$�%�%� �H�T� �-�z�/A�C� C�
�
��e�O�$�$�
�
� �&� � �K� /�
�
�
�&� � �#�'�'�(9�:�:�E�M�M�^m�os�]t�M�u�u�|� }� }�
�
�
�(� � ��
�d�M�4L� M� M�
�
�
�(� � ��
�d�M�4L� M� M�
�
�% �, �
�
�C�"�H�q�c�B�h���b�� -:�N�� KT� !� !� F�D�$�)�=�,� �
1� �$�!6�!6�#�T�&�'9�#1�"/� &� &� &� � � � � �C�"�H�q�c�B�h���b�� 5B�N�"� ->� �s�4�y�y� )�(5�N�3�t�9�9� %�'3�M�#�d�)�)� $� �M�M��D�� 1�)�'�
�
�
�
�
�
�
�[2
� 2
� 2
� 2
� 2
� 2
� 2
� 2
� 2
� 2
� 2
� 2
���� 2
� 2
� 2
� 2
� 2
� 2
s �J J<�<K �K c # � K � t | � � 5 }|� t j t j � � � t j |k � � � t j � � }t j |d� � � }|D ]\ }}|d� |D � � fV � � ddd� � dS # 1 swxY w Y dS )zA
Get information about durations of requests url-by-url.
c � � | d S r" r r# s r r% z#get_url_durations.<locals>.<lambda>� s
� �D��G� r c � � g | ]\ }}|��S r r )rH r> �durations r rI z%get_url_durations.<locals>.<listcomp>� s � �:�:�:�[�Q���:�:�:r N)
r r( r
r1 rf rU r) r3 r4 r5 )r7 r r8 r] �durations_by_pathr& r; s r �get_url_durationsrh | s# � � � �
�v� � � ;�"��H�H� � � � "�
�
� �&� � �K� /�
�
�
�(� � �
�
� � &� -� �+�+� -� -��+� ;� ;�J�C���:�:�E�:�:�:�:� :� :� :� :� ;�;� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;���� ;� ;� ;� ;� ;� ;s �BB7�7B;�>B;)�__doc__r4 � dataclassesr �typingr r r r rQ r r r
r �ssa.dbr r
r rC rb r r rh r r r �<module>rm s2 ��� � � � � � !� !� !� !� !� !� .� .� .� .� .� .� .� .� .� .� .� .� � � � � 1� 1� 1� 1� 1� 1� 1� 1� 1� 1� 1� 1� /� /� /� /� /� /� /� /� �� � � � � � � ���%��*�!5� %� %� %� %�N6
� 6
� 6
�r;�d�3��c�
�?�.C� ;� ;� ;� ;� ;� ;r
Zerion Mini Shell 1.0