Current File : //usr/lib64/python2.7/site-packages/tornado/httpclient.pyc
�
��L]c @� sO d Z d d l m Z m Z m Z m Z d d l Z d d l Z d d l Z d d l m
Z
d d l m Z m
Z
d d l m Z m Z d d l m Z d d l m Z d e f d
� � YZ d e f d � � YZ d
e f d � � YZ d e f d � � YZ d e f d � � YZ d e f d � � YZ d � Z e d k rKe � n d S( s� Blocking and non-blocking HTTP client interfaces.
This module defines a common interface shared by two implementations,
``simple_httpclient`` and ``curl_httpclient``. Applications may either
instantiate their chosen implementation class directly or use the
`AsyncHTTPClient` class from this module, which selects an implementation
that can be overridden with the `AsyncHTTPClient.configure` method.
The default implementation is ``simple_httpclient``, and this is expected
to be suitable for most users' needs. However, some applications may wish
to switch to ``curl_httpclient`` for reasons such as the following:
* ``curl_httpclient`` has some features not found in ``simple_httpclient``,
including support for HTTP proxies and the ability to use a specified
network interface.
* ``curl_httpclient`` is more likely to be compatible with sites that are
not-quite-compliant with the HTTP spec, or sites that use little-exercised
features of HTTP.
* ``curl_httpclient`` is faster.
* ``curl_httpclient`` was the default prior to Tornado 2.0.
Note that if you are using ``curl_httpclient``, it is highly
recommended that you use a recent version of ``libcurl`` and
``pycurl``. Currently the minimum supported version of libcurl is
7.21.1, and the minimum version of pycurl is 7.18.2. It is highly
recommended that your ``libcurl`` installation is built with
asynchronous DNS resolver (threaded or c-ares), otherwise you may
encounter various problems with request timeouts (for more
information, see
http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTCONNECTTIMEOUTMS
and comments in curl_httpclient.py).
To select ``curl_httpclient``, call `AsyncHTTPClient.configure` at startup::
AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
i ( t absolute_importt divisiont print_functiont with_statementN( t TracebackFuture( t utf8t
native_str( t httputilt
stack_context( t IOLoop( t Configurablet
HTTPClientc B� s5 e Z d Z d d � Z d � Z d � Z d � Z RS( s� A blocking HTTP client.
This interface is provided for convenience and testing; most applications
that are running an IOLoop will want to use `AsyncHTTPClient` instead.
Typical usage looks like this::
http_client = httpclient.HTTPClient()
try:
response = http_client.fetch("http://www.google.com/")
print response.body
except httpclient.HTTPError as e:
# HTTPError is raised for non-200 responses; the response
# can be found in e.response.
print("Error: " + str(e))
except Exception as e:
# Other errors are possible, such as IOError.
print("Error: " + str(e))
http_client.close()
c K� sI t d t � | _ | d k r' t } n | | j | � | _ t | _ d S( Nt make_current( R t Falset _io_loopt Nonet AsyncHTTPClientt
_async_clientt _closed( t selft async_client_classt kwargs( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt __init__J s
c C� s | j � d S( N( t close( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt __del__Q s c C� s3 | j s/ | j j � | j j � t | _ n d S( s2 Closes the HTTPClient, freeing any resources used.N( R R R R t True( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR T s
c K� s+ | j j t j | j j | | � � } | S( s� Executes a request, returning an `HTTPResponse`.
The request may be either a string URL or an `HTTPRequest` object.
If it is a string, we construct an `HTTPRequest` using any additional
kwargs: ``HTTPRequest(request, **kwargs)``
If an error occurs during the fetch, we raise an `HTTPError` unless
the ``raise_error`` keyword argument is set to False.
( R t run_synct functoolst partialR t fetch( R t requestR t response( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR [ s
N( t __name__t
__module__t __doc__R R R R R ( ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR 6 s
R c B� s� e Z d Z e d � � Z e d � � Z e d � � Z d
e d � Z d
d � Z
d � Z d
e d � Z
d � Z e d � � Z RS( s� An non-blocking HTTP client.
Example usage::
def handle_request(response):
if response.error:
print "Error:", response.error
else:
print response.body
http_client = AsyncHTTPClient()
http_client.fetch("http://www.google.com/", handle_request)
The constructor for this class is magic in several respects: It
actually creates an instance of an implementation-specific
subclass, and instances are reused as a kind of pseudo-singleton
(one per `.IOLoop`). The keyword argument ``force_instance=True``
can be used to suppress this singleton behavior. Unless
``force_instance=True`` is used, no arguments other than
``io_loop`` should be passed to the `AsyncHTTPClient` constructor.
The implementation subclass as well as arguments to its
constructor can be set with the static method `configure()`
All `AsyncHTTPClient` implementations support a ``defaults``
keyword argument, which can be used to set default values for
`HTTPRequest` attributes. For example::
AsyncHTTPClient.configure(
None, defaults=dict(user_agent="MyUserAgent"))
# or with force_instance:
client = AsyncHTTPClient(force_instance=True,
defaults=dict(user_agent="MyUserAgent"))
.. versionchanged:: 4.1
The ``io_loop`` argument is deprecated.
c C� s t S( N( R ( t cls( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt configurable_base� s c C� s d d l m } | S( Ni ( t SimpleAsyncHTTPClient( t tornado.simple_httpclientR% ( R# R% ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt configurable_default� s c C� sB d | j } t | | � s5 t | | t j � � n t | | � S( Nt _async_client_dict_( R t hasattrt setattrt weakreft WeakKeyDictionaryt getattr( R# t attr_name( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt _async_clients� s
c K� s� | p t j � } | r! d } n | j � } | d k rM | | k rM | | St t | � j | d | | �} | | _ | d k r� | | | j <n | S( Nt io_loop( R t currentR R/ t superR t __new__t _instance_cacheR0 ( R# R0 t force_instanceR t instance_cachet instance( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR3 � s c C� sG | | _ t t j � | _ | d k r: | j j | � n t | _ d S( N( R0 t dictt HTTPRequestt _DEFAULTSt defaultsR t updateR
R ( R R0 R; ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt
initialize� s
c C� sc | j r
d St | _ | j d k r_ | j j | j � | k rO t d � � n | j | j =n d S( s� Destroys this HTTP client, freeing any file descriptors used.
This method is **not needed in normal use** due to the way
that `AsyncHTTPClient` objects are transparently reused.
``close()`` is generally only necessary when either the
`.IOLoop` is also being closed, or the ``force_instance=True``
argument was used when creating the `AsyncHTTPClient`.
No other methods may be called on the `AsyncHTTPClient` after
``close()``.
Ns" inconsistent AsyncHTTPClient cache( R R R4 R t getR0 t RuntimeError( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR � s
c � s� � j r t d � � n t � t � s< t d � | � � n t j � j � � _ t � � j � � t � � � d k r� t j � � � � � � f d � } � j
| � n � � f d � } � j � | � � S( so Executes a request, asynchronously returning an `HTTPResponse`.
The request may be either a string URL or an `HTTPRequest` object.
If it is a string, we construct an `HTTPRequest` using any additional
kwargs: ``HTTPRequest(request, **kwargs)``
This method returns a `.Future` whose result is an
`HTTPResponse`. By default, the ``Future`` will raise an `HTTPError`
if the request returned a non-200 response code. Instead, if
``raise_error`` is set to False, the response will always be
returned regardless of the response code.
If a ``callback`` is given, it will be invoked with the `HTTPResponse`.
In the callback interface, `HTTPError` is not automatically raised.
Instead, you must check the response's ``error`` attribute or
call its `~HTTPResponse.rethrow` method.
s( fetch() called on closed AsyncHTTPClientt urlc � s� | j � } t | t � r6 | j d k r6 | j } nC | d k rm t � d d | d t j � � j �} n | j � } � j j
� | � d S( NiW t errort request_time( t exceptiont
isinstancet HTTPErrorR R t HTTPResponset timet
start_timet resultR0 t add_callback( t futuret excR ( t callbackR R ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt
handle_future� s c � s3 � r" | j r" � j | j � n
� j | � d S( N( RA t
set_exceptiont
set_result( R ( RK t raise_error( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt handle_response� s N( R R? RD R9 R t HTTPHeaderst headerst
_RequestProxyR; R R R t wrapt add_done_callbackt
fetch_impl( R R RM RQ R RN RR ( ( RM RK RQ R R s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR � s c C� s
t � � d S( N( t NotImplementedError( R R RM ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRX s c K� s t t | � j | | � d S( sG Configures the `AsyncHTTPClient` subclass to use.
``AsyncHTTPClient()`` actually creates an instance of a subclass.
This method may be called with either a class object or the
fully-qualified name of such a class (or ``None`` to use the default,
``SimpleAsyncHTTPClient``)
If additional keyword arguments are given, they will be passed
to the constructor of each subclass instance created. The
keyword argument ``max_clients`` determines the maximum number
of simultaneous `~AsyncHTTPClient.fetch()` operations that can
execute in parallel on each `.IOLoop`. Additional arguments
may be supported depending on the implementation class in use.
Example::
AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
N( R2 R t configure( R# t implR ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRZ s N( R R! R" t classmethodR$ R' R/ R R
R3 R= R R R RX RZ ( ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR j s $ 4 R9 c B� ss e Z d Z e d d d d d e d d d e d d d
e d e � Z d d d d d d d d d d d d d d d d d d d d d d d d d d d d e d d d
� Z e d � � Z
e
j d � � Z
e d � � Z e j d � � Z e d � � Z
e
j d � � Z
e d � � Z e j d � � Z e d � � Z e j d � � Z e d � � Z e j d � � Z RS( s HTTP client request object.t connect_timeoutg 4@t request_timeoutt follow_redirectst
max_redirectsi t decompress_responset proxy_passwordt t allow_nonstandard_methodst
validate_certt GETc! ! C� sX | | _ |
r( t j |
� | j d <n | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _
| | _ | | _ | | _
| | _ | | _ | | _ | | _ |
| _ | d k r� | | _ n | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ t! j! � | _" d S( sC All parameters except ``url`` are optional.
:arg string url: URL to fetch
:arg string method: HTTP method, e.g. "GET" or "POST"
:arg headers: Additional HTTP headers to pass on the request
:type headers: `~tornado.httputil.HTTPHeaders` or `dict`
:arg body: HTTP request body as a string (byte or unicode; if unicode
the utf-8 encoding will be used)
:arg body_producer: Callable used for lazy/asynchronous request bodies.
It is called with one argument, a ``write`` function, and should
return a `.Future`. It should call the write function with new
data as it becomes available. The write function returns a
`.Future` which can be used for flow control.
Only one of ``body`` and ``body_producer`` may
be specified. ``body_producer`` is not supported on
``curl_httpclient``. When using ``body_producer`` it is recommended
to pass a ``Content-Length`` in the headers as otherwise chunked
encoding will be used, and many servers do not support chunked
encoding on requests. New in Tornado 4.0
:arg string auth_username: Username for HTTP authentication
:arg string auth_password: Password for HTTP authentication
:arg string auth_mode: Authentication mode; default is "basic".
Allowed values are implementation-defined; ``curl_httpclient``
supports "basic" and "digest"; ``simple_httpclient`` only supports
"basic"
:arg float connect_timeout: Timeout for initial connection in seconds
:arg float request_timeout: Timeout for entire request in seconds
:arg if_modified_since: Timestamp for ``If-Modified-Since`` header
:type if_modified_since: `datetime` or `float`
:arg bool follow_redirects: Should redirects be followed automatically
or return the 3xx response?
:arg int max_redirects: Limit for ``follow_redirects``
:arg string user_agent: String to send as ``User-Agent`` header
:arg bool decompress_response: Request a compressed response from
the server and decompress it after downloading. Default is True.
New in Tornado 4.0.
:arg bool use_gzip: Deprecated alias for ``decompress_response``
since Tornado 4.0.
:arg string network_interface: Network interface to use for request.
``curl_httpclient`` only; see note below.
:arg callable streaming_callback: If set, ``streaming_callback`` will
be run with each chunk of data as it is received, and
``HTTPResponse.body`` and ``HTTPResponse.buffer`` will be empty in
the final response.
:arg callable header_callback: If set, ``header_callback`` will
be run with each header line as it is received (including the
first line, e.g. ``HTTP/1.0 200 OK\r\n``, and a final line
containing only ``\r\n``. All lines include the trailing newline
characters). ``HTTPResponse.headers`` will be empty in the final
response. This is most useful in conjunction with
``streaming_callback``, because it's the only way to get access to
header data while the request is in progress.
:arg callable prepare_curl_callback: If set, will be called with
a ``pycurl.Curl`` object to allow the application to make additional
``setopt`` calls.
:arg string proxy_host: HTTP proxy hostname. To use proxies,
``proxy_host`` and ``proxy_port`` must be set; ``proxy_username`` and
``proxy_pass`` are optional. Proxies are currently only supported
with ``curl_httpclient``.
:arg int proxy_port: HTTP proxy port
:arg string proxy_username: HTTP proxy username
:arg string proxy_password: HTTP proxy password
:arg bool allow_nonstandard_methods: Allow unknown values for ``method``
argument?
:arg bool validate_cert: For HTTPS requests, validate the server's
certificate?
:arg string ca_certs: filename of CA certificates in PEM format,
or None to use defaults. See note below when used with
``curl_httpclient``.
:arg string client_key: Filename for client SSL key, if any. See
note below when used with ``curl_httpclient``.
:arg string client_cert: Filename for client SSL certificate, if any.
See note below when used with ``curl_httpclient``.
:arg ssl.SSLContext ssl_options: `ssl.SSLContext` object for use in
``simple_httpclient`` (unsupported by ``curl_httpclient``).
Overrides ``validate_cert``, ``ca_certs``, ``client_key``,
and ``client_cert``.
:arg bool allow_ipv6: Use IPv6 when available? Default is true.
:arg bool expect_100_continue: If true, send the
``Expect: 100-continue`` header and wait for a continue response
before sending the request body. Only supported with
simple_httpclient.
.. note::
When using ``curl_httpclient`` certain options may be
inherited by subsequent fetches because ``pycurl`` does
not allow them to be cleanly reset. This applies to the
``ca_certs``, ``client_key``, ``client_cert``, and
``network_interface`` arguments. If you use these
options, you should pass them on every request (you don't
have to always use the same values, but it's not possible
to mix requests that specify these options with ones that
use the defaults).
.. versionadded:: 3.1
The ``auth_mode`` argument.
.. versionadded:: 4.0
The ``body_producer`` and ``expect_100_continue`` arguments.
.. versionadded:: 4.2
The ``ssl_options`` argument.
s If-Modified-SinceN(# RT R t format_timestampt
proxy_hostt
proxy_portt proxy_usernameRb R@ t methodt bodyt
body_producert
auth_usernamet
auth_passwordt auth_modeR] R^ R_ R` t
user_agentR Ra t network_interfacet streaming_callbackt header_callbackt prepare_curl_callbackRd Re t ca_certst
allow_ipv6t
client_keyt client_certt ssl_optionst expect_100_continueRG RH (! R R@ Rk RT Rl Rn Ro Rp R] R^ t if_modified_sinceR_ R` Rq t use_gzipRr Rs Rt Ru Rh Ri Rj Rb Rd Re Rv Rw Rx Ry Rm R{ Ra Rz ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR , sH x c C� s | j S( N( t _headers( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRT � s c C� s+ | d k r t j � | _ n | | _ d S( N( R R RS R~ ( R t value( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRT � s c C� s | j S( N( t _body( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRl � s c C� s t | � | _ d S( N( R R� ( R R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRl � s c C� s | j S( N( t _body_producer( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRm � s c C� s t j | � | _ d S( N( R RV R� ( R R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRm � s c C� s | j S( N( t _streaming_callback( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRs � s c C� s t j | � | _ d S( N( R RV R� ( R R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRs � s c C� s | j S( N( t _header_callback( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRt � s c C� s t j | � | _ d S( N( R RV R� ( R R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRt � s c C� s | j S( N( t _prepare_curl_callback( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRu � s c C� s t j | � | _ d S( N( R RV R� ( R R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRu � s N( R R! R" R8 R R
R: R R t propertyRT t setterRl Rm Rs Rt Ru ( ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR9 sH �RF c B� sS e Z d Z d d d d d d d d � Z d � Z e e � Z d � Z d � Z RS( s� HTTP Response object.
Attributes:
* request: HTTPRequest object
* code: numeric HTTP status code, e.g. 200 or 404
* reason: human-readable reason phrase describing the status code
* headers: `tornado.httputil.HTTPHeaders` object
* effective_url: final location of the resource after following any
redirects
* buffer: ``cStringIO`` object for response body
* body: response body as string (created on demand from ``self.buffer``)
* error: Exception object, if any
* request_time: seconds from request start to finish
* time_info: dictionary of diagnostic timing information from the request.
Available data are subject to change, but currently uses timings
available from http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html,
plus ``queue``, which is the delay (if any) introduced by waiting for
a slot under `AsyncHTTPClient`'s ``max_clients`` setting.
c
C� s* t | t � r | j | _ n | | _ | | _ | pH t j j | d � | _ | d k rf | | _ n t j
� | _ | | _ d | _ | d k r� | j
| _ n | | _ | d k r| j d k s� | j d k r� t | j d | j d | �| _ qd | _ n | | _ | | _ | p i | _ d S( Nt Unknowni� i, t messageR ( RD RU R t codeR t responsesR> t reasonR RT RS t bufferR� R@ t
effective_urlRE RA RB t time_info(
R R R� RT R� R� RA RB R� R� ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR s* c C� s> | j d k r d S| j d k r7 | j j � | _ n | j S( N( R� R R� t getvalue( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt _get_body: s
c C� s | j r | j � n d S( s; If there was an error on the request, raise an `HTTPError`.N( RA ( R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt rethrowD s c C� s< d j d � t | j j � � D� � } d | j j | f S( Nt ,c s� s | ] } d | Vq d S( s %s=%rN( ( t .0t i( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pys <genexpr>J s s %s(%s)( t joint sortedt __dict__t itemst __class__R ( R t args( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt __repr__I s (N(
R R! R" R R R� R� Rl R� R� ( ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRF � s RE c B� s e Z d Z d d d � Z RS( s� Exception thrown for an unsuccessful HTTP request.
Attributes:
* ``code`` - HTTP error integer error code, e.g. 404. Error code 599 is
used when no HTTP response was received, e.g. for a timeout.
* ``response`` - `HTTPResponse` object, if any.
Note that if ``follow_redirects`` is False, redirects become HTTPErrors,
and you can look at ``error.response.headers['Location']`` to see the
destination of the redirect.
c C� sN | | _ | p! t j j | d � } | | _ t j | d | j | f � d S( NR� s HTTP %d: %s( R� R R� R> R t ExceptionR ( R R� R� R ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR \ s N( R R! R" R R ( ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRE N s
RU c B� s e Z d Z d � Z d � Z RS( so Combines an object with a dictionary of defaults.
Used internally by AsyncHTTPClient implementations.
c C� s | | _ | | _ d S( N( R R; ( R R R; ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyR h s c C� sL t | j | � } | d k r" | S| j d k rD | j j | d � Sd Sd S( N( R- R R R; R> ( R t namet request_attr( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt __getattr__l s ( R R! R" R R� ( ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyRU c s c C� s6 d d l m } m } m } | d d t d t �| d d t d t �| d d t d t �| d d t d t �| � } t � } x� | D]� } y% | j | d | j d | j
�} Wn1 t k
r� } | j d k r� | j } q� � n X| j rt | j � n | j r� t t | j � � q� q� W| j � d S( Ni ( t definet optionst parse_command_linet
print_headerst typet defaultt
print_bodyR_ Re ( t tornado.optionsR� R� R� t boolR
R R R R_ Re RE R R R� t printRT R� R Rl R ( R� R� R� R� t clientt argR t e( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt mainv s*
t __main__( R" t
__future__R R R R R RG R+ t tornado.concurrentR t tornado.escapeR R t tornadoR R t tornado.ioloopR t tornado.utilR
t objectR R R9 RF R� RE RU R� R ( ( ( s8 /usr/lib64/python2.7/site-packages/tornado/httpclient.pyt <module>' s$ " 4��P
Mini Shell