.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "11_demos\python_packages\parallelization_threading\demo_ThreadPoolExecutor.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_11_demos_python_packages_parallelization_threading_demo_ThreadPoolExecutor.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_11_demos_python_packages_parallelization_threading_demo_ThreadPoolExecutor.py:

Thread pool executor
=======================

``Asyncio``/``concurrent`` heavily changed from python ``3.4`` to ``3.7``, better read the docs
and do some tutorials. Asyncio is preferred over plain concurrent module.

.. GENERATED FROM PYTHON SOURCE LINES 6-19

.. code-block:: Python


    import concurrent.futures
    import urllib.request

    URLS = [
        "http://www.foxnews.com/",
        "http://www.cnn.com/",
        "http://europe.wsj.com/",
        "http://www.bbc.co.uk/",
        "http://some-made-up-domain.com/",
    ]









.. GENERATED FROM PYTHON SOURCE LINES 21-22

Retrieve a single page and report the url and contents

.. GENERATED FROM PYTHON SOURCE LINES 22-27

.. code-block:: Python

    def load_url(url, timeout):
        with urllib.request.urlopen(url, timeout=timeout) as conn:
            return conn.read()









.. GENERATED FROM PYTHON SOURCE LINES 28-29

We can use a with statement to ensure threads are cleaned up promptly

.. GENERATED FROM PYTHON SOURCE LINES 29-40

.. code-block:: Python

    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        # Start the load operations and mark each future with its URL
        future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
        for future in concurrent.futures.as_completed(future_to_url):
            url = future_to_url[future]
            try:
                data = future.result()
            except Exception as exc:
                print(f"{url!r} generated an exception: {exc}")
            else:
                print("%r page is %d bytes" % (url, len(data)))




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    'http://www.foxnews.com/' page is 718244 bytes
    'http://europe.wsj.com/' generated an exception: HTTP Error 403: Forbidden
    'http://www.cnn.com/' page is 3063439 bytes
    'http://www.bbc.co.uk/' page is 557992 bytes
    'http://some-made-up-domain.com/' generated an exception: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)>





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.831 seconds)


.. _sphx_glr_download_11_demos_python_packages_parallelization_threading_demo_ThreadPoolExecutor.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: demo_ThreadPoolExecutor.ipynb <demo_ThreadPoolExecutor.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: demo_ThreadPoolExecutor.py <demo_ThreadPoolExecutor.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: demo_ThreadPoolExecutor.zip <demo_ThreadPoolExecutor.zip>`