I’m working in a project with Python and AIOHTTP, so inevitably I was going to hit some threading issues that forced me to learn more about asyncio. I’ve collected in this blogpost some useful links. Please contact me for any other good stuff that you can suggest :)
- Official docs:
- Async Python: The Different Forms of Concurrency: very good summary about the current state of Python parallelism, including GIL explanation and a “Making the Right Choice” recommendation.
- How to deal with blocking code within asyncio event loop: recommendations, including lower-level suggestions than usual.
- Async IO in Python: A Complete Walkthrough.
- Explanation about
ThreadPoolExecutor, which is affected by the GIL.
- Yet another async/await tutorial.
- uvloop, an alternative implementation of the built-in asyncio event loop.
- asyncio snippets.
- To explore: Python 3.8 multiprocessing shared memory: Python 3.8, currently in beta, will have a
multiprocessing.shared_memorymodule. That might make feasible sharing the connection among processes. I haven’t done any research about this, so this can be plain wrong. In addition, the overhead of creating a new process might make this approach prohibitive.
Fast.ai is a library for neural networks that has a useful helper for triggering processes (via @antor, thanks!):
from fastai.core import parallel def process_one(item,i): return (item**2, i) result = parallel(process_one, range(100000))
- Dynamic languages and Parallelism: How to Go from Broken or Slow to Safe and Efficient: not about Python but dynamic languages in general.