Concurrency with gevent

Introduction

The gevent homepage describes it a coroutine -based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libev or libuv event loop.

Features include:

• Fast event loop based on libev or libuv.

• Lightweight execution units based on greenlets.

• API that reuses concepts from the Python standard library (for examples there are events and queues).

• Cooperative sockets with SSL support

• Cooperative DNS queries performed through a threadpool, dnspython, or c-ares.

• Monkey patching utility to get 3rd party modules to become cooperative

• TCP/UDP/HTTP servers

• Subprocess support (through gevent.subprocess)

• Thread pools

gevent is inspired by eventlet but features a more consistent API, simpler implementation and better performance. Read why others use gevent and check out the list of the open source projects based on gevent.

Enabling gevent

You can enable the gevent pool by using the celery worker -P gevent or celery worker --pool=gevent worker option.

$ celery-Aprojworker-Pgevent-c1000

Examples

See the gevent examples directory in the Celery distribution for some examples taking use of Eventlet support.

Known issues

There is a known issue using python 3.11 and gevent. The issue is documented here and addressed in a gevent issue. Upgrading to greenlet 3.0 solves it.