NOTHING HERE IS FINISHED! IF YOU WANT A WORKING PRODUCT, GO AWAY! IT'S BARELY STARTED! IT MAY NEVER BE FINISHED! This is intended to be a Python interface to PostgreSQL. I want one that supports all of the following: 1. asynchronous operation (Twisted-friendly!) 2. server-side prepared statements 3. incremental results from queries, flushing memory usage periodically 4. asynchronous notification 5. prompt handling of signals (KeyboardInterrupt!) 6. thread safety 7. fast bulk copies - "copy from stdin" and "copy to stdin" The other Python PostgreSQL interfaces (PyGreSQL, pyscopg) do not support #1-#5. [*] psycopg also has severely broken threading support. It attempts to support DB-API's threadsafety=2, which PEP-249 describes as: 2 Threads may share the module and connections. ... Sharing in the above context means that two threads may use a resource without wrapping it using a mutex semaphore to implement resource locking. Note that you cannot always make external resources thread safe by managing access using a mutex: the resource may rely on global variables or other external sources that are beyond your control. This support is badly broken, managed by an ill-thought-out mutex scheme that has race conditions (using resources without proper locking) and deadlocks (acquiring Python's Global Interpreter Lock and the connection mutexes in inconsistent order, resulting in hangs). It is also of limited utility, since libpq (which pyscopg uses internally) is not intended for this level: One restriction is that no two threads attempt to manipulate the same PGconn object at the same time. In particular, you cannot issue concurrent commands from different threads through the same connection object. (If you need to run concurrent commands, use multiple connections.) The only proper way to support threadsafety=2 is wrap libpq with mutexes. This module will do the same. However, the locking will be done in the Python code, rather than the C extension, because higher-level code is easier to get right. Since a primary goal is to support the asynchronous API, the C code will simplify things by _only_ supporting the asynchronous API. Synchronous behavior can be implemented as a wrapper, but not the other way around. Furthermore, this simplifies locking dramatically. It never blocks in the C API, so it never needs to release the Global Interpreter Lock. The C API is inherently thread-safe. The more minor problems of inconsistent state in the Python layer are easily solved by per-connection threading.Lock instances. The C layer will be as thin as possible. Python modules will be responsible for making programming easy by providing whatever interface is desired: - A synchronous DB-API 2.0 implementation for compatibility. - Perhaps a synchronous database API that makes sense. DB-API sucks. A subset of JDBC would be much more useful. - An asynchronous twisted implementation. The entire implementation will support the latest PostgreSQL and Python versions. Other versions working would be a bonus, but not necessary. [*] - Actually, #3 can be supported by SQL-level "DECLARE CURSOR ..." and "FETCH ..." commands. This requires additional PostgreSQL-specific code for each query, however.