Use our own Python implementation of thread-local data even if a faster implementation is available, since the C implementation does not work with mod_wsgi, but added a parameter to make this configurable.

Author: Cito

DBUtils/Docs/RelNotes-1.0.html

@@ -24,6 +24,11 @@ <h2>Changes:</h2>
 <li>Added a <tt>version</tt> attribute to all exported classes.</li>
 <li>Where <tt>0</tt> has the meaning "unlimited", parameters can now be also
 set to <tt>None</tt> instead.</li>
+<li>It turned out that <tt>threading.local</tt> does not work properly with
+<tt>mod_wsgi</tt>, so we use the Python implementation for thread-local data
+even when a faster <tt>threading.local</tt> implementation is available.
+A new parameter <tt>threadlocal</tt> allows you to pass an arbitrary class
+such as <tt>threading.local</tt> if you know it works in your environment.</li>
 </ul>
 
 <h2>Bugfixes and Improvements:</h2>

DBUtils/Docs/UsersGuide.de.txt

@@ -278,6 +278,10 @@ Parameter angeben müssen:
 * ``closeable``: wenn dies auf ``True`` gesetzt wird, dann wird das Schließen
   von Verbindungen erlaubt, normalerweise wird es jedoch ignoriert
 
+* ``threadlocal``: eine optionale Klasse zur Speicherung thread-lokaler Daten,
+  die anstelle unserer Python-Implementierung benutzt wird (threading.local
+  ist schneller, kann aber nicht in allen Fällen verwendet werden)
+
 * Die als ``creator`` angegebene Funktion oder die Funktion ``connect``
   des DB-API-2-Datenbankadapter-Moduls erhalten alle weiteren Parameter,
   wie ``host``, ``database``, ``user``, ``password`` usw. Sie können einige
@@ -308,6 +312,11 @@ Stattdessen wird die Verbindung automatisch dann geschlossen, wenn der Thread
 endet. Sie können dieses Verhalten ändern, indem Sie den Parameter namens
 ``closeable`` setzen.
 
+Beachten Sie, dass das Holen einer Verbindung etwas beschleunigt werden kann,
+indem Sie den Parameter ``threadlocal`` auf ``threading.local`` setzen; dies
+könnte aber in einigen Umgebungen nicht funktionieren (es ist zum Beispiel
+bekannt, dass ``mod_wsgi`` hier Probleme bereitet, da es Daten, die mit ``threading.local`` gespeichert wurden, zwischen Requests löscht).
+
 PooledDB
 --------
 Wenn Sie das ``PooledDB``-Modul einsetzen möchten, müssen Sie zuerst einen

DBUtils/Docs/UsersGuide.txt

@@ -257,6 +257,10 @@ of ``PersistentDB``, passing the following parameters:
 * ``closeable``: if this is set to true, then closing connections will
   be allowed, but by default this will be silently ignored
 
+* ``threadlocal``: an optional class for representing thread-local data
+  that will be used instead of our Python implementation
+  (threading.local is faster, but cannot be used in all cases)
+
 * The creator function or the connect function of the DB-API 2 compliant
   database module specified as the creator will receive any additional
   parameters such as the host, database, user, password etc. You may
@@ -285,6 +289,11 @@ contrary to the intent of having persistent connections. Instead,
 the connection will be automatically closed when the thread dies.
 You can change this behavior be setting the ``closeable`` parameter.
 
+Note that by setting the ``threadlocal`` parameter to ``threading.local``,
+getting connections may become a bit faster, but this may not work in
+all environments (for instance, ``mod_wsgi`` is known to cause problems
+here since it clears the ``threading.local`` data between requests).
+
 PooledDB
 --------
 In order to make use of the ``PooledDB`` module, you first need to set up the

DBUtils/PersistentDB.py

@@ -41,6 +41,9 @@
 		if the default (OperationalError, InternalError) is not adequate
 	closeable: if this is set to true, then closing connections will
 		be allowed, but by default this will be silently ignored
+	threadlocal: an optional class for representing thread-local data
+		that will be used instead of our Python implementation
+		(threading.local is faster, but cannot be used in all cases)
 
 	The creator function or the connect function of the DB-API 2 compliant
 	database module specified as the creator will receive any additional
@@ -70,10 +73,15 @@
 the connection will be automatically closed when the thread dies.
 You can change this behavior be setting the closeable parameter.
 
+Note that by setting the threadlocal parameter to threading.local,
+getting connections may become a bit faster, but this may not work in
+all environments (for instance, mod_wsgi is known to cause problems
+since it clears the threading.local data between requests).
+
 
 Requirements:
 
-Minimum requirement: Python 2.2. Recommended: Python 2.4.3.
+Python >= 2.2, < 3.0.
 
 
 Ideas for improvement:
@@ -99,6 +107,7 @@
 __date__ = "$Date$"
 
 
+import ThreadingLocal
 from DBUtils.SteadyDB import connect
 
 
@@ -120,8 +129,8 @@ class PersistentDB:
 	version = __version__
 
 	def __init__(self, creator,
-		maxusage=None, setsession=None, failures=None, closeable=False,
-		*args, **kwargs):
+			maxusage=None, setsession=None, failures=None,
+			closeable=False, threadlocal=None, *args, **kwargs):
 		"""Set up the persistent DB-API 2 connection generator.
 
 		creator: either an arbitrary function returning new DB-API 2
@@ -136,6 +145,9 @@ def __init__(self, creator,
 			if the default (OperationalError, InternalError) is not adequate
 		closeable: if this is set to true, then closing connections will
 			be allowed, but by default this will be silently ignored
+		threadlocal: an optional class for representing thread-local data
+			that will be used instead of our Python implementation
+			(threading.local is faster, but cannot be used in all cases)
 		args, kwargs: the parameters that shall be passed to the creator
 			function or the connection constructor of the DB-API 2 module
 
@@ -158,7 +170,7 @@ def __init__(self, creator,
 		self._failures = failures
 		self._closeable = closeable
 		self._args, self._kwargs = args, kwargs
-		self.thread = local()
+		self.thread = (threadlocal or ThreadingLocal.local)()
 
 	def steady_connection(self):
 		"""Get a steady, non-persistent DB-API 2 connection."""
@@ -187,81 +199,3 @@ def connection(self, shareable=False):
 	def dedicated_connection(self):
 		"""Alias for connection(shareable=False)."""
 		return self.connection()
-
-
-try: # import a class for representing thread-local objects
-	from threading import local
-except ImportError: # for Python < 2.4, use the following simple implementation
-	from threading import currentThread, enumerate, RLock
-	class _localbase(object):
-		__slots__ = '_local__key', '_local__args', '_local__lock'
-		def __new__(cls, *args, **kwargs):
-			self = object.__new__(cls)
-			key = '_local__key', 'thread.local.' + str(id(self))
-			object.__setattr__(self, '_local__key', key)
-			object.__setattr__(self, '_local__args', (args, kwargs))
-			object.__setattr__(self, '_local__lock', RLock())
-			if args or kwargs and (cls.__init__ is object.__init__):
-				raise TypeError("Initialization arguments are not supported")
-			d = object.__getattribute__(self, '__dict__')
-			currentThread().__dict__[key] = d
-			return self
-	def _patch(self):
-		key = object.__getattribute__(self, '_local__key')
-		d = currentThread().__dict__.get(key)
-		if d is None:
-			d = {}
-			currentThread().__dict__[key] = d
-			object.__setattr__(self, '__dict__', d)
-			cls = type(self)
-			if cls.__init__ is not object.__init__:
-				args, kwargs = object.__getattribute__(self, '_local__args')
-				cls.__init__(self, *args, **kwargs)
-		else:
-			object.__setattr__(self, '__dict__', d)
-	class local(_localbase):
-		def __getattribute__(self, name):
-			lock = object.__getattribute__(self, '_local__lock')
-			lock.acquire()
-			try:
-				_patch(self)
-				return object.__getattribute__(self, name)
-			finally:
-				lock.release()
-		def __setattr__(self, name, value):
-			lock = object.__getattribute__(self, '_local__lock')
-			lock.acquire()
-			try:
-				_patch(self)
-				return object.__setattr__(self, name, value)
-			finally:
-				lock.release()
-		def __delattr__(self, name):
-			lock = object.__getattribute__(self, '_local__lock')
-			lock.acquire()
-			try:
-				_patch(self)
-				return object.__delattr__(self, name)
-			finally:
-				lock.release()
-		def __del__():
-			threading_enumerate = enumerate
-			__getattribute__ = object.__getattribute__
-			def __del__(self):
-				try:
-					key = __getattribute__(self, '_local__key')
-					threads = list(threading_enumerate())
-				except Exception:
-					return
-				for thread in threads:
-					try:
-						__dict__ = thread.__dict__
-					except AttributeError:
-						continue
-					if key in __dict__:
-						try:
-							del __dict__[key]
-						except KeyError:
-							pass
-			return __del__
-		__del__ = __del__()

DBUtils/PersistentPg.py

@@ -39,6 +39,9 @@
 		prepare the session, e.g. ["set datestyle to german", ...]
 	closeable: if this is set to true, then closing connections will
 		be allowed, but by default this will be silently ignored
+	threadlocal: an optional class for representing thread-local data
+		that will be used instead of our Python implementation
+		(threading.local is faster, but cannot be used in all cases)
 
 	Additionally, you have to pass the parameters for the actual
 	PostgreSQL connection which are passed via PyGreSQL,
@@ -65,11 +68,15 @@
 the connection will be automatically closed when the thread dies.
 You can change this behavior be setting the closeable parameter.
 
+Note that by setting the threadlocal parameter to threading.local,
+getting connections may become a bit faster, but this may not work in
+all environments (for instance, mod_wsgi is known to cause problems
+since it clears the threading.local data between requests).
+
 
 Requirements:
 
-Minimum requirement: Python 2.2 and PyGreSQL 3.4.
-Recommended: Python 2.4.3 and PyGreSQL 3.8.
+Python >= 2.2, < 3.0, PyGreSQL >= 3.4.
 
 
 Ideas for improvement:
@@ -95,6 +102,7 @@
 __date__ = "$Date$"
 
 
+import ThreadingLocal
 from DBUtils.SteadyPg import SteadyPgConnection
 
 
@@ -108,8 +116,8 @@ class PersistentPg:
 
 	version = __version__
 
-	def __init__(self, maxusage=None, setsession=None, closeable=False,
-			*args, **kwargs):
+	def __init__(self, maxusage=None, setsession=None,
+			closeable=False, threadlocal=None, *args, **kwargs):
 		"""Set up the persistent PostgreSQL connection generator.
 
 		maxusage: maximum number of reuses of a single connection
@@ -120,6 +128,9 @@ def __init__(self, maxusage=None, setsession=None, closeable=False,
 			the session, e.g. ["set datestyle to ...", "set time zone ..."]
 		closeable: if this is set to true, then closing connections will
 			be allowed, but by default this will be silently ignored
+		threadlocal: an optional class for representing thread-local data
+			that will be used instead of our Python implementation
+			(threading.local is faster, but cannot be used in all cases)
 		args, kwargs: the parameters that shall be used to establish
 			the PostgreSQL connections using class PyGreSQL pg.DB()
 
@@ -128,7 +139,7 @@ def __init__(self, maxusage=None, setsession=None, closeable=False,
 		self._setsession = setsession
 		self._closeable = closeable
 		self._args, self._kwargs = args, kwargs
-		self.thread = local()
+		self.thread = (threadlocal or ThreadingLocal.local)()
 
 	def steady_connection(self):
 		"""Get a steady, non-persistent PyGreSQL connection."""
@@ -144,81 +155,3 @@ def connection(self):
 			con = self.steady_connection()
 			self.thread.connection = con
 		return con
-
-
-try: # import a class for representing thread-local objects
-	from threading import local
-except ImportError: # for Python < 2.4, use the following simple implementation
-	from threading import currentThread, enumerate, RLock
-	class _localbase(object):
-		__slots__ = '_local__key', '_local__args', '_local__lock'
-		def __new__(cls, *args, **kwargs):
-			self = object.__new__(cls)
-			key = '_local__key', 'thread.local.' + str(id(self))
-			object.__setattr__(self, '_local__key', key)
-			object.__setattr__(self, '_local__args', (args, kwargs))
-			object.__setattr__(self, '_local__lock', RLock())
-			if args or kwargs and (cls.__init__ is object.__init__):
-				raise TypeError("Initialization arguments are not supported")
-			d = object.__getattribute__(self, '__dict__')
-			currentThread().__dict__[key] = d
-			return self
-	def _patch(self):
-		key = object.__getattribute__(self, '_local__key')
-		d = currentThread().__dict__.get(key)
-		if d is None:
-			d = {}
-			currentThread().__dict__[key] = d
-			object.__setattr__(self, '__dict__', d)
-			cls = type(self)
-			if cls.__init__ is not object.__init__:
-				args, kwargs = object.__getattribute__(self, '_local__args')
-				cls.__init__(self, *args, **kwargs)
-		else:
-			object.__setattr__(self, '__dict__', d)
-	class local(_localbase):
-		def __getattribute__(self, name):
-			lock = object.__getattribute__(self, '_local__lock')
-			lock.acquire()
-			try:
-				_patch(self)
-				return object.__getattribute__(self, name)
-			finally:
-				lock.release()
-		def __setattr__(self, name, value):
-			lock = object.__getattribute__(self, '_local__lock')
-			lock.acquire()
-			try:
-				_patch(self)
-				return object.__setattr__(self, name, value)
-			finally:
-				lock.release()
-		def __delattr__(self, name):
-			lock = object.__getattribute__(self, '_local__lock')
-			lock.acquire()
-			try:
-				_patch(self)
-				return object.__delattr__(self, name)
-			finally:
-				lock.release()
-		def __del__():
-			threading_enumerate = enumerate
-			__getattribute__ = object.__getattribute__
-			def __del__(self):
-				try:
-					key = __getattribute__(self, '_local__key')
-					threads = list(threading_enumerate())
-				except Exception:
-					return
-				for thread in threads:
-					try:
-						__dict__ = thread.__dict__
-					except AttributeError:
-						continue
-					if key in __dict__:
-						try:
-							del __dict__[key]
-						except KeyError:
-							pass
-			return __del__
-		__del__ = __del__()

DBUtils/PooledDB.py

@@ -155,10 +155,10 @@ class PooledDB:
 	version = __version__
 
 	def __init__(self, creator,
-		mincached=0, maxcached=0,
-		maxshared=0, maxconnections=0, blocking=False,
-		maxusage=None, setsession=None, failures=None,
-		*args, **kwargs):
+			mincached=0, maxcached=0,
+			maxshared=0, maxconnections=0, blocking=False,
+			maxusage=None, setsession=None, failures=None,
+			*args, **kwargs):
 		"""Set up the DB-API 2 connection pool.
 
 		creator: either an arbitrary function returning new DB-API 2

DBUtils/PooledPg.py

@@ -129,10 +129,10 @@ class PooledPg:
 	version = __version__
 
 	def __init__(self,
-		mincached=0, maxcached=0,
-		maxconnections=0, blocking=False,
-		maxusage=None, setsession=None,
-		*args, **kwargs):
+			mincached=0, maxcached=0,
+			maxconnections=0, blocking=False,
+			maxusage=None, setsession=None,
+			*args, **kwargs):
 		"""Set up the PostgreSQL connection pool.
 
 		mincached: initial number of connections in the pool