How to insert Python GIL manipulation code?

I'm using SWIG/Python to wrap a large C++ library.  The library's
problem domain is well suited to shared-memory parallelism, which I'd
like Python client code to be able to exploit.  As you probably know,
virtually all access to Python's C API must be serialized by a
process-wide lock called the Global Interpreter Lock (GIL).

Python's threads are genuine native threads, and the interpreter can be
manipulated by multiple threads, but only in a serial manner, regulated
by the GIL.  To take advantage of the underlying C++ library's
parallelism, a Python client program should be able to start multiple
threads via the Python thread libraries, then "kick off" requests to
the underlying library from each Python thread.  Since a Python thread
is just a native thread with some tags attached, each request to the
C++ library can run in a fully parallel manner after it's clear of (and
before it returns to) the wrapper code between Python and C++.

Although I'm not a stranger to Python GIL manipulation in C, I'm not
sure how to cause SWIG to insert GIL manipulation snippets at
appropriate places in the SWIG-generated wrapper code.  I could use the
'%exception' directive to globally generate GIL release/acquisition
code around every wrapped call to the underlying C++ library, but there
are some problems with that approach:

1.  Directors
  If the '%exception' directive has been used to trigger a GIL-release
before each call to a wrapped C++ method, a call to a "directorized"
C++ method will potentially call back into Python without reacquiring
the GIL, won't it?
  The Director implementation for Python (Lib/python/director.swg)
doesn't appear to protect against this.  How can I cause SWIG to insert
arbitrary code at the beginning of and before all exits from director
methods?

2.  Calls during which the GIL shouldn't be released
  I use the '%extend' directive extensively, and frequently call the
Python C API from the '%extend'ed code.  If I use the '%exception'
directive to globally generate GIL release/acquisition code around
every wrapped call, I'll have to turn around and reacquire/rerelease
the GIL in each '%extend'ed method.  This will cause at least two GIL
acquisitions and two releases per '%extend'ed method call, when in most
cases I'd prefer that there be none.
  There are also some computationally trivial calls to the underlying
library around which GIL manipulation would introduce pointless
overhead.


How can I overcome these obstacles, short of modifying SWIG itself?

Thanks.


        
                
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail 
_______________________________________________
Swig maillist  -  Swig@xxxxxxxxxxxxxxx
http://mailman.cs.uchicago.edu/mailman/listinfo/swig