Subject: How to insert Python GIL manipulation code? - msg#00078

List: programming.swig

Date: Prev Next Index Thread: Prev Next Index

I'm using SWIG/Python to wrap a large C++ library. The library's
problem domain is well suited to shared-memory parallelism, which I'd
like Python client code to be able to exploit. As you probably know,
virtually all access to Python's C API must be serialized by a
process-wide lock called the Global Interpreter Lock (GIL).

Python's threads are genuine native threads, and the interpreter can be
manipulated by multiple threads, but only in a serial manner, regulated
by the GIL. To take advantage of the underlying C++ library's
parallelism, a Python client program should be able to start multiple
threads via the Python thread libraries, then "kick off" requests to
the underlying library from each Python thread. Since a Python thread
is just a native thread with some tags attached, each request to the
C++ library can run in a fully parallel manner after it's clear of (and
before it returns to) the wrapper code between Python and C++.

Although I'm not a stranger to Python GIL manipulation in C, I'm not
sure how to cause SWIG to insert GIL manipulation snippets at
appropriate places in the SWIG-generated wrapper code. I could use the
'%exception' directive to globally generate GIL release/acquisition
code around every wrapped call to the underlying C++ library, but there
are some problems with that approach:

1. Directors
If the '%exception' directive has been used to trigger a GIL-release
before each call to a wrapped C++ method, a call to a "directorized"
C++ method will potentially call back into Python without reacquiring
the GIL, won't it?
The Director implementation for Python (Lib/python/director.swg)
doesn't appear to protect against this. How can I cause SWIG to insert
arbitrary code at the beginning of and before all exits from director
methods?

2. Calls during which the GIL shouldn't be released
I use the '%extend' directive extensively, and frequently call the
Python C API from the '%extend'ed code. If I use the '%exception'
directive to globally generate GIL release/acquisition code around
every wrapped call, I'll have to turn around and reacquire/rerelease
the GIL in each '%extend'ed method. This will cause at least two GIL
acquisitions and two releases per '%extend'ed method call, when in most
cases I'd prefer that there be none.
There are also some computationally trivial calls to the underlying
library around which GIL manipulation would introduce pointless
overhead.

How can I overcome these obstacles, short of modifying SWIG itself?

Thanks.

__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail
_______________________________________________
Swig maillist - Swig@xxxxxxxxxxxxxxx
http://mailman.cs.uchicago.edu/mailman/listinfo/swig

Was this page helpful?

Thread at a glance:

Previous Message by Date: click to view message preview

problem linking python libs

Im very newbie to swig and python, I made my first test app a couple of days ago and I havent found the correct flags to link required libs to my C++ program in linux. Can somebody give me a help with that? _______________________________________________ Swig maillist - Swig@xxxxxxxxxxxxxxx http://mailman.cs.uchicago.edu/mailman/listinfo/swig

Next Message by Date: click to view message preview

Re: how to use the autodoc feature of new swig-1.3.22?

Bo Peng wrote: Dear list, I applied the docstring patch. '%feature("autodoc",1");' is working as expected but I am not sure how to attach a string to a specific function when namespace and templates are involved. For example, class A is a template class in namespace N, is node name for function func 'N::A::func'? I find that %ignore N::A::somefunc; works fine but %feature("docstring","some doc") N::A::func; does not work. Try it like this: %feature("docstring") N::A::func "some doc"; -- Robin Dunn Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython! _______________________________________________ Swig maillist - Swig@xxxxxxxxxxxxxxx http://mailman.cs.uchicago.edu/mailman/listinfo/swig

Previous Message by Thread: click to view message preview

Next Message by Thread: click to view message preview

C++ class inheritance using shadow class

Hi there, I've been using Swig successfully for a while now but I'm stuck with this one! I've got a basic C++ class: class System { public: System(void); float Dt; float VoxelSize; int NVoxelX, NVoxelY,NVoxelZ; Vector SimulationBoxOrigin; }; inline System::System(void) { Dt = 0; VoxelSize = 0; NVoxelX = NVoxelY = NVoxelZ = 0; SimulationBoxOrigin = Vector(0,0,0); } I then swig it using shadow classes, that works fine. Now I would like to add extra python features to this classes, so I want to use inheritance. >>> class Sys(System): ... def __init__(self): ... print Sys.VoxelSize ... >>> c = Sys() <property object at 0x818ce24> >>> c.VoxelSize Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: Type error. Expected _p_System >>> The inheritance doesn't seem to work! Any reason why that should be?? Looking forward to your suggestions, Alex _______________________________________________ Swig maillist - Swig@xxxxxxxxxxxxxxx http://mailman.cs.uchicago.edu/mailman/listinfo/swig

Loading Comments...