osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Calling an unbound method in C using the Public API


Matthieu Dartiailh schrieb am 29.08.2018 um 16:33:
> I am one of the maintainer of the atom library (https://github.com/nucleic/atom <https://github.com/nucleic/atom>). This library provides low-memory footprint Python objects, descriptors and containers enforcing type validation, implements the observer pattern. 
> 
> For list, we basically subclass the standard Python list in C++ and add the required checks/notifications to some methods (in particular insert, pop and sort). To call the method of standard list object on our custom object, the original author chose to directly access the c-function inside the MethodDef and call it (by first casting to the proper function pointer type).
> 
> Starting with Python 3.7, insert, pop and sort use the FASTCALL calling convention, which means that the signature of the function stored inside the MethodDef have changed. I have a working solution, but it involves to use a lot of the CPython private C-API. In particular I needed to access:
> _PyCFunctionFast
> _PyCFunctionFastWithKeywords
> _PyStack_UnpackDict
> 
> I tried to look at the public C API for a way to call an unbound method with a minimal cost (in term of speed and memory). It seems to me, but please correct me if I am wrong, that one cannot call a MethodDef using only the public API. To use the public C API, one has to use PyCFunction_Call (or a variant) that expect a PyCFunctionObject which binds a the MethodDef to an instance. In my case, to avoid creating a temporary PyCFunctionObject each time I call list.insert on my custom subclass instance, I have to store that PyCFunctionObject for each instance. But this means storing  7 PyCFunctionObject per instance (one for each method of list I need to wrap). So I can either use the public API and increase the memory footprint or slow down the code by creating PyCFunctionObject for each call
>  , or use large amount of the private API.

These functions are very unlikely to change within the Py3.7.x releases,
but they are hopefully going to change for 3.8. See PEP 580. Your feedback
on it is welcome.

https://www.python.org/dev/peps/pep-0580/

Stefan