Re: cytpes and non executable memory

Andrew MacIntyre <Andrew.MacIntyre@xxxxxxxxxx> writes:

>> But: mmap (or VirtualAlloc on Windows) return memory blocks 
>> with a size
>> rounded up to be a multiple of the system's pagesize - and it is
>> somewhat wasteful to use a 4k block when only 20 or 30 bytes 
>> are really
>> needed.  So, it seems a very simple memory manager is needed. 
>>  Since the
>> block size is always the same, it probably wouldn't be too complicated
>> to hack one, but isn't there a library I can reuse?
>
> It occurs to me that you might be able to take the PyMalloc code from 
> the Python core and hack it to use mmap() etc (instead of malloc()/free()) 
> if you're only needing allocations up to 256 bytes...

Good idea, but certainly overkill.

I came up with a very simple solution because the memory blocks needed
have always the same size.

I simply allocate an array of several of these ffi_closure things (which
will contain the few bytes of customized exeutable code), and build a
linked list of them.  Then allocating and freeing the ffi_closure things
become trivial and fast list operations.

> I'm not sure whether thread safety is an issue - if ctypes doesn't release 
> the GIL until it actually calls the foreign function it probably isn't 
> an issue.

> I came across another reference (may have been on python-dev) that some 
> systems have limits on the number of mmap()ed regions you can have open 
> at any given time.

The GIL is held, and the size of the raw mmap'd memory blocks can be adjusted.

Thomas



-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl