FDs will be closed after exception automatically in python2.7?
On 10Jun2019 12:47, Peter J. Holzer <hjp-python at hjp.at> wrote:
>On 2019-06-10 18:18:07 +0800, lampahome wrote:
>> I confused will fd will be close after exception automatically?
>> fd.get() //any useless function of fd
>> except Exception:
>> print 'hi'
>Please try to preserve indentation when sending code to the list.
>Indentation is part of the Python syntax and if you smash everything to
>the left margin your program is not only hard to read but syntactically
>To answer your question:
>No, an exception will not automatically close open file descriptors
>(which file descriptors should it close and why?).
>However, while handling an exception the variable holding the file
>descriptor may go out of scope (for example if you leave a "with"
>block or return from a function). In that case the file descriptor
>will be closed (in the case of a with block immediately, otherwise when
>the garbage collector gets around to it).
We want to be more precise here.
By using the term "file descriptor" above you've conflated 2 things: the
Python file object and the OS file descriptor. They're different things
with different behaviours.
What you get from "open()" is a python "file object". And what the OP is
getting in his example code is also a file object. And as you say, when
all references to that are removed (variable goes out of scope), the
Python interpreter will close the file.
So far so good.
However the term "file descriptor", at least in POSIX (UNIX and Linux),
is _not_ the same as a Python "file object". Instead, it is the integer
you get from a OS level file open such as returned by os.open(). It is
_not_ managed by the Python interpreter and does not get closed when
nobody references it, because it is not an object.
Now, for the OP's edification: when you use Python's open() function you
get a file object, which includes a reference to the _underlying_ OS
level file descriptor. When that object get recovered _it_ will close
the file descriptor as part of the close operation.
However, if you do an OS level file open which returns _just_ the file
descriptor, this does not happen because there's no object cleanup and
nothing to close the descriptor, because nothing is managing it.
fd = os.open('/dev/null',os.O_RDONLY)
os.system("lsof -p "+str(os.getpid()))
Here's some code which does an OS level open, getting a fie descriptor.
The opening function raises an exception. After we've caught it, we run
"lsof" on our own process. here's it on my Mac:
[~]fleet*1> /usr/bin/python fd.py
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
Python 97417 cameron cwd DIR 1,5 9214 934731 /Users/cameron
Python 97417 cameron txt REG 1,5 25152 199261280 /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
Python 97417 cameron txt REG 1,5 2633024 199261275 /System/Library/Frameworks/Python.framework/Versions/2.7/Python
Python 97417 cameron txt REG 1,5 52832 199261366 /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_locale.so
Python 97417 cameron txt REG 1,5 643792 199264919 /usr/lib/dyld
Python 97417 cameron txt REG 1,5 560200432 200219840 /private/var/db/dyld/dyld_shared_cache_x86_64
Python 97417 cameron 0u CHR 16,47 0t1529 1991 /dev/ttys047
Python 97417 cameron 1u CHR 16,47 0t1529 1991 /dev/ttys047
Python 97417 cameron 2u CHR 16,47 0t1529 1991 /dev/ttys047
Python 97417 cameron 3r CHR 3,2 0t0 305 /dev/null
See the last line. That is file descriptor 3, attached to /dev/null. It
is still open.
When your programme exits the OS will clean this up, but until then the
_descriptor_ remains open.
If you rewrote this with plain Python "open("/dev/null")" instead of the
os.open you would not see the file descriptor lying around because the
close of the _file object_ would have cleaned up the related OS file
Cameron Simpson <cs at cskk.id.au>