Re: Re: urllib and ftplib hang in bundle but not when run directly

Russell E Owen <owen@xxxxxxxxxxxxxxxxxxxx> writes:

> At 11:56 AM +0100 5/14/04, Michael Hudson wrote:
>>"Russell E. Owen" <rowen@xxxxxxxxxxx> writes:
>>
>>>  In article <rowen-C76C8E.09571605052004@xxxxxxxxxxxxx>,
>>>   "Russell E. Owen" <rowen@xxxxxxxxxxx> wrote:
>>>
>>>>I am having trouble with some ftp code. On some platforms it works fine
>>>>and on others it reliably hangs....(including MacOS X command-line
>>>>it works, but a bundled app version of the identical code fails)
>>>
>>>  After some experimentation, I found the hanging was occurring in ftplib.
>>>  It delays importing re and compiling two regular expressions until they
>>>  are needed, and the thread hangs both on the import and (if I modify
>>>  ftplib to import re in advance) on the compilation. Modifying ftplib to
>>>  do both jobs in advance (when first imported) fixes the problem.
>>
>>Um.  Python has an 'import lock' so only one thread can be importing a
>>module at the same time.
>>
>>Do you have code like this:
>>
>>def main():
>>     ....
>>
>>main()
>>
>>i.e. that does things as a result of being imported?  You should stop
>>that :-)
>
>
> It's an interesting idea. I didn't know about any import lock (just
> the global interpreter lock, which I doubt is relevant).
>
> I do have code like this:
> def _createMyConst():
>     ...
> _MyConst = _creatMyConst()
>
> that runs as a result of being imported. I thought that was normal
> python -- initialization can safely be done when the module is
> imported. I don't understand why that is considered dangerous or how
> it could run afoul of an importer lock.

I think that so long as _creatMyConst() doesn't start any threads, you
should be OK.

> Also, my own code has fully loaded and finished initializing well
> before I try any ftp transfers and run into the problem.

Ah.  Then it's not what I suspected.

> The hang occurs inside ftplib itself. I use it to fetch a file in a
> separate (background) thread. If "too much other stuff" is going on,
> that thread hangs. With ftplib as written, it hangs while executing
> "import re" (which is only done once the ftp starts). If I modify
> ftplib to import re right away at the beginning, then the thread hangs
> in ftplib compiling a regular expression. Modifying ftplib to
> pre-compile the two regular expressions it uses makes it stop hanging
> and my code starts working again.

A deadlock!  Have fun!

It can hardly be a coincidence that the sre compiler is written in
Python...

> It seems strange to me that a thread could either work fine or 100%
> totally hang like that. It always works fine when run on MacOS X (when
> run as a command-line app). It always hangs forever when the identical
> code is run as a macOS X bundled app or when run on RedHat linux. When
> the same code is run as part of a smaller app, it runs fine on linux.

Oddness.

Cheers,
mwh

-- 
  One of the great skills in using any language is knowing what not
  to use, what not to say.  ... There's that simplicity thing again.
                                                       -- Ron Jeffries

_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@xxxxxxxxxx
http://mail.python.org/mailman/listinfo/pythonmac-sig