osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Python-Dev] Help preventing SIGPIPE/SIG_DFL anti-pattern.


(sorry for the double post, looks like maybe attachments are dropped, 
inlined the attachment this time.)

Hello,

I'm looking for someone in the python community to help with a problem 
of anti-patterns showing up dealing with SIGPIPE.

Specifically I've noticed an anti-pattern developing where folks will 
try to suppress broken pipe errors written to stdout by setting 
SIGPIPE's disposition to SIG_DFL.? This is actually very common, and 
also rather broken due to the fact that for all but the most simple text 
filters this opens up a problem where the process can exit unexpectedly 
due to SIGPIPE being generated from a remote connection the program makes.

I have attached a test program which shows the problem.

to use this program it takes several args.

# 1. Illustrate the 'ugly output to stderr' that folks want to avoid:

% python3 t0.py nocatch | head -1


# 2. Illustrate the anti-pattern, the program exits on about line 47 
which most folks to not understand

% python3 t0.py dfl | head -1

# 3. Show a better solution where we catch the pipe error and cleanup to 
avoid the message:

% python3 t0.py | head -1


I did a recent audit of a few code bases and saw this pattern pop often 
often enough that I am asking if there's a way we can discourage the use 
of "signal(SIGPIPE, SIG_DFL)" unless the user really understands what 
they are doing.

I do have a pull req here: https://github.com/python/cpython/pull/6773 
where I am trying to document this on the signal page, but I can't sort 
out how to land this doc change.

thank you,

-Alfred


=== CUT HERE ===

#
# Program showing the dangers of setting the SIG_PIPE handler to the 
default handler (SIG_DFL).
#
# To illustrate the problem run with:
# ./foo.py dfl
#
# The program will exit in do_network_stuff() even though there is a an 
"except" clause.
# The do_network_stuff() simulates a remote connection that closes 
before it can be written to
# which happens often enough to be a hazard in practice.
#
#
#

import signal
import sys
import socket
import os

def sigpipe_handler(sig, frame):
 ??? sys.stderr.write("Got sigpipe \n\n\n")
 ??? sys.stderr.flush()

def get_server_connection():
 ??? # simulate making a connection to a remote service that closes the 
connection
 ??? # before we can write to it.? (In practice a host rebooting, or 
otherwise exiting while we are
 ??? # trying to interact with it will be the true source of such behavior.)
 ??? s1, s2 = socket.socketpair()
 ??? s2.close()
 ??? return s1


def do_network_stuff():
 ??? # simulate interacting with a remote service that closes its connection
 ??? # before we can write to it.? Example: connecting to an http 
service and
 ??? # issuing a GET request, but the remote server is shutting down between
 ??? # when our connection finishes the 3-way handshake and when we are able
 ??? # to write our "GET /" request to it.
 ??? # In theory this function should be resilient to this, however if 
SIGPIPE is set
 ??? # to SIGDFL then this code will cause termination of the program.
 ??? if 'dfl' in sys.argv[1:]:
 ??????? signal.signal(signal.SIGPIPE, signal.SIG_DFL)

 ??? for x in range(5):
 ??????? server_conn = get_server_connection()
 ??????? sys.stderr.write("about to write to server socket...\n")
 ??????? try:
 ??????????? server_conn.send(b"GET /")
 ??????? except BrokenPipeError as bpe:
 ??????????? sys.stderr.write("caught broken pipe on talking to server, 
retrying...")

def work():
 ??? do_network_stuff()
 ??? for x in range(10000):
 ??????? print("y")
 ??? sys.stdout.flush()


def main():
 ??? if 'nocatch' in sys.argv[1:]:
 ??????? work()
 ??? else:
 ??????? try:
 ??????????? work()
 ??????? except BrokenPipeError as bpe:
 ??????????? signal.signal(signal.SIGPIPE, signal.SIG_DFL)
 ??????????? os.kill(os.getpid(), signal.SIGPIPE)


if __name__ == '__main__':
 ??? main()