[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sat, Jun 23, 2018 at 12:22:33AM +1000, Nick Coghlan wrote:
> * for the reactions to my description of the currently proposed parent
> local scoping behaviour in comprehensions, I'd use the word
> "horrified", and feel I wasn't overstating the response :)
Without knowing how you worded the question, and the reasons for this
horrified reaction, I'm afraid that isn't really helpful. It is nothing
more than an appeal to emotion:
Such strong emotions as "horrified" are typically a sign of an
immediate, emotional gut reaction, not careful thought. We often see
those sorts of reactions attached to the most objectively trivial
matters. Immediate gut reactions are rarely a good guide because they
tend to over-value the status quo, exaggerate the difficulty and costs
of change, and under-estimate the benefits.
Speaking personally, I've learned to question my immediately gut
reaction. (And I remember to do so at least half the time.) PEP 572 is
an example: when the issue was first raised back in February, my gut
reaction was "Not in MY Python!!!" but by taking it seriously and
running through some examples over the course of the discussion, I
realised that, actually, I cautiously favour the idea.
Of course, matters of *personal taste* cannot be anything but gut
reaction, but in those matters, what one person holds strongly another
can legitimately reject strongly. We ought to try to look beyond
personal taste, and try (even if only imperfectly) to consider rational
reasons for and against a proposal. If we do, reactions like "horrified"
are rarely justified. It's just a minor feature in a programming
language, the world will go on one way or the other, and Python already
has trickier gotchas.
> While I try to account for the fact that I implemented the current
> comprehension semantics for the 3.x series, and am hence biased
> towards considering them the now obvious interpretation,
While we certainly don't want to make "non-obvious" a virtue for its own
sake, obviousness (obvious to who?) ought to take a distant second place
to *useful*. Otherwise we'd have to give up an awful lot of existing
Python, starting with the fundamental execution model.
(Oh, the number and length of arguments about whether Python uses call
by value or call by reference, why mutable defaults and []*3 are
"broken"... if you think Python's execution model is "obvious" you've
been using Python too long ;-)
But as Tim Peters has said on a number of occasions, nobody is
suggesting changing the interpretation of current comprehension
semantics. Comprehension loop variables will continue to remain
isolated to the comprehension.
(And for the record, that makes *comprehensions* a weird special case,
not assignment expressions. All other expressions run in the current
lexical scope. Comprehensions introduce an implicit, invisible,
sub-local scope that doesn't match up with a change in indentation as
class and def statements do.)
The behaviour in question is a matter of *assignment expression*
semantics, not comprehensions. And honestly, I don't see why the
proposed behaviour is "horrifying". Here's the high-level overview:
- at the top level of a module, assignment expressions assign in
the global scope;
- inside a class, assignment expressions assign in the class scope;
- inside a function, assignment expressions assign in the function
local scope (unless declared global or nonlocal);
- inside a comprehension, assignment expressions assign in the
surrounding lexical scope (the surrounding function, class or
The first three are the same as ordinary statement assignment. The last
one is what you would expect if you treat comprehensions as any other
expression which run in the current lexical scope. (The current function
or class or module.) Even if we treat it as a "weird special case" (I
don't think it is, but for the sake of the argument let's say it is) its
not hard to explain.
As I discuss below, you can get a very long way indeed working with
comprehensions without once thinking about the scope they run in. By the
time you need to think about comprehension scope, it shouldn't be hard
to deal with the rule:
- loop variables are hidden in a comprehension private scope;
- explicit assignment expression variables are not.
This is not async, or metaclasses, or even Unicode.
> plenty of
> functional-language-inspired documentation to instead encourage folks
> to view comprehensions as tightly encapsulated declarative container
> construction syntax.
I can't say I've done a broad survey, but the third-party documentation
I've read on comprehensions typically glosses over the scoping issues
without mentioning them. To the extent that scoping is even hinted at,
comprehensions are treated as expressions which are exactly equivalent
to re-writing them as a for-loop in the current scope.
This is a typical example, found as the top result on googling for
Nothing is mentioned about scope, and it repeats the inaccurate but
for item in list:
But perhaps that tutorial is too old. Okay this recent one is only a
little more than a year old:
Again, no mention of scoping issues, comprehensions are simply
expressions which presumably run in the same scope as any other
I think you over-estimate how many newcomers to Python are even aware
that the scope of comprehensions is something to consider.
> I'm currently working on a concept proposal at
> https://github.com/ncoghlan/peps/pull/2 that's much closer to PEP 572
> than any of my previous `given` based suggestions:
I look forward to reading it, and I promise I won't go by my gut