[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Python-Dev] PEP 575 (Unifying function/method classes) update

On 05/15/18 05:15, Jeroen Demeyer wrote:
> On 2018-05-14 19:56, Petr Viktorin wrote:
>> It does quite a lot of things, and the changes are all intertwined,
>> which will make it hard to get reviewed and accepted.
> The problem is that many things *are* already intertwined currently. You 
> cannot deal with functions without involving methods for example.
> An important note is that it was never my goal to create a minimal PEP. 
> I did not aim for changing as little as possible. I was thinking: we are 
> changing functions, what would be the best way to implement them?

That might be a problem. For the change to be accepted, a core developer 
will need to commit to maintaining the code, understand it, and accept 
responsibility for anything that's broken. Naturally, large-scale 
changes have less of a chance there.

With such a "finished product" PEP, it's hard to see if some of the 
various problems could be solved in a better way -- faster, more 
maintainable, or less disruptive.
It's also harder from a psychological point of view: you obviously 
already put in a lot of good work, and it's harder to waste that work if 
an even better solution is found. (I always tell Marcel to view 
large-scale changes as a hands-on learning experiment -- more likely to 
be thrown away than accepted -- rather than as creating a finished project.)

> The main goal was fixing introspection but a secondary goal was fixing 
> many of the existing warts with functions. Probably this secondary goal 
> will in the end be more important for the general Python community.
> I would argue that my PEP may look complicated, but I'm sure that the 
> end result will be a simpler implementation than we have today. Instead 
> of having four related classes implementing similar functionality 
> (builtin_function_or_method, method, method_descriptor and function), we 
> have just one (base_function). The existing classes like method still 
> exist with my PEP but a lot of the core functionality is implemented in 
> the common base_function.

Is a branching class hierarchy, with quite a few new of flags for 
feature selection, the kind of simplicity we want?
Would it be possible to first decouple things, reducing the complexity, 
and then tackle the individual problems?

> This is really one of the key points: while my PEP *could* be 
> implemented without the base_function class, the resulting code would be 
> far more complicated.
>> Are there parts that can be left to a subsequent PEP, to simplify the
>> document (and implementation)?
> It depends. The current PEP is more or less a finished product. You can 
> of course pick parts of the PEP and implement those, but then those 
> parts will be somewhat meaningless individually.
> But if PEP 575 is accepted "in principle" (you accept the new class 
> hierarchy for functions), then the details could be spread over several 
> PEPs. But those individual PEPs would only make sense in the light of 
> PEP 575.

Well, that's the thing I'm not sure about.
The class hierarchy still makes it hard to decouple the introspection 
side (how functions look on the outside) from the calling mechanism (how 
the calling works internally). It fear that it is replacing complexity 
with a different kind of complexity.
So my main question now is, can this all be *simplified* rather than 
*reorganized*? It's a genuine question ? I don't know, but I feel it 
should be explored more.

> A few small details could be left out, such as METH_BINDING. But that 
> wouldn't yield a significant simplification.
>> It seems to me that the current complexity is (partly) due to the fact
>> that how functions are *called* is tied to how they are *introspected*.
> The *existing* situation is that introspection is totally tied to how 
> functions are called. So I would argue that my PEP improves on that by 
> removing some of those ties by moving __call__ to a common base class.
>> Maybe we can change `inspect` to use duck-typing instead of isinstance?
> That was rejected on https://bugs.python.org/issue30071
>> Then, if built-in functions were subclassable, Cython functions could
>> need to provide appropriate __code__/__defaults__/__kwdefaults__
>> attributes that inspect would pick up.
> Of course, that's possible. I don't think that it would be a *better* 
> solution than my PEP though.
> Essentially, my PEP started from that idea. But then you realize that 
> you'll need to handle not only built-in functions but also method 
> descriptors (unbound methods of extension types). And you'll want to 
> allow __get__ for the new subclasses. For efficiency, you really want to 
> implement __get__ in the base classes (both builtin_function_or_method 
> and method_descriptor) because of optimizations combining __get__ and 
> __call__ (the LOAD_METHOD and CALL_METHOD opcodes). And then you realize 
> that it makes no sense to duplicate all that functionality in both 
> classes. So you add a new base class. You already end up with a major 
> part of my PEP this way.

Starting from an idea and ironing out the details it lets you (and, if 
since you published results, everyone else) figure out the tricky 
details. But ultimately it's exploring one path of doing things ? it 
doesn't necessarily lead to the best way of doing something.

> That still leaves the issue of what inspect.isfunction() should do. 
> Often, "isfunction" is used to check for "has introspection" so you 
> certainly want to allow for custom built-in function classes to satisfy 
> inspect.isfunction(). So you need to involve Python functions too in the 
> class hierarchy. And that's more or less my PEP.

That's a good question. Maybe inspect.isfunction() serves too many use 
cases to be useful. Cython functons should behave like "def" functions 
in some cases, and like built-in functions in others. *Can* a single 
boolean usefully distinguish between these? The current docs are 
unclear, and before we change how inspect.isfunction ultimately behaves, 
I'd like to make its purpose clearer (and try to check how that meshes 
with the current use cases).

What is your ultimate use case? Is it documentation tools like pydoc? If 
we design for them, what other uses of isfunction() will be left out?

I hope this doesn't read as too negative. I'm grateful for the work you 
put in, and it is useful (even in case we do end up settling on a 
different solution).