[Python-Dev] The future of the wchar_t cache
On 22Oct2018 0928, Victor Stinner wrote:
>> Also, I'm
>> proposing keeping the 'kind' as UCS-2 when the string is created from
>> UCS-2 data that is likely to be used as UCS-2.
> Oh. That's a major change in the PEP 393 design. You would have to
> modify many functions in CPython. Currently, the PEP 393 requires that
> a string always use the most efficient storage, and many optimizations
> and code paths rely on that assumptions.
I don't know that it requires that many modifications - those functions
already have to handle UCS-2 content anyway (e.g. if I get a path from
scandir() that includes a non-ASCII character), and they're only using
the assumption of most efficient storage to determine the resulting
storage size of a string operation (which I'm proposing should also be
UCS-2 when the source strings are UCS-2, since that's the best indicator
we have that it'll be used as UCS-2 later, as well as being the current
implementation :) ).
> I'm against this change.
> Moreover, it's hard to guess how a string will be used later...
Agreed. There are some heuristics we can use, but it's definitely only a
guess. That's the nature of this problem - guessing that it *won't* be
used as UCS-2 later on is also a guess.