osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

ubuntu python 2.7 variations


I help maintain apps on a number of machines and recently a bug surfaced which 
affected some of them using the preppy template module; the bug ended up being 
the way tokenize worked as in the following

#############################
from StringIO import StringIO
import tokenize, token
L = []
s='i'
tokenize.tokenize(StringIO(s.strip()).readline,lambda *a: L.append(a))
print('L=%s' % repr(L))
#############################
Older working machines
Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)\n[GCC 7.3.0] on linux2
L=[(1, 'i', (1, 0), (1, 1), 'i'), (0, '', (2, 0), (2, 0), '')]

upgraded machines showing a bug
Python 2.7.15+ (default, Nov 27 2018, 23:36:35)\n[GCC 7.3.0] on linux2
L=[(1, 'i', (1, 0), (1, 1), 'i'), (4, '', (1, 1), (1, 2), ''), (0, '', (2, 0), 
(2, 0), '')]

The extra token represents a linefeed; however StringIO('i').readline() doesn't 
contain a linefeed.

so the problem is that parsing a simple string 'i' results in two different 
parses. On my Arch machines running 2.7.16 I see the original parse.

The fix was simple and involved using more robust code.

My question is that since the package builders feel able to modify and affect 
behaviour in such a simple case will they do it randomly elsewhere perhaps 
leading to other less obvious changes.

Should I always be using self build python versions?

It seems that ubuntu feels able to provide packages which are rc versions or 
have a + indicating they're modified. They'll probably argue that this improves 
things and I shouldn't be using such low level code ....... :(
--
Robin Becker