Re: Assignment from a list is slow in Numarray

Here's another possible I/O approach which uses numarray.strings to
create and evaluate a CharArray using the new for 1.1 method,
fasteval().  This method is dependent on fixed length fields of data and
doesn't currently handle 64-bit integer types or complex numbers the way
I'd like,  but it may be useful for your case.

I created my test file like this:

>>> for i in range(10**5):
...     f.write("%10d %10d %10d\n" % (i, i, i))
...


Then I created a CharArray from the file like this:

>>> import numarray.strings as str
>>> f = open("test.dat", "r")
>>> c = str.fromfile(f, shape=(10**5, 3), itemsize = 11)
>>> c
CharArray([['         0', '         0', '         0'],
           ['         1', '         1', '         1'],
           ['         2', '         2', '         2'],
           ...,
           ['     99997', '     99997', '     99997'],
           ['     99998', '     99998', '     99998'],
           ['     99999', '     99999', '     99999']])

Finally, I converted it into a NumArray, with reasonable performance,
like this:

>>> n = c.fasteval(type=Int32)
>>> n
array([[    0,     0,     0],
       [    1,     1,     1],
       [    2,     2,     2],
       ...,
       [99997, 99997, 99997],
       [99998, 99998, 99998],
       [99999, 99999, 99999]])

Hope this helps,
Todd





-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM.
Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php