osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Reading 'scientific' csv using Pandas?


Den 2018-11-18 skrev Shakti Kumar <shakti.shrivastava13 at gmail.com>:
> On Sun, 18 Nov 2018 at 18:18, Martin Sch??n <martin.schoon at gmail.com> wrote:
>>
>> Now I hit a bump in the road when some of the data is not in plain
>> decimal notation (xxx,xx) but in 'scientific' (xx,xxxe-xx) notation.
>>
>
> Martin, I believe this should be done by pandas itself while reading
> the csv file,
> I took an example in scientific notation and checked this out,
>
> my sample.csv file is,
> col1,col2
> 1.1,0
> 10.24e-05,1
> 9.492e-10,2
>
That was a quick answer!

My pandas is up to date.

In your example you use the US convention of using "." for decimals
and "," to separate data. This works perfect for me too.

However, my data files use European conventions: decimal "," and TAB
to separate data:

col1	col2
1,1	0
10,24e-05	1
9,492e-10	2

I use 

EUData = pd.read_csv('file.csv', skiprows=1, sep='\t',
decimal=',', engine='python')

to read from such files. This works so so. 'Common floats' (3,1415 etc)
works just fine but 'scientific' stuff (1,6023e23) does not work.

/Martin