osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Efficient Pandas serialization for mixed object and numeric DataFrames


How are you serializing the dataframe? If you use *pyarrow.serialize(df)*,
then each column should be serialized separately and numeric columns will
be handled efficiently.

On Thu, Oct 18, 2018 at 9:10 PM Mitar <mmitar@xxxxxxxxx> wrote:

> Hi!
>
> It seems that if a DataFrame contains both numeric and object columns,
> the whole DataFrame is pickled and not that only object columns are
> pickled? Is this right? Are there any plans to improve this?
>
>
> Mitar
>
> --
> http://mitar.tnode.com/
> https://twitter.com/mitar_m
>