osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (ARROW-4057) [Python] Revamp handling of file URIs in pyarrow.parquet


Wes McKinney created ARROW-4057:
-----------------------------------

             Summary: [Python] Revamp handling of file URIs in pyarrow.parquet
                 Key: ARROW-4057
                 URL: https://issues.apache.org/jira/browse/ARROW-4057
             Project: Apache Arrow
          Issue Type: Improvement
          Components: Python
            Reporter: Wes McKinney
             Fix For: 0.13.0


The way this is being handled currently is pretty brittle. If the HDFS cluster being used to run the unit tests does not support writes from {{$USER}} then the tests fail (e.g. the only permissioned user in the docker-compose cluster is "root", so the unit tests cannot be run)

I'm inserting various hacks to get the tests passing for now, but they are temporary. There is code relating to path and URI handling spread throughout the parquet module; it would be much better to centralize and clean this up



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)