osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [JAVA] Arrow performance measurement


Hi all - quick update on the performance investigation:

- I spent some time looking at performance profile for a binary blob column
(1024 bytes of byte[]) and found a few favorable settings for delivering up
to 168 Gbps from in-memory reading benchmark on 16 cores. These settings
(NUMA, JVM settings, Arrow holder API, and batch size, etc.) are documented
here:
https://github.com/animeshtrivedi/blog/blob/master/post/2018-10-03-arrow-binary.md
- these setting also help to improved the last number that reported (but
not by much) for the in-memory TPC-DS store_sales table from ~39 Gbps up to
~45-47 Gbps (note: this number is just in-memory benchmark, i.e., w/o any
networking or storage links)

A few follow up questions that I have:
1. Arrow reads a batch size worth of data in one go. Are there any
recommended batch sizes? In my investigation, small batch size help with a
better cache profile but increase number of instructions required (more
looping). Larger one do otherwise. Somehow ~10MB/thread seem to be the best
performing configuration, which is also a bit counter intuitive as for 16
threads this will lead to 160 MB of memory footprint. May be this is also
tired to the memory management logic which is my next question.
2. Arrow use's netty's memory manager. (i) what are decent netty memory
management settings for "io.netty.allocator.*" parameters? I don't find any
decent write-up on them; (ii) Is there a provision for ArrowBuf being
re-used once a batch is consumed? As it looks for now, read read allocates
a new buffer to read the whole batch size.
3. Are there examples of Arrow in C++ read/write code that I can have a
look?

Cheers,
--
Animesh


On Wed, Sep 19, 2018 at 8:49 PM Wes McKinney <wesmckinn@xxxxxxxxx> wrote:

> On Wed, Sep 19, 2018 at 2:13 PM Animesh Trivedi
> <animesh.trivedi@xxxxxxxxx> wrote:
> >
> > Hi Johan, Wes, and Jacques - many thanks for your comments:
> >
> > @Johan -
> > 1. I also do not suspect that there is any inherent drawback in Java or
> C++
> > due to the Arrow format. I mentioned C++ because Wes pointed out that
> Java
> > routines are not the most optimized ones (yet!). And naturally one would
> > expect better performance in a native language with all
> pointer/memory/SIMD
> > instruction optimizations that you mentioned. As far as I know, the
> > off-heap buffers are managed in ArrowBuf which implements an abstract
> netty
> > class. But there is nothing unusual, i.e., netty specific, about these
> > unsafe routines, they are used by many projects. Though there is cost
> > associated with materializing on-heap Java values from off-heap memory
> > regions. I need to benchmark that more carefully.
> >
> > 2. When you say "I've so far always been able to get similar performance
> > numbers" - do you mean the same performance of my case 3 where 16 cores
> > drive close to 40 Gbps, or the same performance between your C++ and Java
> > benchmarks. Do you have some write-up? I would be interested to read up
> :)
> >
> > 3. "Can you get to 100 Gbps starting from primitive arrays in Java" ->
> that
> > is a good idea. Let me try and report back.
> >
> > @Wes -
> >
> > Is there some benchmark template for C++ routines I can have a look? I
> > would be happy to get some input from Java-Arrow experts on how to write
> > these benchmarks more efficiently. I will have a closer look at the JIRA
> > tickets that you mentioned.
> >
> > So, for now I am focusing on the case 3, which is about establishing
> > performance when reading from a local in-memory I/O stream that I
> > implemented (
> >
> https://github.com/animeshtrivedi/benchmarking-arrow/blob/master/src/main/java/com/github/animeshtrivedi/benchmark/MemoryIOChannel.java
> ).
> > In this case I first read data from parquet files, convert them into
> Arrow,
> > and write-out to a MemoryIOChannel, and then read back from it. So, the
> > performance has nothing to do with Crail or HDFS in the case 3. Once, I
> > establish the base performance in this setup (which is around ~40 Gbps
> with
> > 16 cores) I will add Crail to the mix. Perhaps Crail I/O streams can take
> > ArrowBuf as src/dst buffers. That should be doable.
>
> Right, in any case what you are testing is the performance of using a
> particular Java accessor layer to JVM off-heap Arrow memory to sum the
> non-null values of each column. I'm not sure that a single bandwidth
> number produced by this benchmark is very informative for people
> contemplating what memory format to use in their system due to the
> current state of the implementation (Java) and workload measured
> (summing the non-null values with a naive algorithm). I would guess
> that a C++ version with raw pointers and a loop-unrolled, branch-free
> vectorized sum is going to be a lot faster.
>
> >
> > @Jacques -
> >
> > That is a good point that "Arrow's implementation is more focused on
> > interacting with the structure than transporting it". However, in any
> > distributed system one needs to move data/structure around - as far as I
> > understand that is another goal of the project. My investigation started
> > within the context of Spark/SQL data processing. Spark converts incoming
> > data into its own in-memory UnsafeRow representation for processing. So
> > naturally the performance of this data ingestion pipeline cannot
> outperform
> > the read performance of the used file format. I benchmarked Parquet, ORC,
> > Avro, JSON (for the specific TPC-DS store_sales table). And then
> curiously
> > benchmarked Arrow as well because its design choices are a better fit for
> > modern high-performance RDMA/NVMe/100+Gbps devices I am investigating.
> From
> > this point of view, I am trying to find out can Arrow be the file format
> > for the next generation of storage/networking devices (see Apache Crail
> > project) delivering close to the hardware speed reading/writing rates. As
> > Wes pointed out that a C++ library implementation should be memory-IO
> > bound, so what would it take to deliver the same performance in Java ;)
> > (and then, from across the network).
> >
> > I hope this makes sense.
> >
> > Cheers,
> > --
> > Animesh
> >
> > On Wed, Sep 19, 2018 at 6:28 PM Jacques Nadeau <jacques@xxxxxxxxxx>
> wrote:
> >
> > > My big question is what is the use case and how/what are you trying to
> > > compare? Arrow's implementation is more focused on interacting with the
> > > structure than transporting it. Generally speaking, when we're working
> with
> > > Arrow data we frequently are just interacting with memory locations and
> > > doing direct operations. If you have a layer that supports that type of
> > > semantic, create a movement technique that depends on that. Arrow
> doesn't
> > > force a particular API since the data itself is defined by its
> in-memory
> > > layout so if you have a custom use or pattern, just work with the
> in-memory
> > > structures.
> > >
> > >
> > >
> > > On Wed, Sep 19, 2018 at 7:49 AM Wes McKinney <wesmckinn@xxxxxxxxx>
> wrote:
> > >
> > > > hi Animesh,
> > > >
> > > > Per Johan's comments, the C++ library is essentially going to be
> > > > IO/memory bandwidth bound since you're interacting with raw pointers.
> > > >
> > > > I'm looking at your code
> > > >
> > > > private void consumeFloat4(FieldVector fv) {
> > > >     Float4Vector accessor = (Float4Vector) fv;
> > > >     int valCount = accessor.getValueCount();
> > > >     for(int i = 0; i < valCount; i++){
> > > >         if(!accessor.isNull(i)){
> > > >             float4Count+=1;
> > > >             checksum+=accessor.get(i);
> > > >         }
> > > >     }
> > > > }
> > > >
> > > > You'll want to get a Java-Arrow expert from Dremio to advise you the
> > > > fastest way to iterate over this data -- my understanding is that
> much
> > > > code in Dremio interacts with the wrapped Netty ArrowBuf objects
> > > > rather than going through the higher level APIs. You're also dropping
> > > > performance because memory mapping is not yet implemented in Java,
> see
> > > > https://issues.apache.org/jira/browse/ARROW-3191.
> > > >
> > > > Furthermore, the IPC reader class you are using could be made more
> > > > efficient. I described the problem in
> > > > https://issues.apache.org/jira/browse/ARROW-3192 -- this will be
> > > > required as soon as we have the ability to do memory mapping in Java
> > > >
> > > > Could Crail use the Arrow data structures in its runtime rather than
> > > > copying? If not, how are Crail's runtime data structures different?
> > > >
> > > > - Wes
> > > > On Wed, Sep 19, 2018 at 9:19 AM Johan Peltenburg - EWI
> > > > <J.W.Peltenburg@xxxxxxxxxx> wrote:
> > > > >
> > > > > Hello Animesh,
> > > > >
> > > > >
> > > > > I browsed a bit in your sources, thanks for sharing. We have
> performed
> > > > some similar measurements to your third case in the past for C/C++ on
> > > > collections of various basic types such as primitives and strings.
> > > > >
> > > > >
> > > > > I can say that in terms of consuming data from the Arrow format
> versus
> > > > language native collections in C++, I've so far always been able to
> get
> > > > similar performance numbers (e.g. no drawbacks due to the Arrow
> format
> > > > itself). Especially when accessing the data through Arrow's raw data
> > > > pointers (and using for example std::string_view-like constructs).
> > > > >
> > > > > In C/C++ the fast data structures are engineered in such a way
> that as
> > > > little pointer traversals are required and they take up an as small
> as
> > > > possible memory footprint. Thus each memory access is relatively
> > > efficient
> > > > (in terms of obtaining the data of interest). The same can
> absolutely be
> > > > said for Arrow, if not even more efficient in some cases where object
> > > > fields are of variable length.
> > > > >
> > > > >
> > > > > In the JVM case, the Arrow data is stored off-heap. This requires
> the
> > > > JVM to interface to it through some calls to Unsafe hidden under the
> > > Netty
> > > > layer (but please correct me if I'm wrong, I'm not an expert on
> this).
> > > > Those calls are the only reason I can think of that would degrade the
> > > > performance a bit compared to a pure JAva case. I don't know if the
> > > Unsafe
> > > > calls are inlined during JIT compilation. If they aren't they will
> > > increase
> > > > access latency to any data a little bit.
> > > > >
> > > > >
> > > > > I don't have a similar machine so it's not easy to relate my
> numbers to
> > > > yours, but if you can get that data consumed with 100 Gbps in a pure
> Java
> > > > case, I don't see any reason (resulting from Arrow format / off-heap
> > > > storage) why you wouldn't be able to get at least really close. Can
> you
> > > get
> > > > to 100 Gbps starting from primitive arrays in Java with your
> consumption
> > > > functions in the first place?
> > > > >
> > > > >
> > > > > I'm interested to see your progress on this.
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > >
> > > > > Johan Peltenburg
> > > > >
> > > > > ________________________________
> > > > > From: Animesh Trivedi <animesh.trivedi@xxxxxxxxx>
> > > > > Sent: Wednesday, September 19, 2018 2:08:50 PM
> > > > > To: dev@xxxxxxxxxxxxxxxx; dev@xxxxxxxxxxxxxxxx
> > > > > Subject: [JAVA] Arrow performance measurement
> > > > >
> > > > > Hi all,
> > > > >
> > > > > A week ago, Wes and I had a discussion about the performance of the
> > > > > Arrow/Java implementation on the Apache Crail (Incubating) mailing
> > > list (
> > > > >
> http://mail-archives.apache.org/mod_mbox/crail-dev/201809.mbox/browser
> > > ).
> > > > In
> > > > > a nutshell: I am investigating the performance of various file
> formats
> > > > > (including Arrow) on high-performance NVMe and RDMA/100Gbps/RoCE
> > > setups.
> > > > I
> > > > > benchmarked how long does it take to materialize values (ints,
> longs,
> > > > > doubles) of the store_sales table, the largest table in the TPC-DS
> > > > dataset
> > > > > stored on different file formats. Here is a write-up on this -
> > > > > https://crail.incubator.apache.org/blog/2018/08/sql-p1.html. I
> found
> > > > that
> > > > > between a pair of machine connected over a 100 Gbps link, Arrow
> (using
> > > > as a
> > > > > file format on HDFS) delivered close to ~30 Gbps of bandwidth with
> all
> > > 16
> > > > > cores engaged. Wes pointed out that (i) Arrow is in-memory IPC
> format,
> > > > and
> > > > > has not been optimized for storage interfaces/APIs like HDFS; (ii)
> the
> > > > > performance I am measuring is for the java implementation.
> > > > >
> > > > > Wes, I hope I summarized our discussion correctly.
> > > > >
> > > > > That brings us to this email where I promised to follow up on the
> Arrow
> > > > > mailing list to understand and optimize the performance of
> Arrow/Java
> > > > > implementation on high-performance devices. I wrote a small
> stand-alone
> > > > > benchmark (https://github.com/animeshtrivedi/benchmarking-arrow)
> with
> > > > three
> > > > > implementations of WritableByteChannel, SeekableByteChannel
> interfaces:
> > > > >
> > > > > 1. Arrow data is stored in HDFS/tmpfs - this gives me ~30 Gbps
> > > > performance
> > > > > 2. Arrow data is stored in Crail/DRAM - this gives me ~35-36 Gbps
> > > > > performance
> > > > > 3. Arrow data is stored in on-heap byte[] - this gives me ~39 Gbps
> > > > > performance
> > > > >
> > > > > I think the order makes sense. To better understand the
> performance of
> > > > > Arrow/Java we can focus on the option 3.
> > > > >
> > > > > The key question I am trying to answer is "what would it take for
> > > > > Arrow/Java to deliver 100+ Gbps of performance"? Is it possible? If
> > > yes,
> > > > > then what is missing/or mis-interpreted by me? If not, then where
> is
> > > the
> > > > > performance lost? Does anyone have any performance measurements
> for C++
> > > > > implementation? if they have seen/expect better numbers.
> > > > >
> > > > > As a next step, I will profile the read path of Arrow/Java for the
> > > option
> > > > > 3. I will report my findings.
> > > > >
> > > > > Any thoughts and feedback on this investigation are very welcome.
> > > > >
> > > > > Cheers,
> > > > > --
> > > > > Animesh
> > > > >
> > > > > PS~ Cross-posting on the dev@xxxxxxxxxxxxxxxx list as well.
> > > >
> > >
>