OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Building and visualizing the Beam SQL graph


I was also wondering the same thing. I don't think there is any visualization tool for Beam. :(

On Mon, Jun 11, 2018 at 11:39 AM Andrew Pilloud <apilloud@xxxxxxxxxx> wrote:
We are currently converting the Calcite Rel tree to Beam by recursively building a tree of nested PTransforms. This results in a weird nested graph in the dataflow UI where each node contains its inputs nested inside of it. I'm going to change the internal data structure for converting the tree from a PTransform to a PCollection, which will result in a more accurate representation of the tree structure being built and should simplify the code as well. This will not change the public interface to SQL, which will remain a PTransform. Any thoughts or objections?

I was also wondering if there are tools for visualizing the Beam graph aside from the dataflow runner UI. What other tools exist?

Andrew