OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Flink Cluster Overview Dashboard Improvement Proposal


Hi everyone, thx for all the comments and feedback. Let me address
everything individually:

@Till: yes, for the start my plan would be to just touch the
flink-runtime-web/web-dashboard repo/folder.

@Jin Sun:

   - smaller icons on increasing server counts: yes, thats also something i
   already thought about. will keep it in mind when realizing the first
   version!
   - about locality: i searched quickly through the docs, but i could not
   find anything regarding flink featuring rack awareness. Is this something
   already implemented? If not, i think this will bloat the size of this
   initial proposal. If its somewhere already included, we could implement it
   for sure.

@Zhijiang:the focus of this redesign was not yet including the job list in
the lower half of the overview. as part of the redesign we can also think
about optimising this list though, and removing unnecessary columns is
usually the most easy thing to do. we can maybe create a separate ticket
for this as well and discuss this issue there, to not bloat the initial
discussion with too much topics.

@Robert:

   - Agreed that it might make sense to also show this on job level. Since
   these metrics are probably gonna be introduced later only anyways, we can
   discuss this maybe then separately after FLINK-9050
   <https://issues.apache.org/jira/browse/FLINK-9050> (linked the wrong
   ticket in my initial mail) is done.
   - Rewriting the whole thing while doing this also came to my mind. What
   i would like to do anyways (even if we stick for now to A1) is to remove
   bower as a package manager (since its deprecated) and update bootstrap to
   V4. I will check what the additional effort is to move to React/Redux.
   We're working with this here at work as well, so implementing at least a
   first MVP might be feasible as well, before getting to deep into A1
   specifics. But that basically means that you guys are open to change the
   underlying web/JS technology, yeah?

Cheers

--


*Fabian WollertZalando SE*

E-Mail: fabian@xxxxxxxxxx


Am Mi., 10. Okt. 2018 um 08:41 Uhr schrieb Robert Metzger <
rmetzger@xxxxxxxxxx>:

> Hey Fabian,
> thanks a lot for reaching out to the Flink community with this proposal!
> (Posting to the ML instead of creating a JIRA is a good idea for such
> questions -- you can create a ticket/tickets once the discussion here has
> come to a conclusion)
>
> I have two comments:
> - You are listing Records/Kb in and Records/Kb out as cluster-wide metrics.
> I wonder whether we should rather show these metrics for each job, instead
> of the entire cluster? (or maybe both). My concern is that the cluster-wide
> metric is not really relevant as soon as you have jobs with different
> characteristics running on one cluster
> - You mention that the Flink UI is based on Angular 1. I've been thinking
> for quite a while now whether we should actually rewrite / migrate the
> Flink UI to React.
> Do you think we can re-use most of the work you'd be doing for this change
> when we migrate to React?
>
> Best,
> Robert
>
>
>
> On Wed, Oct 10, 2018 at 8:24 AM Zhijiang(wangzhijiang999)
> <wangzhijiang999@xxxxxxxxxx.invalid> wrote:
>
> > Thanks Fabian for proposing this topic.
> >
> > It is very worth improving the web dashborad for showing more useful
> > informations which can benefit flink users a lot.
> >
> > Just two small personal concerns:
> > 1. The start time and end time are already given, so it is easy to
> > estimate the rough duration time. Is it necessary to show the duration
> > information to occupy the space?
> > 2. The job name given by users can be used for identification, and the
> > job id is automatically generated in random. I am not sure whether this
> id
> > is useful for further debugging. If not maybe we can ignore the job id
> from
> > the dashboard?
> >
> > Best,
> > Zhijiang
> >
> > ------------------------------------------------------------------
> > 发件人:Jin Sun <isunjin@xxxxxxxxx>
> > 发送时间:2018年10月10日(星期三) 01:10
> > 收件人:dev <dev@xxxxxxxxxxxxxxxx>
> > 主 题:Re: [DISCUSS] Flink Cluster Overview Dashboard Improvement Proposal
> >
> > Great job! That would very helpful for debug.
> >
> >
> >    - I would suggest to use small icons for this Job Manager/Managers
> >    when there are too many instances (like a thousand)
> >    - May be we can also introduce locality,  that task managers belongs
> >    to same rack shows together?
> >
> >
> >
> >
> >
> > Small icons can be like this:
> >
> >
> >
> >
> > On Oct 9, 2018, at 8:49 PM, Till Rohrmann <trohrmann@xxxxxxxxxx> wrote:
> >
> > mation on the front
> > page. Your mock looks really promising to me since it shows some basic
> > metrics and cluster information at a glance. Apart from the the source
> > input and sink output metrics, all other required information should be
> > available to display it in the dashboard. Thus, your proposal should only
> > affect flink-runtime-web which should make it easier to realize.
> >
> > I'm in favour of adding this feature to Flink's dashboard to make it
> > available to the whole community.
> >
> >
> >
>