[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

For your interest, here are some snapshots of Flink web UI (with angular 7.0) we recently reconstructed.

We choose Angular 7.0 due to the following reasons:
  1. It is easy to migrate the current codebase to Angular 7.0 following the official guide.
  2. It is easy for current Web UI developers who are experienced with Angular 1.x to get familiar with 7.0 reading the official doc.
  3. Migrating other than rewriting is helpful to ensure its functional integrity.
  4. There are over 1 million Angular developers now (2.0 and above, still increasing rapidly), which means there are potentially more developers willing to join the community.
  5. No bower or complex node / webpack configuration needed (only angular cli).
We split Angular 1.x and 7.0 versions into two folders based on the following consideration:
  1. 1.x version could be the default page because there is no breakchange for both users and developers.
  2. Make the upgrade progressive. Users can switch between two versions by clicking the button on the top right corner.
  3. Reduce maintenance costs and keep the performance of web without bootstrap both 1.x and 7.0 versions in the same page. 
  4. Won't interrupt the current development process. The code can still be submitted to 1.x version folder, we could help to sync all features and bug fix to 7.0 version if needed.
  5. When the 7.0 version is ready and stable, it is easy to change default page to it, and remove the old version.


Here is the brief list of what we have done:
  1. Create a new folder named next-web-dashboard in flink-runtime-web.
  2. Migrate all Angular 1.x controllers, services, config etc one by one into 7.0 version with ngMigration.
  3. Modify frontend build scripts in order to package two versions together.
  4. All works listed above are limited in the flink-runtime-web folder. One folder was added and no file or folder was deleted.
The new version of the Web UI is running stable for several months in Alibaba. We are happy to contribute it to the Flink community if everyone like the work.


On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <trohrmann@xxxxxxxxxx> wrote:
I agree with Chesnay that the web UI is already fairly isolated in Flink's
code base (see flink-runtime-web module). Moreover, imo it is very
important that Flink comes with a nice web UI which works out of the box
because it lowers the entrance hurdle massively. What one could argue is to
make it easier to plug in your own web UI if you want to develop it. But I
would see this as an orthogonal issue.

I would actually argue against Robert's proposal to make a grand rewrite if
this is by any means possible. Experience shows that these kind of efforts
are usually set up to fail or take super long. If possible, then I would
rather prefer to convert one component at a time to React. But I'm not an
expert here and cannot tell whether this is possible or not.

What we would need is active help from the community to review these
changes. I think that is the crucial piece to make such an effort work or


On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <chesnay@xxxxxxxxxx>

> The WebUI goes entirely through the REST API.
> I don't see how moving it out would make it easier to develop/contribute.
> The UI is already encapsulated in it's own module (flink-runtime-web)
> and changes can be made to it independently.
> Historically the issue was simply that contributions to the UI were not
> reviewed in a timely manner /  not at all, which mind you
> applies to virtually all components. This also won't change by moving it
> out of Flink.
> Having it in Flink also guarantees that there is an easily-discoverable
> version of the UI that is compatible with your Flink distribution.
> On 31.10.2018 09:53, dyana.rose wrote:
> > Re: who's using the web ui
> >
> > Though many mature solutions, with a fair amount of time/resources
> available are likely running their own front ends, for teams like mine
> which are smaller and aren't focused solely on working with Flink, having
> the web ui available removes a large barrier to getting up and running
> quickly.
> >
> > But, with the web UI being in core Flink, it does make it a bit more of
> a chore to contribute.
> >
> > I'm unaware of how necessary it is for the UI to deploy with Flink? Is
> it making any calls that aren't through the REST API (and if so, can those
> endpoints be added to the REST API)?
> >
> > In general I'd support moving it to its own package, making it easier to
> develop with your standard UI dev tools. This also allows the web UI to be
> developed and released independently to core Flink.
> >
> > Dyana
> >
> > On 2018/10/31 07:47:50, Fabian Wollert <fabian@xxxxxxxxxx> wrote:
> >> Hi Till, I basically agree with all your points. i would stress the
> >> "dustiness" of the current architecture: the package manager used
> (bower)
> >> is deprecated since a long time, the chance for the builds of the flink
> web
> >> dashboard not working anymore is increasing every day.
> >>
> >> About the knowledge in the community: Two days is not a lot of time, but
> >> interest in this topic seems to be minor anyways. Is someone using the
> >> Flink Web Dashboard at all, or is everyone running their own custom
> >> solutions? Because if there is no interest in using the Web UI AND no
> one
> >> interested in developing, there would be no need to package this as
> part of
> >> the official Flink package, but rather develop an independent solution
> (I'm
> >> not voting for this right now, just putting it out), if at all. The
> >> official package could then just ship with the API, which other
> solutions
> >> can build upon. This solution could be from an agile point of view also
> the
> >> best (enhanced testing, independent and more effective dev workflow,
> etc.),
> >> but is bad for the usage of the Flink UI, because people need to install
> >> two things individually (Flink and the web dashboard).
> >>
> >> How did the choice for Angular1 happen back then? Who was writing the
> >> Dashboard in the first place?
> >>
> >> Cheers
> >>
> >> --
> >>
> >>
> >> *Fabian WollertZalando SE*
> >>
> >> E-Mail: fabian@xxxxxxxxxx
> >>
> >>
> >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
> >> trohrmann@xxxxxxxxxx>:
> >>
> >>> Thanks for starting this discussion Fabian! I think our web UI
> technology
> >>> stack is quite dusty by now and it would be beneficial to think about
> its
> >>> technological future.
> >>>
> >>> On the one hand, our current web UI works more or less reliable and
> >>> changing the underlying technology has the risk of breaking things.
> >>> Moreover, there might be the risk that the newly chosen technology
> will be
> >>> deprecated at some point in time as well.
> >>>
> >>> On the other hand, we don't have much Angular 1 knowledge in the
> community
> >>> and extending the web UI is, thus, quite hard at the moment. Maybe by
> using
> >>> some newer web technologies we might be able to attract more people
> with a
> >>> web technology background to join the community.
> >>>
> >>> The lack of people working on the web UI is for me the biggest problem
> I
> >>> would like to address. If there is interest in the web UI, then I'm
> quite
> >>> sure that we will be able to even migrate to some other technology in
> the
> >>> future. The next important issue for me is to do the change
> incrementally
> >>> if possible. Ideally we never break the web UI in the process of
> migrating
> >>> to a new technology. I'm not an expert here so it might or might not be
> >>> possible. But if it is, then we should design the implementation steps
> in
> >>> such a way.
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fabian@xxxxxxxxxx>
> wrote:
> >>>
> >>>> Hi everyone,
> >>>>
> >>>> in this email thread
> >>>> <
> >>>>
> >>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
> >>>> and the tickets FLINK-10705
> >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and FLINK-10706
> >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
> came
> >>> up
> >>>> whether to change the underlying architecture of Flink's Web Dashboard
> >>> from
> >>>> Angular1 to something else. This email thread should be solely to
> discuss
> >>>> the pro's and con's of this, and what could be the target
> architecture.
> >>>>
> >>>> My choice would be React. Personally I agree with Till's comments in
> the
> >>>> ticket, Angular 1 being basically outdated and is not having a large
> >>>> following anymore. From my experience the choice between Angular 2-7
> or
> >>>> React is subjective, you can get things done with both. I personally
> only
> >>>> have experience with React, so I  personally would be faster to
> develop
> >>>> with this one. I currently have not planned to learn Angular as well
> >>> (being
> >>>> a more backend focused developer in general) so if the decision would
> be
> >>> to
> >>>> go with Angular, i would be unfortunately out of this rework of the
> Flink
> >>>> Dashboard most certainly.
> >>>>
> >>>> Additionally i would like to get rid of bower, since its officially
> >>>> deprecated <
> https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
> >>>> my
> >>>> idea would be to just use a create-react-app package with npm and
> webpack
> >>>> under the hood. no need for additional lib's here imho. But again:
> thats
> >>>> mostly what i've been working with recently, so thats a subjective
> >>> point. I
> >>>> could imagine getting used to yarn in the future as well.
> >>>>
> >>>> Cheers
> >>>> Fabian
> >>>>
> >>>> --
> >>>>
> >>>>
> >>>> *Fabian WollertZalando SE*
> >>>>
> >>>> E-Mail: fabian@xxxxxxxxxx
> >>>>

Yadong Xie
Ministry of Education Key Lab for Intelligent Networks and Network Security
Dept. of Computer Science and Technology
School of electronic information and engineering
Xi'an Jiaotong University
Xi'an, 710049, P.R. China