osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard


+1 to keep the web UI stack to Angular. This will make the stack upgrading
much easier and won't break the on-going efforts already made towards the
improvements of Flink Web UI.

Till, Robert, and Fabian,
Please take a look at the upgrading plan from Angular 1.x to 7.0 proposed
by Yadong. Any feedback is appreciated.

Regards,
Shaoxuan


On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vthinkxie@xxxxxxxxx> wrote:

> For your interest, here are some snapshots of Flink web UI (with angular
> 7.0) we recently reconstructed.
> [image: snapshot.001.jpeg]
>
> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing
>
> We choose Angular 7.0 due to the following reasons:
>
>    1. It is easy to migrate the current codebase to Angular 7.0 following
>    the official guide.
>    2. It is easy for current Web UI developers who are experienced with
>    Angular 1.x to get familiar with 7.0 reading the official doc.
>    3. Migrating other than rewriting is helpful to ensure its functional
>    integrity.
>    4. There are over 1 million Angular developers now (2.0 and above,
>    still increasing rapidly), which means there are potentially more
>    developers willing to join the community.
>    5. No bower or complex node / webpack configuration needed (only
>    angular cli).
>
> We split Angular 1.x and 7.0 versions into two folders based on the
> following consideration:
>
>    1. 1.x version could be the default page because there is no
>    breakchange for both users and developers.
>    2. Make the upgrade progressive. Users can switch between two versions
>    by clicking the button on the top right corner.
>    3. Reduce maintenance costs and keep the performance of web without
>    bootstrap both 1.x and 7.0 versions in the same page.
>    4. Won't interrupt the current development process. The code can still
>    be submitted to 1.x version folder, we could help to sync all features and
>    bug fix to 7.0 version if needed.
>    5. When the 7.0 version is ready and stable, it is easy to change
>    default page to it, and remove the old version.
>
> [image: test.jpg]
>
>
> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing
>
> Here is the brief list of what we have done:
>
>    1. Create a new folder named next-web-dashboard in flink-runtime-web.
>    2. Migrate all Angular 1.x controllers, services, config etc one by
>    one into 7.0 version with ngMigration.
>    3. Modify frontend build scripts in order to package two versions
>    together.
>    4. All works listed above are limited in the flink-runtime-web folder.
>    One folder was added and no file or folder was deleted.
>
> The new version of the Web UI is running stable for several months in
> Alibaba. We are happy to contribute it to the Flink community if everyone
> like the work.
>
> Best
> Yadong
>
> On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <trohrmann@xxxxxxxxxx>
> wrote:
>
>> I agree with Chesnay that the web UI is already fairly isolated in Flink's
>> code base (see flink-runtime-web module). Moreover, imo it is very
>> important that Flink comes with a nice web UI which works out of the box
>> because it lowers the entrance hurdle massively. What one could argue is
>> to
>> make it easier to plug in your own web UI if you want to develop it. But I
>> would see this as an orthogonal issue.
>>
>> I would actually argue against Robert's proposal to make a grand rewrite
>> if
>> this is by any means possible. Experience shows that these kind of efforts
>> are usually set up to fail or take super long. If possible, then I would
>> rather prefer to convert one component at a time to React. But I'm not an
>> expert here and cannot tell whether this is possible or not.
>>
>> What we would need is active help from the community to review these
>> changes. I think that is the crucial piece to make such an effort work or
>> fail.
>>
>> Cheers,
>> Till
>>
>> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler <chesnay@xxxxxxxxxx>
>> wrote:
>>
>> > The WebUI goes entirely through the REST API.
>> >
>> > I don't see how moving it out would make it easier to
>> develop/contribute.
>> >
>> > The UI is already encapsulated in it's own module (flink-runtime-web)
>> > and changes can be made to it independently.
>> > Historically the issue was simply that contributions to the UI were not
>> > reviewed in a timely manner /  not at all, which mind you
>> > applies to virtually all components. This also won't change by moving it
>> > out of Flink.
>> >
>> > Having it in Flink also guarantees that there is an easily-discoverable
>> > version of the UI that is compatible with your Flink distribution.
>> >
>> > On 31.10.2018 09:53, dyana.rose wrote:
>> > > Re: who's using the web ui
>> > >
>> > > Though many mature solutions, with a fair amount of time/resources
>> > available are likely running their own front ends, for teams like mine
>> > which are smaller and aren't focused solely on working with Flink,
>> having
>> > the web ui available removes a large barrier to getting up and running
>> > quickly.
>> > >
>> > > But, with the web UI being in core Flink, it does make it a bit more
>> of
>> > a chore to contribute.
>> > >
>> > > I'm unaware of how necessary it is for the UI to deploy with Flink? Is
>> > it making any calls that aren't through the REST API (and if so, can
>> those
>> > endpoints be added to the REST API)?
>> > >
>> > > In general I'd support moving it to its own package, making it easier
>> to
>> > develop with your standard UI dev tools. This also allows the web UI to
>> be
>> > developed and released independently to core Flink.
>> > >
>> > > Dyana
>> > >
>> > > On 2018/10/31 07:47:50, Fabian Wollert <fabian@xxxxxxxxxx> wrote:
>> > >> Hi Till, I basically agree with all your points. i would stress the
>> > >> "dustiness" of the current architecture: the package manager used
>> > (bower)
>> > >> is deprecated since a long time, the chance for the builds of the
>> flink
>> > web
>> > >> dashboard not working anymore is increasing every day.
>> > >>
>> > >> About the knowledge in the community: Two days is not a lot of time,
>> but
>> > >> interest in this topic seems to be minor anyways. Is someone using
>> the
>> > >> Flink Web Dashboard at all, or is everyone running their own custom
>> > >> solutions? Because if there is no interest in using the Web UI AND no
>> > one
>> > >> interested in developing, there would be no need to package this as
>> > part of
>> > >> the official Flink package, but rather develop an independent
>> solution
>> > (I'm
>> > >> not voting for this right now, just putting it out), if at all. The
>> > >> official package could then just ship with the API, which other
>> > solutions
>> > >> can build upon. This solution could be from an agile point of view
>> also
>> > the
>> > >> best (enhanced testing, independent and more effective dev workflow,
>> > etc.),
>> > >> but is bad for the usage of the Flink UI, because people need to
>> install
>> > >> two things individually (Flink and the web dashboard).
>> > >>
>> > >> How did the choice for Angular1 happen back then? Who was writing the
>> > >> Dashboard in the first place?
>> > >>
>> > >> Cheers
>> > >>
>> > >> --
>> > >>
>> > >>
>> > >> *Fabian WollertZalando SE*
>> > >>
>> > >> E-Mail: fabian@xxxxxxxxxx
>> > >>
>> > >>
>> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann <
>> > >> trohrmann@xxxxxxxxxx>:
>> > >>
>> > >>> Thanks for starting this discussion Fabian! I think our web UI
>> > technology
>> > >>> stack is quite dusty by now and it would be beneficial to think
>> about
>> > its
>> > >>> technological future.
>> > >>>
>> > >>> On the one hand, our current web UI works more or less reliable and
>> > >>> changing the underlying technology has the risk of breaking things.
>> > >>> Moreover, there might be the risk that the newly chosen technology
>> > will be
>> > >>> deprecated at some point in time as well.
>> > >>>
>> > >>> On the other hand, we don't have much Angular 1 knowledge in the
>> > community
>> > >>> and extending the web UI is, thus, quite hard at the moment. Maybe
>> by
>> > using
>> > >>> some newer web technologies we might be able to attract more people
>> > with a
>> > >>> web technology background to join the community.
>> > >>>
>> > >>> The lack of people working on the web UI is for me the biggest
>> problem
>> > I
>> > >>> would like to address. If there is interest in the web UI, then I'm
>> > quite
>> > >>> sure that we will be able to even migrate to some other technology
>> in
>> > the
>> > >>> future. The next important issue for me is to do the change
>> > incrementally
>> > >>> if possible. Ideally we never break the web UI in the process of
>> > migrating
>> > >>> to a new technology. I'm not an expert here so it might or might
>> not be
>> > >>> possible. But if it is, then we should design the implementation
>> steps
>> > in
>> > >>> such a way.
>> > >>>
>> > >>> Cheers,
>> > >>> Till
>> > >>>
>> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert <fabian@xxxxxxxxxx>
>> > wrote:
>> > >>>
>> > >>>> Hi everyone,
>> > >>>>
>> > >>>> in this email thread
>> > >>>> <
>> > >>>>
>> > >>>
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html
>> > >>>> and the tickets FLINK-10705
>> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and
>> FLINK-10706
>> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the discussion
>> > came
>> > >>> up
>> > >>>> whether to change the underlying architecture of Flink's Web
>> Dashboard
>> > >>> from
>> > >>>> Angular1 to something else. This email thread should be solely to
>> > discuss
>> > >>>> the pro's and con's of this, and what could be the target
>> > architecture.
>> > >>>>
>> > >>>> My choice would be React. Personally I agree with Till's comments
>> in
>> > the
>> > >>>> ticket, Angular 1 being basically outdated and is not having a
>> large
>> > >>>> following anymore. From my experience the choice between Angular
>> 2-7
>> > or
>> > >>>> React is subjective, you can get things done with both. I
>> personally
>> > only
>> > >>>> have experience with React, so I  personally would be faster to
>> > develop
>> > >>>> with this one. I currently have not planned to learn Angular as
>> well
>> > >>> (being
>> > >>>> a more backend focused developer in general) so if the decision
>> would
>> > be
>> > >>> to
>> > >>>> go with Angular, i would be unfortunately out of this rework of the
>> > Flink
>> > >>>> Dashboard most certainly.
>> > >>>>
>> > >>>> Additionally i would like to get rid of bower, since its officially
>> > >>>> deprecated <
>> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>.
>> > >>>> my
>> > >>>> idea would be to just use a create-react-app package with npm and
>> > webpack
>> > >>>> under the hood. no need for additional lib's here imho. But again:
>> > thats
>> > >>>> mostly what i've been working with recently, so thats a subjective
>> > >>> point. I
>> > >>>> could imagine getting used to yarn in the future as well.
>> > >>>>
>> > >>>> Cheers
>> > >>>> Fabian
>> > >>>>
>> > >>>> --
>> > >>>>
>> > >>>>
>> > >>>> *Fabian WollertZalando SE*
>> > >>>>
>> > >>>> E-Mail: fabian@xxxxxxxxxx
>> > >>>>
>> >
>> >
>>
>
>
> --
> Yadong Xie
> Ministry of Education Key Lab for Intelligent Networks and Network Security
> Dept. of Computer Science and Technology
> School of electronic information and engineering
> Xi'an Jiaotong University
> Xi'an, 710049, P.R. China
> Email: vthinkxie@xxxxxxxxx
>