OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 1.10.0beta1 now available for download


Thanks for the pointer! I went through and set this up today, using Google
OAuth as the RBAC provider. Overall I'm quite enthusiastic about this move,
but I thought that it might be helpful to collect feedback as someone who
hasn't been following the overall process and is therefore coming at it
with fresh eyes.

- The Flask appbuilder security documentation is poor quality (e.g.,
there's some broken sentences); if Airflow is to send people there, it
might be worth PRing some of the docs to at least look more professional.

- There's not much documentation out there on how to properly set up an
OAuth app in Google (in my case, using the G+ API). From an adoption POV,
it would be good to screenshot the (current) steps in the process, and
point out which values should be used in which fields on Google. For
example, I had to grep the code base to find the callback URL.

- The initial login UI seems over-complex: you have to click the provider
icon, and then click either login or register. The standard for this
workflow is that you login by clicking the desired provider's icon, and
doing so will register you automatically if you aren't already. In my case
I only have one provider, so this menu was even more confusing.

- It was not clear to me that the "Public" role has absolutely no
permissions. When I set this as the default role and registered, I could no
longer access the site until I cleared cookies. I thought it was an OAuth
error at first, but it turns out the Public role has fewer effective
permissions than an anonymous user; this resulted in a redirect loop
because I could not even view the homepage. I had to correct this in the
database to be able to log in.

- The roles list (at roles/list/ ) is intimidatingly large and hard to
parse. For instance, I couldn't tell at a glance what "user" allows
relative to "viewer". It would be good to have a narrative description of
what each of these roles is intended for, and to present the list of
permissions in a more clustered or diffable way. Permissions lists tend to
only grow, after all.

- A "Viewer" currently lacks enough access to see their own profile.

- "User Statistics" (userstatschartview/chart/) uses the internal name,
rather than firstname/lastname - which in my case is a `google_idnumber`
name. Should probably show both names.

Unrelatedly to RBAC (I think), on this branch on my sandbox instance, tasks
appear to be failing with the only logs present in the UI as:

[{'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True},
{'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True}]


Finally, in case anyone else wanted to test run a similar setup, here is
the webserver_config.py that I ended up using (note that it has Jinja
templating via Ansible):

import os
from airflow import configuration as conf
from flask_appbuilder.security.manager import AUTH_OAUTH
basedir = os.path.abspath(os.path.dirname(__file__))

# The SQLAlchemy connection string.
SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')

# Flask-WTF flag for CSRF
CSRF_ENABLED = True

# The name to display, e.g. "Airflow Staging Sandbox"
APP_NAME = "Airflow {{ env }} {{ app_config | capitalize }}"

# Use OAuth
AUTH_TYPE = AUTH_OAUTH

# Will allow user self registration
AUTH_USER_REGISTRATION = True

# The default user self registration role
AUTH_USER_REGISTRATION_ROLE = "{{ airflow_rbac_registration_role |
default('Viewer') }}"

# Google OAuth:
OAUTH_PROVIDERS = [{
# The name of the provider
'name': 'google',
# The icon to use
'icon': 'fa-google',
# The name of the key that the provider sends
'token_key': 'access_token',
# Just in case, whitelist to only @quantopian.com emails
'whitelist': ['@quantopian.com'],
# Define the remote app:
'remote_app': {
'base_url': 'https://www.googleapis.com/oauth2/v2/',
'access_token_url': 'https://accounts.google.com/o/oauth2/token',
'authorize_url': 'https://accounts.google.com/o/oauth2/auth',
'request_token_url': None,
'request_token_params': {
# Uses the Google+ API, requestingf the 'email' and 'profile' scope
'scope': 'email profile'
},
'consumer_key': '{{ vault_airflow_google_oauth_key }}',
'consumer_secret': '{{ vault_airflow_google_oauth_secret }}'
}
}]



On Mon, Apr 30, 2018 at 12:54 PM, Jørn A Hansen <jornhansen@xxxxxxxxx>
wrote:

> On Mon, 30 Apr 2018 at 15.56, James Meickle <jmeickle@xxxxxxxxxxxxxx>
> wrote:
>
> > Installed this off of the branch, and I do get the Kubernetes executor
> > (incl. demo DAG) and some bug fixes - but I don't see any RBAC feature
> > anywhere I'd think to look. Do I need to set up some config to get that
> to
> > show up?
>
>
> See
> https://github.com/apache/incubator-airflow/blob/v1-10-
> test/UPDATING.md#new-webserver-ui-with-role-based-access-control
>
> It had me left wondering as well - so I decided to go hunt for it in the
> RBAC PR. And there it was :-)
>
> Cheers,
> JornH
>
> >
> >
> > On Mon, Apr 23, 2018 at 2:06 PM, Bolke de Bruin <bdbruin@xxxxxxxxx>
> wrote:
> >
> > > Hi All,
> > >
> > > I am really happy that Fokko and I have created the v1-10-test branch
> and
> > > subsequently build the first beta of Apache Airflow 1.10!
> > >
> > > It is available for testing here:
> > >
> > > https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0beta1/
> > >
> > > Highlights include:
> > >
> > > * New RBAC web interface in beta
> > > * Timezone support
> > > * First class kubernetes operator
> > > * Experimental kubernetes executor
> > > * Documentation improvements
> > > * Performance optimizations for large DAGs
> > > * many GCP and S3 integration improvements
> > > * many new operators
> > > * many many many bug fixes
> > >
> > > We are aiming for a fully compliant Apache release so we should be able
> > to
> > > kick off the graduation process after this release. I hope you help us
> > out
> > > getting there!
> > >
> > > Kind regards,
> > >
> > > Bolke & Fokko
> >
>