osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 1.10.0beta1 now available for download


Hi Jakob,

Understood. But isn’t that in this case not just wording? Ie. this is a tar-ball that we think is beyond just developer testing (alpha) but more towards the enthusiasts (beta) but not a version of the tarball that is for the general public to test (RC) and not a Release (release)? Ie. is the issue in calling it a ‘release’ which in this case is just meta for a tarball? In the original email in never mentioned the word release in conjunction with the beta I think.

Cheers
Bolke


> On 1 May 2018, at 22:01, Jakob Homan <jghoman@xxxxxxxxx> wrote:
> 
> Hey all-
>  With my Mentor hat on, I need to point out that ASF doesn't really
> have beta releases.  This work is awesome, but really needs to go
> through the proper steps.  The Release Candidate process is pretty
> well described:
> https://incubator.apache.org/policy/incubation.html#releases.  This is
> particularly important since, as was mentioned, graduation should be
> imminent and this process will be heavily scrutinized.
> 
> -Jakob
> 
> On 1 May 2018 at 12:41, James Meickle <jmeickle@xxxxxxxxxxxxxx> wrote:
>> Thanks for the pointer! I went through and set this up today, using Google
>> OAuth as the RBAC provider. Overall I'm quite enthusiastic about this move,
>> but I thought that it might be helpful to collect feedback as someone who
>> hasn't been following the overall process and is therefore coming at it
>> with fresh eyes.
>> 
>> - The Flask appbuilder security documentation is poor quality (e.g.,
>> there's some broken sentences); if Airflow is to send people there, it
>> might be worth PRing some of the docs to at least look more professional.
>> 
>> - There's not much documentation out there on how to properly set up an
>> OAuth app in Google (in my case, using the G+ API). From an adoption POV,
>> it would be good to screenshot the (current) steps in the process, and
>> point out which values should be used in which fields on Google. For
>> example, I had to grep the code base to find the callback URL.
>> 
>> - The initial login UI seems over-complex: you have to click the provider
>> icon, and then click either login or register. The standard for this
>> workflow is that you login by clicking the desired provider's icon, and
>> doing so will register you automatically if you aren't already. In my case
>> I only have one provider, so this menu was even more confusing.
>> 
>> - It was not clear to me that the "Public" role has absolutely no
>> permissions. When I set this as the default role and registered, I could no
>> longer access the site until I cleared cookies. I thought it was an OAuth
>> error at first, but it turns out the Public role has fewer effective
>> permissions than an anonymous user; this resulted in a redirect loop
>> because I could not even view the homepage. I had to correct this in the
>> database to be able to log in.
>> 
>> - The roles list (at roles/list/ ) is intimidatingly large and hard to
>> parse. For instance, I couldn't tell at a glance what "user" allows
>> relative to "viewer". It would be good to have a narrative description of
>> what each of these roles is intended for, and to present the list of
>> permissions in a more clustered or diffable way. Permissions lists tend to
>> only grow, after all.
>> 
>> - A "Viewer" currently lacks enough access to see their own profile.
>> 
>> - "User Statistics" (userstatschartview/chart/) uses the internal name,
>> rather than firstname/lastname - which in my case is a `google_idnumber`
>> name. Should probably show both names.
>> 
>> Unrelatedly to RBAC (I think), on this branch on my sandbox instance, tasks
>> appear to be failing with the only logs present in the UI as:
>> 
>> [{'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True},
>> {'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True}]
>> 
>> 
>> Finally, in case anyone else wanted to test run a similar setup, here is
>> the webserver_config.py that I ended up using (note that it has Jinja
>> templating via Ansible):
>> 
>> import os
>> from airflow import configuration as conf
>> from flask_appbuilder.security.manager import AUTH_OAUTH
>> basedir = os.path.abspath(os.path.dirname(__file__))
>> 
>> # The SQLAlchemy connection string.
>> SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')
>> 
>> # Flask-WTF flag for CSRF
>> CSRF_ENABLED = True
>> 
>> # The name to display, e.g. "Airflow Staging Sandbox"
>> APP_NAME = "Airflow {{ env }} {{ app_config | capitalize }}"
>> 
>> # Use OAuth
>> AUTH_TYPE = AUTH_OAUTH
>> 
>> # Will allow user self registration
>> AUTH_USER_REGISTRATION = True
>> 
>> # The default user self registration role
>> AUTH_USER_REGISTRATION_ROLE = "{{ airflow_rbac_registration_role |
>> default('Viewer') }}"
>> 
>> # Google OAuth:
>> OAUTH_PROVIDERS = [{
>> # The name of the provider
>> 'name': 'google',
>> # The icon to use
>> 'icon': 'fa-google',
>> # The name of the key that the provider sends
>> 'token_key': 'access_token',
>> # Just in case, whitelist to only @quantopian.com emails
>> 'whitelist': ['@quantopian.com'],
>> # Define the remote app:
>> 'remote_app': {
>> 'base_url': 'https://www.googleapis.com/oauth2/v2/',
>> 'access_token_url': 'https://accounts.google.com/o/oauth2/token',
>> 'authorize_url': 'https://accounts.google.com/o/oauth2/auth',
>> 'request_token_url': None,
>> 'request_token_params': {
>> # Uses the Google+ API, requestingf the 'email' and 'profile' scope
>> 'scope': 'email profile'
>> },
>> 'consumer_key': '{{ vault_airflow_google_oauth_key }}',
>> 'consumer_secret': '{{ vault_airflow_google_oauth_secret }}'
>> }
>> }]
>> 
>> 
>> 
>> On Mon, Apr 30, 2018 at 12:54 PM, Jørn A Hansen <jornhansen@xxxxxxxxx>
>> wrote:
>> 
>>> On Mon, 30 Apr 2018 at 15.56, James Meickle <jmeickle@xxxxxxxxxxxxxx>
>>> wrote:
>>> 
>>>> Installed this off of the branch, and I do get the Kubernetes executor
>>>> (incl. demo DAG) and some bug fixes - but I don't see any RBAC feature
>>>> anywhere I'd think to look. Do I need to set up some config to get that
>>> to
>>>> show up?
>>> 
>>> 
>>> See
>>> https://github.com/apache/incubator-airflow/blob/v1-10-
>>> test/UPDATING.md#new-webserver-ui-with-role-based-access-control
>>> 
>>> It had me left wondering as well - so I decided to go hunt for it in the
>>> RBAC PR. And there it was :-)
>>> 
>>> Cheers,
>>> JornH
>>> 
>>>> 
>>>> 
>>>> On Mon, Apr 23, 2018 at 2:06 PM, Bolke de Bruin <bdbruin@xxxxxxxxx>
>>> wrote:
>>>> 
>>>>> Hi All,
>>>>> 
>>>>> I am really happy that Fokko and I have created the v1-10-test branch
>>> and
>>>>> subsequently build the first beta of Apache Airflow 1.10!
>>>>> 
>>>>> It is available for testing here:
>>>>> 
>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0beta1/
>>>>> 
>>>>> Highlights include:
>>>>> 
>>>>> * New RBAC web interface in beta
>>>>> * Timezone support
>>>>> * First class kubernetes operator
>>>>> * Experimental kubernetes executor
>>>>> * Documentation improvements
>>>>> * Performance optimizations for large DAGs
>>>>> * many GCP and S3 integration improvements
>>>>> * many new operators
>>>>> * many many many bug fixes
>>>>> 
>>>>> We are aiming for a fully compliant Apache release so we should be able
>>>> to
>>>>> kick off the graduation process after this release. I hope you help us
>>>> out
>>>>> getting there!
>>>>> 
>>>>> Kind regards,
>>>>> 
>>>>> Bolke & Fokko
>>>> 
>>>