[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 1.10.0beta1 now available for download

Hey Bolke-
   To be clear, I'm not suggesting anyone is trying to do anything
wrong.  Release wasn't mentioned, but a new tar ball with a new
version number with a 'beta' tag is published in some way for people
to come and test.  How is that different than the expected release/RC
process (specify a git point, offer a tar ball, add an RCx tag and
invite people to test that)?  Seems like a parallel process with lots
of similarities that could confuse both our end users and the IPMC.


On 1 May 2018 at 13:08, Bolke de Bruin <bdbruin@xxxxxxxxx> wrote:
> Hi Jakob,
> Understood. But isn’t that in this case not just wording? Ie. this is a tar-ball that we think is beyond just developer testing (alpha) but more towards the enthusiasts (beta) but not a version of the tarball that is for the general public to test (RC) and not a Release (release)? Ie. is the issue in calling it a ‘release’ which in this case is just meta for a tarball? In the original email in never mentioned the word release in conjunction with the beta I think.
> Cheers
> Bolke
>> On 1 May 2018, at 22:01, Jakob Homan <jghoman@xxxxxxxxx> wrote:
>> Hey all-
>>  With my Mentor hat on, I need to point out that ASF doesn't really
>> have beta releases.  This work is awesome, but really needs to go
>> through the proper steps.  The Release Candidate process is pretty
>> well described:
>> https://incubator.apache.org/policy/incubation.html#releases.  This is
>> particularly important since, as was mentioned, graduation should be
>> imminent and this process will be heavily scrutinized.
>> -Jakob
>> On 1 May 2018 at 12:41, James Meickle <jmeickle@xxxxxxxxxxxxxx> wrote:
>>> Thanks for the pointer! I went through and set this up today, using Google
>>> OAuth as the RBAC provider. Overall I'm quite enthusiastic about this move,
>>> but I thought that it might be helpful to collect feedback as someone who
>>> hasn't been following the overall process and is therefore coming at it
>>> with fresh eyes.
>>> - The Flask appbuilder security documentation is poor quality (e.g.,
>>> there's some broken sentences); if Airflow is to send people there, it
>>> might be worth PRing some of the docs to at least look more professional.
>>> - There's not much documentation out there on how to properly set up an
>>> OAuth app in Google (in my case, using the G+ API). From an adoption POV,
>>> it would be good to screenshot the (current) steps in the process, and
>>> point out which values should be used in which fields on Google. For
>>> example, I had to grep the code base to find the callback URL.
>>> - The initial login UI seems over-complex: you have to click the provider
>>> icon, and then click either login or register. The standard for this
>>> workflow is that you login by clicking the desired provider's icon, and
>>> doing so will register you automatically if you aren't already. In my case
>>> I only have one provider, so this menu was even more confusing.
>>> - It was not clear to me that the "Public" role has absolutely no
>>> permissions. When I set this as the default role and registered, I could no
>>> longer access the site until I cleared cookies. I thought it was an OAuth
>>> error at first, but it turns out the Public role has fewer effective
>>> permissions than an anonymous user; this resulted in a redirect loop
>>> because I could not even view the homepage. I had to correct this in the
>>> database to be able to log in.
>>> - The roles list (at roles/list/ ) is intimidatingly large and hard to
>>> parse. For instance, I couldn't tell at a glance what "user" allows
>>> relative to "viewer". It would be good to have a narrative description of
>>> what each of these roles is intended for, and to present the list of
>>> permissions in a more clustered or diffable way. Permissions lists tend to
>>> only grow, after all.
>>> - A "Viewer" currently lacks enough access to see their own profile.
>>> - "User Statistics" (userstatschartview/chart/) uses the internal name,
>>> rather than firstname/lastname - which in my case is a `google_idnumber`
>>> name. Should probably show both names.
>>> Unrelatedly to RBAC (I think), on this branch on my sandbox instance, tasks
>>> appear to be failing with the only logs present in the UI as:
>>> [{'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True},
>>> {'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True}]
>>> Finally, in case anyone else wanted to test run a similar setup, here is
>>> the webserver_config.py that I ended up using (note that it has Jinja
>>> templating via Ansible):
>>> import os
>>> from airflow import configuration as conf
>>> from flask_appbuilder.security.manager import AUTH_OAUTH
>>> basedir = os.path.abspath(os.path.dirname(__file__))
>>> # The SQLAlchemy connection string.
>>> # Flask-WTF flag for CSRF
>>> # The name to display, e.g. "Airflow Staging Sandbox"
>>> APP_NAME = "Airflow {{ env }} {{ app_config | capitalize }}"
>>> # Use OAuth
>>> # Will allow user self registration
>>> # The default user self registration role
>>> AUTH_USER_REGISTRATION_ROLE = "{{ airflow_rbac_registration_role |
>>> default('Viewer') }}"
>>> # Google OAuth:
>>> # The name of the provider
>>> 'name': 'google',
>>> # The icon to use
>>> 'icon': 'fa-google',
>>> # The name of the key that the provider sends
>>> 'token_key': 'access_token',
>>> # Just in case, whitelist to only @quantopian.com emails
>>> 'whitelist': ['@quantopian.com'],
>>> # Define the remote app:
>>> 'remote_app': {
>>> 'base_url': 'https://www.googleapis.com/oauth2/v2/',
>>> 'access_token_url': 'https://accounts.google.com/o/oauth2/token',
>>> 'authorize_url': 'https://accounts.google.com/o/oauth2/auth',
>>> 'request_token_url': None,
>>> 'request_token_params': {
>>> # Uses the Google+ API, requestingf the 'email' and 'profile' scope
>>> 'scope': 'email profile'
>>> },
>>> 'consumer_key': '{{ vault_airflow_google_oauth_key }}',
>>> 'consumer_secret': '{{ vault_airflow_google_oauth_secret }}'
>>> }
>>> }]
>>> On Mon, Apr 30, 2018 at 12:54 PM, Jørn A Hansen <jornhansen@xxxxxxxxx>
>>> wrote:
>>>> On Mon, 30 Apr 2018 at 15.56, James Meickle <jmeickle@xxxxxxxxxxxxxx>
>>>> wrote:
>>>>> Installed this off of the branch, and I do get the Kubernetes executor
>>>>> (incl. demo DAG) and some bug fixes - but I don't see any RBAC feature
>>>>> anywhere I'd think to look. Do I need to set up some config to get that
>>>> to
>>>>> show up?
>>>> See
>>>> https://github.com/apache/incubator-airflow/blob/v1-10-
>>>> test/UPDATING.md#new-webserver-ui-with-role-based-access-control
>>>> It had me left wondering as well - so I decided to go hunt for it in the
>>>> RBAC PR. And there it was :-)
>>>> Cheers,
>>>> JornH
>>>>> On Mon, Apr 23, 2018 at 2:06 PM, Bolke de Bruin <bdbruin@xxxxxxxxx>
>>>> wrote:
>>>>>> Hi All,
>>>>>> I am really happy that Fokko and I have created the v1-10-test branch
>>>> and
>>>>>> subsequently build the first beta of Apache Airflow 1.10!
>>>>>> It is available for testing here:
>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0beta1/
>>>>>> Highlights include:
>>>>>> * New RBAC web interface in beta
>>>>>> * Timezone support
>>>>>> * First class kubernetes operator
>>>>>> * Experimental kubernetes executor
>>>>>> * Documentation improvements
>>>>>> * Performance optimizations for large DAGs
>>>>>> * many GCP and S3 integration improvements
>>>>>> * many new operators
>>>>>> * many many many bug fixes
>>>>>> We are aiming for a fully compliant Apache release so we should be able
>>>>> to
>>>>>> kick off the graduation process after this release. I hope you help us
>>>>> out
>>>>>> getting there!
>>>>>> Kind regards,
>>>>>> Bolke & Fokko