[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Call for fixes for Airflow 1.10.1


Hi Ash,

I've cherry-picked two commits on top of 1.10-test branch:

https://github.com/apache/incubator-airflow/commit/4d2f83b19af3489d6c9563d51210a3dab2f38b26

https://github.com/apache/incubator-airflow/commit/0d2bb5cd215bd3e2d4f47033230ea03b14b36634


This was on request of Yuan of the Amazon SageMaker team, because some
companies are waiting for this operator. Since the commits don't modify any
existing code, I took the liberty of cherry-picking these on top of the
1.10.1 branch.


Cheers, Fokko



Op zo 9 sep. 2018 om 20:37 schreef Ash Berlin-Taylor <ash@xxxxxxxxxx>:

> I've (re)created the v1-10-test branch with some of the fixes
> cherry-picked in. I can't give much time this week (as what spare time I
> have is being used up working on my talk) but I'll work more on this
> towards the end of next week.
>
> I'll look at resolved Jira tickets targeted with a fix version of 1.10.1
> (i.e. if you want it in 1.10.1, merge the pr into master and also mark the
> Jira as fix in 1.10.1 and I'll work on cherry-picking the fixes. If they
> can be. If it is diffucult/has other things to cherry pick in I might
> change the fix version on you.)
>
> -ash
>
>
> > On 9 Sep 2018, at 19:22, Ash Berlin-Taylor <ash@xxxxxxxxxxxxxx> wrote:
> >
> > On 9 September 2018 18:19:40 BST, Bolke de Bruin <bdbruin@xxxxxxxxx>
> wrote:
> > You can already add them to v1-10-test.
> >
> > Normally we are a bit cautious to this if you are not the release
> manager to ensure that he/she knows what the state is.
> >
> > B
> >
> > Op zo 9 sep. 2018 18:02 schreef Driesprong, Fokko <fokko@xxxxxxxxxxxxxx
> >:
> > Can we add this one as well?
> >
> > https://github.com/apache/incubator-airflow/pull/3862 <
> https://github.com/apache/incubator-airflow/pull/3862>
> > https://issues.apache.org/jira/browse/AIRFLOW-1917 <
> https://issues.apache.org/jira/browse/AIRFLOW-1917>
> >
> > I'm happy to cherry pick them onto the 1.10.1 by myself as well. Any idea
> > when we will start this branch?
> >
> > Cheers, Fokko
> >
> > Op do 6 sep. 2018 om 08:08 schreef Deng Xiaodong <xd.deng.r@xxxxxxxxx
> <mailto:xd.deng.r@xxxxxxxxx>>:
> >
> > > Hi Ash,
> > >
> > >
> > > May you consider including JIRA ticket 2848 (PR 3693, Ensure dag_id in
> > > metadata "job" for LocalTaskJob) in 1.10.1 as well?
> > >
> > > https://issues.apache.org/jira/browse/AIRFLOW-2848 <
> https://issues.apache.org/jira/browse/AIRFLOW-2848>
> > >
> > > https://github.com/apache/incubator-airflow/pull/3693 <
> https://github.com/apache/incubator-airflow/pull/3693>
> > >
> > >
> > > This is a bug in terms of metadata, which also affects the UI
> > > “Browse->Jobs”.
> > >
> > >
> > > Thanks.
> > >
> > >
> > > Regards,
> > >
> > > XD
> > >
> > > On Wed, Sep 5, 2018 at 23:55 Bolke de Bruin <bdbruin@xxxxxxxxx
> <mailto:bdbruin@xxxxxxxxx>> wrote:
> > >
> > > > You should push these to v1-10-test not to stable. Only once we start
> > > > cutting RCs you should push to -stable. See the docs. This ensures a
> > > stable
> > > > “stable”branch.
> > > >
> > > > Cheers
> > > > Bolke.
> > > >
> > > > > On 3 Sep 2018, at 14:20, Ash Berlin-Taylor <ash@xxxxxxxxxx
> <mailto:ash@xxxxxxxxxx>> wrote:
> > > > >
> > > > > Hi everyone,
> > > > >
> > > > > I'm starting the process of gathering fixes for a 1.10.1. So far
> the
> > > > list of issues I have that we should pull in are
> > > >
> > >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20AIRFLOW%20AND%20fixVersion%20%3D%201.10.1%20ORDER%20BY%20key%20ASC
> <
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20AIRFLOW%20AND%20fixVersion%20%3D%201.10.1%20ORDER%20BY%20key%20ASC
> >
> > > > (reproduces below)
> > > > >
> > > > > I will start pushing these as cherry-picked commits to the
> v1-10-stable
> > > > branch today.
> > > > >
> > > > > If you have something that is not in the list below let me know.
> I'd
> > > > like to keep this to bug fixes against 1.10.0 only if possible.
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2145 <
> https://issues.apache.org/jira/browse/AIRFLOW-2145> Deadlock after
> > > > clearing a running task
> > > > > https://github.com/apache/incubator-airflow/pull/3657 <
> https://github.com/apache/incubator-airflow/pull/3657>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2476 <
> https://issues.apache.org/jira/browse/AIRFLOW-2476> update tabulate dep
> > > > to 0.8.2
> > > > > https://github.com/apache/incubator-airflow/pull/3835 <
> https://github.com/apache/incubator-airflow/pull/3835>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2778 <
> https://issues.apache.org/jira/browse/AIRFLOW-2778> Bad Import in
> > > > collect_dag in DagBag
> > > > > https://github.com/apache/incubator-airflow/pull/3624 <
> https://github.com/apache/incubator-airflow/pull/3624>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2900 <
> https://issues.apache.org/jira/browse/AIRFLOW-2900> Show code for
> > > > packaged DAGs
> > > > > https://github.com/apache/incubator-airflow/pull/3749 <
> https://github.com/apache/incubator-airflow/pull/3749>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2949 <
> https://issues.apache.org/jira/browse/AIRFLOW-2949> Add syntax
> > > highlight
> > > > for single quote strings
> > > > > https://github.com/apache/incubator-airflow/pull/3795 <
> https://github.com/apache/incubator-airflow/pull/3795>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2984 <
> https://issues.apache.org/jira/browse/AIRFLOW-2984> Cannot convert
> > > > naive_datetime when task has a naive start_date/end_date
> > > > > https://github.com/apache/incubator-airflow/pull/3822 <
> https://github.com/apache/incubator-airflow/pull/3822>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2979 <
> https://issues.apache.org/jira/browse/AIRFLOW-2979> Deprecated Celery
> > > > Option not in Options list
> > > > > https://github.com/apache/incubator-airflow/pull/3832 <
> https://github.com/apache/incubator-airflow/pull/3832>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2989 <
> https://issues.apache.org/jira/browse/AIRFLOW-2989> No Parameter to
> > > > change bootDiskType for DataprocClusterCreateOperator
> > > > > https://github.com/apache/incubator-airflow/pull/3825 <
> https://github.com/apache/incubator-airflow/pull/3825>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2990 <
> https://issues.apache.org/jira/browse/AIRFLOW-2990> Docstrings for
> > > > Hooks/Operators are in incorrect format
> > > > > https://github.com/apache/incubator-airflow/pull/3820 <
> https://github.com/apache/incubator-airflow/pull/3820>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2994 <
> https://issues.apache.org/jira/browse/AIRFLOW-2994> flatten_results in
> > > > BigQueryOperator/BigQueryHook should default to None
> > > > > https://github.com/apache/incubator-airflow/pull/3829 <
> https://github.com/apache/incubator-airflow/pull/3829>
> > > > >
> > > > >
> > > > > In addition to those PRs which are already marked with Fix Version
> of
> > > > 1.10.1 I think we should also pull in these:
> > > > >
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2713 <
> https://issues.apache.org/jira/browse/AIRFLOW-2713> Rename async
> > > > variable for Python 3.7.0 compatibility
> > > > > https://github.com/apache/incubator-airflow/pull/3561 <
> https://github.com/apache/incubator-airflow/pull/3561>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2895 <
> https://issues.apache.org/jira/browse/AIRFLOW-2895> Prevent scheduler
> > > > from spamming heartbeats/logs
> > > > > https://github.com/apache/incubator-airflow/pull/3747 <
> https://github.com/apache/incubator-airflow/pull/3747>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2921 <
> https://issues.apache.org/jira/browse/AIRFLOW-2921> A trivial
> > > > incorrectness in CeleryExecutor()
> > > > > https://github.com/apache/incubator-airflow/pull/3773 <
> https://github.com/apache/incubator-airflow/pull/3773>
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2866 <
> https://issues.apache.org/jira/browse/AIRFLOW-2866> Missing CSRF Token
> > > > Error on Web RBAC UI Create/Update Operations
> > > > > https://github.com/apache/incubator-airflow/pull/3804 <
> https://github.com/apache/incubator-airflow/pull/3804>
> > > > >
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-2951 <
> https://issues.apache.org/jira/browse/AIRFLOW-2951>
> > > > > https://github.com/apache/incubator-airflow/pull/3798 <
> https://github.com/apache/incubator-airflow/pull/3798> Update dag_run
> > > > table end_date when state change
> > > > > (though as written it has a few other deps to cherry pick in, so
> will
> > > > see about this one)
> > > > >
> > > >
> > > >
> > >
>
>