osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Duplicate key unique constraint error


Max,

We always trigger the DAG externally. I am not sure if there is still any
backfill involved.

Is there a way where I can find out in logs, if more than one instance of
scheduler is running?


On 29 October 2018 at 10:43:19 PM, Maxime Beauchemin (
maximebeauchemin@xxxxxxxxx) wrote:

The stacktrace seems to be pointing in that direction. Id check that first.
It seems like it **could** be a race condition with a backfill as well,
unclear.

It's still a bug though, and the scheduler should make sure to handle this
and not raise/crash.

On Mon, Oct 29, 2018, 10:05 AM Abhishek Sinha <abhishek@xxxxxxxxxxxx> wrote:

> Max,
>
> I do not think there was more than one instance of scheduler running.
> Since the scheduler crashed and it has been restarted, I cannot confirm it
> now. Is there any log that can provide this information?
>
> Could there be a different cause apart from multiple scheduler instances
> running?
>
>
> On 29 October 2018 at 9:30:56 PM, Maxime Beauchemin (
> maximebeauchemin@xxxxxxxxx) wrote:
>
> Abhishek, are you running more than one scheduler instance at once?
>
> Max
>
> On Mon, Oct 29, 2018 at 8:17 AM Abhishek Sinha <abhishek@xxxxxxxxxxxx>
> wrote:
>
> > The issue is happening more frequently now. Can someone please look into
> > this?
> >
> >
> >
> >
> > On 24 September 2018 at 12:42:49 PM, Abhishek Sinha (
> abhishek@xxxxxxxxxxxx
> > )
> > wrote:
> >
> > Can someone please help in looking into this issue? It is critical since
> > this has come up in one of our production environment. Also, this issue
> has
> > appeared only once till now.
> >
> >
> >
> >
> > Regards,
> >
> > Abhishek
> >
> > On 20-Sep-2018, at 10:18 PM, Abhishek Sinha <abhishek@xxxxxxxxxxxx>
> wrote:
> >
> > Any update on this?
> >
> >
> >
> >
> > Regards,
> >
> > Abhishek
> >
> > On 18-Sep-2018, at 12:48 AM, Abhishek Sinha <abhishek@xxxxxxxxxxxx>
> wrote:
> >
> > Pastebin: https://pastebin.com/K6BMTb5K
> >
> >
> >
> >
> > Regards,
> >
> > Abhishek
> >
> > On 18-Sep-2018, at 12:31 AM, Stefan Seelmann <mail@xxxxxxxxxxxxxxxxxx>
> > wrote:
> >
> > On 9/17/18 8:19 PM, Abhishek Sinha wrote:
> >
> > Any update on this?
> >
> > Please find the scheduler error log attached.
> >
> > Can you share the full python stack trace?
> >
> >
> > Seems the mailing list doesn't allow attachments. Either post the
> > stacktrace inline, or post it somewhere at pastebin or so.
> >
>
>