osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: excessive java precommit logging


Interestingly, I was thinking exactly the same thing the other day.

If we could drop the info logs for passing tests, that would be ideal.
Regardless, tests should fail (when possible) with actionable
messages. I think the rare case of not being able to reproduce the
error locally if info logs are needed makes it OK to go and add
logging to jenkins as a one-off. (If it's about jenkins build errors,
perhaps we could build with higher verbosity before testing with a
lower one.)
On Thu, Dec 20, 2018 at 11:24 AM Maximilian Michels <mxm@xxxxxxxxxx> wrote:
>
> Thanks Udi for bringing this up. I'm also for dropping INFO. It's just
> incredible verbose. More importantly, from my experience the INFO level doesn't
> help debugging problems, but it makes finding errors messages or warnings harder.
>
> That said, here's what I do to search through the log:
>
> 1) curl <build_url>/consoleText | less
>
> This is when I just want to quickly look for something.
>
> 2) curl <build_url>/consoleText > log.txt
>     less log.txt
>
> Here we store the log to a file first, then use 'less' or 'grep' to search it.
>
> When in 'less', I use '/' to grep through the lines. Pressing 'n' or 'N' gets
> you forward and back in the search results.
>
> That works pretty well, but I think we would do us a favor by dropping the log
> level. Shall we try it out?
>
> -Max
>
> On 19.12.18 23:27, Udi Meiri wrote:
> > The gradle scan doesn't pinpoint the error message, and it doesn't contain all
> > the lines: https://scans.gradle.com/s/ckhjrjdexpuzm/console-log
> >
> > The logs might be useful, but usually not from passing tests. Doesn't gradle log
> > output from failed tests by default?
> >
> > On Wed, Dec 19, 2018 at 1:22 PM Thomas Weise <thw@xxxxxxxxxx
> > <mailto:thw@xxxxxxxxxx>> wrote:
> >
> >     I usually follow the download procedure outlined by Scott to look at the logs.
> >
> >     These logs are big, but when there is a problem it is sometimes essential to
> >     have the extra output, especially for less frequent flakes.
> >
> >     Reducing logs would then require the author to add extra logging to the PR
> >     (and attempt to reproduce), which is also not nice.
> >
> >     Thomas
> >
> >
> >     On Wed, Dec 19, 2018 at 11:47 AM Scott Wegner <scott@xxxxxxxxxx
> >     <mailto:scott@xxxxxxxxxx>> wrote:
> >
> >         I'm not sure what we lose by dropping the --info flag, but I generally
> >         worry about reducing log output since logs are the main resource for
> >         diagnosing Jenkins build errors.
> >
> >         It seems the issue is that Chrome doesn't scale well to large log files.
> >         A few alternative solutions:
> >
> >         1. Use the produced Build Scan (example: [1]) instead of the raw console
> >         log. The build scan is quite useful at pointing to what actually failed,
> >         and filtering log output for only that task.
> >         2. Instead of consoleFull, use consoleText ("View as plain text" link in
> >         Jenkins), which seems to be much easier on Chrome
> >         3. Download the consoleText output locally and use your favorite log
> >         viewer that can scale to large files.
> >
> >         [1] https://gradle.com/s/ckhjrjdexpuzm
> >
> >         On Wed, Dec 19, 2018 at 10:42 AM Udi Meiri <ehudm@xxxxxxxxxx
> >         <mailto:ehudm@xxxxxxxxxx>> wrote:
> >
> >             Hi all,
> >             I'd like to reduce precommit log sizes on Jenkins. For example:
> >             https://builds.apache.org/job/beam_PreCommit_Java_Commit/3181/consoleFull
> >             is 79M, which makes Chrome sluggish to use on it (tab is constantly
> >             using a whole cpu core).
> >
> >             I know this might be controversial, but I'd like to propose to
> >             remove the --info flag from the gradlew command line.
> >
> >
> >
> >         --
> >
> >
> >
> >
> >         Got feedback? tinyurl.com/swegner-feedback
> >         <https://tinyurl.com/swegner-feedback>
> >