osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Build failure on 0.13.SNAPSHOT


OOME seems to be showing up in some of the Travis testing as well for group
by related stuff. Unsure what's going on there.

On Tue, Jul 24, 2018 at 9:46 PM Dongjin Lee <dongjin@xxxxxxxxxx> wrote:

> After some experiments, I figured out the following:
>
> 1. Druid uses above 8gb of memory for testing. (building-druid.png)
> 2. With 8gb(physical)+4gb(swap) of memory, the test succeeds regardless of
> maven version (3.3.9, 3.5.2, 3.5.4) or MAVEN_OPTS. However, with
> 8gb(physical)+2gb(swap) of memory[^1], some tests failed. The list of
> failing tests differs between maven 3.3.9 and 3.5.2.
>
> In short, retaining sufficient memory solved the problem - *It seems like
> 12gb of memory is a recommended setting for building druid.* (I guess
> lots of you are working with the MacBook Pro with 16gb RAM, right? In that
> case, you must not have encountered this problem.)
>
> If you are okay, may I update the building documentation for the newbies
> like me?
>
> Thanks,
> Dongjin
>
> +1. While building Druid, I found another problem. But this issue should
> be discussed in another thread.
>
> [^1]: You know, the other processes also occupy the memory.
>
>
> On Tue, Jul 24, 2018 at 3:07 AM Jihoon Son <jihoonson@xxxxxxxxxx> wrote:
>
>> I'm also using Maven 3.5.2 and not using any special configurations for
>> Maven, but I have never seen that error too.
>> Most of our Travis jobs have been working with only 512 MB of direct
>> memory. Only the 'strict compilation' Travis job requires 3 GB of memory.
>>
>> I think it's worthwhile to look into this more. Maybe we somehow use more
>> memory when we run all tests by 'mvn install'. Maybe this relates to the
>> frequent transient failures of 'processing module test', one of our Travis
>> jobs.
>>
>> Jihoon
>>
>> On Mon, Jul 23, 2018 at 9:32 AM Gian Merlino <gian@xxxxxxxxxx> wrote:
>>
>> > Interesting. Fwiw, I am using Maven 3.5.2 for building Druid and it has
>> > been working for for me. I don't think I"m using any special Maven
>> > overrides (at least, I don't see anything interesting in my ~/.m2
>> directory
>> > or in my environment variables). It might have to do with how much
>> memory
>> > our machines have? I do most of my builds on a Mac with 16GB RAM. Maybe
>> try
>> > checking .travis.yml in the druid repo. It sets -Xmx3000m for mvn
>> install
>> > commands, which might be needed for more low memory environments.
>> >
>> > $ mvn --version
>> > Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
>> > 2017-10-18T00:58:13-07:00)
>> > Maven home: /usr/local/Cellar/maven/3.5.2/libexec
>> > Java version: 1.8.0_161, vendor: Oracle Corporation
>> > Java home:
>> > /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre
>> > Default locale: en_US, platform encoding: UTF-8
>> > OS name: "mac os x", version: "10.13.5", arch: "x86_64", family: "mac"
>> >
>> > On Mon, Jul 23, 2018 at 6:40 AM Dongjin Lee <dongjin@xxxxxxxxxx> wrote:
>> >
>> > > Finally, it seems like I found the reason. It was a composition of
>> > several
>> > > problems:
>> > >
>> > > - Druid should not be built with maven 3.5.x. With 3.5.2, Test suites
>> > like
>> > > `GroupByQueryRunnerFailureTest` fails. After I switched into 3.3.9
>> which
>> > is
>> > > built in the latest version of IntelliJ, those errors disappeared. It
>> > seems
>> > > like maven 3.5.x is not stable yet - it applied a drastic change, and
>> it
>> > is
>> > > also why they skipped 3.4.x.
>> > > - It seems like Druid requires some MaxDirectMemorySize configuration
>> for
>> > > some test suites. With some JVM parameter like
>> > `-XX:MaxDirectMemorySize=4g`
>> > > some test suites were passed, but not all. I am now trying the other
>> > > options with enlarged swap space.
>> > >
>> > > Question: How much MaxDirectMemorySize configuration are you using?
>> > >
>> > > Best,
>> > > Dongjin
>> > >
>> > > On Sat, Jul 21, 2018 at 3:01 AM Jihoon Son <ghoonson@xxxxxxxxx>
>> wrote:
>> > >
>> > > > Hi Dongjin,
>> > > >
>> > > > that is weird. It looks like the vm crashed because of out of memory
>> > > while
>> > > > testing.
>> > > > It might be a real issue or not.
>> > > > Have you set any memory configuration for your maven?
>> > > >
>> > > > Jihoon
>> > > >
>> > > > On Thu, Jul 19, 2018 at 7:09 PM Dongjin Lee <dongjin@xxxxxxxxxx>
>> > wrote:
>> > > >
>> > > > > Hi Jihoon,
>> > > > >
>> > > > > I ran `mvn clean package` following development/build
>> > > > > <
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/incubator-druid/blob/master/docs/content/development/build.md
>> > > > > >
>> > > > > .
>> > > > >
>> > > > > Dongjin
>> > > > >
>> > > > > On Fri, Jul 20, 2018 at 12:30 AM Jihoon Son <jihoonson@xxxxxxxxxx
>> >
>> > > > wrote:
>> > > > >
>> > > > > > Hi Dongjin,
>> > > > > >
>> > > > > > what maven command did you run?
>> > > > > >
>> > > > > > Jihoon
>> > > > > >
>> > > > > > On Wed, Jul 18, 2018 at 10:38 PM Dongjin Lee <
>> dongjin@xxxxxxxxxx>
>> > > > wrote:
>> > > > > >
>> > > > > > > Hello. I am trying to build druid, but it fails. My
>> environment
>> > is
>> > > > like
>> > > > > > the
>> > > > > > > following:
>> > > > > > >
>> > > > > > > - CPU: Intel(R) Core(TM) i7-7560U CPU @ 2.40GHz
>> > > > > > > - RAM: 7704 MB
>> > > > > > > - OS: ubuntu 18.04
>> > > > > > > - JDK: openjdk version "1.8.0_171" (default configuration,
>> with
>> > > > > > MaxHeapSize
>> > > > > > > = 1928 MB)
>> > > > > > > - Branch: master (commit: cd8ea3d)
>> > > > > > >
>> > > > > > > The error message I got is:
>> > > > > > >
>> > > > > > > [INFO]
>> > > > > > > >
>> > > > > >
>> > > >
>> > ------------------------------------------------------------------------
>> > > > > > > > [INFO] Reactor Summary:
>> > > > > > > > [INFO]
>> > > > > > > > [INFO] io.druid:druid .....................................
>> > > > SUCCESS [
>> > > > > > > > 50.258 s]
>> > > > > > > > [INFO] java-util ..........................................
>> > > SUCCESS
>> > > > > > > [03:57
>> > > > > > > > min]
>> > > > > > > > [INFO] druid-api ..........................................
>> > > > SUCCESS [
>> > > > > > > > 22.694 s]
>> > > > > > > > [INFO] druid-common .......................................
>> > > > SUCCESS [
>> > > > > > > > 14.083 s]
>> > > > > > > > [INFO] druid-hll ..........................................
>> > > > SUCCESS [
>> > > > > > > > 17.126 s]
>> > > > > > > > [INFO] extendedset ........................................
>> > > > SUCCESS [
>> > > > > > > > 10.856 s]
>> > > > > > > >
>> > > > > > > > *[INFO] druid-processing ...................................
>> > > > FAILURE
>> > > > > > > > [04:36 min]*[INFO] druid-aws-common
>> > > > > ...................................
>> > > > > > > > SKIPPED
>> > > > > > > > [INFO] druid-server .......................................
>> > > SKIPPED
>> > > > > > > > [INFO] druid-examples .....................................
>> > > SKIPPED
>> > > > > > > > ...
>> > > > > > > > [INFO]
>> > > > > > > >
>> > > > > >
>> > > >
>> > ------------------------------------------------------------------------
>> > > > > > > > [INFO] BUILD FAILURE
>> > > > > > > > [INFO]
>> > > > > > > >
>> > > > > >
>> > > >
>> > ------------------------------------------------------------------------
>> > > > > > > > [INFO] Total time: 10:29 min
>> > > > > > > > [INFO] Finished at: 2018-07-19T13:23:31+09:00
>> > > > > > > > [INFO] Final Memory: 88M/777M
>> > > > > > > > [INFO]
>> > > > > > > >
>> > > > > >
>> > > >
>> > ------------------------------------------------------------------------
>> > > > > > > >
>> > > > > > > > *[ERROR] Failed to execute goal
>> > > > > > > > org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test
>> > > > > > (default-test)
>> > > > > > > > on project druid-processing: Execution default-test of goal
>> > > > > > > > org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test
>> > > failed:
>> > > > > The
>> > > > > > > > forked VM terminated without properly saying goodbye. VM
>> crash
>> > or
>> > > > > > > > System.exit called?*[ERROR] Command was /bin/sh -c cd
>> > > > > > > > /home/djlee/workspace/java/druid/processing &&
>> > > > > > > > /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx3000m
>> > > > > > > -Duser.language=en
>> > > > > > > > -Duser.country=US -Dfile.encoding=UTF-8 -Duser.timezone=UTC
>> > > > > > > >
>> > > -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
>> > > > > > > > -Ddruid.indexing.doubleStorage=double -jar
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /home/djlee/workspace/java/druid/processing/target/surefire/surefirebooter1075382243904099051.jar
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /home/djlee/workspace/java/druid/processing/target/surefire/surefire5444459351134757209tmp
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> /home/djlee/workspace/java/druid/processing/target/surefire/surefire_5173894389718744688tmp
>> > > > > > >
>> > > > > > >
>> > > > > > > It seems like it fails when it runs tests on
>> `druid-processing`
>> > > > module
>> > > > > > but
>> > > > > > > I can't certain. Is there anyone who can give me some hints?
>> > Thanks
>> > > > in
>> > > > > > > advance.
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Dongjin
>> > > > > > >
>> > > > > > > --
>> > > > > > > *Dongjin Lee*
>> > > > > > >
>> > > > > > > *A hitchhiker in the mathematical world.*
>> > > > > > >
>>
> > > > > > > *github:  <http://goog_969573159/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__goog-5F969573159_&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=S55krYhaUwTC-7EGIgllQk3Jl9k8T3fZxfZAS_5nPA4&e=>
>> >github.com/dongjinleekr
>> > > > > > > <http://github.com/dongjinleekr>linkedin:
>> > > > > > kr.linkedin.com/in/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kr.linkedin.com_in_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=G-v0Qp9IHM8nieX8oUBvnQfbqWN_OC_MHzk52goEzqM&e=>
>> > > > > > > <http://kr.linkedin.com/in/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kr.linkedin.com_in_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=G-v0Qp9IHM8nieX8oUBvnQfbqWN_OC_MHzk52goEzqM&e=>
>> >slideshare:
>> > > > > > > www.slideshare.net/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=SKr88hVIw32a2NGp3MDw-_EUvn0nFf_yISLrwD_ySYk&e=>
>> > > > > > > <http://www.slideshare.net/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=SKr88hVIw32a2NGp3MDw-_EUvn0nFf_yISLrwD_ySYk&e=>
>> >*
>
>
>> > > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > *Dongjin Lee*
>> > > > > >
>> > > > > > *A hitchhiker in the mathematical world.*
>> > > > > >
>>
> > > > > > *github:  <http://goog_969573159/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__goog-5F969573159_&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=S55krYhaUwTC-7EGIgllQk3Jl9k8T3fZxfZAS_5nPA4&e=>
>> >github.com/dongjinleekr
>> > > > > > <http://github.com/dongjinleekr>linkedin:
>> > > > > kr.linkedin.com/in/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kr.linkedin.com_in_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=G-v0Qp9IHM8nieX8oUBvnQfbqWN_OC_MHzk52goEzqM&e=>
>> > > > > > <http://kr.linkedin.com/in/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kr.linkedin.com_in_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=G-v0Qp9IHM8nieX8oUBvnQfbqWN_OC_MHzk52goEzqM&e=>
>> >slideshare:
>> > > > > www.slideshare.net/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=SKr88hVIw32a2NGp3MDw-_EUvn0nFf_yISLrwD_ySYk&e=>
>> > > > > > <http://www.slideshare.net/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=SKr88hVIw32a2NGp3MDw-_EUvn0nFf_yISLrwD_ySYk&e=>
>> >*
>
>
>> > > > > >
>> > > > >
>> > > >
>> > > > --
>> > > > *Dongjin Lee*
>> > > >
>> > > > *A hitchhiker in the mathematical world.*
>> > > >
>>
> > > > *github:  <http://goog_969573159/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__goog-5F969573159_&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=S55krYhaUwTC-7EGIgllQk3Jl9k8T3fZxfZAS_5nPA4&e=>
>> >github.com/dongjinleekr
>> > > > <http://github.com/dongjinleekr>linkedin:
>> > > kr.linkedin.com/in/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kr.linkedin.com_in_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=G-v0Qp9IHM8nieX8oUBvnQfbqWN_OC_MHzk52goEzqM&e=>
>> > > > <http://kr.linkedin.com/in/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kr.linkedin.com_in_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=G-v0Qp9IHM8nieX8oUBvnQfbqWN_OC_MHzk52goEzqM&e=>
>> >slideshare:
>> > > www.slideshare.net/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=SKr88hVIw32a2NGp3MDw-_EUvn0nFf_yISLrwD_ySYk&e=>
>> > > > <http://www.slideshare.net/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=SKr88hVIw32a2NGp3MDw-_EUvn0nFf_yISLrwD_ySYk&e=>
>> >*
>> > > >
>> > >
>> >
>>
>> --
>> *Dongjin Lee*
>>
>> *A hitchhiker in the mathematical world.*
>>
>> *github:
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__goog-5F969573159_&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=S55krYhaUwTC-7EGIgllQk3Jl9k8T3fZxfZAS_5nPA4&e=>github.com/dongjinleekr
>> <http://github.com/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__kr.linkedin.com_in_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=G-v0Qp9IHM8nieX8oUBvnQfbqWN_OC_MHzk52goEzqM&e=>slideshare: www.slideshare.net/dongjinleekr
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slideshare.net_dongjinleekr&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_w&r=HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m=thcH_tBRIMD3fQlav3ogOGyksut5JF_p3v21U7VlKyM&s=SKr88hVIw32a2NGp3MDw-_EUvn0nFf_yISLrwD_ySYk&e=>*
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxx
> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxx