osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Empire-db build problems <-- brooklyn-master-build-docker_ws-cleanup workspace left behind


This is pretty much what Apache Yetus was built to do. (I’d go into more depth but about to board a plane. ) :D

Sent from my ☎️

> On Sep 24, 2018, at 3:20 PM, Aled Sage <aled.sage@xxxxxxxxxxxxxxxxx> wrote:
> 
> Hi Chris, all,
> 
> Thomas has just pointed me at https://issues.apache.org/jira/browse/INFRA-16417, in which you recommend:
> 
>   "The problem is the bind mount for .m2 and permissions conflicts
>   between the docker
>   image and jenkins. Please don't use a bind mount for filesystems you
>   intend to write to.
>   It will make the build run a bit longer since it has to download the
>   artifacts within the
>   container, but it will prevent you from having problems and
>   conflicts with the
>   jenkins .m2 directory."
> 
> Apologies for us not running with this advice previously - I'm looking into how we could do this.
> 
> Are there any good example configurations of other Apache projects building and running tests (ideally using maven) in docker containers, on apache jenkins slaves?
> 
> Aled
> 
> 
>> On 21/09/2018 10:03, Aled Sage wrote:
>> Hi all,
>> 
>> Do any brooklyn folk know what the `brooklyn-master-build-docker_ws-cleanup_.*` workspaces are for, and why they are not being cleaned up?
>> 
>> It sounds like this bug in ws-cleanup-plugin [1]. It was fixed, but then folk reported that they were still experiencing the same problem [2].
>> 
>> A workaround described in [3] is:
>> 
>>   ws-cleanup is leaking workspaces because the dynamic nodes are unavailable at the time
>>   the dispose is ran. I've installed a cron which is run every 1/2/h which is removing
>>   ws-cleanup folders that aren't modified the last hour on ovh docker hosts.
>> 
>> I'm not sure what we could do on the Brooklyn side.
>> 
>> ---
>> Looking in the console output of our brooklyn-master-build-docker jobs, I see #700 [4] says at the very start:
>> 
>>   Building remotely on H24 (ubuntu xenial) in workspace /home/jenkins/jenkins-slave/workspace/brooklyn-master-build-docker
>>   [WS-CLEANUP] Deleting project workspace...
>>   [WS-CLEANUP] Done
>> 
>> In contrast, builds #702 [5] doesn't have the line `[WS-CLEANUP] Done`. Does that mean it will have left behind one of these directories perhaps?
>> 
>> Looking at [6], it looks like we only keep a history of 5 builds in jenkins. We therefore should really not have 28 workspace directories on the server H24!
>> 
>> If it might be a problem with ws-cleanup having permissions to delete the workspace, could it be related to the way we mount the workspace as a volume in the docker container?
>> 
>>   docker run -i --rm -v /home/jenkins:/usr/home
>>       -v /home/jenkins/jenkins-slave/workspace/brooklyn-master-build-docker/.m2:/root/.m2 \
>>       -v /home/jenkins/jenkins-slave/workspace/brooklyn-master-build-docker:/usr/build \
>> -w /usr/build brooklyn:latest mvn clean install -s /usr/home/.m2/settings.xml
>> 
>> Aled
>> 
>> [1] https://issues.jenkins-ci.org/browse/JENKINS-24824
>> [2] https://issues.jenkins-ci.org/browse/JENKINS-53579
>> [3] https://jira.nuxeo.com/browse/NXBT-2205
>> [4] https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-master-build-docker/700/consoleFull
>> [5] https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-master-build-docker/700/consoleFull
>> [6] https://builds.apache.org/job/brooklyn-master-build-docker/
>> 
>> 
>>> On 20/09/2018 20:20, Chris Lambertus wrote:
>>> 
>>> 
>>> 
>>> 
>>>> On Sep 20, 2018, at 11:51 AM, Dave Brondsema <dave@xxxxxxxxxxxxx> wrote:
>>>> 
>>>> Moving to builds@
>>>> 
>>>> I'm seeing this type of disk error on Allura builds that have used H27 and H24
>>>> today.
>>> Here’s the raw usage output from H24. Not sure what’s going on with those Brooklyn jobs. CC to dev@brooklyn so they can have a look.
>>> 
>>> 
>>> 
>>> root@asf924:/home/jenkins/jenkins-slave/workspace# du -sh -- *
>>> 4.3G    ActiveMQ-Java8
>>> 2.5G    ActiveMQ-Java8-All-UnitTests
>>> 1.2G    ActiveMQ-PR-Build
>>> 4.5G    Ambari-branch-2.7
>>> 8.0G    Ambari-trunk-Commit
>>> 127M    Ant-Build-Matrix-1.9.x-Linux
>>> 129M    Ant-Build-Matrix-master-Linux
>>> 88M    Ant-Master SonarQube
>>> 4.7M    Apache%20Jackrabbit%20Oak%20matrix
>>> 15G    Apache Jackrabbit Oak matrix
>>> 12M    Aries jax-rs whiteboard
>>> 14M    Aries-rsa master deploy
>>> 816M    aurora-packaging-nightly
>>> 4.0K aven-assembly-plugin_master-SP4DXUL4GVN3Q5KJHI5ED6WPWUEC2L2PUW2RCT2SFVGTCXHUIFDQ@tmp
>>> 4.0K aven-box_maven-doxia_master-4SHBBM33BQC2SGN65AA3AW6GPXIK6RA462YQND5AN3YPZ5U2VGSQ@tmp
>>> 4.0K aven-enforcer_MENFORCER-288-GL5B4ERVFARMDDY5GUJ32NFJSOP7ZR6QLATDJG7AYHOHJMZGS6NA@2@tmp
>>> 4.0K aven-enforcer_MENFORCER-288-GL5B4ERVFARMDDY5GUJ32NFJSOP7ZR6QLATDJG7AYHOHJMZGS6NA@tmp
>>> 4.0K aven-enforcer_MENFORCER-312-H6BWWMKU3O4RIVLVOSDDUVD7CT73H5PSEGRJICFTH5FH5BR7L3XQ@tmp
>>> 122M    Bigtop-trunk-testartifacts
>>> 924M    bookkeeper_postcommit_master_java8
>>> 261M    bookkeeper_precommit_bookie_tests
>>> 302M    bookkeeper_precommit_client_tests
>>> 754M    bookkeeper_precommit_pullrequest_java8
>>> 758M    bookkeeper_precommit_pullrequest_java9
>>> 272M    bookkeeper_precommit_pullrequest_validation
>>> 400M    bookkeeper_precommit_pullrequest_validation@2
>>> 334M    bookkeeper_precommit_remaining_tests
>>> 257M    bookkeeper_precommit_replication_tests
>>> 259M    bookkeeper_precommit_tls_tests
>>> 907M    bookkeeper_release_branch_48_integrationtests
>>> 886M    bookkeeper_release_branch_48_java8
>>> 4.0K box_maven-ant-plugin_master-N743QKOWXTZ36LZCK2YK3A6N4ZY3UG7X5RV3MH7PEH646A2IMDHA@tmp
>>> 4.0K -box_maven-archetype_master-7AVMJNUR3VZV7RPVUBCE43H65NYTRWAOYFWSDDPCNSCWDYBAGBVQ@2@tmp
>>> 4.0K -box_maven-archetype_master-7AVMJNUR3VZV7RPVUBCE43H65NYTRWAOYFWSDDPCNSCWDYBAGBVQ@tmp
>>> 4.0K box_maven-ejb-plugin_master-D5MGOBWWYZCBR23TA7FG7MUJQK233O27B5TSPA7YUOAEWLP5Z72Q@tmp
>>> 4.0K -box_maven-filtering_master-7JLC4RFGGUFPJD5SZA5D3WNJVBYMDYSH6XEWGGYSHCAMQDN7VZQA@2@tmp
>>> 4.0K -box_maven-filtering_master-7JLC4RFGGUFPJD5SZA5D3WNJVBYMDYSH6XEWGGYSHCAMQDN7VZQA@tmp
>>> 5.7M -box_maven_pre-reset-master-AF72GWR2EBH4F7AYOLQPRDHKOEZISK7V6BEF33K5DPJKW2VE4ATA
>>> 4.0K box_maven-rar-plugin_master-CZJVOZNWNEWP3FI6FQDKOMWKKMP7HVM2MPSDE36B4TYFH2HNQANA@2@tmp
>>> 4.0K box_maven-rar-plugin_master-CZJVOZNWNEWP3FI6FQDKOMWKKMP7HVM2MPSDE36B4TYFH2HNQANA@tmp
>>> 4.0K -box_maven-shared-io_master-CKUVSWZXBCH4QK62Z7UJO2FSLE5X6ZCFSHJWVQPSCR7EQORIBC7Q@2@tmp
>>> 4.0K -box_maven-shared-io_master-CKUVSWZXBCH4QK62Z7UJO2FSLE5X6ZCFSHJWVQPSCR7EQORIBC7Q@tmp
>>> 4.0K box_maven-war-plugin_master-QSYAUU4F27NQH3VKQKYR34QIY6P36LH6ACXOADA2HDWCZVVU4TYA@tmp
>>> 155M    brooklyn-library-master-docker
>>> 4.4G    brooklyn-master-build-docker_ws-cleanup_1535791870056
>>> 4.4G    brooklyn-master-build-docker_ws-cleanup_1535929381454
>>> 4.4G    brooklyn-master-build-docker_ws-cleanup_1535991522759
>>> 4.4G    brooklyn-master-build-docker_ws-cleanup_1536015781206
>>> 4.4G    brooklyn-master-build-docker_ws-cleanup_1536050912838
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536188581290
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536350369118
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536361380993
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536447781251
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536534181007
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536620581304
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536706981637
>>> 451M    brooklyn-master-build-docker_ws-cleanup_1536793381496
>>> 4.7G    brooklyn-master-build-docker_ws-cleanup_1536966181681
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1536974386104
>>> 733M    brooklyn-master-build-docker_ws-cleanup_1537055864223
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537079895664
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537087960702
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537092848128
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537116168718
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537138981521
>>> 3.1G    brooklyn-master-build-docker_ws-cleanup_1537166290489
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537252717591
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537260673390
>>> 4.8G    brooklyn-master-build-docker_ws-cleanup_1537289677014
>>> 5.0G    brooklyn-master-build-docker_ws-cleanup_1537425657042
>>> 5.0G    brooklyn-master-build-docker_ws-cleanup_1537433463888
>>> 5.0G    brooklyn-master-build-docker_ws-cleanup_1537461822531
>>> 1.4G    Calcite-Master
>>> 170M _CAMEL-12549-olingo2_update-PZQQDS6CDS5DORR2RC63COME3FNUAH4R6CMNQPYMRZXGMI47DS3Q
>>> 8.0K _CAMEL-12549-olingo2_update-PZQQDS6CDS5DORR2RC63COME3FNUAH4R6CMNQPYMRZXGMI47DS3Q@tmp
>>> 2.8G Camel_2.20.x-TFEL6P7XFNLJIRPYRL6K22RDL2QKGR5FX4M6GKSE64EU4BICYPVA
>>> 4.0K Camel_2.20.x-TFEL6P7XFNLJIRPYRL6K22RDL2QKGR5FX4M6GKSE64EU4BICYPVA@tmp
>>> 171M Camel_CAMEL-12622-IBFTSXAOWKDWEBFQ3GOXANPDGQSGYTS3GUJR3ZXLZJQRPUOAZYEQ
>>> 4.0K Camel_CAMEL-12622-IBFTSXAOWKDWEBFQ3GOXANPDGQSGYTS3GUJR3ZXLZJQRPUOAZYEQ@tmp
>>> 60M Camel_camel-2.20.x-SQQNE7QN3DLJQV6AZB4KZEBHA7N2TZZRYU4SJGXEFFCLXWX5UQ2A
>>> 4.0K Camel_camel-2.20.x-SQQNE7QN3DLJQV6AZB4KZEBHA7N2TZZRYU4SJGXEFFCLXWX5UQ2A@tmp
>>> 1.9G Camel_camel-2.21.x-2HLLW2P24PAMSWCRX5CGD52T7NJ5PZVNS47EDMN4CZFYHCFKKPQQ
>>> 4.0K Camel_camel-2.21.x-2HLLW2P24PAMSWCRX5CGD52T7NJ5PZVNS47EDMN4CZFYHCFKKPQQ@tmp
>>> 2.5G Camel_camel-2.22.x-BCWNGKYBULUM6XGPMTXCH5J6NJFJGFAO43BCTMPRQM6SZTDNLMYQ
>>> 4.0K Camel_camel-2.22.x-BCWNGKYBULUM6XGPMTXCH5J6NJFJGFAO43BCTMPRQM6SZTDNLMYQ@tmp
>>> 3.3G    Camel.daily
>>> 4.0K    Camel.daily@tmp
>>> 2.4G Camel_master-CIGY56HHEDNGVJBWJRRHBZ64GKNJTCSYDKOPJS46W5IZTMAS3I2Q
>>> 4.0K Camel_master-CIGY56HHEDNGVJBWJRRHBZ64GKNJTCSYDKOPJS46W5IZTMAS3I2Q@tmp
>>> 2.3G Camel_website-YC7K6UZ6PNVBP5RVOX3M6D2U4MRT66KUPRNB45L2G7LLE63BZYVQ
>>> 4.0K Camel_website-YC7K6UZ6PNVBP5RVOX3M6D2U4MRT66KUPRNB45L2G7LLE63BZYVQ@tmp
>>> 810M    carbondata-master-spark-2.1
>>> 4.0K    carbondata-master-spark-2.1@tmp
>>> 584K    commons-geometry
>>> 4.0K CouchDB_master-FP4LRZTK2IWL4WCWS25JHGLJVPORIFX6ES6POVJARPVUTCZMU7SA@tmp
>>> 190M    CXF-Master-Analysis
>>> 188M    CXF-Master-JDK10
>>> 24M    CXF-Master-JDK9
>>> 1.7G    CXF-trunk-deploy
>>> 2.2G    CXF-Trunk-JDK18
>>> 74M    DeltaSpike Deploy
>>> 22M    DeltaSpike for Java EE
>>> 53M    DeltaSpike OWB 1.7.2
>>> 70M    DeltaSpike OWB 2.0.6
>>> 4.0K -dependency-plugin_MDEP-617-ZQLYQZECSCXYYD7MV7RDO5BGII4AACSGUIH7FWUOUWHLCGKN7XPQ@2@tmp
>>> 4.0K -dependency-plugin_MDEP-617-ZQLYQZECSCXYYD7MV7RDO5BGII4AACSGUIH7FWUOUWHLCGKN7XPQ@tmp
>>> 7.1M    Derby-JaCoCo
>>> 7.3M    dir-ldap-api-jdk8-linux SONAR
>>> 23M    dir-ldap-api-ubuntu-deploy
>>> 2.0G    dir-studio
>>> 2.7G    dir-studio-ui-tests-in-docker
>>> 1.3M    Empire-db reports
>>> 19M    Empire-db snapshot
>>> 4.0K en-artifact-resolver_master-TCRLMCKTTCPXV2IPJG6T7K54FQQY2O7AFHHEFILBSO6ZFSIUCGRQ@tmp
>>> 4.0K en-box_maven-mapping_master-VTBNHOZQB7UZ4ORPSWLQNN55JVWPXJBJNR54WLBERZKMDQGEPMXA@tmp
>>> 4.0K en-jxr_asfMavenTlpPlgnBuild-WCSOEZUCROETKKJCNPTYSFNDV52TA7MIFL63KMUYVDEPFRT5OK4Q@tmp
>>> 4.0K en-patch-plugin_MNGSITE-332-OEGOX4KAMT3HNPJNZR44Z57IABRBYYJTCNVPYZHJPINSHFM6IBJQ@2@tmp
>>> 4.0K en-patch-plugin_MNGSITE-332-OEGOX4KAMT3HNPJNZR44Z57IABRBYYJTCNVPYZHJPINSHFM6IBJQ@tmp
>>> 4.0K en-toolchains-plugin_master-JSJRH7N7WHA5OITGTMSQVCI6AVNUPHTOLBYXKJXWNGGNGCYHSF5A@tmp
>>> 287M g-site_scm-project-url-list-PACBIBTTXIHT44AKXWHSV4UWSDD4I7ADSILT76TRJE5VCUTX2CTA
>>> 4.0K g-site_scm-project-url-list-PACBIBTTXIHT44AKXWHSV4UWSDD4I7ADSILT76TRJE5VCUTX2CTA@tmp
>>> 419M    guacamole-rest-api-test
>>> 997M HBase_Nightly_branch-1.3-7YSYBQ4PRBNZXQ26CWR7INGGD64I37BNOE6TLABRP5HTPURGHBPQ
>>> 4.0K HBase_Nightly_branch-1.3-7YSYBQ4PRBNZXQ26CWR7INGGD64I37BNOE6TLABRP5HTPURGHBPQ@tmp
>>> 333M HBase_Nightly_branch-1-AUHDHKRYNR4KK4DI4KZ6UHJ26DVLDRQOZOFPOYKX6D6EO3KYD56A
>>> 4.0K HBase_Nightly_branch-1-AUHDHKRYNR4KK4DI4KZ6UHJ26DVLDRQOZOFPOYKX6D6EO3KYD56A@tmp
>>> 2.6G HBase_Nightly_branch-2.1-S5GA5UYEBNAEMXMEVWGCXQO7HNXNX47M5KZBXWNNXMFDW4LAP27Q
>>> 8.0K HBase_Nightly_branch-2.1-S5GA5UYEBNAEMXMEVWGCXQO7HNXNX47M5KZBXWNNXMFDW4LAP27Q@tmp
>>> 36M HBase_Nightly_branch-2-MO2I3RYUAQ2OT52Z2B54XSO6W34ZHP3JU2DA5UQYCAVNS747RZWQ
>>> 4.0K HBase_Nightly_branch-2-MO2I3RYUAQ2OT52Z2B54XSO6W34ZHP3JU2DA5UQYCAVNS747RZWQ@tmp
>>> 1.4G HBase_Nightly_HBASE-20112-STZFPQJNFOBXVX6TM3W4MPKJAH6TZRFVCCDUMAA7ASRQ6ONM4G3Q
>>> 4.0K HBase_Nightly_HBASE-20112-STZFPQJNFOBXVX6TM3W4MPKJAH6TZRFVCCDUMAA7ASRQ6ONM4G3Q@tmp
>>> 596K HBase_Nightly_HBASE-20364-Q5VZYXODTSMLHQ4BICHKC6YELVAH3QMO5DND6NDAF3UTNDVMWXAQ
>>> 4.0K HBase_Nightly_HBASE-20364-Q5VZYXODTSMLHQ4BICHKC6YELVAH3QMO5DND6NDAF3UTNDVMWXAQ@tmp
>>> 2.8G HBase_Nightly_master-4PMG3QPNOXT5YRQZS7HMZP3GLNX6XSF6DVHYXYIB5BWQ75VW3CPA
>>> 4.0K HBase_Nightly_master-4PMG3QPNOXT5YRQZS7HMZP3GLNX6XSF6DVHYXYIB5BWQ75VW3CPA@tmp
>>> 538M    Hive-trunk
>>> 3.2G    incubator-netbeans-linux
>>> 744M    incubator-netbeans-linux-experiment
>>> 4.0K -info-reports-plugin_master-32Y3P743XLZN3V3HHV4NR6ZSZMFDM7Z5PXO3RSN6SA3U6NVU4ZTA@tmp
>>> 108K    infra-test-ant-ubuntu
>>> 12K    infra-test-docker-ubuntu
>>> 60K    infra-test-jdk-ubuntu
>>> 20K    infra-test-jdk-ubuntu-harmony
>>> 40K    infra-test-jdk-ubuntu-ibm
>>> 56K    infra-test-maven-ubuntu
>>> 12K    infra-test-snap-ubuntu
>>> 12K    infra-test-xvfb
>>> 1.4M    Ivy-Snapshot-Deploy
>>> 99M    Ivy-tests-ubuntu
>>> 16M    Ivy-tests-Windows
>>> 12M    Jackrabbit-trunk
>>> 4.0K javadoc-plugin_MJAVADOC-444-JPYTCQM73CF7UEOSYS6Z2MLTMHWVHYCB5QGVIGAH4MONP6SX7OHA@tmp
>>> 4.0K javadoc-plugin_MJAVADOC-516-Z27AS7O3YE3KP4HN2X3EMP56QPDDJR4P5EBKM2B5MKLXDNLXZ3DA@tmp
>>> 19M    Jena_Development_Test
>>> 731M    Jena_JDK11
>>> 264M    kafka-1.0-jdk7
>>> 290M    kafka-1.1-jdk7
>>> 501M    kafka-2.0-jdk8
>>> 628M    kafka-pr-jdk10-scala2.12
>>> 582M    kafka-pr-jdk8-scala2.11
>>> 523M    kafka-trunk-jdk8
>>> 60M    karaf-cave-master
>>> 4.8G    karaf-master
>>> 344M    Log4j 2 3.x
>>> 173M Log4jCore_master-7CCUSWJNA426OZKLRJWUDV3PDMWZHLNIFTRT7HQZXCV5SQ2476XQ
>>> 93M Log4jCore_master-7CCUSWJNA426OZKLRJWUDV3PDMWZHLNIFTRT7HQZXCV5SQ2476XQ@2
>>> 4.0K Log4jCore_master-7CCUSWJNA426OZKLRJWUDV3PDMWZHLNIFTRT7HQZXCV5SQ2476XQ@2@tmp
>>> 4.0K Log4jCore_master-7CCUSWJNA426OZKLRJWUDV3PDMWZHLNIFTRT7HQZXCV5SQ2476XQ@tmp
>>> 4.0K ly-plugin_FIXING-JDK-ISSUES-IEX46HBDBJ6NRUCALBPK7SVRYXYL7WLKQAOOVWNB63FAXVHXQWMQ@tmp
>>> 11M maticModuleNameImprovements-5CTZJI7MCAG23DNAIMMOTYFEI4EKBOZ2AOP3EOG34Q46K3EBQVBA
>>> 4.0K _maven-archiver_MSHARED-721-QXH4Z3WHL2IJTFJJIFNELXYZM6YOEOOXEJCOQA7JK5JMIS42PNSA@tmp
>>> 4.0K maven-box_maven-jxr_JXR-110-R2OIXR7IG7PF3QUMPP2IV53ZAKEOGWXENXIP7G2LROL7YOEOZZTA@tmp
>>> 4.0K maven-box_maven-jxr_JXR-125-KFC5JZL777UAPG4UY6F434EV6BJ2ZRPG7HBO32YUS3UJ56NWROZQ@tmp
>>> 4.0K maven-box_maven-jxr_master-HV7BHOKN2YNGQ4SOGMW6QYUNHKCZDFS3I74XW3IMRQXBAUKY6MSQ@tmp
>>> 8.0K maven-box_maven_master-DVEN5KAF4FLE3WDK7SWFNYUY4YZ7X45JVWYY6FZT5ZEB3CLWABVQ
>>> 5.7M maven-box_maven_MNG-5868-TIXVUCFHOPHJ7LSBIS4NVVVTXHOVMMQO4W2BXVF455AQYGYZ26SA
>>> 8.0K maven-box_maven_MNG-6391-BFSVDMPDHLGZMTGMVMMLWPKG4K2P6CHHPIPVW27SUVBB6UQWV7YA
>>> 8.0K maven-box_maven_MNG-6475-LMWI4CBOH2JNDHX4PPHOWPHEASG5JKLTTZW3URJSHEBTPFA2447Q
>>> 4.0K maven-box_maven-osgi_master-537V4ALFOX2ZNQAIHHNQL5TC7LIKAHUUTBK4WE42PW2CUAN3GTBQ@tmp
>>> 4.0K maven-box_maven-scm_master-LZ7UVRJQGWEFPDXKGVXHDWRQ4JZLLSJWYQRZLEZSFZG4LONP4W2A@2@tmp
>>> 4.0K maven-box_maven-scm_master-LZ7UVRJQGWEFPDXKGVXHDWRQ4JZLLSJWYQRZLEZSFZG4LONP4W2A@tmp
>>> 4.0K maven-changes-plugin_master-O3PQ7WZZHAA2T5PBIGGQZFFXFD6PLMGLZHAF4U525J6NJFJA5IMQ@tmp
>>> 4.0K maven-install-plugin_master-3R4FSR6ITADJX4LXJM433BEETYXHTGVGRU6VEEEHH2PU3W5LNT6A@tmp
>>> 4.0K _maven-jlink-plugin_CLEANUP-YYRC5VLF7HAIAIEICXGYPF25BOQW6NXDQRTJOEZNHEUF2NVU67EA@2@tmp
>>> 4.0K _maven-jlink-plugin_CLEANUP-YYRC5VLF7HAIAIEICXGYPF25BOQW6NXDQRTJOEZNHEUF2NVU67EA@tmp
>>> 4.0K _maven-project-utils_master-YFJOAALTBBMYPILDHBYJYDFNTT6HPH3JYBY5B7IKICAXCAIG7IYA@2@tmp
>>> 4.0K _maven-project-utils_master-YFJOAALTBBMYPILDHBYJYDFNTT6HPH3JYBY5B7IKICAXCAIG7IYA@tmp
>>> 4.0K maven-reporting-exec_master-XY3GEBFQ42VNXZOS5MVD7PZKI6HQ7WOMNLQJ5IUZ7XSHVEI5U6RA@tmp
>>> 5.0G    Mesos-Buildbot
>>> 1.2M    Mesos-Reviewbot
>>> 4.0K mpiler-plugin_MCOMPILER-320-LDDLDH7HQQYXD6B2RHESP35HSEBJ26NA355LSJP2PHGG3DW43OPQ@tmp
>>> 4.0K m-publish-plugin_MSCMPUB-38-ULK6I7N2ZA3FU5JGEWCSL4ODBTZDTTC77NU7CUHQJKPNE7QVYS7A@tmp
>>> 4.0K n-archiver_MSHARED-668-test-WG4DQP6EJS2N73J3VGM2JH6SMRRHF477RDRLQRWF4WAHIMWMG5OA@tmp
>>> 4.0K n-box_maven-enforcer_master-KHI64AG3D4VZKUKLMNAYHKONI5AIQBLJ5ZKYGGSW7DAZVXCJBFHQ@tmp
>>> 4.0K n-box_maven-release_cleanup-YGG2KC5W5NBVLT77VULX2CHSYMVGRKDIZ7FJM54R7TMFDTDO6JYA@tmp
>>> 4.0K n-box_maven-surefire_master-CK52RLZV7R5PVUUYTYVCEEENMKBFPX7NV3GV6YB4YGV3EM4ZIPOQ@2@tmp
>>> 4.0K n-box_maven-surefire_master-CK52RLZV7R5PVUUYTYVCEEENMKBFPX7NV3GV6YB4YGV3EM4ZIPOQ@tmp
>>> 4.0K n-box_maven-wagon_WAGON-482-U5UQDLHLBEYZ6YKA2XFN2FQ74OK76EBRADZ4AK62MZWPJOQW2HDQ@tmp
>>> 4.0K n-box_maven-wagon_WAGON-526-MVYPB7RAMW5VZ4RA4NSMECE376IR6WO6PB7AIWCZTHT5KJGFZFHQ@2@tmp
>>> 4.0K n-box_maven-wagon_WAGON-526-MVYPB7RAMW5VZ4RA4NSMECE376IR6WO6PB7AIWCZTHT5KJGFZFHQ@tmp
>>> 4.0K n-box_maven-wagon_WAGON-527-KWYDRBGRPK7Q2YVCYHYJDNDOSEOHGQPJHV5AHJB4GNG66RKBHCBA@tmp
>>> 28K    Nemo
>>> 4.0K nfo-reports-plugin_MPIR-366-S464FI4ATSVOULE3N7MCKV5FYYG6GVHDD7OLISMYBGUEPLQM2UNA@tmp
>>> 5.7M _not-updated-MJAR-MCOMPILER-EYMH2UKFHCTFDWW4GZGG3AX4OJLGLPGZO6TAQGLBDBFX4K6CQ5GQ
>>> 4.0K n-resolver-ant-tasks_master-PZI36OPNRAJBXJOAPEQN5EKT3YVHZTR3KX72Y4FQGBMQE6GFTARQ@2@tmp
>>> 4.0K n-resolver-ant-tasks_master-PZI36OPNRAJBXJOAPEQN5EKT3YVHZTR3KX72Y4FQGBMQE6GFTARQ@tmp
>>> 4.0K n-scm-publish-plugin_master-2GNHI3CFRHSCHWWMDIRTSWMJIE3KK6DFWPSDNLSGKYY2ZVCACIZA@tmp
>>> 4.0K n-testing_MPLUGINTESTING-55-AWMCSLLBQGJRKNRZR27EZX2B5SVGADVMVK4FGA6YEL5S4OR3TOZQ@tmp
>>> 4.0K n-testing_MPLUGINTESTING-58-UONNAARVYXAHZ4RVLUBM5ODIM6A4TFB5CEC4ELRKMGU3KDYXAPIQ@tmp
>>> 800K nts-spring-taskqueue_master-DEHCJRMAJ4RCCKE7MZYOUSHRWFQYJUMVRFF7GSAKS7JROVA4UVRA
>>> 4.0K nts-spring-taskqueue_master-DEHCJRMAJ4RCCKE7MZYOUSHRWFQYJUMVRFF7GSAKS7JROVA4UVRA@tmp
>>> 343M    OpenWhisk-DockerHub
>>> 4.0K    OpenWhisk-DockerHub@tmp
>>> 4.0K ox_maven-doap-plugin_master-K56SP7WQZT7DLJESN4Z67EH56OK5QEQZKI2M7VJKWLIVTWMNHWNA@2@tmp
>>> 4.0K ox_maven-doap-plugin_master-K56SP7WQZT7DLJESN4Z67EH56OK5QEQZKI2M7VJKWLIVTWMNHWNA@tmp
>>> 4.0K ox_maven-fluido-skin_master-6RM5JATOOS6MTECVSVHV7YNCQIINRWOBDST2UCUIKNH64TYTME6Q@tmp
>>> 4.0K ox_maven-help-plugin_master-U66QJWH3TRYC4M2J55DEV6AO7VQ2JW77H5GHKBUOBKZTW32WDTTA@tmp
>>> 4.0K ox_maven-jmod-plugin_master-35YKHMTP6DUYFXQHM4NKZCKZAUADGHEV3PUSPURFALNQQYRXE5BQ@2@tmp
>>> 4.0K ox_maven-jmod-plugin_master-35YKHMTP6DUYFXQHM4NKZCKZAUADGHEV3PUSPURFALNQQYRXE5BQ@tmp
>>> 4.0K ox_maven-site-plugin_master-LZWAKOPRMO5JAZW5HRA2IQ56LLTFU3PLKDIE2JNFFXEGASBYKK3Q@tmp
>>> 667M    PDFBox-trunk
>>> 171M    PDFBox-Trunk-jdk10
>>> 52M    Phoenix-4.x-HBase-1.2
>>> 1.6G    Phoenix-4.x-HBase-1.3
>>> 6.7M    PreCommit-Admin
>>> 2.4G pulsar_precommit_integrationtests_ws-cleanup_1537313772888
>>> 4.4G pulsar_precommit_integrationtests_ws-cleanup_1537469839992
>>> 331M    Qpid-Broker-J-7.0.x-TestMatrix
>>> 331M    Qpid-Broker-J-Artefact-Release
>>> 339M    Qpid-Broker-J-JoramJMSTest
>>> 733M    Qpid-Broker-J-Python-Test
>>> 240M    Qpid-Broker-J-Test-IBMJDK1.8
>>> 12M    Qpid-Java-Java-6.1.x-TestMatrix
>>> 29M    Qpid-proton-c
>>> 1.3M    Qpid-proton-j
>>> 12M    ranger-0.7-snapshot-publish
>>> 256M    Ratis-Master
>>> 4.0K rces-plugin_MRRESOURCES-110-PEC2YE7NKYL6E63GN7TSRMPI5Q3DHSCF7W5XK77IXP5MYQPGBVDA@2@tmp
>>> 4.0K rces-plugin_MRRESOURCES-110-PEC2YE7NKYL6E63GN7TSRMPI5Q3DHSCF7W5XK77IXP5MYQPGBVDA@tmp
>>> 125M    samza-pr-checks
>>> 4.0K sembly-plugin_MASSEMBLY-892-AIWNXXLMVTBQ62GDGAZ5RR52AYYD2PS4MNOQBOW2K6A6AS4WHTGQ@tmp
>>> 50M    ServiceMix-Bundles
>>> 36M sling-site_master-RGGZSQNSRE77KJSAWGA5ISC4OHTBYZLT56LJVWLLL57HSX2R5R2A
>>> 4.0K sling-site_master-RGGZSQNSRE77KJSAWGA5ISC4OHTBYZLT56LJVWLLL57HSX2R5R2A@tmp
>>> 282M    Struts-2-3-JDK6
>>> 4.0K -surefire_V30_SUREFIRE-1535-6LAZP67UAOYQ7KUSYGA75JJSKWQN3XZICMKBDXXTUO5K7RMMATWQ@2@tmp
>>> 4.0K -surefire_V30_SUREFIRE-1535-6LAZP67UAOYQ7KUSYGA75JJSKWQN3XZICMKBDXXTUO5K7RMMATWQ@tmp
>>> 16M    Syncope-2_0_X-deploy
>>> 4.3G    Syncope-2_1_X-deploy
>>> 16M    Syncope-master-deploy
>>> 21M    Tamaya-Javadoc-Master
>>> 23M    Tamaya-Master-branch
>>> 74M    Tobago Trunk
>>> 81M    Tobago Trunk Sonar
>>> 4.0K    Tomcat-7.x-Maven
>>> 350M    trafficcontrol-master-build
>>> 4.0K    trafficcontrol-master-build@tmp
>>> 115M    trafficcontrol-PR-rat
>>> 41M    trafficcontrol-PR-trafficops-test
>>> 162M    trafficcontrol-traffic_ops-test
>>> 4.0K    trafficcontrol-traffic_ops-test@tmp
>>> 590M    unomi-master
>>> 4.0K ven-archetype_ARCHETYPE-551-HRU2Q2KOTZQ5SCJRJRFH54HWHKWPNEX6PLYZFXKJEPWSWHHELAOA@2@tmp
>>> 4.0K ven-archetype_ARCHETYPE-551-HRU2Q2KOTZQ5SCJRJRFH54HWHKWPNEX6PLYZFXKJEPWSWHHELAOA@tmp
>>> 4.0K ven-box_maven-surefire_1532-O6EC5ODCIKDKZL25QDLH6QFMFZD5LDHASAYEMDZSNREZV4N37X7A@tmp
>>> 4.0K ven-scripting-plugin_master-F7DFWWXZD7KAQGDPAQML3SYNOE2CUDOJDQGTJN7NKKPXY3SWUHCQ@tmp
>>> 4.0K ven-site-plugin_MNGSITE-332-4LYPVCGK77QO5G6UKI6IR7JSEN5EMK7EVKG6TK2D3VYBAHH6QPBA@tmp
>>> 4.0K x_maven-clean-plugin_master-I6NGCLTWS42DBQD52EFLHANYNHM7TDPGSN6PDGRJS5IR6XUDW46Q@2@tmp
>>> 4.0K x_maven-clean-plugin_master-I6NGCLTWS42DBQD52EFLHANYNHM7TDPGSN6PDGRJS5IR6XUDW46Q@tmp
>>> 4.0K x_maven-invoker_MSHARED-752-O2LI4W4PL6ARSMCGVOOVX6GZJCOYVGXMTNU5KTGD65M6CNMTDSRQ@tmp
>>> 4.0K x_maven-jdeps-plugin_master-NK4576UNJJCYQVC667D46I6VDU64O3SNLDEXNWG3R2K63JIOVMYA@2@tmp
>>> 4.0K x_maven-jdeps-plugin_master-NK4576UNJJCYQVC667D46I6VDU64O3SNLDEXNWG3R2K63JIOVMYA@tmp
>>> 4.0K x_maven-jlink-plugin_master-TWLBBVLGC7P5GUJSJEP6SD46MW2WP64SO5MED7OJBUOM66RIHIJQ@tmp
>>> 4.0K x_maven-patch-plugin_master-AJNCC2MTSOLL2XNT6A33RFSXCRCGBAJM5IANLZY75KP6MNCEGTWQ@2@tmp
>>> 4.0K x_maven-patch-plugin_master-AJNCC2MTSOLL2XNT6A33RFSXCRCGBAJM5IANLZY75KP6MNCEGTWQ@tmp
>>> 4.0K x_maven-release_IMPROVE_POM-KZP6VA3ZSGGYNTVVOCGLUBHYECJZQ34VIS6H2U5HNYVKLJFCFQKQ@tmp
>>> 4.0K x_maven-shared-utils_master-2E2DRWROHLNJSGQVJQOXBED6MNLVPJLSLU6DGTX6YKEIJNHAYWWA@2@tmp
>>> 4.0K x_maven-shared-utils_master-2E2DRWROHLNJSGQVJQOXBED6MNLVPJLSLU6DGTX6YKEIJNHAYWWA@tmp
>>> 4.0K x_maven-stage-plugin_master-DUIB3VWMFIP4K7X4DPTLYSUTRVJKIKKQPIMXJ3UK6V5VUGFJHXHQ@tmp
>>> 4.0K x_maven-wagon_wagon-scm-git-JFLMMYBLUJZHZUM4KST663ZOB5W6QQB2OF5JLK4YVV3OMUVRSJDA@tmp
>>> 340M    Yetus - OWASP dependency-check cache
>>> 119M    zeppelin-pull-request
>>> 93M    ZooKeeper-Find-Flaky-Tests
>>> 465M    ZooKeeper-trunk-openjdk10
>>> 157M    ZooKeeper-trunk-owasp
>>> 119M    ZooKeeper_UT_Stress
>> 
>