osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: kahadb index file larger than journal file


Christopher,


Thank you very much!

:), I will track this bug and verify the fix after 5.14.4 released.


Regards,

Scorpio

On 2017/2/2 19:59, christopher.l.shannon [via ActiveMQ] wrote:
> Sorry for taking a while to get back to you.
>
> Based on your originally message and after analyzing some index files I
> have actually discovered a bug with KahaDB that causes the index file to
> grow too large when things like out of memory errors occur or any unclean
> shutdown.  On recovery the free pages in the index file are not being
> properly tracked which is leading to excessive index file usage.  So this
> is probably the issue that you are seeing.  I will be fixing this in
> version 5.14.4.  Upon upgrade you will need to delete the old index and
> have it be rebuilt and then going forward it should work as intended. 
>  You
> can track the status of the bug here:
> https://issues.apache.org/jira/browse/AMQ-6590
>
> In terms of the in memory cache, only a fixed number of pages are cached.
> The default is 10 thousand pages and each page is 4KB.  So roughly 40 
> MB of
> memory will be used at most.  If the number of pages in use grows beyond
> that then the cache will swap pages based on the defined eviction
> strategy.  You can increase the number of pages in memory (which would 
> help
> performance at the cost of eating up memory) by tweaking the 
> indexCacheSize
> value of KahaDB.  See http://activemq.apache.org/kahadb.html for more 
> info.
>
> On Tue, Jan 24, 2017 at 11:00 PM, Scorpio <[hidden email] 
> </user/SendEmail.jtp?type=node&node=4721502&i=0>> wrote:
>
> > Christopher:
> >
> >
> > Thank you for your explanation!
> >
> > Hope there would be other proper ways to shrink the index file in 
> future
> > ActiveMQ releases.
> >
> > Once we monitored that our broker is killed by system due to "out of
> > memory".
> >
> > I know that meta cache in memory will be synced into index file at
> > checking points. _*Does this mean there is as much memory as  the
> > db.data file being used?*_
> >
> > Even though our broker is idle,  we still find that it uses up much
> > memory. Could they be relevant associated? Thanks
> >
> >
> > Kind regards,
> >
> > Scorpio
> >
> > On 2017/1/24 20:16, christopher.l.shannon [via ActiveMQ] wrote:
> > > The index grows based off the amount of data it tracks in the 
> journal,
> > > but doesn't shrink.  This means if you had a lot of data in your
> > > journal at some point (several gigabytes, etc) then the index would
> > > have grown in size with enough page files to keep track of all of the
> > > data.  This is usually not an issue because the index size generally
> > > stabilizes at some point relative to the work load of the broker. 
>   If
> > > you really want to shrink it the only way right now is to stop the
> > > broker, delete the index and restart so it does a replay and rebuilds
> > > the index but the size would just grow again as messages flow.
> > >
> > > On Mon, Jan 23, 2017 at 8:06 PM, Scorpio <[hidden email]
> > > </user/SendEmail.jtp?type=node&node=4721300&i=0>> wrote:
> > >
> > > > Hi, All,
> > > >
> > > > In my activemq instance, I find that the kahadb index file is 
> larger
> > > than
> > > > the journal file.
> > > >
> > > > As far as I know, /db.data/ is the meta store, index for the 
> journal
> > > file,
> > > > which should means it is much smaller than the journal file.
> > > However, it
> > > > does not look like that. The index file seems to keep growing 
> while the
> > > > journal file is rotated. Do I miss something? Thanks.
> > > >
> > > > Log attached:
> > > > root@host:~/run/apache-activemq-5.13.1/data/kahadb# du * -sh
> > > > 16M     db-16.log
> > > > 1.2G    db.data
> > > > 3.2M    db.redo
> > > > 4.0K    lock
> > > > root@host:~/run/apache-activemq-5.13.1/data/kahadb#
> > > >
> > > > Kind regards,
> > > > Scorpio
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > http://activemq.2283324.n4.nabble.com/kahadb-index-file-
> > larger-than-journal-file-tp4721284.html
> > > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> > >
> > >
> > > 
> ------------------------------------------------------------------------
> > > If you reply to this email, your message will be added to the
> > > discussion below:
> > > http://activemq.2283324.n4.nabble.com/kahadb-index-file-
> > larger-than-journal-file-tp4721284p4721300.html
> > >
> > > To start a new topic under ActiveMQ - User, email
> > > [hidden email] </user/SendEmail.jtp?type=node&node=4721502&i=1>
> > > To unsubscribe from kahadb index file larger than journal file, click
> > > here
> > > <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=
> > unsubscribe_by_code&node=4721284&code=bnlAeGlhbmdqaWFiYW8uY29tfDQ3Mj
> > EyODR8MTkzNzMzOTQ5NQ==>.
> > > NAML
> > > <http://activemq.2283324.n4.nabble.com/template/
> > NamlServlet.jtp?macro=macro_viewer&id=instant_html%
> > 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> > BasicNamespace-nabble.view.web.template.NabbleNamespace-
> > nabble.view.web.template.NodeNamespace&breadcrumbs=
> > notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> > 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> > >
> >
> >
> >
> >
> >
> > --
> > View this message in context: http://activemq.2283324.n4.
> > nabble.com/kahadb-index-file-larger-than-journal-file-
> > tp4721284p4721315.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
>
>
> ------------------------------------------------------------------------
> If you reply to this email, your message will be added to the 
> discussion below:
> http://activemq.2283324.n4.nabble.com/kahadb-index-file-larger-than-journal-file-tp4721284p4721502.html 
>
> To start a new topic under ActiveMQ - User, email 
> ml-node+s2283324n2341805h23@xxxxxxxxxxxxx
> To unsubscribe from kahadb index file larger than journal file, click 
> here 
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4721284&code=bnlAeGlhbmdqaWFiYW8uY29tfDQ3MjEyODR8MTkzNzMzOTQ5NQ==>.
> NAML 
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> 
>





--
View this message in context: http://activemq.2283324.n4.nabble.com/kahadb-index-file-larger-than-journal-file-tp4721284p4721557.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.