Re: RE: [Plone-developers] Plone 1.1 - TODO

<snipppety-snip />

> > The immediate fix for this is easy - just change the size of the random
> > number pool in _genId() to a larger number.  This should not break 
> > backward
> >
> > compatibility with old uids, I think.  For example:
> > >>> import time
> > >>> i = int((time.time() % 1.0) * 10000)
> > >>> i
> >
> > 8646  #won't be larger than 10k
> >
> > >>> i = int((time.time() % 1.0) * 1000000)
> > >>> i
> >
> > 528131  #won't be larger than 1Million
>
> Your rigth, It is 10000 not 10000.
> Neverthelesse I did run the following loop:
> x = {}
> while 1:
> x[int(time.time() % 10000)] = 1
>
> after running five minutes on a moderately fast machine it had produced 
> only 335 entries in the range from 7034 to 7438.
>
> I intended to batch create a couple of tousand users with each an 
> index_html in their home folder. This is just not possible with the 
> current implementation.

Correct.

Attempting to use time as anything other than the *seed* of a random number 
generator is a futile exercise. No matter what floating-point precision you 
use, you're assigning numbers based on a pattern that repeats every second. 
In a system where the assignment of an ID can take a fraction of a 
millisecond, this is obviously going to cause problems.

The immediate answer (based on fewest lines of code) is to use the actual 
output of time.time() (floating point representation of seconds since 
epoch), which should prevent any ID conflicts unless you're assigning at a 
rate faster than the precision of time.time() (which is to 6 decimal places 
on my kit, but could be only to 3 DP on systems with low-resolution 
counters). Again, this is totally CPU dependent and is a situation that may 
occur...

> > Another thought: Right now, I'm working on a patch to ReferenceEngine 
> > that
> > (a) replaces PersistentMapping with OOBTrees, (b) migrates existing
> > PersitentMapping reference attributes to OOBTrees to support upgrade, 
> > and
> > (c) adds methods for aggregate ref/bref querying (putting proper
> > names/labels on brefs via lookup table).   For the future: In 
> > approaching
> > (a) I was thinking about how BTreeFolder2 does its UID generation: 
> > random
> > ISN and sequential IDs afterward.  This is a bucket/thread affinity and
> > conflict-minimization mechanism, as it is likely that the ISN that each
> > thread/client picks will be in a different bucket than other threads
> > updating the btree and each thread tends to stay within its own btree. 
> > This may be a YAGNI thing for references, but I suppose there could be a
> > high-write situation on a site where all content heavily use references 
> > and
> > there are high-frequency updates.

Yep. I'm currently examining a situation with a number of objects that is 
several orders of magnitude more than required to make the current ID 
assignment explode.

As Zope/Plone increasingly makes its way into the enterprise, this kind of 
situation is going to become more and more common. Nice to know that large- 
scale architecture is being taken into account.

Regards,

- seb

--
Seb Potter
Lead Developer
Getfrank Limited



-------------------------------------------------------
This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
thread debugger on the planet. Designed with thread debugging features
you've never dreamed of, try TotalView 6 free at www.etnus.com.