Subject: Re: Chaining MapReduce function pairs - msg#00348

List: couchdb-user

Mail Archive Navigation:
by Date: Prev Next Date Index by Thread: Prev Next Thread Index

It doesn't have to be a separate DB though. Example would be: you want
to run a view (report), say once a day, and then trend on this (using
another view).

It's possible to generate the view results, add that back to couchdb
as a document with a timestamp and then have meta-views that
map/reduce these views to pull out trends and what not. I've done this
and works like a charm. This kinda view is awesome from a motion chart
perspective
(http://code.google.com/apis/visualization/documentation/gallery/motionchart.html)
where you can see 4-dimensional data transforming (time being one of
the dimensions).

While couch doesn't "natively" support this, this kind of thing is
possible since the results of a view is just JSON which itself can be
another native document.

K.

On Tue, Jul 28, 2009 at 8:00 PM, Paul Joseph
Davis<paul.joseph.davis@xxxxxxxxx> wrote:
> There's nothing builtin. Me and a couple others have tried to do such things
> but nothing has ever stuck. At the moment your best bet would be to write
> som helpers to copy view output to a new db and go from there.
>
>
>
> On Jul 28, 2009, at 4:53 PM, Cortland Klein <me@xxxxxxxxxxxxx> wrote:
>
>> I'm interested in creating views that chain multiple pairs of MapReduce
>> functions together. This would allow me, for example, to do complex data
>> analysis that's not possible with only one pair of MapReduce functions.
>> When I say chain I mean the second map function takes as its input the
>> key-value pairs outputted by the first reduce functions.
>>
>> Is this something CouchDB can do or will be able to do in the future?
>>
>> --
>> Cortland Klein <me@xxxxxxxxxxxxx> +1 408 506 9791
>> http://pixelcort.com/
>> 2260 California Street #13
>> Mountain View, CA, USA 94040
>

Thread at a glance:

Previous Message by Date:

Unique human readable ID

I get the whole point of letting couchdb generate id's. At the same time I need a way to create a human memorable/readable/communicative id: - ID: 45060, 45061, 45062, etc That is a whole lot simpler to relay over a phone call compared to: - 1d98d82f5b6a628527344ba991cb7e2f What have people done to satisfy the users of their systems with a simple id and also be able to replicate the documents between different couchdb instances? I've come up with 2 ideas: # 1 Have a single couchdb that has 50 documents (tickets) (figure out a way to always have 50 available) in it like what is mentioned in the wiki to do transactions[1] #2 For each instance of the application prefix the id with it's instance id so 1.45060 & 2.45060 have the same id but were created on different instances. Therefore when the two instances of the couchdb's replicate with each other there are two unique id's I can see this adding a fair amount of overhead if it gets to the point of more then a few instances... If I drop the requirement of replication everything becomes simple. Then I have to wonder - why am I using CouchDB then?? Nick [1] - http://wiki.apache.org/couchdb/Frequently_asked_questions#transactions

Next Message by Date:

Re: Unique human readable ID

> > If I drop the requirement of replication everything becomes simple. Then I > have to wonder - why am I using CouchDB then?? > load balancing, backup, http api, caching, schemaless and probably many other reasons ;) I usually generate slugs when I need to use easy doc ids. You can add a timestamp to your slugs, prepend them by your node id, verify their uniqueness in your local DB etc... It works great for me. - Matt On Tue, Jul 28, 2009 at 8:33 PM, Nicholas Orr <nicholas.orr@xxxxxxxxx>wrote: > I get the whole point of letting couchdb generate id's. > At the same time I need a way to create a human > memorable/readable/communicative id: > - ID: 45060, 45061, 45062, etc > That is a whole lot simpler to relay over a phone call compared to: > - 1d98d82f5b6a628527344ba991cb7e2f > > What have people done to satisfy the users of their systems with a simple > id > and also be able to replicate the documents between different couchdb > instances? > > I've come up with 2 ideas: > > # 1 > Have a single couchdb that has 50 documents (tickets) (figure out a way to > always have 50 available) in it like what is mentioned in the wiki to do > transactions[1] > > #2 > For each instance of the application prefix the id with it's instance id so > 1.45060 & 2.45060 have the same id but were created on different instances. > Therefore when the two instances of the couchdb's replicate with each other > there are two unique id's > I can see this adding a fair amount of overhead if it gets to the point of > more then a few instances... > > If I drop the requirement of replication everything becomes simple. Then I > have to wonder - why am I using CouchDB then?? > > Nick > > > [1] - > http://wiki.apache.org/couchdb/Frequently_asked_questions#transactions >

Previous Message by Thread:

Re: Chaining MapReduce function pairs

There's nothing builtin. Me and a couple others have tried to do such things but nothing has ever stuck. At the moment your best bet would be to write som helpers to copy view output to a new db and go from there. On Jul 28, 2009, at 4:53 PM, Cortland Klein <me@xxxxxxxxxxxxx> wrote: I'm interested in creating views that chain multiple pairs of MapReduce functions together. This would allow me, for example, to do complex data analysis that's not possible with only one pair of MapReduce functions. When I say chain I mean the second map function takes as its input the key-value pairs outputted by the first reduce functions. Is this something CouchDB can do or will be able to do in the future? -- Cortland Klein <me@xxxxxxxxxxxxx> +1 408 506 9791 http://pixelcort.com/ 2260 California Street #13 Mountain View, CA, USA 94040

Next Message by Thread:

Re: Chaining MapReduce function pairs

One solution I see would be the output of a view being treated as a first class (read-only albeit) database that itself could contain _design docs. The MR functions of the first _design doc would have to carefully include the _design doc in the resulting reduced set so it could be used.