Re: data/olap cubes

Adam,

I am interested in your OLAP ideas.  I've haven't used the existing ofbiz
olap tables directly, but I have used them to model new Star schemas.  I
capture a lot of performance testing data and needed a way to query and
report the results.

A lot of the work in data mining is breaking down the data to work
appropriately in a start schema topology.  Creating a standard way to do
this would be very helpful.  I like the idea of a dynamic view.

My approach to date has been the following:

- Define Fact Table and reuse DimensionTables as much as possible
- Create common Java utils to handle dimension table look up from a flat
file of imported data
- Create Groovy scripts to tie the Java utils and file parsing together

I have also considered just using a common Fact Table entity with just
generic column names and types to capture most common data imports.  Then
use a meta table dimension table to define the columns used for the
particular fact.

If anyone has thoughts on a generic approach to data mining I would
appreciate the feedback.

Thanks Adam for bringing up the topic.


Brett



On Wed, Jul 29, 2009 at 10:14 AM, Adam Heath <doogie@xxxxxxxxxxxxx> wrote:

> Has anyone had any dealings with using data/olap cubes with ofbiz?
> I've recently had some thoughts on how to maintain such data
> structures automatically.
>
> This is somewhat of a brainstorm, but this is what I'm thinking.
>
> Define the cube with a dynamic view entity(possibly using my sql
> representation).  Have this dve serialized to a DataCubeDescriptor
> table.  Then, during ofbiz startup, it would read from
> DataCubeDescriptor, auto-create any missing summary tables, then add
> dynamic reverse ecas, so that any time the underlying data is
> modified, the summary table would be auto-updated with new data.
>
> This would allow data cube querying to be instantaneous, because it
> would just be dumping the summary table's contents.
>
> Does anyone else understand what I am describing?  Is anyone else
> interested?
>