Download Firefox: WindowsMac OS X
logo       
Google Custom Search
    AddThis Social Bookmark Button

Re: General questions about Harvesting - installation - setup - exchangemen: msg#00007

Subject: Re: General questions about Harvesting - installation - setup - exchangements
Christoffer Landtman wrote:

Luk Vanlanduyt - vi-host.net BVBA wrote:
Hello Christoffer,

Thank you for your kindly answers. I put my remarks here. Luk
==>



Hello,
sorry that this mail comes in a general list, but there is no contactperson on the homepage.
Hello,

This is actually a very good place to discuss this, keeping in mind the
aspects to be discussed below.


We are working on a project in Belgium that has several goals.

PROJECT

Different institutes have a catalogue. The goal is to make a union catalogue and to harvest on regular times the OAI-compliant institutional repositories. As format we have concluded to use MARC21.
you might look at cern document server.

emilda is primarily aimed as a library-loan system !!!

Emilda seems interesting for us: MARC capabilities / Z 39.50 etc. We can program in several languages and PHP is no problem. Translations can be done in dutch. We see this as a normal gift back for all work
already done.

there are different oai/z3950 harvesters/gateways/whatever-thingies out there - google is your friend :-)

QUESTIONS
Some questions should be resolved before we go on in further participation or using Emilda. a) Is it possible to make the database, containing the bibliographic data in Emilda to a OAI-compliant repository? Is this already provided, or can it be fixed with other tools, someone with experience?
possible in principle, since you can populate dc/oai-relevant fields as needed. per-se emilda is not aimed to be oai-compliant or to deliver dc - i think this could eventually be a 1.4 release thing ? christoffer ?

Emilda uses the Zebra server from Indexdata
(http://www.indexdata.dk/zebra) for bibliographic data. This means that
Emilda itself is not relying on any particular bibliographic format, only
the interface between Zebra and Emilda. Zebra is fully compliant with all
possible standards, and you can even create your own using e.g. regexps etc. For example Zebra can easily index Dublin core records and
present these, MARCXML, you name it. Thus, it should not be too difficult to
create a filter for Zebra to create OAI compliant output of the
bibliographic data. You might find some more concrete information on the
Zebra homepage.

yes it is a zebra-issue. indexdata has no interest to unleak infos for free. one has to get paid support ... i googled around a lot but only found some zebra-configs to deliver dc. nothing for mab.
best place is to look around in opensource-sw which utilizes zebra.

As I understood from the above project description, you would use MARC21
natively, but support OAI-harvesters from reading this data, thus creating
the possibility to present the MARC21 as OAI compliant. Would this be
correct?

======> The project is as follow: A) Some institutions can create a MARCXML output, but have no system to set
it up as OAI Data provider. We have to find a system for them (the most
easiest way is we can put their MARCXML files on a server and those files
can be directly read in a system (f.e. Emilda or other). B) Some institutions can set up a OAI dataproviding server, so we can harvest them. C) Some institutions are looking for a library system. If Emilda can import
data (bibliographic), can create MARC XML, can play the role as Data
provider it would be great. If it can also harvest or read in MARC XML
files, it would be very great.
c)
emilda does not create any data :-) it uses data.
importing is just a matter of physical presence and zebra- and marc-setup (and to count the numbers of items for an existing marc-record in sql ...)
harvesting is a matter of zebra-setup and has nothin to do with emilda-core.
if you decide to use marc-xml as basic bibliographic system it would be easy to adapt emilda as "editing-system" to process xml instead of usmarc - just replace a few lines (ok, some more) in perl ...


Emilda can import e.g. MARCXML and also create MARCXML. Emilda can
ofcourse import other formats, its just a matter of configuration.

as said above i think it is easy to run emilda over macr-xml instead of usmarc if MARC.pl is adapted - am i right christofffer ?
and of course zebra setup will be different ...


b) Can Emilda harvest OAI-compliant repositories?Is this already provided, or can it be fixed with other tools, someone with experience?
Emilda uses the Yaz toolkit also from Indexdata
(http://www.indexdata.dk/yaz) to communicate to both its native Zebra server
but also to other international Z39.50 servers. As this is only a "module"
for getting the data, and by harvesting OAI repositories we get XML-like
data, it should be no problem to create a new module to the Z39.50 gateway
of Emilda to support OAI: convert the OAI to MARC21 via MARCXML for example
should work.

======> We will do some tests, but do you have a lot of experience of those
tools?

Not explicit experience of OAI, but we have e.g. been tackling import
with MAB as Wolfgang may tell you. As bibliographic data is separated in
Emilda, one can easily create external scripts that can load data from
various sources, convert these to Emilda-understandable and then tell
Emilda to use these.

i must object. no one out there either
- did the "implicit" MAB-MARC conversion
or
- willing to unleak detailed infos on how to configure zebra to achieve this

i fear i will have to dig very deep into zebra to get the knowledge.
of course i will not tell anyone how to do it because it was a hard work and it is MY money and time :-))))))))))))

c) I was amazed by the tool to add Marc tags. It was not clear for me how or where you can define which tag should become searchable. For example; If you add 440_a (subtitle of a serial), can you define in the searchscreen that this field should be searchable. Our goal is to create a special searchscreen where we can define by ourselves which marc-tags should be searchable. We have some extra tags used for special information.
best way to add 9xx is to edit marc sql-tables by hand :-). i can supply some examples if you like ...

The beauty of Emilda is that it supports complete MARC transparency. What this in practice implies is that all MARC fields that have been
defined, can be displayed, edited, deleted, searched, sorted etc. Thus
Emilda is fully configurable and transparent against the chosen
MARC-flavour, as long as you tell it to be ;-) Thus using e.g. custom tags
or custom MARC flavours is no problem.



====> Is it possible to import data in Emilda? F.e. users, or books? In
which format?

Users can be imported from e.g. LDAP/AD. We are working on refined LDAP
support currently, but the basics already exist.

yes :-)

Emilda natively supports import of bibliographic data from any Z39.50
server that can provide MARC. However with some minor tweaking, it
should be no problem to support also other sources, as the functionality
already exists.

minor tweaking ? hey lets try MAB first :-)

d) Are there users in Belgium or the Netherlands?
Officially we have not been reported of any users in neither Belguim or the
Netherlands. The closest verified reference case would probably be Wolfgang
Pichlers Emilda project in Austria. But due to the nature of OSS, there
might very well be users in either Belgium or the Netherlands without us
knowing.

=======> It should be great we find some one to set up server etc. Can you
provide a support f.e. using remote desktop? We can do a lot for emilda
(translation, php programmation, but we should as soon be sure the whole
project can succeed. At this moment for the harvesting tool, we are now
looking to i-tor (www.i-tor.org) using Lucene / TEA). Harvesting was no
problem and also the display is working very well. The problem is only
A) i-tor is not a library solution B) i-tor is not familiar with marcxml harvesting
Important for us is that -because we're not familiar with zebra and yaz - we
can during the installation proces have quick answers. We will decide
tomorrow or Wednesday what we will do, depending on answers here.

you are a very fast company.
as said before try cern ...
but it is no library system either.
you will not find any system you like ready to use with key-turn ...
there are simply no mature and/or usable GPL systems out there except emilda and koha.

We will assist you to our best extent possible. If you need assistance
with the server we can give some tips and assistance depending on the needs.

Could you still shed some more light on the purpose of the project; what
is the goal? Will all participants start using the same database, or
just harvest using the same tool? This is very interesting concerning
the "big picture".

Hopefully you will decide to choose Emilda.

Regards,




e) do you think Emilda can be used in our project.
I think this really sounds like something interesting and would show others
how well Emilda can adapt to different media and standards. If you continue
to pursue, which I sincerely hope you do, we will help you to our best
extent to get it complete.


f) is there a possibility for structural support?
The community is ofcourse the primary source of support; most questions
concerning setup and similar can be handled. There however is also a legal
entity behind Emilda: Realnode Oy. Realnode Oy offers official support for
Emilda and also tailored development if the situation so requires. If you
are interested about the official offering of Realnode Oy what Emilda
concerns, the please drop me a note.

All in all I think this sounds like a very interesting project and that both
Emilda and you could benefit from the synergies that it might create.

Best regards,


===> An extra question, we had the impression when you look on the demo,
that the results -when you search f.e. on bach- are coming from different
institutes. I suppose that the data of the local library is also quick
accessible. Which library is here local in the demo?

"Native" is the local database. Others are remote Z39.50 servers.

thanks
Luk

_______________________________________________
Emilda mailing list
Emilda@xxxxxxxxxxxxxxxxxx
http://lists.realnode.com/mailman/listinfo/emilda






<Prev in Thread] Current Thread [Next in Thread>