Re: Gracefull stop in the middle of a fetch phase ?

Alex McLintock wrote:
> I am not sure if it solves your problem but you might do something
> like disconnect your machines from the internet - preferably by making
> your dns server return "dont know that domain"
>
> This will relatively quickly cause the remaining part of the fetch to fail.
>
> Just a suggestion...

I solved this once by implementing a check in Fetcher.run() for a marker 
file on HDFS. If the presence of this file was detected, the 
FetcherThreads would be stopped one by one (again, by setting a flag in 
their run() methods to terminate the loop).

It's a hack but it works well.

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com