nutch-user.lucene.apache.org (thread)

Specific fetch list based on url status or score, MilleBii
Focussed Web Crawling with Nutch, Alex McLintock
- Re: Focussed Web Crawling with Nutch, Ken Krugler
  - Re: Focussed Web Crawling with Nutch, MilleBii
Re: Hadoop java.io.IOException: Job failed! at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232) while indexing., Filipe Antunes
- RE: Hadoop java.io.IOException: Job failed! at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232) while indexing., Davide.D'ALESSANDRO
denied by robots.txt rules, Saurabh Suman
- denied by robots.txt rules, Saurabh Suman
Plugin development, Paul Tomblin
- Re: Plugin development, Alexander Aristov
  - Re: Plugin development, Paul Tomblin
    - Re: Plugin development, Alexander Aristov
    - Re: Plugin development, Paul Tomblin
Dumping Crawl DB with XML, schroedi
Meaning of ProtocolStatus.ACCESS_DENIED, Saurabh Suman
Nutch and Solr, Paul Tomblin
- Nutch in C++, alxsss
  - how to exclude some external links, alxsss
    - Re: how to exclude some external links, Paul Tomblin
How fetcher works, Saurabh Suman
- Re: How fetcher works, reinhard schwab
Include/exclude lists, Paul Tomblin
- Re: Include/exclude lists, reinhard schwab
How to add new field in indexing in SolrIndexer.java, Saurabh Suman
- Re: How to add new field in indexing in SolrIndexer.java, DoÄacan GÃney
Dumping what I have?, Paul Tomblin
- Re: Dumping what I have?, reinhard schwab
  - Re: Dumping what I have?, Paul Tomblin
- Re: Dumping what I have?, schroedi
Development support, Koch Martina
Host specific parsing, Koch Martina
- Re: Host specific parsing, Andrzej Bialecki
- Re: Host specific parsing, Sudhi Seshachala
Support needed, sf30098
- Re: Support needed, Sudhi Seshachala
Using Nutch (w/custom plugin) to crawl vs. custom Lucene app, ohaya
question, Jair Piedrahita Vargas
- Re: question, reinhard schwab
Nutch crawling status, caezar
- Re: Nutch crawling status, caezar
How to index other fields in solr, Saurabh Suman
- Re: How to index other fields in solr, Paul Tomblin
- Re: How to index other fields in solr, DoÄacan GÃney
crawl-tool.xml, reinhard schwab
- Re: crawl-tool.xml, reinhard schwab
How to search in one specific field?, xiao yang
Nutch 1.0 and Hadoop 0.20, Hrishikesh Agashe
Dumping CrawlDB into database, schroedi
Why did my crawl fail?, Paul Tomblin
- RE: Why did my crawl fail?, Arkadi.Kosmynin
  - Re: Why did my crawl fail?, Paul Tomblin
    - RE: Why did my crawl fail?, Arkadi.Kosmynin
- Re: Why did my crawl fail?, xiao yang
  - Re: Why did my crawl fail?, Paul Tomblin
    - Re: Why did my crawl fail?, xiao yang
    - Re: Why did my crawl fail?, Paul Tomblin
Can I "chunk" during the crawl?, Paul Tomblin
IO exception while adding field in Parsedata parsemeta., Saurabh Suman
- Re: IO exception while adding field in Parsedata parsemeta., DoÄacan GÃney
IO exception while adding field in Parsedata contentmeta., Saurabh Suman
adding [-numFetchers numFetchers] to crawl, Brian Tingle
Gracefull stop in the middle of a fetch phase ?, MilleBii
- Re: Gracefull stop in the middle of a fetch phase ?, DoÄacan GÃney
- Re: Gracefull stop in the middle of a fetch phase ?, Alex McLintock
  - Re: Gracefull stop in the middle of a fetch phase ?, Andrzej Bialecki
    - Re: Gracefull stop in the middle of a fetch phase ?, Alex McLintock
Pages with Specific URLS., Zaihan
- Re: Pages with Specific URLS., reinhard schwab
How to add new field in parseData, Saurabh Suman
- Re: How to add new field in parseData, DoÄacan GÃney
Querying nutch content using Pig Latin, Ninad Raut
nutch -threads in hadoop, Brian Tingle
- Re: nutch -threads in hadoop, Andrzej Bialecki
  - RE: nutch -threads in hadoop, Brian Tingle
    - Re: nutch -threads in hadoop, Andrzej Bialecki
[ApacheCon US] Travel Assistance, Grant Ingersoll
nutch 0.9 with jetty 6 and jdk 1.6, Michaela Moesenbacher
Using Nutch to crawl PubMed, Arshad Khan
- Re: Using Nutch to crawl PubMed, Magnús Skúlason
Nutch 1.0 Fetch failure..., Fred Kuipers
- Re: Nutch 1.0 Fetch failure..., DoÄacan GÃney
  - Re: Nutch 1.0 Fetch failure..., Fred Kuipers
Reminder: NYC Lucene et. al Meetup next week, Grant Ingersoll
indexing meta tags in 1.0, Will Daley
Crawling, Neeti Gupta
different urlfilter for different seeds, Beats
- Re: different urlfilter for different seeds, Beats
  - RE: different urlfilter for different seeds, Devang Shah
directories needed for a merge, Alex Basa
Entities.encode is not UTF-8 compliant, MilleBii
- Re: Entities.encode is not UTF-8 compliant, MilleBii
error in using generate command, Beats
- Re: error in using generate command, Alex McLintock
  - Re: error in using generate command, Beats
- error in using generate command, Beats
  - Re: error in using generate command, Beats
    - Re: error in using generate command, DoÄacan GÃney
wrong outlinks, reinhard schwab
- Re: wrong outlinks, DoÄacan GÃney
  - Re: wrong outlinks, reinhard schwab
    - Re: wrong outlinks, reinhard schwab
dump all outlinks, reinhard schwab
- Re: dump all outlinks, kevin chen
  - Re: dump all outlinks, reinhard schwab
Why cant I inject a google link to the database?, Larsson85
- Re: Why cant I inject a google link to the database?, reinhard schwab
  - Re: Why cant I inject a google link to the database?, reinhard schwab
    - Re: Why cant I inject a google link to the database?, reinhard schwab
  - Re: Why cant I inject a google link to the database?, Larsson85
    - Re: Why cant I inject a google link to the database?, DoÄacan GÃney
    - Re: Why cant I inject a google link to the database?, DoÄacan GÃney
    - Re: Why cant I inject a google link to the database?, reinhard schwab
    - Re: Why cant I inject a google link to the database?, Dennis Kubes
    - Re: Why cant I inject a google link to the database?, reinhard schwab
    - Re: Why cant I inject a google link to the database?, Larsson85
    - Re: Why cant I inject a google link to the database?, Jake Jacobson
    - Re: Why cant I inject a google link to the database?, Brian Ulicny
    - Re: Why cant I inject a google link to the database?, Andrzej Bialecki
    - Re: Why cant I inject a google link to the database?, reinhard schwab
Issue with Parse metaData while crawling RSSFeed URL, Saurabh Suman
- Re: Issue with Parse metaData while crawling RSSFeed URL, DoÄacan GÃney
How segment depends on depth, Saurabh Suman
- Re: How segment depends on depth, MilleBii
Difference between Feed parser and Rss Parser, Saurabh Suman
- Re: Difference between Feed parser and Rss Parser, DoÄacan GÃney
Question about crawling local filesystem and directories, ohaya
java heap space problem when using the language identifier, MilleBii
- Re: java heap space problem when using the language identifier, MilleBii
  - Re: java heap space problem when using the language identifier, DoÄacan GÃney
    - Re: java heap space problem when using the language identifier, MilleBii
    - Re: java heap space problem when using the language identifier, MilleBii
    - Re: java heap space problem when using the language identifier, MilleBii
    - Re: java heap space problem when using the language identifier, DoÄacan GÃney
Meta tag plugin for 1.0, wadaley
Problem crawling local filesystem, ohaya
- Re: Problem crawling local filesystem, ohaya
Crawling with a PKI Cert, Jake Jacobson
Add new conf file., Beats
how to filter pages before indexing, Beats
- Re: how to filter pages before indexing, DoÄacan GÃney
  - Re: how to filter pages before indexing, Beats
    - Nutch download speed, Hrishikesh Agashe
    - Re: Nutch download speed, DoÄacan GÃney
  - Re: how to filter pages before indexing, Beats
Use of lock file, Saurabh Suman
How nutch use ontology, Saurabh Suman
Local or Distributed mode?, Rodrigo Reyes C.
- Re: Local or Distributed mode?, xiao yang
Errorr when using language-identifier plugin ?, MilleBii
mergesegs disk space, Tomislav Poljak
- Re: mergesegs disk space, DoÄacan GÃney
  - Re: mergesegs disk space, MilleBii
    - Re: mergesegs disk space, DoÄacan GÃney
    - Re: mergesegs disk space, Tomislav Poljak
    - Re: mergesegs disk space, DoÄacan GÃney
    - Re: mergesegs disk space, reinhard schwab
    - Re: mergesegs disk space, DoÄacan GÃney
    - Re: mergesegs disk space, reinhard schwab
[REMINDER] NYC Meetup July 22nd, Grant Ingersoll
How to manage the urls in crawlDB?, xiao yang
- Re: How to manage the urls in crawlDB?, DoÄacan GÃney
Tutorial followup - Nutch webapp not seeing stuff?, ohaya
- Re: Tutorial followup - Nutch webapp not seeing stuff?, ohaya
  - Re: Tutorial followup - Nutch webapp not seeing stuff?, ohaya
    - Re: Tutorial followup - Nutch webapp not seeing stuff?, ohaya
    - Re: Tutorial followup - Nutch webapp not seeing stuff?, DoÄacan GÃney
    - Re: Tutorial followup - Nutch webapp not seeing stuff?, ohaya
    - Re: Tutorial followup - Nutch webapp not seeing stuff?, Alex McLintock
    - Re: Tutorial followup - Nutch webapp not seeing stuff?, ohaya
How to crawl page displayed as response to search query in solr, Beats
A few questions about crawl-urlfilter.txt, Hrishikesh Agashe
- Re: A few questions about crawl-urlfilter.txt, Ken Krugler
  - RE: A few questions about crawl-urlfilter.txt, Pravin Karne
- Re: A few questions about crawl-urlfilter.txt, reinhard schwab
job failed for "java.io.IOException: Task process exit with nonzero status of 255.", lei wang
- Re: job failed for "java.io.IOException: Task process exit with nonzero status of 255.", lei wang
Ignoring robots.txt, Beats
- Re: Ignoring robots.txt, Beats
  - Re: Ignoring robots.txt, Dennis Kubes
Re: recrawling, Neeti Gupta
- Re: recrawling, Sjaiful Bahri
- recrawling, Neeti Gupta
url normalizer, Neeti Gupta
Just getting started w/tutorial- errors in crawl.log, ohaya
- Re: Just getting started w/tutorial- errors in crawl.log, Alex McLintock
  - Re: Just getting started w/tutorial- errors in crawl.log, ohaya
- Re: Just getting started w/tutorial- errors in crawl.log, Beats
- Re: Just getting started w/tutorial- errors in crawl.log, xiao yang
Nutch Tutorial 1.0 based off of the French Version, Jake Jacobson
- Re: Nutch Tutorial 1.0 based off of the French Version, alxsss
  - Re: Nutch Tutorial 1.0 based off of the French Version, Jake Jacobson
    - Re: Nutch Tutorial 1.0 based off of the French Version, Alex McLintock
- Re: Nutch Tutorial 1.0 based off of the French Version, schroedi
- Re: Nutch Tutorial 1.0 based off of the French Version, Jake Jacobson
Search History and Top Searches, Kenan Azam
- Re: Search History and Top Searches, Kenan Azam
Integrating Nutch frontend with Backend., Zaihan
- Re: Integrating Nutch frontend with Backend., Alex McLintock
Job failed help, Jake Jacobson
- Re: Job failed help, SunGod
  - Re: Job failed help, Jake Jacobson
    - Re: Job failed help, Jake Jacobson
    - Re: Job failed help, DoÄacan GÃney
    - Re: Job failed help, Jake Jacobson
    - Re: Job failed help, DoÄacan GÃney
    - Re: Job failed help, MilleBii
prune tool query, Beats
- prune tool query, Beats
  - Re: prune tool query, MilleBii
Nutch OutPut in which UTF format, Saurabh Suman
- Re: Nutch OutPut in which UTF format, DoÄacan GÃney
Deleting indexes, Beats
- Re: Deleting indexes, DoÄacan GÃney
  - Re: Deleting indexes, Beats
    - Re: Deleting indexes, DoÄacan GÃney
Nutch Character encoding converter, Saurabh Suman
- Re: Nutch Character encoding converter, Ken Krugler
  - Re: Nutch Character encoding converter, Saurabh Suman
Changing fieldsNorm at query time, ilayaraja
Problem with nutch, Pranay Gunna
How to search part of words?, stefan . kaifer
Search results return 0, Zaihan
Too many fether failures, lei wang
how to crawl a page but not index it, Beats
- Re: how to crawl a page but not index it, Beats
  - Re: how to crawl a page but not index it, SunGod
    - Re: how to crawl a page but not index it, SunGod
    - Re: how to crawl a page but not index it, Beats
    - Re: how to crawl a page but not index it, Jake Jacobson
job failed for "Too many fetch-failures", lei wang
Ontology-Clearing Cache..., gunnapranay
how to allow every url to b accepted, Beats
- Re: how to allow every url to b accepted, lei wang
How to search for part of words?, stefan . kaifer
- Re: How to search for part of words?, DoÄacan GÃney
Re: How to parse and index content field of RSS-Feed?, Beats
[ANN] Luke + Hadoop, alpha version, Andrzej Bialecki
how to change encoding, Saurabh Suman
- Re: how to change encoding, DoÄacan GÃney
indexing each item in seperate page, Beats
- Re: indexing each item in seperate page, DoÄacan GÃney
  - Re: indexing each item in seperate page, Beats
    - Re: indexing each item in seperate page, DoÄacan GÃney
Arc to segements failed for " Task attempt_200907091108_0001_m_000520_0 failed to report status for 602 seconds. Killing!", lei wang
- Re: Arc to segements failed for " Task attempt_200907091108_0001_m_000520_0 failed to report status for 602 seconds. Killing!", Ken Krugler
Script to crawl web, Jake Jacobson
call for answer, postusenet
Weighting different html text nodes - h1,h2 etc.., Joel Halbert
- Re: Weighting different html text nodes - h1,h2 etc.., Ken Krugler
Index weightings of different types of text node...h1, h2 anchor etc.., Joel Halbert
- Re: Index weightings of different types of text node...h1, h2 anchor etc.., Magnús Skúlason
How to crawl URLs getting from RSSParser, Saurabh Suman
Show db_gone in crawlDB, schroedi
- Re: Show db_gone in crawlDB, Xiangjun(XJ) Wang
Running Nutch on VMs, Jake Jacobson
- Re: Running Nutch on VMs, schroedi
How to add chinese segment feature to Nutch-1.0, xiao yang
How to Parse Rss Feed URL, Saurabh Suman
- Re: How to Parse Rss Feed URL, DoÄacan GÃney
- Re: How to Parse Rss Feed URL, Saurabh Suman
Solr Integration since v1.0 ?, Alex McLintock
error nutch recrawl, Maurizio Croci
- Re: error nutch recrawl, xiao yang
Writing Plugins - Documentation?, Alex McLintock
how parse chm files, Yaidel Guedes Beltran
Authentication Not Occuring, youyou wu
- Re: Authentication Not Occuring, Susam Pal
what is Non DFS Used in cluster summary? how to delete Non DFS Used data, Pravin Karne
what is Non DFS Used in cluster summary ?how to delete it?, Pravin Karne
Hoe to search Nutch DB, Saurabh Suman
- Re: Hoe to search Nutch DB, Xiangjun(XJ) Wang
  - Re: How to search Nutch DB, Saurabh Suman
Nutch-1.0: Cannot lock storage error, xiao yang
Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out., xiao yang
- Re: Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out., lei wang
  - Re: Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out., lei wang
    - Re: Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out., Andrzej Bialecki
How to get lastModified or create-date content from html pages?, postusenet
Favorite Linux Distribution for Nutch, schroedi
- Re: Favorite Linux Distribution for Nutch, ben bouzid mohamed
- Re: Favorite Linux Distribution for Nutch, SunGod
- Re: Favorite Linux Distribution for Nutch, Dennis Kubes
  - Re: Favorite Linux Distribution for Nutch, Marcus Herou
  - Re: Favorite Linux Distribution for Nutch, schroedi
- Re: Favorite Linux Distribution for Nutch, éäå
Getting Nutch1.0 example working in tomcat 6 (on ubuntu), Alex McLintock
Re: Storing a serialized object ?, MilleBii
- Re: Storing a serialized object ?, MilleBii
Problems when deploy nutch-1.0.war, xiao yang
- Re: Problems when deploy nutch-1.0.war, schroedi
  - Re: Problems when deploy nutch-1.0.war, xiao yang
    - Re: Problems when deploy nutch-1.0.war, Alex McLintock
- Re: Problems when deploy nutch-1.0.war, Alex McLintock
  - Problems when index .chm files, Yaidel Guedes Beltran
    - Re: Problems when index .chm files, Ken Krugler
  - Re: Problems when deploy nutch-1.0.war, claus westerkamp
what's the relationship between nutch, solr, lucene, and hadoop, xiao yang
- Re: what's the relationship between nutch, solr, lucene, and hadoop, johan . sjoberg
NYC Apache Lucene/Solr/Nutch/etc. Meetup, Grant Ingersoll
Nutch 1.0 on the limits of the data, Polsnet
- Re: Nutch 1.0 on the limits of the data, Otis Gospodnetic
- Re: Nutch 1.0 on the limits of the data, Dennis Kubes
Optimal size of a segments sub-directory and a couple of other questions relating to Nutch response times, Vijay
How To Generate the JavaDoc, schroedi
- Re: How To Generate the JavaDoc, Neeti Gupta
nutch crawldb failed for java heap space, lei wang
- Re: nutch crawldb failed for java heap space, lei wang
- Re: nutch crawldb failed for java heap space, Julien Nioche
  - Re: nutch crawldb failed for java heap space, lei wang
    - Re: nutch crawldb failed for java heap space, lei wang
Re: How torunning nutch on 2G memory tasknode, lei wang
How to tell Nutch that text files are text files?, Hannu VÃisÃnen