|
|
Choosing A Webhost: |
Spanish stemmer with accents stripped before stemming: msg#00003search.snowball
Hello! We're trying to fine-tune an applicaiton we're building that searches mainly Spanish content. We've been working with Lucene and the included Spanish snowball stemmer. We are removing accents before stemming because misplaced-accent errors are extremely common--we cannot expect users to enter correctly accented terms in their queries. Our approach so far has been to simply replace accented characters in the Spanish stemmer Java code with unaccented ones. This has produced a stemmer that works--to a degree--but some simple terms are no longer stemmed correctly. Specifically, regular plural masculine nouns ("libros" or "datos") are not stemmed at all. Still, after reading the explanation of the Spanish algorithm [1] I can't see why this would be the case--or why the removal of accented characters from both the code and stemmer input would affect the algorithm's effectivenes at all. You can see our code here [2] and here [3]. Any suggestions? Thanks in advance, Andrew Green [1] http://snowball.tartarus.org/algorithms/spanish/stemmer.html [2] http://200.67.231.185/svn/pescador/trunk/java_source/net/sf/snowball/ext/Spanish2Stemmer.java [3] http://200.67.231.185/svn/pescador/trunk/java_source/org/apache/lucene/analysis/snowball/SnowballAnalyzerWithoutAccents.java
|
|
| <Prev in Thread] | Current Thread | [Next in Thread> |
|---|---|---|
| Previous by Date: | Re: Snowball API versioning, Richard Boulton |
|---|---|
| Next by Date: | Re: Spanish stemmer with accents stripped before stemming, Martin Porter |
| Previous by Thread: | Snowball API versioning, Oleg Bartunov |
| Next by Thread: | Re: Spanish stemmer with accents stripped before stemming, Martin Porter |
| Indexes: | [Date] [Thread] [Top] [All Lists] |
Free MagazinesCisco NewsReceive a free quarterly e-newsletter with exclusive articles on how Cisco IT uses its own products and solutions to enable the business. subscribe Systems Management News, the newspaper for IT systems administration and data center managers! Each issue of Systems Management News is chock-full of news and analysis to help you understand what's happening in your field. subscribe The Enterprise Newsweekly eWeek is the essential technology information source for builders of e-business. subscribe Oracle Magazine Oracle Magazine contains technology strategy articles, sample code, tips, Oracle and partner news, how to articles for developers and DBAs, and more. Oracle (NASDAQ: ORCL) is the world's largest enterprise software company. subscribe Total Telecom Total Telecom is "The Economist of the communications industry". subscribe |