Lucene 4.x Spellcheck使用說明

2019-11-14 23:36:07

字體：大中小

來源：轉載

供稿：網(wǎng)友

　　Spellcheck是Lucene新版本的功能，在介紹spellcheck之前，我們需要弄清楚Spellcheck支持幾種數(shù)據(jù)源。Spellcheck構造函數(shù)需要傳入Dictionary接口：

package org.apache.lucene.search.spell;/* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements.  See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License.  You may obtain a copy of the License at * *     http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either exPRess or implied. * See the License for the specific language governing permissions and * limitations under the License. */import java.io.IOException;import org.apache.lucene.search.suggest.InputIterator;/** * A simple interface representing a Dictionary. A Dictionary * here is a list of entries, where every entry consists of * term, weight and payload. *  */public interface Dictionary {  /**   * Returns an iterator over all the entries   * @return Iterator   */  InputIterator getEntryIterator() throws IOException;}

　　常用的Dictionary主要有以下幾種，常用的主要有基于文本型的和基于lucene索引構建的：

　　下面是我測試用的一段代碼，代碼包括索引構建和索引查詢：

package com.tianditu.com.search;import java.io.File;import java.io.IOException;import org.apache.lucene.index.DirectoryReader;import org.apache.lucene.index.IndexWriterConfig;import org.apache.lucene.search.spell.LuceneDictionary;import org.apache.lucene.search.spell.SpellChecker;import org.apache.lucene.store.Directory;import org.apache.lucene.store.FSDirectory;import org.apache.lucene.store.MMapDirectory;import org.apache.lucene.util.Version;public class GlobalSuggest {//拼寫檢查構建的索引private  final String SPELL_CHECK_FOLDER = "c://spellcheck//";//根據(jù)已有的索引private final String GLOBAL_PINYIN_SUGGEST = "O://searchwork_custom//data_index//pinyin2008//";//構建索引public void testIndexPinyin2008() throws IOException{long start = System.currentTimeMillis();//北京吉威時代軟件股份有限公司//String indexDir ="O://searchwork_custom//data_index//GlobalIndex//";Directory direct = new MMapDirectory(new File(GLOBAL_PINYIN_SUGGEST));LuceneDictionary ld = new LuceneDictionary(DirectoryReader.open(direct), "name");ld.getEntryIterator();Directory spd = FSDirectory.open(new File(SPELL_CHECK_FOLDER));SpellChecker sc = new SpellChecker(spd);//sc.inIndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_30,null);//往spellcheck目錄下寫索引--------------sc.indexDictionary(ld, iwc, true);sc.close();long end = System.currentTimeMillis();System.out.println("索引完畢,耗時:"+(end-start)+"ms");}public void testIndex() throws IOException{long start = System.currentTimeMillis();//北京吉威時代軟件股份有限公司String indexDir ="O://searchwork_custom//data_index//GlobalIndex//";Directory direct = new MMapDirectory(new File(indexDir));LuceneDictionary ld = new LuceneDictionary(DirectoryReader.open(direct), "name");ld.getEntryIterator();Directory spd = FSDirectory.open(new File(SPELL_CHECK_FOLDER));SpellChecker sc = new SpellChecker(spd);//sc.inIndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_30,null);sc.indexDictionary(ld, iwc, true);sc.close();long end = System.currentTimeMillis();System.out.println("索引完畢,耗時:"+(end-start)+"ms");}public void testSearch(String wd) throws IOException{//構建DirectoryDirectory spd = FSDirectory.open(new File(SPELL_CHECK_FOLDER));//實例化 spellcheck組件SpellChecker sc = new SpellChecker(spd);//根據(jù)輸入關鍵字  獲得N條最相近的幾率 第三個鄙視精確度 越大越匹配 安裝實際需要調(diào)整String[] suggests = sc.suggestSimilar(wd, 10,0.6f);if(suggests!=null){for(String Word:suggests){System.out.println("Dou you mean:"+word);}}}/** * @param args * @throws IOException  */public static void main(String[] args) throws IOException {GlobalSuggest spellcheck = new GlobalSuggest();//spellcheck.testIndexPinyin2008();spellcheck.testSearch("beijing京鴨");//spellcheck.testSearch("beijng");}}

　　其中索引構建處代碼：

//構建索引public void testIndexPinyin2008() throws IOException{long start = System.currentTimeMillis();//北京吉威時代軟件股份有限公司//String indexDir ="O://searchwork_custom//data_index//GlobalIndex//";Directory direct = new MMapDirectory(new File(GLOBAL_PINYIN_SUGGEST));LuceneDictionary ld = new LuceneDictionary(DirectoryReader.open(direct), "name");ld.getEntryIterator();Directory spd = FSDirectory.open(new File(SPELL_CHECK_FOLDER));SpellChecker sc = new SpellChecker(spd);//sc.inIndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_30,null);//往spellcheck目錄下寫索引--------------sc.indexDictionary(ld, iwc, true);sc.close();long end = System.currentTimeMillis();System.out.println("索引完畢,耗時:"+(end-start)+"ms");}

　　此處代碼，就是根據(jù)已有的索引來構建Spellcheck所需的索引。

Spellcheck查詢索引代碼片段如下：

//構建DirectoryDirectory spd = FSDirectory.open(new File(SPELL_CHECK_FOLDER));//實例化 spellcheck組件SpellChecker sc = new SpellChecker(spd);//根據(jù)輸入關鍵字  獲得N條最相近的幾率 第三個鄙視精確度 越大越匹配 安裝實際需要調(diào)整String[] suggests = sc.suggestSimilar(wd, 10,0.6f);if(suggests!=null){for(String word:suggests){System.out.println("Dou you mean:"+word);}}

　相關算法：默認是LevensteinDistance 。