一、 Oracle Text 索引文檔時所使用的主要邏輯步驟如下:
(1)數據存儲邏輯搜索表的所有行,並讀取列中的數據。通常,這只是列數據,但有些數據存儲使用列數據作為文檔數據的指針。例如,URL_DATASTORE 將列數據作為URL使用。
(2)過濾器提取文檔數據並將其轉換為文本表示方式。存儲二進制文檔 (如 Word 或 Acrobat 文件) 時需要這樣做。過濾器的輸出不必是純文本格式 -- 它可以是 XML 或 HTML 之類的文本格式。
(3)分段器提取過濾器的輸出信息,並將其轉換為純文本。包括 XML 和 HTML 在內的不同文本格式有不同的分段器。轉換為純文本涉及檢測重要文檔段標記、移去不可見的信息和文本重新格式化。
(4)詞法分析器提取分段器中的純文本,並將其拆分為不連續的標記。既存在空白字符分隔語言使用的詞法分析器,也存在分段復雜的亞洲語言使用的專門詞法分析器。
(5)索引引擎提取詞法分析器中的所有標記、文檔段在分段器中的偏移量以及被稱為非索引字的低信息含量字列表,並構建反向索引。倒排索引存儲標記和含有這些標記的文檔。
每個索引的許多選項組成功能組,稱為“類”
每個類集中體現配置的某一方面,可以認為這些類就是與文檔數據庫有關的一些問題。例如:數據存儲、過濾器、詞法分析器、相關詞表、存儲等。
每個類具有許多預定義的行為,稱之為對象。每個對象是類問題可能具有的答案,並且大多數對象都包含有屬性。通過屬性來定制對象,從而使對索引的配置更加多變以適應於不同的應用。
(1)存儲(Storage)類
存儲類指定構成Oracle Text索引的數據庫表和索引的表空間參數和創建參數。它僅有一個基本對象:BASIC_STORAGE,其屬性包括:I_Index_Clause、I_Table_Clause、K_Table_Clause、N_Table_Clause、P_Table_Clause、R_Table_Clause。
(2)數據存儲(Datastore)類
數據存儲:關於列中存儲文本的位置和其他信息。默認情況下,文本直接存儲到列中,表中的每行都表示一個單獨的完整文檔。其他數據存儲位置包括存儲在單獨文件中或以其 URL 標識的 Web 頁上。七個基本對象包括:Default_Datastore、Detail_Datastore、Direct_Datastore、File_Datastore、Multi_Column_Datastore 、URL_Datastore、User_Datastore,。
(3)文檔段組(Section Group)類
文檔段組是用於指定一組文檔段的對象。必須先定義文檔段,然後才能使用索引通過 WITHIN 運算符在文檔段內進行查詢。文檔段定義為文檔段組的一部分。包含七個基本對象:AUTO_SECTION_GROUP、BASIC_SECTION_GROUP、HTML_SECTION_GROUP、NEWS_SECTION_GROUP、NULL_SECTION_GROUP、XML_SECTION_GROUP、PATH_SECTION_GROUP。
(4)相關詞表(Wordlist)類
相關詞表標識用於索引的詞干和模糊匹配查詢選項的語言,只有一個基本對象BASIC_WORDLIST,其屬性有:Fuzzy_Match、Fuzzy_Numresults、Fuzzy_Score、Stemmer、Substring_Index、Wildcard_Maxterms、Prefix_Index、Prefix_Max_Length、Prefix_Min_Length。
(5)索引集(Index Set)
索引集是一個或多個Oracle 索引 (不是Oracle Text索引) 的集合,用於創建 CTXCAT類型的Oracle Text索引,只有一個基本對象BASIC_INDEX_SET。
(6)詞法分析器(Lexer)類
詞法分析器類標識文本使用的語言,還確定在文本中如何標識標記。默認的詞法分析器是英語或其他西歐語言,用空格、標准標點和非字母數字字符標識標記,同時禁用大小寫。包含8個基本對象:BASIC_LEXER、CHINESE_LEXER、CHINESE_VGRAM_LEXER、JAPANESE_LEXER、JAPANESE_VGRAM_LEXER、KOREAN_LEXER、KOREAN__MORPH_ LEXER、MULTI_LEXER。
(7)過濾器(Filter)類
過濾器確定如何過濾文本以建立索引。可以使用過濾器對文字處理器處理的文檔、格式化的文檔、純文本和 HTML 文檔建立索引,包括5個基本對象:CHARSET_FILTER、INSO_FILTER INSO、NULL_FILTER、PROCEDURE_FILTER、USER_FILTER。
(8)非索引字表(Stoplist)類
非索引字表類是用以指定一組不編入索引的單詞 (稱為非索引字)。有兩個基本對象:BASIC_STOPLIST (一種語言中的所有非索引字) 、 MULTI_STOPLIST (包含多種語言中的非索引字的多語言非索引字表)。
二、使用Oracle Text建立全文索引的完整步驟,歸納起來如下:
(1)建表並裝載文本(包含帶有需要檢索的文本字段)
(2)配置索引
(3)建立索引
(4)發出查詢
(5)索引維護:同步與優化(將在後面介紹)
三、索引類型
(1)CONTEXT
A CONTEXT
index is the basic type of Oracle Text index. This is an index on a text column. A CONTEXT
index is useful when your source text consists of many large, coherent documents. Query this index with the CONTAINS
operator in the WHERE
clause of a SELECT
statement. This index requires manual synchronization after DML. See Syntax for CONTEXT Index Type.
(2)CTXCAT
The CTXCAT
type of index is a combined index on a text column and one or more other columns. CTXCAT
is typically used to index small documents or text fragments, such as item names, prices and descriptions found in catalogs. Query this index with the CATSEARCH
operator in the WHERE
clause of a SELECT
statement. This type of index is optimized for mixed queries. This index is transactional, automatically updating itself with DML to the base table. See Syntax for CTXCAT Index Type.
(3)CTXRULE
A CTXRULE
index is used to build a document classification application. The CTXRULE
index is an index created on a table of queries or a column containing a set of queries, where the queries serve as rules to define the classification criteria. Query this index with theMATCHES
operator in the WHERE
clause of a SELECT
statement. See Syntax for CTXRULE Index Type.
(4)CTXXPATH
Create this index when you need to speed up existsNode()
queries on an XMLType column. See Syntax for CTXXPATH Index Type.
四、詞法分析器類型
Lexer Types
Type
Description
AUTO_LEXER
Lexer for indexing columns that contain documents of different languages.
BASIC_LEXER
Lexer for extracting tokens from text in languages, such as English and most western European languages that use white space delimited words.
MULTI_LEXER
Lexer for indexing tables containing documents of different languages such as English, German, and Japanese.
CHINESE_VGRAM_LEXER
Lexer for extracting tokens from Chinese text.
CHINESE_LEXER
Lexer for extracting tokens from Chinese text. This lexer offers benefits over the CHINESE_VGRAM
lexer:
· Generates a smaller index
· Better query response time
· Generates real world tokens resulting in better query precision
· Supports stop words
JAPANESE_VGRAM_LEXER
Lexer for extracting tokens from Japanese text.
JAPANESE_LEXER
Lexer for extracting tokens from Japanese text. This lexer offers the following advantages over the JAPANESE_VGRAM
lexer:
· Generates smaller index
· Better query response time
· Generates real world tokens resulting in better precision
KOREAN_MORPH_LEXER
Lexer for extracting tokens from Korean text.
USER_LEXER
Lexer you create to index a particular language.
WORLD_LEXER
Lexer for indexing tables containing documents of different languages; autodetects languages in a document.
在這著重介紹WORLD_LEXER:
Use the WORLD_LEXER
to index text columns that contain documents of different languages. For example, use this lexer to index a text column that stores English, Japanese, and German documents.
WORLD_LEXER
differs from MULTI_LEXER
in that WORLD_LEXER
automatically detects the language(s) of a document. Unlike MULTI_LEXER
, WORLD_LEXER
does not require you to have a language column in your base table nor to specify the language column when you create the index. Moreover, it is not necessary to use sub-lexers, as with MULTI_LEXER
. (See MULTI_LEXER.)
WORLD_LEXER supports all database character sets, and for languages whose character sets are Unicode-based, it supports the Unicode 5.0 standard. For a list of languages that WORLD_LEXER
can work with, see "World Lexer Features".
The WORLD_VGRAM_LEXER
has the following attribute:
Attribute
Attribute Value
mixed_case
Enable mixed-case (upper- and lower-case) searches of text (for example, cat and Cat). Allowable values are YES
and NO
(default).
Here is an example of creating an index using WORLD_LEXER
.
exec ctx_ddl.create_preference('MYLEXER', 'world_lexer');
create index doc_idx on doc(data)
indextype is CONTEXT
parameters ('lexer MYLEXER
stoplist CTXSYS.EMPTY_STOPLIST');
五、創建全文索引語法