2.1 Topic Detection

The term topic is usually defined as the aboutness of a unit of discourse ^[22]. Topic Detection (TD) is a part of the study content of Topic Detection and Tracking (TDT). It is a new skill by which a given set of documents from the data stream, such as news reports, newswires, blogs and social media, are classified into a given set of documents into thematic categories.

In the following, we exclusively consider the single-Topic Detection version, since it is more general than multi-Topic Detection: the latter can be split into several binary (i.e., single-label) detection problems, but the contrary is not possible. Also, we are interested in using a fixed set of topics instead of an open one. Our technique allows the specification of topics of interest and attempts to classify documents within those topics only, based on two major steps: topic vocabulary generation and topic assignment.

TD is based on supervised or unsupervised learning using a training corpus to represent each topic with a specific model obtained using a wide range of text processing approaches, text representation models, and detection methods to estimate similarities between topics and documents vectors.

2.2 Text Representation

Text representation is used to reduce the complexity of the documents, capture the meaning of them and make them easier to handle. Extensive work is carried out to propose various text representations. As definition of a document is that it is made of a joint membership of terms which have various patterns of occurrence. Thus it has to be converted from the full text version to a standard representation. Bag of Word (BoW) happens to be the basic representation model where a document is typically represented as a vector of term weights (word features) from a set of terms, using the frequency count of each term in the document. This model of document representation is also called a Vector Space Model (VSM) ^[26].

However, the BOW/VSM representation has its own limitations, namely: the extremely high dimensionality of text data, loss of correlation with adjacent words and loss of semantic relationship that exist among the terms in a document ^[8].To overcome these problems, we present and experiment in this paper many text representation models while trying especially to preserve semantic relations between words by using Named Entities or Multi-words Terms.

3.1 Stems

Terms have many morphological variants that will not be recognized by term matching algorithm without additional text processing. In most cases, these variants have similar semantic interpretation and can be treated as equivalence in text mining. Stemming has become an important step in text mining and information retrieval in order to make the information more accurate and effective.

Stemming in Arabic language can be defined as the process or removing prefixes or/and suffixes from words to recover their stems. The aim of the Stemming or Light Stemming is not to produce the root of a given Arabic word, rather is to remove the most frequent suffixes and prefixes. However, till now there is almost no standard algorithm for Arabic light stemming, as there is no definite list suffixes and prefixes that must be remove.

Experiments have shown that using stems is more efficient than roots or the full words in Arabic Topic Detection ^[16]. We used the morphological analyzer Alkhalil ^[9] to recover a list of stems for each document. Alkhalil realizes a morphological analysis for each word in the corpus and returns among other morphological information all possibly related stems to the considered word. So, we implemented a Viterbi algorithm to keep only the stems that are relevant to the context.

3.2 Arabic Multi-Word Terms

Although multi-word term has no totally agreed upon definition, it can be understood as a sequence of two or more consecutive individual words, forming a semantic unit ^[24]. In fact, the exact meaning of a multi-word term could not be fully achieved by its individual parts.

MWTs Extraction represents an important task of Automatic term recognition and is employed in numerous NLP fields such as: Text Mining ^[27], Syntactic Parsing ^[20,3], Machine Translation ^[11] and Text Classification ^[34]. The MWTs extraction task covers detection and extraction of a set of consecutive semantically related words and the techniques used can be classified into four major categories: (a) Statistical approaches based on frequency, probability and co-occurrence measures ^[31], (b) Symbolic approaches using parsers, morphological analysis, MWTs boundaries detection and patterns ^[33], (c) Hybrid approaches combining statistical and morphological methods ^[12] and (d) Word alignment approaches ^[18].

We built our multi-word extraction system based on the hybrid approach which performs in two steps:

The implemented MWTs extraction system achieved an overall of 90.25% in term of precision.

3.3 Arabic Named Entities

The objective of Named Entity Recognition (NER) task is to identify and classify mentions of rigid designators from text belonging to named entity types such as: persons, organizations, locations and miscellaneous names (date, time, percentage and monetary expressions) within an open-domain text ^[19]. The NER is a key technique of Information Extraction and Question Answering systems. The techniques proposed in the literature of NER fall within three major categories: (a)Rule-based approaches ^[23]: is a language dependent approach that uses hand crafted linguistic rules, (b) Machine learning based approaches ^[15]: is a language independent approach based on machine learning algorithms and (c) Hybrid approaches ^[10]: combines linguistic patterns and machine learning techniques.

We developed an Arabic NER system implementing the hybrid approach. We used ANERCorpus ^[7] for the training and test corpus which is an annotated corpus following the Conferences on the Computational Natural Language Learning (CoNLL) 2002 and 2003 shared task, formed by tags falling into the following four categories: Person, Location, Organization and Miscellaneous names. Also, we used a SVM based software for sequence tagging using Hidden Markov Models ^[2], along with a combination of language independent and language specific binary features to capture the essence of the Arabic language such as: lexical, contextual, morphological features and gazetteers,... We boosted the system with an automatic pattern extraction framework in order to enhance the ANER system. The developed system achieved an overall of Fl-measure of 83.20%.

4.1 The Dataset

For the setup of our experiments, we used a corpus of over 20.291 articles, collected from the Arabic newspaper Wattan of the year 2004 ^[1]. The corpus contains articles covering the six following topics: culture, economics, international, local, religion and sport. The repartition of documents is described in Table 1. The corpus was divided into two subsets of documents. Thus, 9/10 of the corpus was dedicated to training the feature selection system (Topic vocabulary construction), whereas 1/10 of the overall documents formed the evaluation corpus.

4.2 Experiments Setup

Text preprocessing is the first step in a Topic Detection process. It aims to reduce the noise in documents by removing all the unnecessary terms and mistyped words. We process the corpus using the following operations:

We use the preprocessed training corpus to generate Topic Representation Model for each of the three models presented in section 3 as follow:

To evaluate the performance of the developed system, we process the test corpus to extract vectors representing each test document according to the preprocessing steps detailed earlier for each one of the three text representation models. Then, we compute similarity between each test document and the generated topics vocabulary vectors using Cosine similarity. The cosine similarity value lies within the real interval of [0; 1], where 1 indicates a perfect match between the two vectors and 0 indicates a complete mismatch. The cosine similarity formula is expressed as follows:

Where Tj=vj1,vj2,…vjn represents the jth topic feature vector and Di=wi1,wi2,…win represents features vector of test document i. The test document is assigned to the topic with the maximums cosine similarity measure.

4.3 Evaluation Methods

In order to evaluate the classifiers performance, three standard set-based metrics are used: recall and precision ^[4]. For a given topic Ti, precision and recall are defined as follows:

The value of precision and recall often depend on parameter tuning; there is a tradeoff between them. This why we also use another measure that combine both of precision and recall: the F1-measure ^[32], defined as:

We also calculate the macro average of each metric which is given by the average on the metric scores of all topics.

4.4 Results

Figure 1 shows the Topic Detection system performance results using Stems, MWTs and NEs on each topic of the test corpus in term of F1-measure. We notice that the named entities approach realizes higher performances. Among the six topics, all the three models gave the best performance on the "Sport" topic and the poorest performance on the "Culture" topic. This could be explained by the difficulty faced to extract truly representative terms from the training corpus since the "Culture" topic covers a broad range of documents including TV programs, movies, poetry, literature, culinary, museum events... , which is not the case of the "Sport" topic.

Fig. 1 F1-measure per topic 

In general, the performance of MWTs and NEs remain globally more important than Stems. This is due to the influence of the Arabic language nature, In fact, one word can be used in many sentences with different meaning. For example, the word elected in Arabic can be used in local documents as an adjective referring to an elected person, and can also be used in sports documents as a noun referring a sport team. This ambiguity can be reduced by using MWTs or NEs as adding one or more words with give more precision about the context of the word. Although Stems outperforms MWTs on "International" topic, this can be explained by the fact that this topic's documents are generally formed by names of cities and countries composed with one word only.

Table 2 presents the global performance of the three approaches based on the macro-average of all measures across the six topics. It can be seen that NEs outperformed the other two approaches in precision, recall and F1-measure. MWTs achieved better performance than Stems in terms of precision and F1-measure.

On one hand, this is due to the fact that the NEs and MWTs carry an important amount of information, which is very benefic to the Arabic Topic Detection rather than using stems only. On the other hand, we notice that the effectiveness of multi-word is strongly dependent on the types of literature. For instance, multi-words as a text representation is effective for documents, in which fixed expressions (terminologies, collocations, etc.) are usually used, such as academic papers, but may be not effective for the documents with extensive topics, in which fixed expressions are not usually used such as poems (culture).

This ambiguity generated in such cases affects the performances of the Topic Detection system, leading to a misjudgment of the similar topics. In fact, great many words are common to different topics leading to imprecision in the Topic Detection process. Although, it can help to differentiate similar topics by named entities, the number of named entities is limited in news documents. Using only named Entities; it may influence the topic detection system as many key words which describe the topics are ignored. As a result, the performance of Topic Detection decrease for related topic such as: culture and local as these topics show a low performance for the three models.

In order to solve the problems related to the difficulty in distinguishing similar topics, we investigate the use of topic oriented combined vocabulary vectors: Stems with NEs and Stems with MWTs. We calculate the similarity as follows:

where:

The α and β values were obtained empirically through running the experiments for various weight values and selecting those ensuring optimal performances, corresponding to α=34 and β=14. As shown in Figure 2, we can clearly notice that using the combined vectors approach has improved the system performances. This improvement is justified by the fact that the combined vectors allow expressing each topic more accurately and highlight the differences between similar topics. Nerveless, the results of combining stems with multi-words terms remain less efficient than those of associating stems with names entities.

Fig. 2 F1-measure per topic for combined vectors 

Table 3 shows that combining stems and multi-word terms vectors has augmented the F1-measure from 83.46% to 84.25%.

The augmentation in the case of named entities categories and stems combined vectors is mush important. Indeed, we have an average of F1-measure of 88.80% against 85.18% realized with named entities vectors only. We conjuncture that the combination method is more effective and deliver better results.