1 Introduction

In recent years, many people express and share their opinions and evaluations on the internet. Knowing this evaluative content is so interesting. For example, in marketing, companies could use the consumers’ opinions and reviews to build their strategies. For individual users, opinions and commetns available on social media about consuming products and services are often consulted before purchasing them.

However, with the rapid growth of web 2.0, the amount of opinionated texts available on the internet is becoming huge which makes its manual exploitation by users a heavy task. For this reason, an automatic mining of such useful information is indispensable.

In this context, the goal of sentiment analysis is to automatically extract and classify document reviews, which express sentiments, emotions and evaluations about various topics such as products, services and persons. Opinion mining is widely applied to social media for a variety of applications like marketing and customer service. Pang et al. [²⁴] cited other applications of using opinion mining such as recommendation systems and document filtering.

Recently, researchers have proposed different methods in order to automatically classify opinionated texts as positive or negative [²³, ²⁴, ³⁰]. This recent research domain is known as opinion mining or sentiment analysis. The reader can find a detailed overview on this research area in the work of [¹⁸].

Basically, two approaches in the literature are proposed for sentiment analysis: machine learning approaches [²⁴] and semantic (linguistic) ones [³¹]. In machine learning approaches, we find supervised methods that utilize a sentiment corpus to train and construct predictive models.

These techniques are extensively proposed and evaluated due to their superior performance results compared with the semantic ones [²⁴]. In the supervised strategy, there are generally two main steps: (i) feature extraction /selection and (ii) sentiment learning/classification [²⁴].

On the other hand, semantic approaches consider the review text as a set of words. These methods use lexical resources, dictionaries like SentiWordNet [¹³] and rules with the aim of identifying the polarity of the word. Then, the text review is classified as positive or negative by comparing the number of positive and negative opinions words in the document review. Such methods provide generally modest results, as they are domain dependent and have low coverage.

Of course, there are methods that combine both machine learning and semantic approaches to increase sentiment classification performance. We can cite some papers performed in Arabic that merge both methods like [¹, ⁴, ¹⁰, ¹¹].

In his doctoral thesis [⁶], showed through an example of a movie review that the global opinion is generally based on selection criteria. Therefore, the extraction of comments for a film allows selecting the most relevant comments, and it permits to eliminate noise by neglecting extra-subject comments.

In addition, the task of mining movie reviews poses a particular challenge. In fact, before expressing the opinion about a given movie, the author often describes the story of the movie in question including informative texts related to facts and events on that story. Such narrative content have to be excluded as it does not serve for the sentiment classification [¹⁹].

This paper is concerned with extracting and predicting sentiments expressed in Arabic movie reviews. The goal of this work is to explore the effect of selecting subjective parts of reviews on opinion classification.

More specifically, we investigate the impact of extracting users’ opinions and comments toward different movie aspects such as director and actor on review classification. In addition, we study the contribution of the lexicon approach in opinion prediction. To improve sentiment classification performance, we proposed a combined approach that integrates the two proposed methods.

This article is organized as follows. In the next section, we provide background information about sentiment analysis. Section three gives an overview of the state of the art on Arabic sentiment analysis. In the fourth section, we describe our proposed approach including document pre-processing and aspect comments extraction. Section five provides experimental environment and the conducted experiments (datasets used, algorithm trained, etc.), and the obtained results are discussed. In the last section, we give our conclusion and future work.

2 Sentiment Analysis

Sentiment analysis (called also opinion mining) is the field that uses text mining techniques including natural language processing and data mining algorithms for the task of automatic extraction and classification of opinionated texts as expressing a positive or negative opinion or sentiment. Opinion mining is named sentiment polarity classification or polarity classification [²⁷].

The main task of opinion mining is polarity classification. However, we can find some other subtasks in this research domain, we can cite for example: subjectivity detection, emotion detection and classification, measuring sentiments intensity and opinion summarization.

Additionally, there exist three different levels to study sentiment analysis: word (feature) level, sentence level and document level. The term (word) level consists of identifying the orientation of each word in the text (positive or negative). The sentence level aims to classify each sentence in the document. Finally, the analysis performed on document level tries to recognize the polarity of the whole document as positive or negative.

In our paper, we use the supervised approach for classifying movie reviews in the Arabic language at the document level meaning that we consider the entire document as the basic information unit.

3 The State of the Art

Most of scientific studies on sentiment analysis focus principally on the English language, while similar studies conducted in the Arabic language and some other Morphologically-Rich Languages (MRL) are still limited. This is due to the lack of public lexical resources and the complexity of the automatic processing of Arabic language [¹, ¹⁹]. Indeed, the agglutinate nature of Arabic makes it more complex compared with the other languages. For instance, one word in Arabic could be a combination of prefixes, stems and suffixes [¹].

In this section, we present the most important research articles that have been performed in Arabic sentiment analysis and in particular in Arabic movie review classification.

To the best of our knowledge, the study of [³] is among the first articles that addressed the sentiment analysis problem in the Arabic language. The authors proposed a system that extracts sentiment-bearing patterns related to financial texts by use of local grammar. The performance rate of their proposed system was between 60% and 75%.

The studies of [²⁴, ³⁰] are considered among original research papers related to the movie reviews domain in English. The authors in [³⁰] proposed an unsupervised learning algorithm for classifying opinions as recommended (thumbs up) or not recommended (thumbs down). This algorithm extracts phrases containing adjectives or adverbs, calculates their semantic orientations, and then classifies the review based on the average semantic orientation of the phrases. The author found that the task of classifying movie reviews is difficult compared with banks and automobiles reviews. The accuracy percentage of automobile reviews was about 74%, whereas this value in movie reviews was 66%.

On the other hand, [²⁴] used standard machine learning algorithms to classify movie reviews. In their work, they found that standard machine learning techniques outperform human generated baselines. The researchers used popular classifiers in machine learning namely, Naive Bayes (NB), Maximum Entropy (MA) and Support Vector Machines (SVM). Experimental results showed that SVM is the best classification algorithm.

For classifying Arabic movie reviews, the researchers in [²⁷] collected Opinion Corpus for Arabic (OCA) from websites and blogs related to movies. The size of this corpus is small; it includes 250 positive reviews and 250 negative ones.

The authors used different pre-processing tasks comprising manual spelling correction, stop-words removal, filtering tokens by length, stemming and n-grams word generation. The experiments indicated that using the SVM classifier provides the superior performance results among the other algorithms.

In their next work, the researchers in [²⁸] translated the OCA corpus into English by using an automatic machine translation tool to build an English version called EVOCA. The authors employed two classifiers which are SVM and NB to classify movie reviews in their corpus. Their results showed that the performance in the translated corpus EVOCA was lower compared with the Arabic corpus OCA.

The authors in [²²] conducted a study in order to investigate the impact of selecting roots of words on the sentiment classification task. Their experiments were done on two datasets (the PATB, Part 1 v 4.1) [¹⁶] and OCA corpus. Results indicated that for the PATB corpus, the Khoja stemmer [¹⁵] with word unigrams was the best solution, whereas using the Tashaphyne stemmer [³²] recorded the best performance in the OCA movie dataset.

The study of [¹⁹] aimed to investigate supervised sentiment classification in the Arabic language. This work was performed at the document level. The researchers employed two collections of documents. The first was ACOM (Arabic Corpus for Opinion Mining) collected by the researchers including two datasets: DS1 consists of 594 movie reviews and DS2, which contains 1492 sport comments. The second dataset tested was OCA. The result of this paper showed that SVM and NB were efficient; whereas the outputs obtained from k-Nearest Neighbor (k-NN) depended on the documents collection.

The researchers in [⁷] investigated different factors that could impact sentiment classification of Arabic reviews such as text representation, feature selection and the algorithm used for predicting reviews. The outcomes of this study indicated that their pre-processing techniques improve sentiment classification performance.

Another scientific study presented in [²¹] that explored the effect of feature selection techniques and machine learning algorithms on Arabic sentiment analysis.

The work compared seven feature selection methods. In addition, three supervised classifiers (SVM, NB and k-NN) were evaluated. The experiment results of this comparative study showed that the SVM classifier coupled with the SVM-based feature selection method was the best option.

The work of [¹²] addressed the problem of lack of resources in the Arabic sentiment analysis domain by generating large datasets in different domains such as movies and products. The results obtained revealed that the best performance was provided by SVM, and the best feature models were the lexicon based feature combined with other features such as term frequency inverse document frequency (TFI-IDF) and count.

A suggestion of a combined approach that merges semantic techniques and machine learning ones was presented in the paper of [⁴]. Another research study [⁸] described a framework for Arabic opinion classification. The researchers compiled a text review dataset including 2591 tweets/comment, and used three algorithms (NB, SVM and k-NN) to predict the polarity of reviews.

The results of their experiments indicated that the three models (binary model, term frequency and TF-IDF) give similar results. In last years, several advanced studies employed deep learning DL models for opinion classification in Arabic. For example, the authors in [⁵] presented a study that explored DL algorithms for sentiment classification at the sentence level. The authors tested deferent DL models such as Deep Belief Networks (DBN), Deep Auto Encoders (DAE) and Recursive Auto Encoders (RAE). The authors found that RAE was the best model for classifying sentences.

More recently, the authors of [¹] proposed a distant supervision algorithm to create a corpus from Twitter called ’TEAD‘, they employed in their approach emojis and sentiment lexicons. Another recent study in [²⁰] that explored the effect of the pre-processing task on opinion classification results in Tunisian dialects. The researchers indicated that integrating named entity recognition NER improves opinion classification.

From this brief overview, we can note that datasets compiled to perform Arabic sentiment analysis about movies are of modest size when we compare them with the English ones. In this perspective, we decided to collect another dataset including 1000 reviews for movies. The second remark is concerning the pre-processing task, which is heavily investigated in Arabic research studies due to its impact on review classification results. The third observation is about the SVM classifier that is proved to be superior in Arabic sentiment classification in terms of performance results.

4 Proposed Approach

This section presents the details of our methodology for the task of sentiment prediction of movie reviews that includes the following sub-tasks: text pre-processing and representation, lexical approach applied to the entire text review, aspect comment extraction as well as the combination of the proposed methods. Figure 1 illustrates the steps of the proposed approach.

Fig. 1 Steps followed in our combined approach 

5 Experimental Study

The present section describes in detail the experimental part of this work; it includes tests and experiments conducted to validate the proposed approach. The methodology for text processing described in Section 4 was applied. In the experiments, we used the Weka tool for the task of pre-processing and classification. Weka is an open source tool for data mining applications that supports different tasks related to text mining like text pre-processing, clustering, classification and prediction [¹⁴].

6 Results

In this section, we present experimental results in order to explore the contribution of our model in sentiment classification. As weighting schemes, we employed TFI-DF. We note that, for example, Win 7 denotes the method that extracts comments using a window size of 7 words, while 7+ lexical means comments extracted by using a window size of 7 words combined with the lexical method that extracts opinion words from the text review.

The experimental results of applying the review models on OCA dataset are shown in Table 4. To extract opinion segments, we varied the size of the window to obtain the best value for sentiment classification. The best results of performance are highlighted in boldface. As it can be observed from Table 4, using bigrams as features improves slightly opinion classification.

In addition, there is not a significant difference in performance when using 9 and 11 words as a window size, while applying a window size of 7 words provides the worst result of classification. This behavior is expected since this length (three words before and after the aspect word) is not able to capture sufficient number of sentiment words, which are surrounding the aspect words.

We can also note that adding lexical features ameliorates slightly classification results, and utilizing the window size 9 combined with the lexical method yield the best results (92.40%).

Table 5 illustrates the obtained results when we applied the frequency based feature reduction method on the entire text reviews of OCA. The best result is obtained when selecting 3,000 as a feature size of words and bigrams of terms.

In next experiments, we investigate the contribution of combining the pruning technique with our method that extracts the most opinionated bodies of document reviews. Table 6 indicates the classification results using the length of 7, as a window size. We can see that 3,000 and 4,000 as threshold value yield the best classification results.

Next, the outcomes of selecting 9 words around the aspect term are depicted in Table 7, which demonstrate that the optimal value of FM is recorded when 3,000 of bigrams are determined as a threshold value.

It can be seen from Table 8 that the size of window 11 for extracting aspects comments (5 words before the aspect word, and 5 words after that word), combined with bigrams is the optimal setting for reaching the best classification results of FM (96%).

Finally, we provide in Table 9 the results of the combined method that extracts sentiment words by using the lexicon and integrates them to the comments about different aspects in movies.

We selected 11 as a window size since it is the best parameter for sentiment classification in the previous tests.

Our remark is that the combined method provides similar results compared with those of Table 8. The best value of FM recorded is 96%.

The results about the second data collection AMRD are summarized in Table 10. The employed features are bigrams of stemmed words. We recall that wind 7 means the method that extracts comments using a window size of 7 words, while wind 11+ lexical is the combined method. F-reduction denotes feature reduction.

The first remark is about opinion classification results, which are lower compared with those of the OCA dataset.

Results in Table 10 confirmed that the combined method (Wind11, lex, F-reduction) outperforms the others as it provides the highest classification outputs in FM (84.60).

From these results, we can conclude that the presented technique for extracting movie aspect comments allows us to mine useful opinions. The best value of window length was 11. We can also say that using bigrams of terms, applying stemming and feature reduction contribute positively in opinion classification.

To show the contribution of this work, we compare here our obtained results with those reported in the literature concerning the OCA dataset. The authors of [²⁷] reported 90% in F-measure, while the approach proposed by [¹⁹] recorded up to 93% in F-measure. We can say that our proposed methods for extracting relevant opinions provide superior results (96% in F measure).

To summarize the obtained results from the experimental study, we concluded the following findings:

Regarding the method for extracting aspect comments of movies, the best value of window size is 11, while the worst length of comment is 7.

Merging the extracted aspect comments of a movie with the sentiment words of lexical method enhances slightly the performance of opinion classification.

Combining the technique of review extraction with the feature reduction technique ameliorates opinion classification results. In addition, the contribution of bigrams is clearly positive. The optimal feature size is 4,000, and the improvement in f-measure is clear, 4% in OCA dataset.

Moreover, performance classification in our combined method depends mainly on the lexicon employed, we think that enriching and adapting the sentiment dictionary could enhance sentiment classification results.

The last remark regarding movie reviews, where negative comments contain sarcastic and ironic sentences comprising positive words used in a negative sense.

The enhancement of the sentiment classification system implies resolving this problem and other related issues such as negation and comparative opinions.

7 Conclusion and Future Work

This article addressed sentiment classification for movie reviews expressed in Arabic. The contribution of this work is to study the impact of selecting subjective parts in reviews and filtering movie reviews from their irrelevant parts including objective information and movie events description. To achieve this goal, we proposed to extract only explicit opinions and subjective parts related to the movie aspects such as film and actor.

In order to improve sentiment classification, we combined the extracted comments with the list of sentiment words found by the lexicon method. The contribution of the lexicon method is to extract implicit sentiments and opinions, which are not directly related to the movie aspect such as the conclusion opinion. In addition, we used Khoja stemmer and a feature reduction technique based on term frequency with the aim of improving the proposed system for review classification.

For the experiments, we applied our methods on two datasets. One of them is OCA, a free collection available on the web, while the other data collection is AMRD collected by the authors. As feature schemes, we used unigrams and bigrams of words.

The classification task was performed by using SVM, and the performance results proved the effectiveness of our combined approach.

As future work, we intend to apply our classification system on other domains such as social and marketing. Moreover, we plan to enhance Arabic sentiment classification by enriching sentiment lexicons, building specific dictionaries and text collections for different domains and aspects in Arabic. We also think that using deep learning models in the proposed system could augment opinion classification performance.