2.1 Text Mining

Text mining is the process of transforming unstructured text into a structured format to identify meaningful patterns and new insights in data [²¹]. The process of text mining involves the use of different research methods for obtaining valuable information from large amounts of data [²⁷].

Some of this methods include techniques for acquiring and analyzing digital documents with NLP or statistics (besides others) [³¹, ⁵]. Researchers today, are using distinct text mining approaches for predicting domain patterns, public opinion and collectible behavior [¹³].

Text mining software has impacted the way that many industries work, allowing them to improve user experience and business decisions. Examples of this impact can be seen in different areas like customer service [²³] where chatbots, and profiling tools are making the user experience faster and simple; Healthcare systems [¹¹], where distinct tools collect massive amounts of medical information for detecting critical insight of patients; and Cyber-security [¹⁵], where spam filtering and automatic intruder detection tools are making possible to identify malicious users. Text mining techniques are becoming more integrated and easier to use on the web.

This in turn, have introduce many approaches related to trending elements. Examples of this include: the analysis of digital marketing [³⁹], recommendation systems [⁴], decision-making [³⁶] and social network analysis [¹⁹]. For this last one, the extraction of non-trivial knowledge related to what is trending and whats is not have made an special effect on how users understand and consume information.

Considering its impact. it can be seen the importance of text mining and how it help others to make the most of their data, which leads to better decisions. Without this kind of tools, it would be impossible to analyze massive amounts of information (mainly on the web) which in consequence will stop the growing-flow of knowledge on the web.

2.2 Text Mining on Political Documents

In the context of social media (X, Facebook, Reddit, etc.), the analysis of political data by text mining techniques [²²] have gained momentum considering the large amount of textual information, the number of interactions and the importance of the topic today (specifically on the U.S.) [²⁹].

Citizenship and law enforcement topics have different text mining approaches that deal with the extraction of relevant users using social network measures (closeness, degree centrality, etc.) [³¹, ¹⁴]. Other approaches [³²] used different ML algorithms (triplets and clustering) for detecting communities that might be important in a specific context. Other kind of approaches use classic statistic metrics like mean, median, mode, etc combined with IR scrapers for detecting structural patterns on social media data [¹⁸].

Immigration and border Security topics [⁷] also have distinct text mining approaches like the use of a friend of a friend (foaf) and co-occurrence graphs for obtaining activity patterns associated to users [¹⁷] or for summarizing/understanding large amounts of textual information [³³]. Other kind of techniques rely on NLP for applying Part of Speech (PoS tags) and entity recognition techniques for scrapping linguistic features that help to understand the structure and purpose of information (attitude, entities, sentiments etc.) [²⁰]. Other approaches [⁶] have deal with the immigration analysis by detecting geo-spatial patterns of users on the Mexico-U.S. border. Additionally, distinct efforts [²] have implemented mining techniques for automatic content-characterization of news, stories and blogs related to the immigration phenomena.

From the different mining approaches implemented for political purposes, it can be seen that is a growing area where more interactions are available everyday. This in turn have made possible to analyze relevant text patterns from different social media sources, which applied in different research and decision-making process have a meaningful impact on how users understand, digest and use knowledge.

4.1 Keyword Selection

In order to obtain suitable information from social media, words related to the immigration topic on July 2016 (dataset starting point) were extracted according to their frequency of occurrence on one thousand web pages^fn. For each language, the fifteen keywords without stopwords^fn or special characters are extracted from the web pages. These words were used as input for diverse social media APIs to obtain relevant text documents. Table 1 summarize words used on each language.

4.2 Social Media Sources

There are several social media APIs to obtain textual information, but most of them do not offer a public streaming to download texts periodically. Among the media channels that do not have this kind of restriction, X (formerly known as Twitter)^fn and Reddit^fn provide robust tools to download streaming data with minimum authentication and authorization from users over different languages.

Additionally, both media channels have real time interaction among users and cover a vast amount of domains around the world (U.S. specifically). Considering the previous features and the free availability of data on both social media channels, it was decided to use X and Reddit as primary data sources for detecting valuable knowledge related to the Immigration topic.

4.3 Text Documents Obtained

Using X and Reddit APIs, a dataset that comprise English and Spanish samples from U.S. users were collected. The dataset contains text documents from July 11 2016 to July 11 2019. All dataset samples were collected in a daily basis using a JSON format^fn with a UTF-8 encoding for storage each text document. Each collected sample contains properties/metadata related to the textual interaction. In the case of X^fn, seventy one properties are obtained for each sample including: name, country, date, number of followers, number of likes, etc.

For Reddit^fn, fifteen properties are collected including: author name, text date, country, topic name, topic score, subreddit name, etc. The dataset also provides some geolocation properties. In the case of X, this provides coordinates and place elements (if these are release by the user) while for Reddit, this does not provide any kind of metadata that can be used for obtaining the latitude and longitude associated to a post due to some API restrictions.

Table 2 summarize the dataset main features emphasizing the number of documents for Spanish and English languages on X and Reddit. From the dataset table, it can be observed that the English subset considered both social media channels while for Spanish it is only used X.

This is due to the lack of Spanish samples and the small amount of texts retrieve from the Reddit API. Additionally, The number of samples collected highlight the amount of interactions in both social media channels. In the case of X, there are more small interactions (140 characters at most) while for Reddit there are less interactions but these condensate more textual information.

6.1 User Interaction Graph

Taking into consideration how users relate to each other and the importance of this relationships to understand social media synergy, a graph that represents the interaction among users that talk about immigration topics on X is proposed (Reddit does not give much importance to users as X does). Formally proposed graph G=(V,LV,LE) has the following attributes:

LE = #hashtags + #URLs + #interactions that have in common two X users (two vertices).

As an example of this graph-based representation, consider the following texts extracted from two texts from X:

Based on proposed properties, user and text information previously preprocessed^fn can be mapped to the user interaction graph shown in Figure 4. In the user interaction graph, the communication (edges) between X users (vertices) that talk about the immigration in English is considered.

Fig. 4 User interaction graph example 

The goal of this graph, is to take advantage of the properties provided by X (and not by Reddit) like the users (@), topics (#) and even the web resources (URLs) to propose a weighting scheme that could be used to uncover meaningful users, communities and new interactions. This graph can be seen as a special foaf graph^fn [³⁸] where the relationships are oriented to model how users relate to each other when they are mentioned in a specific context.

6.2 Co-Occurrence Graph

Keeping in mind the relevance of lexical elements (based on previous section) and the lack of syntactic structure on social media documents. A non-directed and unweighted graph representation based on the co-occurrence [³¹, ⁴⁰] of two words is proposed.

The objective of this graph is to take advantage of all natural interactions between words^fn to extract valuable lexical-syntactical patterns that can not be obtain using traditional statistics. Formally, the proposed co-occurrence graph used in the experiments is represented by G=(V,E,LV),where:

As an example, consider the following sentence ζ extracted from a text T: “Axel Rose needs to just give up. Now. Not later, not soon, not tomorrow.”, which after the preprocessing stage^fn would be as follows: “axel rose needs to just give up now not later not soon not tomorrow”. Based on the proposed representation, preprocessed sentence ζ can be mapped to the co-occurrence graph shown in Figure 5.

Fig. 5 Example of a co-occurrence graph with a window of two words 

The proposed co-occurrence graph captures the syntactic interactions (edges) of all the words (vertices) used when people discussed about immigration in X-Reddit (for English and Spanish languages). The idea of this graph is to obtain relevant words, phrases and even sentences that describe the way individuals write about the topic for understanding attitudes, trends and media consumption.

6.3 Graph Properties

In order to understand the richness of the two proposed graph structures (see Sections 6.1 and 6.2). Table 5 shows main features associated to the co-occurrence graph and the user interaction graph in the context of the social media sites analysed. From table 5, it can be observed the amount of vertices and edges created from the social media dataset. This highlight the diversity of information obtained trough several months as well as the complexity of the data structures created. Considering graph properties, upcoming sections show results examples obtained by optimized graph mining algorithms [³⁰].

6.4 User Interaction Graph Results

As in the statistical phase, the main objective of the user interaction graph is to obtain insightful knowledge from social media.

In this case, there are extract relevant users from X based on distinct social network analysis metrics (centrality, community detection and link prediction). Table 6 show top X users in English and Spanish languages using a reinterpretation of a classic social network analysis called centrality measures^fn [⁹]:

Other elements retrieved from the user interaction graph are triad elements [²⁴]. This provide topological insight of the interaction of three interconnected vertices. These kind of subgraphs helps to understand how users spread information about immigration and how close they are in terms of their interactions.

The triads with highest edge weight are consider as relevant communities that maintain a constant communication flow. Figure 6 display some examples of triads found on English language (something similar is performed for Spanish language). In addition to the triad detection, the uncovering of bigger communities is also performed by the user interaction graph.

Fig. 6 User interaction graph: English triad examples found 

The edge-betweenness centrality [³⁷] is used for detecting high interconnected vertices on the network. The idea of this technique, is to gradually remove the weighted edges with highest betweenness and recalculate the centrality for all edges after every removal. This way sooner or later the network falls off into smaller components which are relevant user groups that talk about immigration.

Figure 7 illustrates the types of user communities obtained on the context of the immigration topic for the Spanish language (something similar is performed for English language). Finally, there is a applied an link prediction approach [²⁸, ⁸] for inferring new relationships among users based on the topological structure of the graph and the weighted scheme proposed.

Fig. 7 User interaction graph: Community of Spanish users found 

This kind of technique analyzed the neighborhood of vertices on the graph. Vertices that have similar neighbors will have new edges in the near future while vertices that do not share many neighbors will remain without change. Figure 8 show some examples of interactions inferred using the link prediction and the user interaction graph on the English language.

Fig. 8 User interaction graph: Link prediction examples found 

6.5 Co-Occurrence Graph Results

The use of a co-occurrence graph permit to obtain insightful knowledge from texts that comes from different social media. The co-occurrence representation allow to extract more complex patterns that do not depend entirely of the frequency of occurrence of texts (like in statistics). Tables 7, 8 and 9 show relevant examples of words, phrases and sentences obtained by using the co-occurrence representation and some of centrality measures^fn. The following interpretations are proposed for extracting elements from the graph:

In table 7, top words are presented without considering stopwords and special symbols. From the words obtained, it can be observed that degree centrality obtained words that people mentioned immediately when the immigration topic is discussed, while in the closeness centrality, the words that play an active and central role in the texts related to immigration are extracted.

In the case of the top phrases related to immigration, Table 8 shows the top collocations with stopwords but not special symbols. Analyzing the collocations obtained using the degree centrality it is possible to see the relevant subtopics used when individuals write about immigration on X and Reddit.

For top sentences related to immigration, Table 9 show some examples that reflect the most important attitudes and ideas related to immigration. These examples are formed of words with high closeness elements which implies that these sentences could be used as a brief summary of what people write about immigration.