1.1 Frames

Frames were introduced by [³], who defines them as “data structures for representing stereotyped situations, like being in a certain kind of living room, an event or object like a cooking pot. Attached to each frame are several kinds of information”. This work uses frames to add semantic knowledge to DESPRE. Briefly, a collection of suitably related frames forms an ontology. Examples of (parts of) ontologies appear in Figures 4 and 5. For example, the frame for a cooking pot shown in Figure 1 is used to identify the type of relationship between the words of a sentence.

Fig. 4 This part of the ontology shows two meanings of the preposition de 

Fig. 5 Another part of the ontology. (a) entidad física {physical entity} is a predecessor of olla {pot}. (b) material {material} is a predecessor of barro {clay} 

This paper is based on [⁴], which uses frames as explained below.

Semantic frameworks (frames) are useful to find the meaning of each word in a sentence, without ambiguities, and ensuring that its syntax and semantics are correctly defined and identified (Figure 1).

They help to identify situations, or the meaning of sentences based on the prepositions used.

1.2 Building the Frame for a Sense of a Preposition

The RAE dictionary gives 26 senses for preposition de {of, from}. As an example, let us build the frame for sense 4 (called d4 in Table 1), the RAE dictionary defines this sense as:

d4: Denota la materia de que está hecho algo.

El vaso de plata. El vestido de seda. {Substance; made of}.

Sense d4 denotes the matter of which something is made. It is used in phrases such as: El vaso de plata {The silver glass} or olla de barro {clay pot}. It describes the material or substance (clay) of which certain things (pot) are made of. A frame is wanted to represent this meaning.

The frame could be as simple as pot made of clay, but it is necessary to generalize as much as possible this meaning, not only for pots, glasses, or clay. WordNet (WN)^fn comes in handy for this. Pot generalizes (a superset, a hypernym) to: *cooking utensil, *kitchen utensil, *utensil, *implement, *instrumentation, *artifact, *object, *physical entity (Figure 5 shows this part of the ontology), while clay generalizes to: soil *earth, *material, *substance. It is necessary to stop the generalizations so that the sense d4 is expressed as generally as possible, but not more.

For instance, substance generalizes to physical entities, but it is clear those physical entities are made of substances but not those physical entities are made of physical entities.

Thus, the frame for sense d4 is physical entity made of substance. When disambiguating olla de barro, {pot of clay} DESPRE locates olla {pot} as a physical entity (using ontology shown in Figure 5), and clay {barro} as a substance, so that the sentence disambiguates to: olla hecho de {made of} barro.

The net effect is that de becomes hecho de {made of}, which is the right meaning in this case.

The sense of the preposition de {of} for sense d4 is hecho de {made of}. These senses of the preposition need to be given to Freeling in a set of adjusted rules (box in the right of Figure 3), so that it can construct a correct dependency tree.

Another sense for de is given by the RAE dictionary as sense 5, which in Table 1 is:

d5 Úsase para señalar lo contenido en algo {Contained in something}.

Sense 5 of de is used to indicate what is contained in something. For example: un vaso de agua {a glass of water}, olla de arroz {rice pot}. In this case, pot generalizes to vessel with its ancestors: *container, *instrumentality, *artifact,

*object, *physical entity, but stops the generalization in this case to container.

Rice generalizes to *grain, *foodstuff, *food, *substance, *physical entity, but the generalization stops in food. Therefore, the sense d5 is expressed by frame container: que contiene {that contains} food.

Sense d5 replaces de by the “correct preposition” que contiene.

Then de becomes hecho de {made of} or que contiene {that contains}, depending on context. Figure 4 shows these two senses of de.

In fact, RAE dictionary says that de has 26 senses, of which DESPRE now disambiguates 13: those shown in Table 1.

1.3 Storing Frames in an Ontology

[⁵] explains in detail how an ontology is formed where the Ontology Merging Notation is explained. Briefly, an ontology can be seen as a network where links or edges relate concepts or nodes. Figure 3 shows the use of the ontology in which a noun represents a concept that is a node in the ontology, while verbs or actions are edges that link two nodes.

Figures 4 and 5 show parts of an ontology. Since each sense of a preposition is stored as a frame in the ontology, which now contains 50 frames: those corresponding to the 50 disambiguated senses, see Table 1.

2.1 FreeLing

FreeLing is a library with a set of Natural Language Processing (NLP) applications, whose purpose is to facilitate the integration of language analysis tools in high-level applications [²].

The online version requests a data entry in natural language text, to which the user chooses the analysis tool he/she wishes to test labeling, morphological analysis, deep analysis, dependency tree, etc. Later, the result of the tool applied to the text is presented.

FreeLing analyzes several languages. The Spanish language has been chosen because it is the most complete and robust module with dictionaries and preprocessing.

The modules that the Spanish language contains are: Lexicographic analyzer, sentence separator, number detection, morphological dictionary, rule applicator, multi-word detection, basic detection of named entities, detection of BIO named entities, quantity detection, labeling PoS, phonetic encoding, WN sense annotation, UKB sense disambiguation, shallow analysis, deep dependency analysis, preference resolution [²].

2.2 Disambiguation of Prepositions in Spanish

[⁸] describes an algorithm and its implementation to disambiguate prepositions in Spanish phrases; some of them are to {a}, before {ante}, under {bajo}, for {para}. They use stand-alone frames, not forming part of an ontology.

Their algorithm tags (with EAGLES labels) each word of a sentence. Then, it looks for a preposition and the word before and after it, much as is done now.

After that, it looks for the frames of these three words. If at least one ‘hypernym’ matches, their algorithm takes the synonymy of the preposition, and it uses this sense as the correct one.

The difference with DESPRE is that [⁸] only uses the ‘hypernym’ of the frame. Since the frames are part of an ontology (see Figure 4), not only the closest ‘hypernym can be used’, but also any predecessor, until a match is found (refer to Figure 4). Another difference: DESPRE ignores the adjectives before and after each word.

For instance, for the sentence: olla verde bonita de hermoso barro {nice green pot of beautiful clay} our algorithm reduces it to olla de barro {pot of clay} and disambiguates that into olla hecha de barro {pot made of clay}, thus finding the correct sense for preposition de {of}.

2.3 PP Attachment

When a parser builds a syntactic tree of a sentence, it faces the problem to decide where to attach a phrase to a list of possible places.

An example of this kind of ambiguity is the PP attachment.

This kind of ambiguity consists in deciding if the attachment is to a verb or a noun, or to which noun.

The PP attachment is considered one of the most difficult constructions for parsers, causing many of their parsing errors [⁹].

Researchers have proposed several methods to solve ‘PP attachment’, and one way to classify their proposals is by identifying the previous knowledge they use.

The corpus-based approach uses annotated examples of sentences with PP attachment ambiguity solved, and these examples are used to train a supervised machine-learning algorithm that learns from annotated sense examples. [¹⁰] use semantic classes to improve the parsing and ‘PP attachment’ tasks in English sentences.

The authors preprocess the input by substituting some words with their corresponding synsets of WN [¹¹], for example, knife is replaced with tool, and then they use three different disambiguation methods.

The first method uses the manually annotated senses in a gold standard corpus created from merging the Penn Treebank [¹²] corpora.

The second method disambiguates by taking the first sense data from WN. The third uses an unsupervised system to predict automatically the most frequent sense of the words. The results showed for the third method an error reduction rate of 20.5 % in the ‘PP attachment’.

In [¹³], a classifier was trained with different features to decide whether the preposition is attached to a verb or a noun. It works with Spanish sentences. The authors extracted 4,764 examples of ‘PP attachment’ ambiguities from the AnCora corpus [¹⁴] and used 90% for training and 10 % for testing. The EsTxala parser in the FreeLing Environment, which obtained 61% of accuracy, was taken as a baseline. The classifier was the best with 87.84% of accuracy, improving by 26% the results of the baseline. Unlike these works, DESPRE does not need previously annotated examples to disambiguate the prepositions.

This is a very important difference because the effort associated with manual tagging could be very high.

In [¹⁵], the authors consider that prepositions may have not only two head candidates (a verb or a noun) but a set of them. For this purpose, they created a dataset for the German language which considers multiple head candidates. A topological field model of that language was used to find regularities in word order across different clause types in German, and this information helped the algorithm to select the correct head of prepositions. DESPRE uses only one head at a time.

The knowledge-based approach uses linguistic resources but does not heed previously annotated examples. Lexical information gathered from large French corpora was used in [¹⁶] to resolve ‘PP attachment’ ambiguities using two methods. In the first method, the author collected around 45,000,000 pairs of verb + preposition and noun + preposition from the Giga-word corpus according to [¹⁷] and calculated the Mutual Information (MI).

The MI provided a measure of association strength and helped to disambiguate the ‘PP attachment’. The second method created a vector-space representation of the context’s words, where the Distributional Semantic Model was a frequency count matrix with a window of size 3 for the context’s words. The cosine is used as a comparison vector, the higher the similarity between vectors, the higher the possibility for a particular attachment. The best result, using the second method, showed an accuracy of 76.9%.

[¹⁸] used word vector representation to build a compositional neural network architecture. They tested it on some English and Arabic datasets.

The word vectors were initially obtained from raw text but in following steps, these vectors were enriched by adding information from WN, VerbNet and the words part-of-speech. All this information trained the neural network, obtaining 88.7% accuracy.

The above approaches –and most of the state-of-the-art in ‘PP attachment’– share the supervised machine learning method. A fresh approach is that of [¹⁹], like DESPRE, uses frames to disambiguate a few prepositions in the Persian language, obtaining a high precisión (99.16%) of its system. [²⁰] says that morph-syntactic information is often insufficient to resolve the ambiguity problem, additional semantic information or even world knowledge is needed.

A PP attachment disambiguation for German system based on biaffine attention and contextual word embeddings is presented. This paper shows that modeling all head-dependent pairs jointly (as done in full parsing) allows the system to make more effective use of the training data and is thus superior to modeling PP attachment as a separate task.

During disambiguation process an Artificial Neural Net (ANN) was used. This ANN was training on the German dataset from the CoNLL corpus 2009 Share Task. They keep ambiguous and unambiguous triples and calculate five association scores. With the same hyper parameters as PP-REP, their system with auxiliary distributions achieves an accuracy of 86.8%.

In [²¹], a novel methodology for preposition sense disambiguation (PSD) in English is proposed, in which does not use any linguistic tools. It uses a supervised setting, where the machine learning model is presented with sentences wherein prepositions have been annotated with ‘senses’. These senses are IDs called ’The Prepositions Project (TPP)’. A Multi-Layer Perceptron is used to classify IDs into the correct sense. The dataset used is from SemEval-2007 Task-6.

The methodology gives an accuracy of 86.85%. In [²²], the authors implemented a module PATCH-COMM, for various conventional parsers, to make attachment decisions, where the commonsense KB infers “out-of-knowledge-base” assertions in a similar manner to the way some NLP systems handle out-of-vocabulary words.

The results suggest that the commonsense knowledge approach can provide the best of both worlds, integrating rule-based and statistical techniques. Also, PATCH-COMM makes the knowledge acquirable by using Machine Learning mechanisms, even if they may be somewhat opaque. Thus, PATCH-COMM focuses on a particular knowledge base and a particular ANN based architecture for extending the knowledge contained in that knowledge base.

His results have been evaluated using Belinkov et al.’s English dataset (2014) [²³]. Only the preposition ‘with’ was evaluated, having the best result from the paper 88.7%.

In [²⁴], the authors provide a way to resolve English ambiguities under the locally factored model by explicitly modeling bi-lexical dependencies. Their proposed model achieves 88.8% in terms of correct labels, which marks the best scores so far.

As is seen in these related works, to solve text analysis problems, it is not enough to use only morpho-syntactic methods. Thus, they are seen complemented with some semantics, through WN, or with classes, tools, specific methods, and algorithms. Thus, FreeLing a semantic module has been chosen.

In this way, it can deduce the meaning of words from its presentation in the text and its semantic context to solve their ‘PP attachment’. As seen below, the semantic information used by DESPRE improves the ‘PP attachment’ phase of FreeLing. It could improve, as well, other text analysis tasks, like dependence analysis and reference resolution.

3.1 DESPRE

DESPRE focuses on Spanish, since it is an important language, spoken by 580 million people. It is the world’s second-most spoken native language after Mandarin Chinese, and the world’s fourth-most spoken language overall after English, Mandarin Chinese, and Hindi.

DESPRE is centered on only one language, since trying to solve PP attachment in a general way for several languages is much more difficult (for instance, some languages do not have prepositions). In fact, [²⁵] argues that cross-language comparisons of the disambiguation results in previous research is impossible because of the different selection procedures when building the training and test sets.

DESPRE uses a knowledge-based approach, as in [²⁶], to disambiguate the PP attachment. To obtain the sense or meaning of a fragment of a sentence, it employs a populated ontology of prepositions and their different senses, instead of using WN or probabilities or statistics.

Freeling is used, since it works very well for Spanish, and it does an excellent job in exploiting its syntax.

DESPRE’s strategy is knowledge-based and takes advantage of the frames of the ontology to disambiguate the preposition, so that the parser can use sense-specific rules to properly build the tree. It does not need a training corpus or a huge amount of text to calculate probabilities or mutual information, and it is able to obtain the sense of the preposition from the word’s context and the ontology.

This facilitates DESPRE’s strategy to be re-targeted to other languages where the appropriate resources (frames) are available. In other words, from the frames of the ontology, DESPRE obtains the necessary knowledge to attach the prepositions correctly, while at the same time using to its advantage, as most approaches do, the syntax of Spanish sentences.

3.2 How DESPRE Works

The purpose of our work is to classify the 22 Spanish prepositions into their different semantic categories or senses, using the senses given by the RAE dictionary. This paper presents the disambiguation of 17 of them.

Nevertheless, this combination of semantic and syntactic methods can be used to analyze all of them once suitable frames are added to the ontology. Their disambiguation permits correct attachment of prepositional phrases, as previously shown in [⁴].

Our applications added to FreeLing are:

Our approach is applied after the FreeLing PoS tagger has been run, and before the rule-based dependency parser is executed.

Frames are used (they indicate whether there is a semantic relationship between two relevant words) to disambiguate the preposition senses given the context.

Then, the tagged sentence with the marked prepositions is fed to the parser, which uses a modified grammar that considers these preposition senses to perform the proper attachment of prepositional phrases. Examples of the use of semantic relationships are:

Nevertheless, people do not speak with all those details of meaning, because they talk to another person, who readily understands the sentences.

Examples d) and e) have a similar morphological and syntactic structure, since nouns surround the preposition with {con} in both cases, but the meanings are different.

Example d) shows that café {coffee} lo bebo en compañía de {is drunk accompanied by} mi hermano {my brother}, while example e) indicates that café {coffee} tiene {has} or está mezclado con {is mixed with}, que tiene leche {milk}.

Figure 2 shows the dependency tree computed by FreeLing for example d), which is correct but lacks the sense of the preposition con {with}. Figure 8 presents the parsing made by DESPRE, which adds the sense of the preposition en compañía de {accompanied by}.

Fig. 6 FreeLing’s default output. The preposition con {with} relates leche {milk} to tomo {to drink}, the highest node, which is incorrect 

Fig. 7 Correct result, showing that preposition con {with} has been replaced by que tiene {that contains} and linked to café {coffee}, instead of linking it to tomo {to drink} 

Fig. 8 DESPRE replaces con {with} by en-compañía-de {accompanied by}, thus linking mi hermano {accompanied by my brother} to the verb tomo {to drink}, resulting in the correct dependency tree} 

However, in example e), milk must be attached to coffee, because the term milk is a modification of coffee. That is to say, drinking coffee que contiene leche {that contains milk}. Figure 7 shows the correct parsing (by DESPRE) for example e).