A TWO-FOLD PROCEDURE
The annotation of the Modal Corpus was performed as a two-step procedure :
(1) firstly the epistemic constructions were identified;
(2) then several semantic, syntactic, pragmatic and discursive features of each epistemic construction were annotated.
IDENTIFYING EPISTEMIC CONSTRUCTIONS
We identified the totality of constructions that explicitly signal the process of shared attribution of a truth value to the propositional tokens that compose a discourse. Here, a theoretical justification of the functional definition of epistemic construction.
ANNOTATING EPISTEMIC CONSTRUCTIONS
We represented the epistemic constructions identified in the corpus as complex triadic constructions consisting of a marker, a scope and a relation between the marker and the scope . Here, a theoretical justification of the formal representation of epistemic constructions as triadic constructions.
We annotated several semantic, syntactic and pragmatic features of the marker, the scope and the relation. Here, a detailed description of the annotation scheme.
OPERATIONALIZING THE ANNOTATION TASK
Here, the decision trees that guided the annotators in the identification of epistemic constructions and in the annotation task.
Inter-annotator AGREEMENT TESTS
The originality of the annotation procedure led us to develop an original technique for testing the inter annotator agreement, essentially based on the percentage of superposition between the spans of text identified as markers or scopes by the annotators. The following article describes this technique.
Ghia, E., L. Kloppenburg, M. Nissim, P. Pietrandrea, V. Cervoni (2016) A Construction-centered approach to the annotation of modality. Proceedings of the 12thISO Workshop on Interoperable Semantic Annotation. Portoroz, 29 may 2016.
Here, the script, developed by Lennart Kloppenburg, used to calculate the inter annotator agreement.
Here, the results of the Inter Annotator Agreement test for the task of identification and annotation of epistemic constructions in the English, French and Italian corpora.