In the “Frame Labeling over Italian Texts” (FLaIT) evaluation exercise systems have to detect the semantic frame “evoked” by a predicate and the major semantic roles explicitly mentioned in an Italian sentence, according to the frame semantics paradigm of (Fillmore, 1985). In particular, the task consists in recognizing words and phrases that evoke semantic frames of the sort defined in the FrameNet project (Baker et al., 1998, http://framenet.icsi.berkeley.edu), and their semantic dependents, which are usually, but not always, their syntactic dependents. We will refer to this problem as Semantic Role Labeling (SRL). As in previous SRL shared tasks (e.g. CoNLL-2004 and CoNLL-2005), the general goal is to come forward with representation models, inductive algorithms and inference methods which address the proposed SRL problem. Previous experiences (as in CoNLL-2004/2005 or Semeval 2007 (Baker et al., 2007)) were focused on developing SRL systems based on partial parsing information and/or increasing the amount of syntactic and semantic input information, aiming to boost the performance of machine learning systems on the SRL task. Accordingly, the Evalita 2011 FLaIT challenge will concentrate on the definition of different tasks, focusing on different aspects of the SRL problem:

• We encourage the adoption of basic resources for Italian that are under development in the iFrame project (http://sag.art.uniroma2.it/iframe/doku.php). These resources will be made publicly available to all groups participating to the FLaIT task
• The exploitation of syntactic information is also encouraged as the challenge is expected to shed some light on the impact of current parsing resources and technologies for Italian on the overall SRL task
• Interested groups that may not rely on proprietary parsing technologies will be supported in their participation as they will be provided with annotations for the development and test data at the morphological and syntactic level (at least lemmas, POS tags and Named Entities are expected). The level of quality of these auxiliary information may not be homogeneous, as no full manual validation in the released material is expected for the 2011 EvalIta edition
• The use of external lexico-semantic knowledge bases for the Italian language is also encouraged, as the impact of the different resources is one of the targeted research aspects of the challenge. All participants are suggested to propose novel learning architectures for better exploiting the data structures, relations and constraints of the problem

Participant systems will be evaluated in different categories, depending on whether they use the auxiliary information strictly contained in the training data (closed challenge) or they make use of external sources of information and/or tools (open challenge). Participants in the open challenge are encouraged to propose novel ideas for using rich semantic information, e.g., Wordnet for Italian, other lexico-semantic resources, such as for example distributional lexical semantic information, or word sense disambiguation tools, etc. The use of unlabeled examples might be also considered.

Task materials

Detailed guidelines can be found at the dedicated web page.

Data Distribution

• Test data will be available from 12 October: for more information, please visit the dedicated web page [05/10/2011]
• A subset of development can be downloaded from the dedicated web page

The full training set is available from the dedicated web page For any problem please contact: Roberto Basili, basili[at]info.uniroma2.it

Organizers

• Roberto Basili (University of Roma, Tor Vergata)
• Alessandro Lenci (University of Pisa)

Steering Committe

• Alessandro Moschitti (University of Trento)
• Sara Tonelli (University of Venice), Diego Decao (University of Roma, Tor Vergata)
• Giampaolo Mazzini (CELI, Torino)

References

Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley frame net project. Collin Baker, Michael Ellsworth, and Katrin Erk. 2007. Semeval-2007 task 19: Frame semantic structure extraction. In Proc. of SemEval-2007, pages 99–104, Czech Republic. Charles J. Fillmore. 1985. Frames and the semantics of understanding. Quaderni di semantica, 6(2):222–254, december.