Following the success of Evalita 2007 and Evalita 2009, we have organised Evalita 2011, the third evaluation campaign of Natural Language Processing and Speech tools for Italian, supported by the NLP working group of AI*IA (Associazione Italiana per l’Intelligenza Artificiale/Italian Association for Artificial Intelligence) and AISV (Associazione Italiana di Scienze della Voce/Italian Association of Speech Science). The participation was open both to academic institutions and industrial organizations and we proposed the following tasks:

• Parsing
  • Dependency Parsing
  • Constituency Parsing
• Domain Adaptation
• Named Entity Recognition on Transcribed Broadcast News:
  • Full Task
  • NER only
• Cross-document Coreference Resolution
• Anaphora Resolution
• Super Sense Tagging
• Frame Labeling over Italian Texts
• Lemmatisation
• Automatic Speech Recognition – Large Vocabulary Transcription
• Forced Alignment on Spontaneous Speech:
  • Phone Segmentation
  • Word Segmentation
• Voice Applications on Mobile – Student Contest

Guidelines describing the different tasks will be distributed among the participants. Participants will also be provided with training data and will have the chance to test their systems with the evaluation metrics and procedures to be used in the formal evaluation.

The results of the evaluation have been disseminated at the final workshop which have been organized in conjunction with the AISV 2012 Conference (Rome, 24th-25th January 2012). The workshop program is available on a dedicated web page.

Final reports describing the participating systems are available in the workshop electronic working notes. After the workshop, the proceedings will be published by Springer in the Lecture Notes in Artificial Intelligence (LNAI), the LNCS subseries.

Post-proceedings have been published by Springer in 2013: “Evaluation of Natural Language and Speech Tools for Italian“.

Dissemination Material – Download Flyer