Speaker
Affilliation
Brighton
Abstract
We have some practise talks scheduled for 12th June at 11:30am in the Chichester 1 NLP meeting room (Ch1 011) . Anja Belz is giving two practice talks for ENLG 07, see the abstracts below. There will hopefully also be a bonus ACL demo feature from Adam Kilgarriff. We anticipate that the 3 talks will each last 20 mins approx + time for questions.
1. Anja Belz, Albert Gatt, Ehud Reiter and Jette Viethen: The First NLG Shared-Task Challenge: Attribute Selection for Referring Expressions Generation, ENLG'07, 18/06
The field of Natural Language Generation (NLG) has strong evaluation traditions, in particular in user-based evaluation of applied systems. However, while in most other NLP fields shared-task evaluation now plays an important role, there are few results of this kind in NLG. The Shared Task Evaluation Campaign (STEC) in Generation of Referring Expressions (GRE) is intended to be a first step in the direction of exploring what is required for shared-task evaluation in NLG. Under the umbrella of this GRE STEC, we are planning to organise a series of evaluation events, involving, over time, a wide range of GRE task definitions, data resources and evaluation methods.
As a first step, and in order to gauge community interest, we are setting up a pilot evaluation in the spirit of a feasibility test: the Attribute Selection for Referring Expressions Generation Challenge. This Challenge will be presented and discussed at this year's UCNLG+MT Workshop in Copenhagen, on 11 September, at MT Summit XI. If successful, we plan to organise a larger-scale event in 2008, extending the remit to cover aspects of GRE beyond attribute selection as well as more data resources and evaluation methods.
This talk will give a brief overview of the shared task itself, organisation of the event and evaluation of the results.
http://www.csd.abdn.ac.uk/research/evaluation
2. Anja Belz and Sebastian Varges: Generation of Repeated References to Discourse Entities, ENLG'07, 18/06
Generation of Referring Expressions is a thriving subfield of Natural Language Generation which has traditionally focused on the task of selecting a set of attributes that unambiguously identify a given referent. In this paper, we address the complementary problem of generating repeated, potentially different referential expressions that refer to the same entity in the context of a piece of discourse longer than a sentence. We describe a corpus of short encyclopaedic texts we have compiled and annotated for reference to the main subject of the text, and report results for our experiments in which we set human subjects and automatic methods the task of selecting a referential expression from a wide range of choices in a full-text context. We find that our human subjects agree on choice of expression to a considerable degree, with three identical expressions selected in 50% of cases. We tested automatic selection strategies based on most frequent choice heuristics, involving different combinations of information about syntactic MSR type and domain type. We find that more information generally produces better results, achieving a best overall test set accuracy of 53.9% when both syntactic MSR type and domain type are known.
http://www.nltg.brighton.ac.uk/home/Anja.Belz/Publications/belz-varges-enlg07-final.pdf