Lexical Semantics: Bridging the gap between semantic theory and computational simulations

Workshop at ESSLLI 2008, Hamburg, August 4-9 2008

http://wordspace.collocations.de/doku.php?id=esslli:start

http://www.illc.uva.nl/ESSLLI2008/

Corpus-based distributional models (such as LSA or HAL) have been claimed to capture interesting aspects of word meaning. However, although they have been proposed as plausible simulations of human semantic space organization, careful and extensive empirical tests of such claim are still lacking.

Systematic evaluation of these models focus on large scale quantitative tasks, often more oriented towards engineering applications (see, e.g., the recent SEMEVAL evaluation campaign) than towards the challenges from formal semantics, linguistic theory, philosophy and cognitive science. This has resulted into a great divide between corpus-driven computational approaches to semantics and more symbolic approaches, typical of the linguistic and of most of the cognitive tradition.

Moreover, whereas human lexical semantic competence is obviously multi-faceted, ranging from free association to taxonomic judgments to relational effects, tests of distributional models tend to focus on a single aspect (typically limited to semantic similarity detection), and few if any models have been tuned to tackle different facets of semantics in an integrated manner.

Our workshop purports to fill these gaps by inviting single scholars and research teams to test their computational models on a variety of small but carefully designed tasks, that aim to bring out linguistically and cognitively interesting aspects of semantics. Specifically, we envisage the following tasks:

- categorization;

• concrete nouns categorization;
• concrete vs. abstract nouns discrimination;
• verb categorization;

- modeling free association;

- modeling generation of salient properties of concepts;

- one or more tasks to test semantic compositionality

The focus is NOT on competition, but on understanding how different models highlight different semantic aspects, and how far we are from integrated models of all such aspects. In fact, we believe that the current state of the art on computational semantics does not need to discover the best model, but rather to enlarge the understanding of the limits and potentialities of different approaches when confronted with cognitively realistic tasks. To this effect, annotated datasets will be distributed to the participants, that will be encouraged to explore them and highlight interesting aspects of their models' performance, perform quantitative and qualitative error analysis, etc.

Theoretical and experimental papers related to the task datasets and simulation results are also invited.

Through collaborative preparatory work on the Word Space wiki and official homepage (http://wordspace.collocations.de/doku.php/esslli:start) and thanks to the ESSLLI multiple-day workshop format, we hope that this initiative will foster collaboration among the nascent community of researchers interested in computational semantics from a theoretical rather than applicative point of view.

Ongoing work on data-set preparation can be monitored at http://wordspace.collocations.de/doku.php/data:start

We ask for expressions of interest from researchers and teams that might take part in the initiative and might want to have a say in test set design.

For further information, please write to lexsem08@gmail.com.

• Late January, 2008: Data-sets available on Workshop website
• Mar 8, 2008: Paper submission deadline
• April, 2008: Notification
• August 4-9, 2008: Workshop in Hamburg (during the first week of ESSLLI)

Marco Baroni (University of Trento), WS co-organizer

Reinhard Blutner (University of Amsterdam)

Gemma Boleda (UPF, Barcelona)

Peter Bosch (University of Osnabrück)

Paul Buitelaar (DFKI, Saarbrücken)

John Bullinaria (University of Birmingham)

Katrin Erk (UT, Austin)

Stefan Evert (University of Osnabrück), WS co-organizer

Patrick Hanks (Masaryk University, Brno)

Anna Korhonen (Cambridge University)

Michiel van Lambalgen (University of Amsterdam)

Alessandro Lenci (University of Pisa), WS co-organizer

Claudia Maienborn (University of Tübingen)

Simonetta Montemagni (ILC-CNR, Pisa)

Rainer Osswald (University of Hagen)

Manfred Pinkal (University of Saarland)

Massimo Poesio (University of Trento)

Reinhard Rapp (University of Mainz)

Magnus Sahlgren (SICS, Kista)

Sabine Schulte im Walde (University of Stuttgart)

Manfred Stede (University of Potsdam)

Suzanne Stevenson (University of Toronto)

Peter Turney (NRC Canada, Ottawa)

Tim Van de Cruys (University of Groningen)

Gabriella Vigliocco (University College, London)

Chris Westbury (University of Alberta)