Lexical Semantics Workshop (ESSLLI 2008)

Start page – Background – Programme and proceedings – Shared tasks – Call for papers

Corpus-based distributional models (such as LSA or HAL) have been claimed to capture interesting aspects of word meaning and provide an explanation for the rapid acquisition of semantic knowledge by human language learners. However, although these models have been proposed as plausible simulations of human semantic space organization, careful and extensive empirical tests of such claims are still lacking.

Systematic evaluations typically focus on large-scale quantitative tasks, often more oriented towards engineering applications (see, e.g., the recent SEMEVAL evaluation campaign) than towards the challenges posed by linguistic theory, philosophy and cognitive science. This has resulted in a great divide between corpus-driven computational approaches to semantics on the one hand and theory-driven symbolic approaches on the other – a situation that is characteristic of the linguistic and of most of the cognitive tradition. Moreover, whereas human lexical semantic competence is obviously multi-faceted – ranging from free association to taxonomic judgments to relational effects – tests of distributional models tend to focus on a single aspect (most typically the detection of semantic similarity), and few if any models have been tuned to tackle different facets of semantics in an integrated manner.

Our workshop purports to fill these gaps by inviting research teams and individual scholars to test their computational models on a variety of small but carefully designed tasks that aim to bring out linguistically and cognitively interesting aspects of semantics (see below for details). To this effect, annotated datasets are available to the participants, who are encouraged to explore them and highlight interesting aspects of their models' performance, conduct quantitative and qualitative error analysis, etc.

The focus is NOT on competition, but on understanding how different models highlight different semantic aspects, how far we are from an integrated model, and which aspects of semantics are beyond the reach of purely distributional approaches. In fact, we believe that at the current state of the art in computational and distributional semantics, our goal should not be to develop the best-performing model for a specific application, but rather to enlarge our understanding of the limits and potentialities of different approaches when confronted with cognitively realistic tasks.

In addition to these practical tasks, theoretical and experimental papers discussing the relation between distributional and symbolic approaches to meaning are also invited. We are particularly interested in papers that analyze our task data sets from a theoretical perspective or that discuss simulation results and their implications for semantic and cognitive theory.

Through collaborative preparatory work on the Word Space wiki (wordspace.collocations.de) and thanks to the ESSLLI multiple-day workshop format, we hope that this initiative will foster collaboration among the nascent community of researchers interested in computational semantics from a theoretical rather than engineering-oriented point of view.