Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
data:start [2008/01/17 22:24] marco |
data:start [2010/02/08 00:50] schtepf |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Data sets for the evaluation of word space models ====== | ====== Data sets for the evaluation of word space models ====== | ||
- | This page contains a developing list of tasks, sub-tasks and corresponding (sub-)data-sets. | ||
- | Other tasks or sub-tasks might be added in the near future. | + | ===== Ordered by events ===== |
+ | |||
+ | * [[: | ||
Line 9: | Line 10: | ||
+ | ==== Semantic classification ==== | ||
+ | * Noun & verb categorization (ESSLLI 2008) | ||
+ | * [[: | ||
+ | * [[: | ||
+ | * [[: | ||
+ | ==== Free association ==== | ||
- | ==== Task 1: Categorization ==== | + | |
- | + | * discrimination: | |
- | Categorization tasks play a prominent role in cognitive research on concepts. In this type of tasks, subjects | + | * correlation: regression modelling of association strength |
- | are typically asked to assign experimental items - objects, images, words - | + | * prediction |
- | to a given category or to group together items belonging to the same category. | + | |
- | Since categorization presupposes an understanding of the relationship between the items in a category, it is regarded as a key source of evidence on the organization and structure of the human conceptual system. | + | |
- | + | ||
- | In the present task, computational models will be tested on their ability to properly group | + | |
- | words into semantic categories. The task is organized into three sub-tasks, focussing on different areas | + | |
- | of the lexicon and/or semantic dimensions: | + | |
- | + | ||
- | | + | |
- | * [[Abstract/ | + | |
- | * [[Verb Categorization]] | + | |
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | ==== Task 2: Free Association ==== | + | |
- | + | ||
- | It is tempting to make a connection between the **statistical association** patterns | + | |
- | + | ||
- | + | ||
- | * [[Correlation with Free Association Norms]] | + | |
- | + | ||
- | ==== Task 3: Property Generation ==== | + | |
- | The ability to describe a concept in terms of its salient properties is an important feature of human conceptual cognition. In this task, we compare human-generated //norms// collected by psychologists to the properties generated by computational models. | ||
- | * [[Comparison with Speaker-Generated Features]] (**preliminary gold standard and evaluation script available!**) | + | ==== Property generation ==== |
+ | * [[: | ||
- | ===== Source corpus ===== | ||
- | While participants are welcome to use models they trained on their corpora, we also invite them, whenever this it suitable, to use the [[http:// | ||