Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
data:start [2008/03/30 13:05] schtepf |
data:start [2010/02/08 00:49] schtepf |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Data sets for the evaluation of word space models ====== | ====== Data sets for the evaluation of word space models ====== | ||
- | This page contains a developing list of tasks, sub-tasks and corresponding (sub-)data-sets. | ||
- | Other tasks or sub-tasks might be added in the near future. | + | ===== Ordered by events ===== |
+ | |||
+ | * [[: | ||
===== Ordered by task categories ===== | ===== Ordered by task categories ===== | ||
- | |||
- | ==== Task 1: Free Association ==== | ||
- | |||
- | It is tempting to make a connection between the **statistical association** patterns of words -- both first-order associations (// | ||
- | |||
- | |||
- | * [[Correlation with Free Association Norms]] | ||
- | |||
- | |||
- | **NB: On March 29th, we fixed a small (but serious) bug in script '' | ||
- | |||
- | ==== Task 2: Categorization ==== | ||
- | |||
- | Categorization tasks play a prominent role in cognitive research on concepts. In this type of tasks, subjects | ||
- | are typically asked to assign experimental items - objects, images, words - | ||
- | to a given category or to group together items belonging to the same category. | ||
- | Since categorization presupposes an understanding of the relationship between the items in a category, it is regarded as a key source of evidence on the organization and structure of the human conceptual system. | ||
- | |||
- | In the present task, computational models will be tested on their ability to properly group | ||
- | words into semantic categories. The task is organized into three sub-tasks, focussing on different areas | ||
- | of the lexicon and/or semantic dimensions: | ||
- | |||
- | * [[Concrete Noun Categorization]] | ||
- | * [[Abstract/ | ||
- | * [[Verb Categorization]] | ||
- | |||
+ | ==== Semantic classification ==== | ||
+ | * Noun & verb categorization (ESSLLI 2008) | ||
+ | * [[: | ||
+ | * [[: | ||
+ | * [[: | ||
+ | ==== Free association ==== | ||
+ | * [[: | ||
+ | * discrimination: | ||
+ | * correlation: | ||
+ | * prediction of most common responses (strongest associations) | ||
- | ==== Task 3: Property Generation ==== | ||
- | The ability to describe a concept in terms of its salient properties is an important feature of human conceptual cognition. In this task, we compare human-generated //norms// collected by psychologists to the properties generated by computational models. | + | ==== Property generation ==== |
* [[Comparison with Speaker-Generated Features]] | * [[Comparison with Speaker-Generated Features]] | ||
- | **NB: ON MARCH 7, WE MADE A SMALL CORRECTION TO THE PROPERTY EXPANSION FILE USED FOR THIS TASK; IF YOU DOWNLOADED THE RELEVANT ARCHIVE BEFORE THIS DATE, PLEASE DOWNLOAD IT AGAIN** | ||
- | ===== Source corpus ===== | ||
- | You can train your word space on your favorite corpus. However, we also invite you, if this is suitable, to experiment with the [[http:// |