Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
data:concrete_nouns_categorization [2008/01/19 17:38] alexlenci |
— (current) | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Task 1a: Concrete Nouns Categorization ====== | ||
| - | |||
| - | ==== Introduction ==== | ||
| - | |||
| - | The goal of the sub-task is to group concrete nouns into semantic categories. | ||
| - | |||
| - | The {{concnouns.categorization.dataset.tar.gz |data set}} consists of 44 concrete nouns, belonging to 7 semantic categories (four animates and two inanimates). All the nouns are included in the feature norms described in McRae et al. (2005) (cf. [[comparison_with_speaker-generated_features|Task3]]). | ||
| - | |||
| - | |||
| - | ==== Task Operationalization ==== | ||
| - | |||
| - | We operationalize the concrete noun categorization as a clustering task. Since the data set is organized hierarchically, | ||
| - | we perform different leves of clustering, to test the model with different number of classes, differenring also for their level of generality: | ||
| - | |||
| - | * 7-way clustering - the model will be tested with respet to its abilityt to cluters the data at the highest levels of granularity. The 7 classes are: bird (peacock), groundAnimal (lion), fruitTree (cherry), green (potato), kitchenware (spoon), instrument (hammer), vehicle (car); | ||
| - | * 4-way clustering - the model will be tested with respet to its abilityt to cluster the data in to 4 classes: animal (superordinate of bird and groundanimal), | ||
| - | |||
| - | * 2-way clustering - the model will be tested with respet to its abilityt to cluster the data in to 2 classes: natural (superordinate of animale and vegentable) and artifact (superordinate of tool and vehichle) | ||
| - | |||
| - | To abtrsct away from differences depending on the particular clustring algoiothm, we ask participants to run their experiments with the imlementation of the k-means algoriothm available in CLUTO (with default parameters). In case you can not use LUTO on your syste, you can provide us with your mdeols | ||
| - | |||
| - | Particpnats ore obvioulty free to experiemnt also with tor ownn favoiurteio model. | ||
| - | |||
| - | |||
| - | ==== Task Evaluation ==== | ||
| - | |||
| - | We envisage e two stage evaluation: | ||
| - | |||
| - | 1. quantitative evaluation - clustering results wil be evaluated with repect to the two standrd meuare used in CLUTO: cluter purity and cluster entropy (for details see?? | ||
| - | |||
| - | 2. qualitative evaluation - particpnats will be asked to focus on a fine-grained process of error analysis, to identify the hardeest nouns to cluster, etc. | ||
| - | |||
| - | |||
| - | Back to [[Start]] | ||
