Task 1a: Concrete Nouns Categorization

The goal of the sub-task is to group concrete nouns into semantic categories.

The data set consists of 44 concrete nouns, belonging to 7 semantic categories (four animates and two inanimates). All the nouns are included in the feature norms described in McRae et al. (2005) (cf. Task3).

We operationalize the concrete noun categorization as a clustering task. Since the data set is organized hierarchically, we perform different leves of clustering, to test the model with different number of classes, differenring also for their level of generality:

• 7-way clustering - the model will be tested with respet to its abilityt to cluters the data at the highest levels of granularity. The 7 classes are: bird (peacock), groundAnimal (lion), fruitTree (cherry), green (potato), kitchenware (spoon), instrument (hammer), vehicle (car);
• 4-way clustering - the model will be tested with respet to its abilityt to cluster the data in to 4 classes: animal (superordinate of bird and groundanimal), vegetable (superordinate of fruitTree and green), tool (superordinate of kitchenware and instrument), vehicle;

• 2-way clustering - the model will be tested with respet to its abilityt to cluster the data in to 2 classes: natural (superordinate of animale and vegentable) and artifact (superordinate of tool and vehichle)

To abtrsct away from differences depending on the particular clustring algoiothm, we ask participants to run their experiments with the imlementation of the k-means algoriothm available in CLUTO (with default parameters). In case you can not use LUTO on your syste, you can provide us with your mdeols

Particpnats ore obvioulty free to experiemnt also with tor ownn favoiurteio model.

We envisage e two stage evaluation:

1. quantitative evaluation - clustering results wil be evaluated with repect to the two standrd meuare used in CLUTO: cluter purity and cluster entropy (for details see??).

2. qualitative evaluation - particpnats will be asked to focus on a fine-grained process of error analysis, to identify the hardeest nouns to cluster, etc.

Back to Start