Task 1a: Concrete Nouns Categorization

The goal of the sub-task is to group concrete nouns into semantic categories.

The data set consists of 44 concrete nouns, belonging to 7 semantic categories (four animates and two inanimates). The nouns are included in the feature norms described in McRae et al. (2005) (cf. Task3).

We operationalize concrete nouns categorization as a clustering task. Since the data set is organized hierarchically, we will run three clustering experiments, varying the number of classes and consequently their level of generality:

• 7-way clustering - models will be tested on their ability to categorize the nouns into the most fine-grained classes of the dataset: bird ("peacock"), groundAnimal ("lion"), fruitTree ("cherry"), green ("potato"), kitchenware ("spoon"), instrument ("hammer"), vehicle ("car");
• 4-way clustering - models will be tested on their ability to categorize the nouns into 4 superordinate classes: animal (superordinate of bird and groundAnimal), vegetable (superordinate of fruitTree and green), tool (superordinate of kitchenware and instrument), vehicle;

• 2-way clustering - models will be tested on their ability to categorize the nouns into the two top classes: natural (superordinate of animal and vegetable) and artifact (superordinate of tool and vehicle)

To abstract away from differences stemming from the particular clustering algorithm, you are asked to run your experiments with k-means algoriothm available in CLUTO. In case you can not use LUTO on your syste, you can provide us with your mdeols

Particpnats ore obvioulty free to experiemnt also with tor ownn favoiurteio model.

We envisage e two stage evaluation:

1. quantitative evaluation - clustering results wil be evaluated with repect to the two standrd meuare used in CLUTO: cluter purity and cluster entropy (for details see??).

2. qualitative evaluation - particpnats will be asked to focus on a fine-grained process of error analysis, to identify the hardeest nouns to cluster, etc.

Back to Start