Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
data:concrete_nouns_categorization [2008/01/19 18:15]
alexlenci 		— (current)
Line 1: Line 1:
-====== Task 1a: Concrete Nouns Categorization ====== 
- 
-==== Introduction ==== 
- 
-The goal of the sub-task is to group concrete nouns into semantic categories. 
- 
-The {{concnouns.categorization.dataset.tar.gz |data set}} consists of 44 concrete nouns, belonging to 7 semantic categories (four animates and two inanimates). The nouns are included in the feature norms described in McRae et al. (2005) (cf. [[comparison_with_speaker-generated_features|Task3]]). 
- 
- 
-==== Task Operationalization ==== 
- 
-We operationalize concrete nouns categorization as a clustering task. Since the data set is organized hierarchically, 
-we will run three clustering experiments, varying the number of classes and consequently their level of generality: 
- 
-  * **7-way clustering** - models will be tested on their ability to categorize the nouns into the most fine-grained classes of the dataset: //bird// ("peacock"), //groundAnimal// ("lion"), //fruitTree// ("cherry"), //green// ("potato"), //kitchenware// ("spoon"), //instrument// ("hammer"), //vehicle// ("car"); 
-  * **4-way clustering** - models will be tested on their ability to categorize the nouns into 4 superordinate classes: //animal// (superordinate of //bird// and //groundAnimal//), //vegetable// (superordinate of //fruitTree// and //green//), //tool// (superordinate of //kitchenware// and //instrument//), //vehicle//; 
- 
-  * **2-way clustering** - models will be tested on their ability to categorize the nouns into the two top classes: //natural// (superordinate of //animal// and //vegetable//) and //artifact// (superordinate of //tool// and //vehicle//) 
- 
-To abstract away from differences stemming from the particular clustering method, you are asked to run your experiments with the //k-means// algorithm available in [[http://glaros.dtc.umn.edu/gkhome/cluto/cluto/overview|CLUTO]]. In case you can not run  [[http://glaros.dtc.umn.edu/gkhome/cluto/cluto/overview|CLUTO]] on your system, the workshop organizers will carry out the clustering experimenst for you, provided you will be able to sedn your data ina a format that will be speciofied later on. Participants are obviouslty free to experiment also with other clustering methotds. Comparisons with the results obtained with [[http://glaros.dtc.umn.edu/gkhome/cluto/cluto/overview|CLUTO]] are also welcome. 
- 
- 
-==== Task Evaluation ==== 
- 
-Evaluation will be carried in two stages: 
- 
-1. quantitative evaluation - results wil be evaluated with respect to the two measures for cluster quality available in CLUTO: //purity// and //entropy// (cf. Zhao, Y. and G. Karypis (2002), "Evaluation of Hierarchical Clustering Algorithms for Document Datasets", in //CIKM 2002//).  
- 
-2. qualitative evaluation - particpnats will be asked to focus on a fine-grained process of error analysis, to identify the hardeest nouns to cluster, etc.  
- 
- 
-Back to [[Start]] 
  

You are here: start » data » concrete_nouns_categorization