Table of Contents

Courses and Tutorials on DSM

Course materials – ESSLLI 2009 – NAACL-HLT 2010 – ESSLLI '16 & '18 – ESSLLI '21 & '22 – U Fribourg 2026/27 – Software & data sets – Bibliography

Software for the course

Practical examples and exercises for these courses and tutorials are based on the user-friendly software package wordspace for the interactive statistical computing environment R. If you want to follow along, please bring your own laptop and set up the required software as follows:

  1. Install up-to-date versions of R (4.6 or newer) and the RStudio GUI
  2. Use the installer built into RStudio (or the standard R GUI) to install the following packages from the CRAN archive:
    • sparsesvd (v0.2-3)
    • wordspace (v0.2-9)
    • recommended: e1071, rsparse, Rtsne, uwot
    • optional: tm, quanteda, data.table, wordcloud, shiny, spacyr, udpipe, coreNLP (don't worry if some of these fail to install)
  3. During the course, you will be asked to install a further package with additional evaluation tasks (wordspaceEval) from a password-protected Web page:
    • wordspaceEval v0.3: Source/Linux – MacOS – Windows (login required)
      • on an older version of R, you may need to fall back to package version 0.2: Source/Linux – MacOS – Windows (login required)
      • if you are stuck with R v3.x, please use package version 0.1: Source/Linux – MacOS – Windows (login required)
    • download a suitable version and select “Install from: Package Archive File” in RStudio
  4. Download the sample data files listed below
  5. Download one or more of the pre-compiled DSMs listed below

Scaling R to large data sets

Most of our hands-on examples work reasonably well in a standard R installation, even on a moderately powerful laptop computer. However, if you intend to work on real-life tasks and process large DSMs, it is important to enable multi-threaded computation in R. Since DSMs build on matrix operations, a multi-threaded linear algebra library (“BLAS”) is key.

  1. In Linux, it should be sufficient to install the OpenBLAS package, e.g. in Ubuntu: sudo apt install libopenblas-dev
  2. In MacOS, you can enable the VecLib BLAS built into MacOS by typing the following commands in a Terminal window 
    cd /Library/Frameworks/R.framework/Libraries
ln -sf libRblas.vecLib.dylib libRblas.dylib
  3. In Windows, you can try installing following these instructions.

In MacOS, you may also want to enable OpenMP for an additional speed boost on expensive distance metrics (but this is less important and will require you to compile the package from source code).

Example data sets

Pre-compiled DSMs

Pre-compiled DSMs for use with the wordspace package for R. Each model is contained in an .rda file, which can be loaded into R with the command load("model.rda") and creates an object with the same name (model).

DSMs based on the English Wikipedia

These models were compiled from WP500, a 200-million word subset of the Wackypedia corpus comprising the first 500 words of each article. Each model covers a vocabulary of the 50,000 most frequent content words (lemmatized) in the corpus and has at least 50,000 feature dimensions. The latent SVD dimensions are based on log-transformed sparse simple-ll scores with L2-normalization. Power scaling with Caron $P = 0$ (i.e. equalization of the latent dimensions) has been applied, but the reduced vectors are not re-normalized.

Neural word embeddings

Some publicly available pre-trained neural embeddings, converted into .rda format for use with the wordspace package.

Web interfaces