The test data has now been released on the CogALex website:
 
http://pageperso.lif.univ-mrs.fr/~michael.zock/CogALex-IV/cogalex-webpage/pst.html
 
It contains 2000 sets of five stimulus words each.
 
Please compute an associative response for each set and, by the deadline (May 9, 2014) send us by e-mail to reinhardrapp (at) gmx.de and with copy to Michael.Zock (at) lif.univ-mrs.fr your list of 2000 responses (in a text file with one response word per line).
 
We will compare your 2000 responses with our gold standard responses and compute an accuracy.
 
Hereby, as allowing alternative responses would require too much subjective judgement, only exact string matches are taken as correct, with the exception that word capitalization is not considered. This conservative evaluation procedure will lead to (compared to other shared tasks) low accuracy values. 
 
Please don't be discouraged if in the training phase your system could solve only a small fraction of the training items correctly. This is to be expected for this difficult task: e.g. in the literature for related tasks often accuracies of less than 5% were reported.
 
In addition to your results, please also provide the following information:
 
- Your name(s)
 
- e-mail-address(es)
 
- Affiliation(s)
 
- Name of your system (if longer than 20 characters, please also provide a shorter form)
 
- What track you are participating in: 
  1) Unrestricted systems: They can use any type of data to compute the results.
  2) Restricted systems: They can only draw on the ukWaC corpus.
  3) Both (in this case we need two lists of responses).