Registration
Participants are required to register in order to obtain the development and test data-sets.
Registration must be done by sending an email to the following address: tweet-norm AT elhuyar.com
Evaluation Period
We will enable the test set the the second week of July (8-15 of July). Each participant will only have 72 hours to work on the test set and to send results. The period to participate will run, by default, from the 10th of July (at 13h) to the 13th of July (at 13h, GMT+2:00). Alternatively participants can ask for another 72h window by email.
Submitting the Results
- Results must be submitted by email to tweet-norm AT elhuyar.com .
- The results shall have the following format (results with invalid format will not be taken into account):
TwitId1 \t OOV-word1 Correct-form1 \t OOV-word2 Correct-form2 TwitId2 \t OOV-word3 Correct-form1 ...
- For each OOV word only one correct form may be proposed.
- Each participant can submit up to two results.
Results
- Download: Results, annotated reference and the script used for evaluation.
- Note: a response period is open until Friday, 26th of July should someone ask for a revision of the results. After that deadline results will be definitive.
Rank | Participant | Best run accuracy | Second run accuracy |
---|---|---|---|
1 | RAE | 0,781 | |
2 | Citius-Imaxin | 0,663 | 0,662 |
3 | UPC | 0,653 | |
4 | Elhuyar | 0,636 | 0,634 |
5 | IXA-EHU_em-phonm | 0,619 | 0,609 |
6 | Vicomtech | 0,606 | |
7 | UniArizona | 0,604 | |
8 | UPF-Havas | 0,548 | 0,491 |
9 | DLSIAlicante | 0,545 | 0,521 |
10 | UniMelbourne | 0,539 | 0,517 |
11 | UniSevilla | 0,396 | |
12 | Ujaen-sinai | 0,376 | |
13 | UniCoruña | 0,335 |