Participation

Registration

Participants are required to register in order to obtain the development and test data-sets.

Registration must be done by sending an email to the following address on or before June 6th: tweetlid AT elhuyar.com

Evaluation Period

We will enable the test set the the first week of July (1-7 of July). Each participant will only have 72 hours to work on the test set and to send results. The period to participate will run, by default, from the 1st of July (12pm CEST) to the 4th of July (12pm CEST). Alternatively participants can apply for another 72h window by email.

Submitting the Results

  • Results must be submitted by email to tweetlid AT elhuyar.com.
  • The results shall have the following format (results with invalid format will not be taken into account):

    TwitId1 \t lang
    TwitId2 \t lang
    ...

    where ‘lang’ can take one of the following forms:

    • ‘lang1′: single language. Possible values are: [es,en,gl,ca,eu,pt,und,other]
    • ‘lang1+lang2[+lang3]‘: multiple languages. Any combination of the abovementioned codes are allowed.

    IMPORTANT:

    • ‘lang1/lang2/lang3′ is not accepted as an answer. If such notation is found only the first language will be taken into account.
    • When using multiple languages, (‘lang1+lang2[+lang3]‘) a maximum number of 3 languages may be included. If more are found only the first 3 languages will be taken into account.

Results

The following two tables include the rankings of the results submitted by participants for the Constrained and Unconstrained tracks.

Constrained

# Team P R F1
1 UPV / ELiRF (2) 0.825 0.744 0.752
2 UPV / ELiRF (1) 0.824 0.730 0.745
3 UB / UPC / URV 0.777 0.719 0.736
4 Citiusimaxin (1) 0.824 0.685 0.726
5 RAE (2) 0.813 0.648 0.711
6 RAE (1) 0.818 0.645 0.710
7 Citiusimaxin (2) 0.689 0.772 0.710
8 CERPAMID (1) 0.716 0.681 0.666
9 UDC / LYS Group (1) 0.732 0.734 0.638
10 IIT-BHU 0.605 0.670 0.615
11 CERPAMID (2) 0.704 0.578 0.605
12 UDC / LYS Group (2) 0.610 0.582 0.498

Unconstrained

# Team P R F1
1 Citiusimaxin (1) 0.802 0.748 0.753
2 UPV / ELiRF (2) 0.737 0.723 0.697
3 UPV / ELiRF (1) 0.742 0.686 0.684
4 Citiusimaxin (2) 0.696 0.659 0.655
5 UDC / LYS Group (1) 0.682 0.688 0.581
6 UB / UPC / URV 0.598 0.625 0.578
7 UDC / LYS Group (2) 0.588 0.590 0.571
8 CERPAMID (1) 0.694 0.461 0.506
9 CERPAMID (2) 0.583 0.537 0.501