BioNLP Open Shared Tasks 2019

A workshop of EMNLP-IJCNLP 2019

About

BioNLP Open Shared Tasks (BioNLP-OST) is organized to facilitate development and sharing of computational tasks of biomedical text mining (TM) and solutions to them. It is a continuation of the previous efforts organized around the BioNLP Shared Task (BioNLP-ST) workshop series (2009, 2011, 2013, 2016).

BioNLP-OST is organized as a reformulation of BioNLP-ST. The reformulation was motivated to make the efforts scientifically sounder and more sustainable. The change, which is represented by the newly added keyword “open” in the title, is summarized below:

  • The opportunity of offering tasks through BioNLP-OST is to be open.
    • Anyone (a task provider; TP) will be allowed to offer a task.
    • The goal of this setup is to observe a wider variety of needs for biomedical TM, and to reduce the bias caused by a small number of organizers.
    • The organizing committee of BioNLP-OST (OC) will provide technical support to potential task providers.
  • The development schedule is to be open.
    • The schedule of data release and solution submission will be managed by individual tasks.
    • The goal of this setup is to reduce the burden on TPs and also participants (or solution providers; SP) to follow a fixed schedule across multiple tasks. By allowing flexible scheduling of individual tasks, we hope the chance of task provision and also scientific investigation for solution provision to be maximized.
    • Instead of a development schedule, the publication schedule for workshop proceedings and journal special issue will motivate the participants to voluntarily organize their own schedule.

Since task registration to BioNLP-OST will be open, it will effectively open the scope of the tasks. While the focus will still be on information extraction (IE) tasks, we also expect relevant tasks to IE, which may include but not limited to the following tasks :

  • Named entity recognition, Normalization, Sense disambiguation
  • Relation extraction, Event extraction, Open IE, Graph representation
  • Sentiment analysis, Discourse analysis
  • Multilingualism

Schedule

  • Feb. 28: Shared task platform ready with initial tasks
  • Mar. 19: Platform open for registration of additional tasks and solutions
  • Aug. 19: Workshop paper submission due
  • Sep. 16: Acceptance notification
  • Sep. 26: Camera-ready submission due
  • Nov. 4, 2019: Workshop to be collocated with EMNLP-IJCNLP 2019 (Hong Kong)
  • Jul. 10, 2020: Journal paper submission due

All deadlines are AoE (Anywhere on Earth)

Mailing Lists

announcements@bionlp-ost.org

(join the mailing list)

Workshop

A Workshop on BioNLP Open Shared Tasks (BioNLP-OST) 2019 will be organized to present and discuss on the results of the shared tasks.

For the workshop, we will solicit original papers of three types:

  • A task paper will present a task. A typical paper of this type will include the task’s use in biomedical applications, its characteristics as a computational task, basic statistics on the training and evaluation data sets, the evaluation methods, the baseline model and its performance, and evaluation on solutions submitted for the task.
  • A solution paper will present a solution for one or more tasks. A typical paper of this type will include the hypotheses implemented in the solution, and their evaluation through the tasks it is registered for. An emphasis will be put on scientific advancement and reproducibility of solutions, separately from engineering aspects.
  • A meta paper will present the newly attempted framework itself. It will include original hypotheses, implementation, and discussions on observations, highlighting both successful and failed features of the framework.

All the submissions go through a single blind review process, where each submission will be reviewed by three program committee members. The reason for single blind review process instead of double blind process is that it is neither possible nor practical to hide the author identity in case of shared task papers. Accepted papers will be presented by the authors in the workshop session either as a talk or a poster, and will also be published in the workshop proceedings.

The International Workshop on BioNLP Open Shared Tasks (BioNLP-OST) 2019 will be collocated with EMNLP-IJCNLP 2019 either on 3rd or 4th of November, in Hong Kong.

Journal Special Issue

In previous experiences of shared task organization, it was observed that investigation is often continued after the workshop, based on new insights obtained through discussions during workshop. The workshop functions as a venue for the different types of contributors, mainly task providers and solution providers, to meet together and exchange their experiences and insights. As a venue to compile the results of follow-up investigation, a journal special issue will be organized to be published a few months after the workshop. The specific journal will be announced after negotiation with publishers.