Annotation Workflow

The Ellogon Annotation Platform offers two deployment channels:

  1. On-line, Cloud Services: This usage type does not involve any form of installation process; users can start using the Ellogon Annotation Platform immediately.

  2. Off-line, On-Premise: In this case, users must download and install the platform on a server of their choice, using technologies such as containers.

Beyond the installation process, which is required only for the second deployment channel (2.), the following steps are typical in an annotation workflow, as supported by the Ellogon Annotation Platform:

  1. Create Accounts: The annotation team (including the annotators and the process managers) must register with the platform.

  2. Define the Annotation Schema: A suitable (for the task) annotation schema (from existing available schemas) must be selected. Alternatively, a new annotation schema must be defined (in XML), if task requirements exceed available schemas capabilities.

  3. Import Data: The data to be annotated must be imported into the platform. Any annotation team member can import data, and can select the rest of the team members that data can be shared.

  4. Annotate Data: Each annotation team member annotates the data assigned for annotation. Annotation can be collaborative (multiple annotators work simultaneously on data) or stand alone (each annotator creates its own annotation set over data).

  5. Review Annotation: Annotations from multiple annotators can be visualised, and compared to identify differences (and even correct inconsistencies to retrieve a “gold” annotation). Standard metrics (like inter-rater agreement) can be automatically calculated.

  6. Export Annotated Data, for further sharing/processing outside the platform.