NTCIR Actionable Knowledge Graph (AKG) Task

Formats

The format for input/output files currently defined by AKG is as follows.

All run files must be submitted to ntcir-akg [at] googlegroups [dot] com via email with the subject: Dry/Formal run submission for AM/AKGG subtask.


Action Mininig subtak (Formal Run)

Format for inputs
  • Elements
    • Query ID
    • Entity
    • Entity Type(s)
    • Wikipedia URL
  • Input (query) file is formatted in the following way.
{
  <!-- Information about this task //-->
  "ntcirtask": "NTCIR-13 Actionable Knowledge Graph (AKG)",
  "subtask": "Action Mining",
  "runtype": "Formal Run",
  <!-- Query data //-->
  "queries": [
    <!-- First query starts //-->
    {
      <!-- Information about 1st query //-->
      "queryid": "ID of this query",
      "entity": "Entity of this query",
      "entitytypes": [
        "Entity types of this query"
      ],
      "entityurl": "URL to a Wikipedia page of this entity",
    }, <!-- First query ends //-->
    <!-- Second query starts //-->
    {
      ...
    },
    ...
  ]
}

Format for outputs (runs)
  • Elements
    • Group ID
    • Run ID
    • System description
    • Query ID
    • Actions
      • Rank
      • Verb
      • Object
  • Participants are required to submit run data using the following format.
    • File name of your run file must be am-formalrun-[groupid].json
{
  <!-- Information about this task //-->
  "ntcirtask": "NTCIR-13 Actionable Knowledge Graph (AKG)",
  "subtask": "Action Mining",
  "runtype": "Formal Run",
  <!-- Information about your group //-->
  "groupid": "ID of your group as registered at NTCIR-13",
  <!-- Run data //-->
  "runs": [
    <!-- First run starts //-->
    {
      <!-- Information about 1st run //-->
      "runid": "ID of this run (string)",
      "systemdescription": "System description for this run",
      <!-- Results of 1st run //-->
      "results": {
        <!-- Results of 1st query //-->
        {
          "queryid": "ID of queries as defined in our input file",
          "actions": [
            <!-- 1st action for this query //-->
            {
              "rank": "Rank position of this action (1-100)"
              "verb": "Verb of this action",
              "object": "Object of this action (null or max 50 chars)"
            },
            <!-- 2nd action for this query //-->
            {
              ...
            },
            ...
          ]
        },
        <!-- Results of 2nd query //-->
        {
          ...
        },
        ...
      }
    }, <!-- First run ends //-->
    <!-- Second run starts //-->
    {
      ...
    },
    ...
  ]
}

Actionable Knowledge Graph Generation subtask (Formal Run)

Format for inputs
  • Elements
    • Query ID
    • Query
    • Entity
    • Entity Type(s)
    • action
  • Input (query) file is formatted in the following way.
    • {
  <!-- Information about this task //-->
  "ntcirtask": "NTCIR-13 Actionable Knowledge Graph (AKG)",
  "subtask": "Actionable Knowledge Graph Generation",
  "runtype": "Formal Run",
  <!-- Query data //-->
  "queries": [
    <!-- First query starts //-->
    {
      <!-- Information about 1st query //-->
      "queryid": "ID of this query",
      "query": "query string",
      "entity": "Entity of this query",
      "entitytypes": [
        "Entity types of this query"
      ],
      "action": "action string",
    }, <!-- First query ends //-->
    <!-- Second query starts //-->
    {
      ...
    },
    ...
  ]
}

    Format for outputs (runs)
    • Elements
      • Group ID
      • Run ID
      • System description
      • Query ID
      • Properties
        • Rank
        • Property
    • Participants are required to submit run data using the following format.
      • File name of your run file must be akgg-formalrun-[groupid].json
      {
  <!-- Information about this task //-->
  "ntcirtask": "NTCIR-13 Actionable Knowledge Graph (AKG)",
  "subtask": "Actionable Knowledge Graph Generation",
  "runtype": "Formal Run",
  <!-- Information about your group //-->
  "groupid": "ID of your group as registered at NTCIR-13",
  <!-- Run data //-->
  "runs": [
    <!-- First run starts //-->
    {
      <!-- Information about 1st run //-->
      "runid": "ID of this run (string)",
      "systemdescription": "System description for this run",
      <!-- Results of 1st run //-->
      "results": {
        <!-- Results of 1st query //-->
        {
          "queryid": "ID of queries as defined in our input file",
          "properties": [
            <!-- 1st property for this query //-->
            {
              "rank": "Rank position of this action (1-100)"
              "property": "property string",
            },
            <!-- 2nd property for this query //-->
            {
              ...
            },
            ...
          ]
        },
        <!-- Results of 2nd query //-->
        {
          ...
        },
        ...
      }
    }, <!-- First run ends //-->
    <!-- Second run starts //-->
    {
      ...
    },
    ...
  ]
}