Abstract
This paper presents a text-based three-step approach to identifying the main action structure in multimodal instructions. As a
worked example we explore the textual elements in 16 first-aid instructions describing the Heimlich manoeuvre. First, the text is split into clauses, then action clauses and clauses with control information concerning those actions are identified, and finally
the actions are categorised with the aim to derive the main action structure for the Heimlich procedure. We discuss methodological considerations and preliminary results for each of these steps. In future research, we will extend this approach to the visual information in multimodal instructions and to other first-aid procedures.
Ultimately, this will allow an integrated analysis of the textual and visual elements in multimodal instructions as well as an
abstraction and comparison of actions in different first-aid procedures.
worked example we explore the textual elements in 16 first-aid instructions describing the Heimlich manoeuvre. First, the text is split into clauses, then action clauses and clauses with control information concerning those actions are identified, and finally
the actions are categorised with the aim to derive the main action structure for the Heimlich procedure. We discuss methodological considerations and preliminary results for each of these steps. In future research, we will extend this approach to the visual information in multimodal instructions and to other first-aid procedures.
Ultimately, this will allow an integrated analysis of the textual and visual elements in multimodal instructions as well as an
abstraction and comparison of actions in different first-aid procedures.
| Original language | English |
|---|---|
| Number of pages | 10 |
| Publication status | Published - 9-Aug-2022 |
| Event | AREA II : Workshop on the Annotation, Recognition and Evaluation of Actions held in conjunction with the 33rd European Summer School in Logic, Language and Information 8-19 August, 2022 - National University of Ireland , Galway, Ireland Duration: 9-Aug-2022 → 11-Aug-2022 https://www.areaworkshop.org/ |
Workshop
| Workshop | AREA II : Workshop on the Annotation, Recognition and Evaluation of Actions held in conjunction with the 33rd European Summer School in Logic, Language and Information 8-19 August, 2022 |
|---|---|
| Abbreviated title | AREA |
| Country/Territory | Ireland |
| City | Galway |
| Period | 09/08/2022 → 11/08/2022 |
| Internet address |
Keywords
- Multimodal instructions
- Action structures
- Annotation model
- Pictures and text