Tool use is currently not part of DLU as, at the time of writing, this requires additional modeling in SOMA and Streaming Construction Grammar (SCG).īack and other concepts requiring keeping track of a history are out of scope for DLU right now.ĭLU does not support actions which manipulate the state of the actor.ĭirections demanding avoidance of actions or indirectly describe actions are also not covered.ĭLU also only handles directions in isolation.
This pipeline employs the ontological Socio-physical Model of Activities (SOMA), which serves not only to define the interfaces of the multi-component pipeline but also to connect numeric data and simulations with symbolic reasoning processes.Įxample directions that DLU does not handle and corresponding explanations The research question in this paper is therefore the following: how can ontological knowledge be used to extract and evaluate parameters from a natural language direction in order to simulate it formally? The solution proposed in this work is the Deep Language Understanding (DLU) processing pipeline. In order to turn an underspecified text, issued or taken from the web, into a detailed robotic action plan, various processing steps are necessary, some based on symbolic reasoning and some on numeric simulations or data sets.
In comparison, artificial agents lack the same depth of symbol grounding between linguistic cues and real world objects, as well as the capacity for insight and prospection to reason about instructions in relation to the real world and the changing states of that world. Humans excel despite many uncertain variables existing in the environment: for example, the cups might all have been put away in a cupboard or the path to the table could be blocked by chairs. Still, asking a human to, for instance, “take the cup to the table” will typically result in a satisfactory outcome. Often, instructions omit vital semantic components such as determiners, quantities, or even the objects they refer to. From the perspective of the robot, these instructions tend to be vague and imprecise as natural language generally employs ambiguous, abstract, and non-verbal cues. For everyday activities, instructions could be given either verbally from a human or through written texts such as recipes and procedures found in online repositories, e.g., from wikiHow. Today, this is mostly done via programming languages or pre-defined user interfaces, but this changes rapidly. Most commonly, any robot activity starts with the robot receiving directions or commands for that specific activity. However, the development of robotic agents that can perform different tasks of increasing complexity is slowly changing this state of affairs, creating new opportunities in the domain of household robotics. Performing household activities such as cooking and cleaning have, until recently, been the exclusive provenance of human participants. This allows for a unified and efficient knowledge retrieval across all pipeline components, providing flexibility and reasoning capabilities as symbolic knowledge is combined with annotated sub-symbolic models. The major advantage of employing an overarching ontological framework is that its asserted facts can be stored alongside the semantics of directions, contextual knowledge, and annotated activity models in one central knowledge base. As a last step, the pipeline simulates the given natural language direction inside a virtual environment. Several reasoning steps formulate simulation plans, in which robot actions are guided by data gathered using human computation. The pipeline includes a natural language parser and a module for natural language grounding.
It uses the ontological Socio-physical Model of Activities (SOMA) as a common interface between its components. In this paper, a processing pipeline to tackle these steps for natural language directions is proposed and implemented. Going from natural language directions to fully specified executable plans for household robots involves a challenging variety of reasoning steps.
0 kommentar(er)
