Transfer of Tool Affordance and Manipulation Cues with 3D Vision Data

Paulo Abelha, Frank Guerin

Research output: Working paper

3 Downloads (Pure)

Abstract

Future service robots working in human environments, such as kitchens, will face situations where they need to improvise. The usual tool for a given task might not be available and the robot will have to use some substitute tool. The robot needs to select an appropriate alternative tool from the candidates available, and also needs to know where to grasp it, how to orient it and what part to use as the end-effector. We present a system which takes as input a candidate tool's point cloud and weight, and outputs a score for how effective that tool is for a task, and how to use it. Our key novelty is in taking a task-driven approach, where the task exerts a top-down influence on how low level vision data is interpreted. This facilitates the type of 'everyday creativity' where an object such as a wine bottle could be used as a rolling pin, because the interpretation of the object is not fixed in advance, but rather results from the interaction between the bottom-up and top-down pressures at run-time. The top-down influence is implemented by transfer: prior knowledge of geometric features that make a tool good for a task is used to seek similar features in a candidate tool. The prior knowledge is learned by simulating Web models performing the tasks. We evaluate on a set of fifty household objects and five tasks. We compare our system with the closest one in the literature and show that we achieve significantly better results
Original languageEnglish
PublisherArXiv
Publication statusSubmitted - 13 Oct 2017

Fingerprint

Robots
Kitchens
Wine
Bottles
End effectors

Keywords

  • cs.RO

Cite this

Transfer of Tool Affordance and Manipulation Cues with 3D Vision Data. / Abelha, Paulo; Guerin, Frank.

ArXiv, 2017.

Research output: Working paper

@techreport{168e38a1bc7d4ca6be9460710b0676ef,
title = "Transfer of Tool Affordance and Manipulation Cues with 3D Vision Data",
abstract = "Future service robots working in human environments, such as kitchens, will face situations where they need to improvise. The usual tool for a given task might not be available and the robot will have to use some substitute tool. The robot needs to select an appropriate alternative tool from the candidates available, and also needs to know where to grasp it, how to orient it and what part to use as the end-effector. We present a system which takes as input a candidate tool's point cloud and weight, and outputs a score for how effective that tool is for a task, and how to use it. Our key novelty is in taking a task-driven approach, where the task exerts a top-down influence on how low level vision data is interpreted. This facilitates the type of 'everyday creativity' where an object such as a wine bottle could be used as a rolling pin, because the interpretation of the object is not fixed in advance, but rather results from the interaction between the bottom-up and top-down pressures at run-time. The top-down influence is implemented by transfer: prior knowledge of geometric features that make a tool good for a task is used to seek similar features in a candidate tool. The prior knowledge is learned by simulating Web models performing the tasks. We evaluate on a set of fifty household objects and five tasks. We compare our system with the closest one in the literature and show that we achieve significantly better results",
keywords = "cs.RO",
author = "Paulo Abelha and Frank Guerin",
note = "24 pages Acknowledgements: Thanks to the University of Aberdeens ABVenture Zone for equipment use. We got a lot of very helpful advice and assistance from the following people (the first four did actual coding work on the project): Nikola Petkov, Krasimir Georgiev, Benjamin Nougier, Severin Fichtl, Subramanian Ramamoorthy, Michael Beetz, Andrei Haidu, John Alexander, Markus Schoeler, Nicolas Pugeault, David Cruickshank, Michael Chung and Naveed Khan Funding: Paulo Abelha is on a PhD studentship supported by the Brazilian agency CAPES through the program Science without Borders. Frank Guerin received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors",
year = "2017",
month = "10",
day = "13",
language = "English",
publisher = "ArXiv",
type = "WorkingPaper",
institution = "ArXiv",

}

TY - UNPB

T1 - Transfer of Tool Affordance and Manipulation Cues with 3D Vision Data

AU - Abelha, Paulo

AU - Guerin, Frank

N1 - 24 pages Acknowledgements: Thanks to the University of Aberdeens ABVenture Zone for equipment use. We got a lot of very helpful advice and assistance from the following people (the first four did actual coding work on the project): Nikola Petkov, Krasimir Georgiev, Benjamin Nougier, Severin Fichtl, Subramanian Ramamoorthy, Michael Beetz, Andrei Haidu, John Alexander, Markus Schoeler, Nicolas Pugeault, David Cruickshank, Michael Chung and Naveed Khan Funding: Paulo Abelha is on a PhD studentship supported by the Brazilian agency CAPES through the program Science without Borders. Frank Guerin received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors

PY - 2017/10/13

Y1 - 2017/10/13

N2 - Future service robots working in human environments, such as kitchens, will face situations where they need to improvise. The usual tool for a given task might not be available and the robot will have to use some substitute tool. The robot needs to select an appropriate alternative tool from the candidates available, and also needs to know where to grasp it, how to orient it and what part to use as the end-effector. We present a system which takes as input a candidate tool's point cloud and weight, and outputs a score for how effective that tool is for a task, and how to use it. Our key novelty is in taking a task-driven approach, where the task exerts a top-down influence on how low level vision data is interpreted. This facilitates the type of 'everyday creativity' where an object such as a wine bottle could be used as a rolling pin, because the interpretation of the object is not fixed in advance, but rather results from the interaction between the bottom-up and top-down pressures at run-time. The top-down influence is implemented by transfer: prior knowledge of geometric features that make a tool good for a task is used to seek similar features in a candidate tool. The prior knowledge is learned by simulating Web models performing the tasks. We evaluate on a set of fifty household objects and five tasks. We compare our system with the closest one in the literature and show that we achieve significantly better results

AB - Future service robots working in human environments, such as kitchens, will face situations where they need to improvise. The usual tool for a given task might not be available and the robot will have to use some substitute tool. The robot needs to select an appropriate alternative tool from the candidates available, and also needs to know where to grasp it, how to orient it and what part to use as the end-effector. We present a system which takes as input a candidate tool's point cloud and weight, and outputs a score for how effective that tool is for a task, and how to use it. Our key novelty is in taking a task-driven approach, where the task exerts a top-down influence on how low level vision data is interpreted. This facilitates the type of 'everyday creativity' where an object such as a wine bottle could be used as a rolling pin, because the interpretation of the object is not fixed in advance, but rather results from the interaction between the bottom-up and top-down pressures at run-time. The top-down influence is implemented by transfer: prior knowledge of geometric features that make a tool good for a task is used to seek similar features in a candidate tool. The prior knowledge is learned by simulating Web models performing the tasks. We evaluate on a set of fifty household objects and five tasks. We compare our system with the closest one in the literature and show that we achieve significantly better results

KW - cs.RO

M3 - Working paper

BT - Transfer of Tool Affordance and Manipulation Cues with 3D Vision Data

PB - ArXiv

ER -