Enable a robot to infer the type of joint between moving clusters of 3d features. This can further be used to build a kinematic structure of an observed object.
The interest in robots that are able to act in unstructured environments is increasing.
Allow Robots to grasp unknown objects while perceive the environment and the objects with a 3D sensors.
Interactively explore and grasp real-world objects using visual feedback.
In this thesis, we study the topic of Lifelong Robotic Object Perception. We propose, as a long-term goal, a framework to recognize known objects and to discover unknown objects in the environment as the robot operates, for as long as the robot operates. We build the foundations for Lifelong Robotic Object Perception by focusing our study on the two critical components of this framework: 1) how to recognize and register known objects for robotic manipulation, and 2) how to automatically discover novel objects in the environment so that we can recognize them in the future.
The thesis describes a novel approach to recognizing objects in RGB-D images and for making this information persistent in a 3D semantic map. It makes two major contributions: Firstly, he proposes a novel approach to object-class segmentation in RGB-D images based on random forest classifiers that provides a pixel-wise class labeling in the image.
Every planning problem in robotics involves constraints. Whether the robot must avoid collision or joint limits, there are always states that are not permissible.Some constraints are straightforward to satisfy while others can be so stringent that feasible states are very difficult to find. What makes planning with constraints challenging is that, for many constraints, it is impossible or impractical to provide the planning algorithm with the allowed states explicitly; it must discover these states as it plans. The goal of this thesis is to develop a framework for representing and exploring feasible states in the context of manipulation planning.
Information on the developed system can also be found at http://robolabwiki.sdu.dk/mediawiki/index.php/Brick_laying_mobile_manipulator
The paper proposes a method to improve flexibility of the motion planning process for mobile manipulators. The approach is based on the exploitation of perception data available only from simple proximity sensors distributed on the robot. Such data are used to correct pre-planned motions to cope with uncertainties and dynamic changes of the scene at execution time. The algorithm computes robot motion commands aimed at fulfilling the mission by combining two tasks at the same time, i.e. following the planned end-effector path and avoiding obstacles in the environment, by exploiting robot redundancy as well as handling priorities among tasks. Moreover, a technique to smoothly switch between the tasks is presented. To show the effectiveness of the method, four experimental case studies have been presented consisting in a place task executed by a mobile manipulator in an increasingly cluttered scene.
Grasping individual objects from an unordered pile in a box has been investigated in static scenarios so far. In this paper, we demonstrate bin picking with an anthropomorphic mobile robot. To this end, we extend global navigation techniques by precise local alignment with a transport box. Objects are detected in range images using a shape primitive-based approach. Our approach learns object models from single scans and employs active perception to cope with severe occlusions. Grasps and arm motions are planned in an efficient local multi-resolution height map. All components are integrated and evaluated in a bin picking and part delivery task.
This paper presents the concept “autonomous industrial mobile manipulation” (AIMM) based on the mobile manipulator “Little Helper” – an ongoing research project at Aalborg University, Denmark, concerning the development of an autonomous and flexible manufacturing assistant.
Abstract: Roboticists are working towards the realization of autonomous mobile manipulators that can perform useful tasks in human environments. These environments pose a significant challenge because of their complexity and inherent uncertainty. They are characterized by having a high dimensional state space. Consequently, performing tasks in these unstructured environments remains a challenge.
Abstract: Robust robotic manipulation and perception remains a difficult challenge, in particular in unstructured environments. To address this challenge, we propose to couple manipulation and perception. The robot observes its own deliberate interactions with the world.
Abstract: We introduce a learning-based approach to manip- ulation in unstructured environments. This approach permits au- tonomous acquisition of manipulation expertise from interactions with the environment. The resulting expertise enables a robot to perform effective manipulation based on partial state information.
Abstract: Previously, we have presented an implementation of impedance control inspired by the Equilibrium Point Hypothesis that we refer to as equilibrium point control (EPC). We have demonstrated that EPC can enable a robot in a fixed position to robustly pull open a variety of doors and drawers, and infer their kinematics without detailed prior models.
Abstract:We describe the architecture, algorithms, and experiments with HERB, an autonomous mobile manipulator that performs useful manipulation tasks in the home.
Robot manipulation tasks for a mobile, personal robot will often be importantly distinct from those of a traditional, factory robot arm; correspondingly, appropriate motion planning solutions may be notably different, as well. This paper introduces novel definitions of and solution methods for optimal kinodynamic planning for mobile robot manipulation applications.
IEEE International Conference on Robotics and Automation, 2009
Abstract - Dynamical systems can generate movement trajectories that are robust against perturbations. This article presents an improved modification of the original dynamic movement primitive (DMP) framework by Ijspeert et al.
In this paper we present a framework for guiding autonomous learning in robot systems. The paradigm we introduce allows a robot to acquire new skills according to an intrinsic motivation function that finds behavioral affordances.
This work addresses the inverse kinematics problem for the 7 Degrees of Freedom Barrett Whole Arm Manipulator with link offsets. The presence of link offsets gives rise to the possibility of the in-elbow & out-elbow poses for a given end-effector pose and is discussed in the paper.
Abstract: We propose a set of control strategies for performing two arm manipulation with the goal of simplifying the task definition. In order to develop these strategies we propose a new representation, derived from the cooperative task-space, in the dual quaternion domain.
At the moment autonomous industrial mobile manipulation (AIMM) is a subject of major focus in development and research environments, as it is a technology with significant potential. However, we are experiencing a lack in calibration techniques, in order to obtain industrially acceptable localization and manipulation tolerances.
Abstract – In this paper we propose a novel approach for interactive manipulation involving a human and a humanoid. The interaction is represented by means of the relative configuration between the human’s and the robot’s hands.
Purpose – The purpose of this paper is to provide a review of the interdisciplinary research field, autonomous industrial mobile manipulation (AIMM),with an emphasis on physical implementations and applications.
Video collage of Technical Committee on Mobile Manipulation (TCMM) member research compiled by Dejan Pangercic. Thanks to all the members that contributed videos!
New visual/tactile control for manipulation contributed by Qiang Li from CITEC Bielefeld:
(1) tactile servoing(related paper published in RSS2013)
(2) unknown object manipulation in-hand(related paper will be published in the forthcoming in IROS2013) :
Autonomous Intelligent Systems
The Autonomous Robotic Collaboration (ARC) Lab
Robotics and Biology Laboratory
Robotics Innovation Center (RIC)
Robotics Laboratory of Ecole Polytechnique
Winning team NimbRo of the RoboCup 2012@Home competition in Mexico City.
The video shows scenes from the 2012 RoboCup@Home competition, which took place in Mexico City. The cognitive service robots Dynamaid and Cosero of team NimbRo (University of Bonn, Germany) perform several tasks in a household environment, including serving drinks and watering a plant.
Winning team NimbRo of the RoboCup 2011@Home competition in Istanbul.
The video shows some scenes of the 2011 RoboCup@Home competition from the winning team NimbRo (University of Bonn, Germany). The robots Cosero and Dynamaid recognize persons, find and fetch objects, clean-up the place, help carrying a table, and finally make an omelet.
Mobile Bin Picking with Cognitive Service Robot Cosero
The video shows the results of the ECHORD experiments ActReMa - Active Recognition and Manipulation of Simple Parts Exploiting 3D Information, which has been carried out by University of Bonn and Metronom Automation GmbH. Our robot Cosero recognizes parts in a transport box, grasps them, and delivers them. We learn object models from examples and actively explore the contents of the box.
Worcester News Tonight Clip of Archie, WPI's PR2 Robot
Archie arriving at WPI and overview of the research we will conduct
AUV
{aviremote}http://csl-ep.mech.ntua.gr/projects/MMC/UAV/Videos/handling%20auv.avi{/aviremote}
In this animation the motion of an AUV is presented. When it spots a particular item for handling, it stops above it and a manipulator grabs it
Turning an underactuated system such as a AUV is not a trivial task. In order to follow a specific circular arc, the vehicle must turn in such a way that its longitudinal axis has been rotated in such way to compensate for the effects of water drag.
MOBILE
The mobile robot tries to reach a predetermined point avoiding a static obstacle.
The mobile robot tries to reach a predetermined point avoiding a dynamic obstacle.
This video presents 2 lego robots. The first searches for a particular area (red circle) and upon finding it, it calls a second robot to reach the particular area.
MICRO
A needle attached on the mobile microrobot moves towards the target under a videomicroscope. A visual servoing algorithm controls the vibration micrormotors, and consequently the motion of the mobile microrobots.
{mp4remote}http://csl-ep.mech.ntua.gr/projects/MMC/Micro/Videos/autonomous_microrobot.mp4{/mp4remote}
The mobile microrobot executes a translational and then a rotational motion. The microrobot is fully autonomous powered by a pair of batteries.
SPACE
{movremote}http://csl-ep.mech.ntua.gr/projects/MMC/Space/videos/general.mov{/movremote}
This video presents all the motions that the robotic emulator can perform. First the use of thrusters for a linear motion is presented. In the next clip the rotation around its center of mass using thrusters is presented while in the third clip the same motion using the reaction wheel is shown. Finally the reaction of the base upon arms motion, due to angula momentum is shown.
Do What You Do
A look back at the Robotics Institute's Manipulation Lab set to the groovy tune "Do What You Do" by Eldridge Gravy & The Court Supreme. This video was created in a special collaboration with the band and the MLab. Thanks for the Gravy!
Push-Grasping with Dexterous Hands
We add to a manipulator’s capabilities a new primitive motion which we term a push-grasp. While significant progress has been made in robotic grasping of objects and geometric path planning for manipulation, such work treats the world and the object being grasped as immovable, often declaring failure when simple motions of the object could produce success. We analyze the mechanics of push-grasping and present a quasi-static tool that can be used both for analysis and simulation. We utilize this analysis to derive a fast, feasible motion planning algorithm that produces stable push-grasp plans for dexterous hands in the presence of object pose uncertainty and high clutter. We demonstrate our algorithm on HERB.
Rearrangement Planning using Pushing Actions
Human environments are cluttered and robots regularly need to solve rearrangement problems by moving certain objects out of the way to reach other objects. We developed an algorithm to rearrange clutter using a library of actions including pushing. The planner can move objects that are not movable by pick-and-place actions, e.g. large or heavy objects.
Efficient Touch Based Localization through Submodularity
We address the handling of uncertainty by finding a sequence of information gathering actions prior to attempting a task. Finding the optimal sequence, which takes the minimum amount of time while providing sufficient information, is generally intractable (e.g. through a POMDP). Instead, we formulate the problem as one of submodular maximization, allowing us to select actions greedily while guaranteeing near-optimality.
Motion Generation for Mobile Manipulators in Unpredictable Environments
This video demonstrates a system for the execution of end-effector tasks while moving a mobile manipulator in dynamically changing environments.
Soft Hand
This video shows an uncut scene of the RBO Hand grasping different objects from a preset spot.
Interactive Segmentation of Articulated Objects in 3D
This video shows a robust perceptual skill for identifying, tracking, and segmenting objects in unstructured scenes.
http://robotik.dfki-bremen.de/en/media/videos/video/detail/Videoplayer/aila-iss-2-1.html
AILA performing an autonomous mission in ISS (Internation Space Station) setup.
http://robotik.dfki-bremen.de/en/media/videos/video/detail/Videoplayer/aila-robofoot-1.html
AILA learning arm motion by operator demonstration.
"http://robotik.dfki-bremen.de/en/media/videos/video/detail/Videoplayer/impera-exploration.html"
A team of two robots demonstrate the abilities to localize and to transport objects.
This video shows several closing sequences on different objects of an industrial size underactuated finger printed with a rapid prototyping machine.
(all our software can be found at http://personalrobotics.ri.cmu.edu/software.php)
Open Wam Driver (OWD): http://personalrobotics.ri.cmu.edu/pr-ros-pkg/owd/html/index.html
Driver for controlling a Barrett WAM and Barrett Hand from ROS
A real-time Object Recognition and Pose Estimation system for camera-based perception of known objects.
Constrained Manipulation Planning Suite (CoMPS), used for planning manipulation motions for robots.
Arm Map-Based Hierarchical Planner (ArmMBHP), an alternate planner for manipulation.
A planner that takes advantage of object-robot and object-object collisions during manipulation.
Software includes The Constrained Manipulation Suite (CoMPS) for motion planning with constraints and LightningROS, a ROS package implementing the Lightning Path Planning Framework. Lightning uses a path library to store previous experience while allowing generality by also planning from scratch.
This source package contains an implementation of Multi-Resolution Surfel Maps for image registration, object modelling and tracking, and SLAM.
The approach is documented in the papers:
Jörg Stückler and Sven Behnke:
Model Learning and Real-Time Tracking using Multi-Resolution Surfel Maps
In Proc. of the AAAI Conference on Artificial Intelligence (AAAI-12), Toronto, Canada, July 2012.
http://www.ais.uni-bonn.de/papers/aaai12_stueckler_behnke.pdf
Jörg Stückler and Sven Behnke:
Integrating Depth and Color Cues for Dense Multi-Resolution Scene Mapping Using RGB-D Cameras.
In Proceedings of the IEEE International Conference on Multisensor Fusion and Information Integration (MFI), Hamburg, Germany, September 2012.
http://www.ais.uni-bonn.de/papers/MFI_2012_SLAM.pdf
The RBO interactive perception project (RBO-IAP) is concerned with the detection and interactive exploration of kinematic objects, i.e. objects with degrees of freedom.
This experiment investigates a hyper-flexible cells scenario. In particular, we consider the following setting: A robot delivers parts to a process station. The parts are made available to the workspace of the robot in transport boxes that are not required to be packed in a systematic way. With its onboard 3D laser scanning sensor, the robot recognizes and identifies the topmost objects in the boxes. It grasps the parts out of the box and moves them to the station for further processing. The parts can be shaped nearly arbitrarily. We only assume that parts can be approximated by geometric shape primitives like e.g. cylinders or spheres. A new part type can be learned in a matter of minutes simply by presenting an exemplar to the robot.
The major challenges arising in this setting which will be addressed in the experiment are:
www.ais.uni-bonn.de/nimbro/@Home">http://www.ais.uni-bonn.de/nimbro/@Home
DFG Research Group "Learning Humanoid Robots"
funded within the Emmy Noether Programme
The cognitive service robots Dynamaid and Cosero have been developed for tasks in domestic environments.
They won the RoboCup 2011 and 2012 competitions.
Overview of projects in the ARM lab.
Selection of projects relative to mobile manipulation (as of End 2012):
Autonomous Servicing of On-Orbit Space Systems from Robotic Systems (NSRF)
Identification and Assessment of Existing Terrestrial Micro-systems and Micro-technologies for Space Robotics (ESA)
Development of a Flexible and Reliable Automated Warehouse System
Modeling and Control of Microrobotic Systems
MiCRoN: MIniature Co-operative RObots advancing towards the Nano-range
Autonomous Inspection of Subsea Telecommunication Cables, Power Cables and Pipelines (EU)
We are developing a grasp testbed for studying the process of grasp acquisition in human and robotic systems. The system is built around our Barrett WAM/Hand system on which we have implemented a hand-frame impedance controller, a Natural Point body tracking system, and multibody dynamic simulation capabiltiy. It will be further enhanced with the results of a recent NRI grant focused on Bayesian filtering for grasp perception.
Our bimanual mobile manipulation platform, with two Barrett WAM arms atop a completely redesigned chassis with a Segway RMP 200 base.
Projects pertaining to mobile manipulation
Currently funded projects:
Human-Robot Interaction for Assistive Applications (HRIAA), funded by ANR Chair d'Excellence Control Software Achitecture for Robust Robot Manipulation (Carroman), funded by ADT INRIA
The main goal of the project BesMan is the development of one- and two-arm manipulation procedures as well as the learning of new situation-specific behaviors by means of a machine learning platform.
The goal of the research project IMPERA is the development of strategies for distributed mission and task planning. An application example is the exploration of unknown, lunar environment using a team of mobile robots