In this thesis, we study the topic of Lifelong Robotic Object Perception. We propose, as a long-term goal, a framework to recognize known objects and to discover unknown objects in the environment as the robot operates, for as long as the robot operates. We build the foundations for Lifelong Robotic Object Perception by focusing our study on the two critical components of this framework: 1) how to recognize and register known objects for robotic manipulation, and 2) how to automatically discover novel objects in the environment so that we can recognize them in the future.
Semantic Mapping Using Object-Class Segmentation of RGB-D Images
The thesis describes a novel approach to recognizing objects in RGB-D images and for making this information persistent in a 3D semantic map. It makes two major contributions: Firstly, he proposes a novel approach to object-class segmentation in RGB-D images based on random forest classifiers that provides a pixel-wise class labeling in the image.
Every planning problem in robotics involves constraints. Whether the robot must avoid collision or joint limits, there are always states that are not permissible.Some constraints are straightforward to satisfy while others can be so stringent that feasible states are very difficult to find. What makes planning with constraints challenging is that, for many constraints, it is impossible or impractical to provide the planning algorithm with the allowed states explicitly; it must discover these states as it plans. The goal of this thesis is to develop a framework for representing and exploring feasible states in the context of manipulation planning.
Performing Masonry with a Mobile Manipulator
By Emil Pedersen and Oluf Skov Nielsen | June 2010
The paper proposes a method to improve flexibility of the motion planning process for mobile manipulators. The approach is based on the exploitation of perception data available only from simple proximity sensors distributed on the robot. Such data are used to correct pre-planned motions to cope with uncertainties and dynamic changes of the scene at execution time. The algorithm computes robot motion commands aimed at fulfilling the mission by combining two tasks at the same time, i.e. following the planned end-effector path and avoiding obstacles in the environment, by exploiting robot redundancy as well as handling priorities among tasks. Moreover, a technique to smoothly switch between the tasks is presented. To show the effectiveness of the method, four experimental case studies have been presented consisting in a place task executed by a mobile manipulator in an increasingly cluttered scene.
Mobile Bin Picking with an Anthropomorphic Service Robot
By Matthias Nieuwenhuisen, David Droeschel, Dirk Holz, Jörg Stückler,Alexander Berner, Jun Li, Reinhard Klein, and Sven Behnke
Grasping individual objects from an unordered pile in a box has been investigated in static scenarios so far. In this paper, we demonstrate bin picking with an anthropomorphic mobile robot. To this end, we extend global navigation techniques by precise local alignment with a transport box. Objects are detected in range images using a shape primitive-based approach. Our approach learns object models from single scans and employs active perception to cope with severe occlusions. Grasps and arm motions are planned in an efficient local multi-resolution height map. All components are integrated and evaluated in a bin picking and part delivery task.
An Autonomous Industrial Mobile Manipulator Concept
This paper presents the concept “autonomous industrial mobile manipulation” (AIMM) based on the mobile manipulator “Little Helper” – an ongoing research project at Aalborg University, Denmark, concerning the development of an autonomous and flexible manufacturing assistant.
Abstract: Assistive mobile robots that autonomously manipulate objects within everyday settings have the potential to improve the lives of the elderly, injured, and disabled. Within this paper, we present the most recent version of the assistive mobile manipulator EL-E with a focus on the subsystem that enables the robot to retrieve objects from and deliver objects to flat surfaces.
Abstract: Roboticists are working towards the realization of autonomous mobile manipulators that can perform useful tasks in human environments. These environments pose a significant challenge because of their complexity and inherent uncertainty. They are characterized by having a high dimensional state space. Consequently, performing tasks in these unstructured environments remains a challenge.
Abstract: Robust robotic manipulation and perception remains a difficult challenge, in particular in unstructured environments. To address this challenge, we propose to couple manipulation and perception. The robot observes its own deliberate interactions with the world.
Abstract: We introduce a learning-based approach to manip- ulation in unstructured environments. This approach permits au- tonomous acquisition of manipulation expertise from interactions with the environment. The resulting expertise enables a robot to perform effective manipulation based on partial state information.
Abstract: Previously, we have presented an implementation of impedance control inspired by the Equilibrium Point Hypothesis that we refer to as equilibrium point control (EPC). We have demonstrated that EPC can enable a robot in a fixed position to robustly pull open a variety of doors and drawers, and infer their kinematics without detailed prior models.
Published in the Proceedings of th IEEE International Conference on Robotics and Automation (ICRA 2010), May 2010. Finalist for Best Manipulation Paper Award.
Abstract: Studies of human manipulation strategies suggest that pre-grasp object manipulation, such as rotation or sliding of the object to be grasped, can improve task performance by increasing both the task success rate and the quality of load-supporting postures.
Robot manipulation tasks for a mobile, personal robot will often be importantly distinct from those of a traditional, factory robot arm; correspondingly, appropriate motion planning solutions may be notably different, as well. This paper introduces novel definitions of and solution methods for optimal kinodynamic planning for mobile robot manipulation applications.
IEEE International Conference on Robotics and Automation, 2009
Abstract - Dynamical systems can generate movement trajectories that are robust against perturbations. This article presents an improved modification of the original dynamic movement primitive (DMP) framework by Ijspeert et al.
In this paper we present a framework for guiding autonomous learning in robot systems. The paradigm we introduce allows a robot to acquire new skills according to an intrinsic motivation function that finds behavioral affordances.
This work addresses the inverse kinematics problem for the 7 Degrees of Freedom Barrett Whole Arm Manipulator with link offsets. The presence of link offsets gives rise to the possibility of the in-elbow & out-elbow poses for a given end-effector pose and is discussed in the paper.
Abstract: We propose a set of control strategies for performing two arm manipulation with the goal of simplifying the task definition. In order to develop these strategies we propose a new representation, derived from the cooperative task-space, in the dual quaternion domain.
At the moment autonomous industrial mobile manipulation (AIMM) is a subject of major focus in development and research environments, as it is a technology with significant potential. However, we are experiencing a lack in calibration techniques, in order to obtain industrially acceptable localization and manipulation tolerances.
Abstract – In this paper we propose a novel approach for interactive manipulation involving a human and a humanoid. The interaction is represented by means of the relative conﬁguration between the human’s and the robot’s hands.
Purpose – The purpose of this paper is to provide a review of the interdisciplinary research field, autonomous industrial mobile manipulation (AIMM),with an emphasis on physical implementations and applications.
Winning team NimbRo of the RoboCup 2012@Home competition in Mexico City.
The video shows scenes from the 2012 RoboCup@Home competition, which took place in Mexico City. The cognitive service robots Dynamaid and Cosero of team NimbRo (University of Bonn, Germany) perform several tasks in a household environment, including serving drinks and watering a plant.
Winning team NimbRo of the RoboCup 2011@Home competition in Istanbul.
The video shows some scenes of the 2011 RoboCup@Home competition from the winning team NimbRo (University of Bonn, Germany). The robots Cosero and Dynamaid recognize persons, find and fetch objects, clean-up the place, help carrying a table, and finally make an omelet.
Mobile Bin Picking with Cognitive Service Robot Cosero
The video shows the results of the ECHORD experiments ActReMa - Active Recognition and Manipulation of Simple Parts Exploiting 3D Information, which has been carried out by University of Bonn and Metronom Automation GmbH. Our robot Cosero recognizes parts in a transport box, grasps them, and delivers them. We learn object models from examples and actively explore the contents of the box.
In this animation the motion of an AUV is presented. When it spots a particular item for handling, it stops above it and a manipulator grabs it
Turning an underactuated system such as a AUV is not a trivial task. In order to follow a specific circular arc, the vehicle must turn in such a way that its longitudinal axis has been rotated in such way to compensate for the effects of water drag.
The mobile robot tries to reach a predetermined point avoiding a static obstacle.
The mobile robot tries to reach a predetermined point avoiding a dynamic obstacle.
This video presents 2 lego robots. The first searches for a particular area (red circle) and upon finding it, it calls a second robot to reach the particular area.
A needle attached on the mobile microrobot moves towards the target under a videomicroscope. A visual servoing algorithm controls the vibration micrormotors, and consequently the motion of the mobile microrobots.
This video presents all the motions that the robotic emulator can perform. First the use of thrusters for a linear motion is presented. In the next clip the rotation around its center of mass using thrusters is presented while in the third clip the same motion using the reaction wheel is shown. Finally the reaction of the base upon arms motion, due to angula momentum is shown.
A look back at the Robotics Institute's Manipulation Lab set to the groovy tune "Do What You Do" by Eldridge Gravy & The Court Supreme. This video was created in a special collaboration with the band and the MLab. Thanks for the Gravy!
We add to a manipulator’s capabilities a new primitive motion which we term a push-grasp. While significant progress has been made in robotic grasping of objects and geometric path planning for manipulation, such work treats the world and the object being grasped as immovable, often declaring failure when simple motions of the object could produce success. We analyze the mechanics of push-grasping and present a quasi-static tool that can be used both for analysis and simulation. We utilize this analysis to derive a fast, feasible motion planning algorithm that produces stable push-grasp plans for dexterous hands in the presence of object pose uncertainty and high clutter. We demonstrate our algorithm on HERB.
Rearrangement Planning using Pushing Actions
Human environments are cluttered and robots regularly need to solve rearrangement problems by moving certain objects out of the way to reach other objects. We developed an algorithm to rearrange clutter using a library of actions including pushing. The planner can move objects that are not movable by pick-and-place actions, e.g. large or heavy objects.
Efficient Touch Based Localization through Submodularity
We address the handling of uncertainty by finding a sequence of information gathering actions prior to attempting a task. Finding the optimal sequence, which takes the minimum amount of time while providing sufficient information, is generally intractable (e.g. through a POMDP). Instead, we formulate the problem as one of submodular maximization, allowing us to select actions greedily while guaranteeing near-optimality.
Robotics and Biology Laboratory (RBO)
Motion Generation for Mobile Manipulators in Unpredictable Environments
This video demonstrates a system for the execution of end-effector tasks while moving a mobile manipulator in dynamically changing environments.
This video shows an uncut scene of the RBO Hand grasping different objects from a preset spot.
Interactive Segmentation of Articulated Objects in 3D
This video shows a robust perceptual skill for identifying, tracking, and segmenting objects in unstructured scenes.
Software includes The Constrained Manipulation Suite (CoMPS) for motion planning with constraints and LightningROS, a ROS package implementing the Lightning Path Planning Framework. Lightning uses a path library to store previous experience while allowing generality by also planning from scratch.
Jörg Stückler and Sven Behnke: Integrating Depth and Color Cues for Dense Multi-Resolution Scene Mapping Using RGB-D Cameras. In Proceedings of the IEEE International Conference on Multisensor Fusion and Information Integration (MFI), Hamburg, Germany, September 2012. http://www.ais.uni-bonn.de/papers/MFI_2012_SLAM.pdf
This experiment investigates a hyper-flexible cells scenario. In particular, we consider the following setting: A robot delivers parts to a process station. The parts are made available to the workspace of the robot in transport boxes that are not required to be packed in a systematic way. With its onboard 3D laser scanning sensor, the robot recognizes and identifies the topmost objects in the boxes. It grasps the parts out of the box and moves them to the station for further processing. The parts can be shaped nearly arbitrarily. We only assume that parts can be approximated by geometric shape primitives like e.g. cylinders or spheres. A new part type can be learned in a matter of minutes simply by presenting an exemplar to the robot.
The major challenges arising in this setting which will be addressed in the experiment are:
Robust and efficient recognition of the composition of objects from measurements of a 3D laser scanning device.
Efficient exploration of the part arrangement in the transport boxes to handle occlusions.
Flexible grasp and motion planning for a robot under real-time constraints in a semi-structured environment, i.e. when the arrangement of parts and transport boxes is variable.
We are developing a grasp testbed for studying the process of grasp acquisition in human and robotic systems. The system is built around our Barrett WAM/Hand system on which we have implemented a hand-frame impedance controller, a Natural Point body tracking system, and multibody dynamic simulation capabiltiy. It will be further enhanced with the results of a recent NRI grant focused on Bayesian filtering for grasp perception.
The goal of the research project IMPERA is the development of strategies for distributed mission and task planning. An application example is the exploration of unknown, lunar environment using a team of mobile robots