Directly to video-clips
Directly to papers
This page has not been updated since 2010. But newer papers are included below.
"Assembly in Motion" is the short form of the title of a project that began in 2005 to investigate assembly of parts to moved objects. The purpose of the project has been to enable robot arms to find their way into assembly lines, e.g. the final assembly in car production lines. Project partners have been: The Institute of Robotics and Mechatronics (DLR), the Institute for Machine Tools and Industrial Management (iwb, an institute of the Technical University of Munich), and the companies BMW, KUKA, QUISS, SCHUNK, and Sturm.
By now, since the first project is terminated, "assembly in motion" stands for control in spite of uncertainties that are caused by a conveyor. This work is funded by the KUKA Roboter GmbH. Assembly to moved objects is different with respect to usual robot programs since a normal teach-in procedure is not possible. In contrast, sensors have to be used to continuously sense the pose of the object. Regarding control it is not more sufficient to guarantee a certain pose accuracy, but the path accuracy has to be as high. In addition, synchronization with the conveyor is crucial.
While existing sensor applications predominantly determine a target pose at which the robot has to arrive, the project at hand combines sensor-based path specification and real-time control to realize an accurate motion of the robot end-effector along online sensed trajectories.
An assembly line in which wheels are mounted to a car body that is moved by a power & free conveyor has been chosen as the demonstration scenario. This set-up (Fig. 1) has been arranged in the application center Augsburg of iwb. The developed approach is general enough to be applied to other assembly tasks as well.
Fig. 1 Set-up at iwb for mounting of wheels to a continuously moved car body
Sensor-based trajectory planning
Assembly in motion cannot be implemented by simply perceiving the target motion and overlaying it to a robot motion that has been programmed for unmoved objects. The configuration has to be extended to allow the perception of objects whose pose is normally implicitly given by the program. This perception in particular has to include oscillations of the car body. As well, control errors of the robot have to be measured, which in the case of unmoved objects could be compensated by teaching slightly displaced poses. Furthermore, in industrial applications the requirements with respect to both accuracy and speed are usually higher than in service or space applications.
A set-up of three kinds of sensors has proved to be generally useful. First, a coarse sensor that informs the system whether the car body is present. Second, a non-contact sensor that surveys the car body. A preferred realization of this sensor is a CCD camera at the robot end-effector, since cameras are flexible sensors and this location leads to more accuracy and less occlusion. The output of the vision system will be used to let the robot end-effector approach the designated contact point. After the impact, the third sensor, a compliant force torque sensor, takes control. A compliance of at least one millimeter or degree is crucial, since conveyors are known to generate small scale oscillations, and also because the robot path accuracy is typically worse than the assembly tolerance.
Fig. 2 Compliant force torque sensor for high forces and torques and high deflection (Exo-Compliance)
Since compliant sensors of the required dimensions (1000N, 300Nm, 2mm) were not available, a small scale sensor has been strengthened by external springs and a second measuring unit, thus it fulfills the requirements (Fig. 2). By now this sensor is commercially available (SCHUNK). Fig. 3 shows a prototype of the resulting multi-sensory end-effector, which includes five power screw drivers as well.
Fig. 3 Multi-sensory end-effector (Courtesy iwb)
Force control is reconsidered with respect to position controlled robots and a compliant sensor (see also the archived page on sensor-based control). Special attention is given to the suppression of rotations that are caused by the asymmetrical arrangement of the sensor (paper).
Trajectory planning is implemented by a Kalman filter that weighs all sensor information with their appropriate accuracies and by a heuristic scheme that defines the transient behavior from the current to the sensed trajectory.
Since filtering is fundamental, pure extrapolation of the last changes will be error-prone. Instead, there are several models used for prediction and the most probable of them is selected in each time step. In this way it is possible to consider a conveyor stop or the restart of the conveyor in an optimal way (video clip, paper).
Predictive trajectory control
Control within assembly lines is different from normal robot control since, first, small control errors have to be guaranteed at significant speed, and second, not only the robot path has to be controlled but also its synchronization with respect to the conveyor. Finally the robot dynamics are more complex since a compliant and thus oscillatory end-effector is present.
Trajectory control is realized using model-based prediction of the sensed desired trajectory as input to a two-stage feed-forward filter (Fig. 4). The upper level is designed as an input shaping filter to suppress any oscillation of the compliant sensor, while the lower level is realized by an adaptive feed-forward controller that compensates for the robot dynamics.
Fig. 4 Signal flow for predictive trajectory control of the robot
This approach allows very small control errors in spite of the robot dynamics as long as the predictions of the motion of the wheel hub are correct. This is arranged by a model of the conveyor and the car oscillations and by a proper calibration of the camera that prevents a hard impact. Sample images of the robot mounted camera during the approach to the wheel hub are provided as a video clip. Note the adjustment of the orientation.
The original robotic set-up includes a redundant linear axis to move the robot in parallel to the conveyor (Fig. 1). For dynamical reasons the speed of this axis is not manipulated as long as the conveyor does not stop. In that case the robot completes assembly while its base is stopped.
Lately, the linear axis has been removed. This demonstrates that for many applications a more cost effective set-up may be used (video_clip, paper).
Task description and programming
Applications for sensor-based control in assembly lines require programming techniques different from the typical industrial robot languages. In addition to the definition of the nominal robot path, for sensor-based tasks the application program has to incorporate the desired sensor values as well. These values usually vary, e.g. during vision-controlled motion towards the tracked object. Therefore, within this project both, the robot trajectory and the object trajectory are used for the online computation of the expected sensor values. In particular, the object motion (the car trajectory) is firstly extracted from the robot path, assuming a conveyor-like motion with ideal contact in the contact phase. In this way, the existing robot languages can still be used with small extensions, e.g. to specify the active sensors or phases.
Video clips of the experiment are available.
Back to the homepage of Friedrich Lange
A more complete list of publications can be found here when giving the author's name (Lange, F.).
F. Lange, W. Bertleff, and M. Suppa, "Force and Trajectory Control of Industrial Robots in Stiff Contact", Proc. 2013 IEEE Int. Conf. on Robotics and Automation (ICRA), Karlsruhe, Germany, May 2013
Position-based force control is presented, incorporating compliance in the robot joints and possibly in a force- / torque-sensor and/or the environment. First, the total compliance is identified. Then, in the control phase, the desired pose of the tool center point is computed from the force control error. Thus standard position control may be applied. This leads to an inherently stable control scheme, even with a low sampling rate of the sensor interface and unknown environmental compliance. The method is designed for applications of industrial robots, e.g. assembly tasks. Parallel control considers the existence of a reference trajectory which allows feedforward in force controlled directions. The paper further examines couplings between forces and torques, which are important for partially constrained configurations. A possible impact force is considered when colliding with an unexpected object.
F. Lange, M. Suppa, and G. Hirzinger, "Control with a Compliant Force-Torque Sensor", Proc. 7th German Conf. on Robotics ROBOTIK 2012, Munich, May 2012, pp. 490-495
There are assembly tasks which require a compliant device at the end-effector since possible disturbances are beyond the bandwidth of robot control. This paper discusses a compliant force-torque sensor for assembly. Two aspects are explained in detail: Force control considering a significant force dependent displacement, and control of an end-effector with an elastic mounting during fast unconstrained motion. The latter uses an adaptive scheme which serves as a further level in a hierarchical position-based control. Experimental results are given which show the limits of industrial robots.
F. Lange, C. Jehle, M. Suppa, and G. Hirzinger, "Revised Force Control Using a Compliant Sensor with a Position Controlled Robot", Proc. 2012 IEEE Int. Conf. on Robotics and Automation (ICRA), St. Paul, MN, USA, May 2012, pp. 1532-1537
paper related video clip
A different way of force control is presented, that is especially advantageous for position controlled robots. Instead of usual force control laws we rely on the well tuned position control loop and just use the force sensor to measure the target pose or to predict the desired trajectory. In combination with a compliant sensor we introduce an inherently stable framework of force control which almost inhibits all control errors. After an unexpected impact the force error is reduced independently from the sensor’s bandwidth or delays in signal processing. Thus the (inevitable) impact force is more significant than the measured force control errors. The special case of a sensor that is mounted far away from a vertex-face contact is discussed, too.
F. Lange, B. Willberg, and G. Hirzinger, "Control of Large Forces and Torques Using an Asymmetrically Arranged Compliant Sensor", Proc. Joint 41th Int. Symp. on Robotics and 6th German Conf. on Robotics ISR/ROBOTIK 2010, Munich, June 2010
paper related video clips
Control of forces and torques seemed to be solved. But the usual method for position controlled robots is not optimal when using a slow interface. In addition, there are differences between control with a compliant sensor and with a stiff sensor, even more if the sensor is distant from the contact point and with respect to the vector of the desired force. The method proposed in this paper further considers a point-surface contact, from which the sensor cannot infer on rotation. We also consider the automatic detection of this case. A further obstacle to force control with compliant sensors is the fact that only small scale sensors are available. Therefore a method for the scaling of existing sensors and an example of its use are presented.
F. Lange, J. Scharrer, and G. Hirzinger, "Is a Linear Axis Really Required When Assembling Parts to a Moving Conveyor", Proc. Joint 41th Int. Symp. on Robotics and 6th German Conf. on Robotics ISR/ROBOTIK 2010, Munich, June 2010
paper related video clips
Assembly tasks in flow assembly lines are usually executed using a robot that is mounted on a linear axis. In this way the robot is moved in parallel to the conveyor, such that control is almost as with a fixed object. Control without such a linear axis is more demanding as the conveyor motion has to be overlaid to the task-specific motion of the robot arm. This requires high path accuracy instead of a high pose accuracy. In addition further kinematical restrictions apply, and a more difficult fault management has to be considered.
F. Lange, J. Scharrer, and G. Hirzinger, "Classification and Prediction for Accurate Sensor-Based Assembly to Moving Objects", Proc. 2010 IEEE Int. Conf. on Robotics and Automation (ICRA), Anchorage, AK, USA, May 2010
paper related video clips
Typical industrial assembly tasks require an accuracy that cannot be realized by only feedback control if a minimum speed is given by a conveyor. Feed-forward has proven to be advantageous, using predictions of the desired trajectory which will be computed from sensor values. These predictions are improved by a model based classification of the sensor data to typical scenarios. In contrast to linear controllers this assures the fastest possible response to external disturbances, in spite of large dynamical delays. The method is demonstrated by assembling wheels to a car body that is moved by a conveyor, fusing sensor data using an extended Kalman filter.
F. Lange, J. Werner, J. Scharrer, and G. Hirzinger, "Assembling Wheels to Continuously Conveyed Car Bodies Using a Standard Industrial Robot", Proc. 2010 IEEE Int. Conf. on Robotics and Automation (ICRA), Anchorage, AK, USA, May 2010
paper related video clips
Within assembly lines, wheel assembly to continuously conveyed car bodies is still executed by human workers using a device that compensates the weight of the wheel. This paper presents a solution in which a robot autonomously assembles and fixes the wheels. The approach uses a sensor-driven control strategy that compensates a possible temporal or spatial offset. Three types of sensors are proposed for adequate perception of the wheel hub. Their signals are fused by a Kalman filter that allows predictions in the time domain. Finally, a feed-forward controller is used, that is designed to consider the predictions in order to minimize dynamical delays. The control is driven by a special task description that extents usual robot programming methods.
B. Stumpp, "Montage in Bewegung", Automobil-Produktion, Ausgabe 04/ April 2009, Seiten 38-39
In Zukunft soll auch der Roboter am bewegten Fließband montieren können. In der ENDMONTAGE der Automobilproduktion dominiert jedoch noch die manuelle Arbeit. Wie in den anderen Produktionsphasen soll auch hier die kostengünstige Arbeitskraft des Roboters besser genutzt werden.
F. Lange und J. Werner, "Montage am laufenden Band", Computer & Automation, 9-2008, Seiten 74-77
Die Montage robotergeführter Anbauteile erfolgt heute in der Regel am ruhenden Basisbauteil. Neue Sensor- und Regelungskonzepte schaffen die Grundlage dafür, Komponenten künftig vollständig synchron zur Fließbandbewegung montieren zu können - und zwar in allen sechs Freiheitsgraden.
A. Kamel, F. Lange, and G. Hirzinger, "An Industrial-Robots Suited Input Shaping Control Scheme", The 9th Int. Conf. on Motion and Vibration Control (MOVIC), Munich, Germany, Sept. 2008
Compliance in robot mounted force/torque sensors is useful for soft mating of parts in many assembly tasks. Nevertheless, it generates nearly undamped oscillations when moving a heavy end-effector in free space. In this paper, input shaping control is investigated to damp such unwanted flexible modes. However, the conventional method presents a major drawback: To eliminate the oscillatory dynamics, the desired motion profiles have to be shaped and thus modified. This means that although the unwanted vibrations are damped, the robot motion does not meet the desired one. In this paper we first review the conventional input shaping technique. Second we show how the mentioned problem may be fixed in the design phase by discretizing the filter and by using a predictive approach that compensates the shaped signals time delay and minimizes the resulting control error. Simulation results are presented.
F. Lange, K. H. Strobl, J. Langwald, S. Jörg, G. Hirzinger, B. Gruber, J. Klein, J. Werner, "Kameragestützte Montage von Rädern an kontinuierlich bewegte Fahrzeuge", Fachtagung Robotik 2008, München, Juni 2008
paper (10 pages) digest (4 pages) related video clips
Betrachtet wird die Montage von Rädern an ein durch ein Förderband transportiertes Fahrzeug. Dazu nimmt ein Industrieroboter Rad und Schrauben auf und führt sie zum Fügen an das Fahrzeug, dessen Radnabe während der Bewegung in allen 6 Freiheitsgraden durch eine robotergeführte Kamera vermessen wird. Die Kamera ist mit einer Ringleuchte ausgestattet und hinter dem Achsloch des gehaltenen Rades angeordnet, das dadurch auch bei schwingender Karosse robust an die Radnabe herangeführt und kraftgeregelt verschraubt wird.
F. Lange, B. Willberg, P. Meusel, J. Langwald, S. Jörg, M. Frommberger, G. Hirzinger, J. Werner, "Montage am kontinuierlich bewegten Band - Sensorkonfiguration und Programmierung", Fachtagung Robotik 2008, München, Juni 2008
paper (10 pages) digest (4 pages) related video clips
Durch Industrieroboter ausgeführte Montageaufgaben im Bereich von Förderbändern erfordern die laufende Erfassung der Lage und der Bewegung des Basisbauteils. Dazu werden sowohl berührungslos messende als auch nachgiebige Sensoren für den Kontaktfall benötigt. Für große Kräfte oder Momente erfordert dies einen besonderen Aufbau. Die Programmierung erfolgt am stehenden Band. Dabei werden automatisch Objekte definiert, deren Relativlage zum Roboter bei der späteren Programmausführung durch die Sensorik geregelt werden soll.
F. Lange, A. Kamel, G. Hirzinger, J. Werner, "Sensorfusion und Regelung eines Roboters am kontinuierlich bewegten Band",
Kongress Automation 2008, Baden-Baden, Juni 2008
paper (10 pages) digest (4 pages) related video clips
Der Artikel beschreibt die Montage eines Anbauteils an ein kontinuierlich bewegtes Basisbauteil, dessen Bewegung durch unterschiedliche Sensoren gemessen wird. Dabei werden u. a. die Ergebnisse einer Bildverarbeitung mit den Auslenkungen eines nachgiebigen Kraft- Momentensensors fusioniert. Aus den Bahnpunkten wird dann eine geglättete Solltrajektorie für das Anbauteil bestimmt. Die Regelung des Roboters wird durch Vorsteuerungen ergänzt, die die dynamischen Verzögerungen und Schwingungen von Roboter und Endeffektor prädiktiv kompensieren.
A. Kamel, F. Lange, and G. Hirzinger, "New Aspects of Input Shaping Control to Damp Oscillations of a Compliant Force Sensor", Proc. 2008 IEEE Int. Conf. on Robotics and Automation (ICRA), Pasadena, CA, USA, May 2008, pp. 2629-2635
Compliance in robot mounted force/torque sensors is useful for soft mating of parts. However it generates nearly undamped oscillations when moving the end-effector in free space. In this paper, input shaping control is investigated to damp such unwanted flexible modes. We present a new design technique that creates long impulse sequences to adapt input shaping to systems with long sampling period and to compensate the resulting time delay. This makes the method feasible for industrial robots. In addition to the conventional input shaping which causes oscillations to stop only after applying the last impulse, we also minimize the quadratic control error until this time step is reached.