Dynamic movement primitives in robotics: A tutorial survey

The classical discrete DMPs, first introduced by Ijspeert et al. (2002c), encapsulate training data into a linear, second-order dynamics (a mass–spring–damper system) with an additive, nonlinear forcing term learned from a single demonstration. A DMP for a single DoF trajectory y of a discrete movement (point-to-point) is defined by the following set of nonlinear differential equations (Ijspeert et al., 2002c, 2013)where x is the phase variable and z is an auxiliary variable. Parameters α_z and β_z define the behavior of the second-order system described by (1) and (2). With the choice τ > 0, α_z = 4β_z, and α_x > 0, the convergence of the underlying dynamic system to a unique attractor point at y = g, z = 0 is ensured (Ijspeert et al., 2013). Alternatively, the gains α_z and β_z can be learned from training data while preserving the convergence of the system (Tan et al., 2016). In the DMP literature, equations (1) and (2), as well as their periodic counterpart (34)–(35), are called the transformation system, while (3) (or (36)) is the canonical system. f(x) is defined as a linear combination of N nonlinear Radial Basis Functions (RBFs), which enables the robot to follow any smooth trajectory from the initial position y₀ to the final configuration gwhere c_i are the centers of Gaussian basis functions distributed along the phase of the movement and h_i their widths. For a given N and setting τ equal to the total duration of the desired movement, we can define $c_i=\exp \left(-{\alpha }_{x}i-1/N-1\right)$, $h_i=1/{\left(c_{i+1}-c_i\right)}^2$, and h_N = h_N−1 where i = 1, …, N. For each DoF, the weights w_i should be adjusted from the measured data so that the desired behavior is achieved. The selection of the number of weights should be based on the desired resolution of the trajectory. For controlling a robotic system with more than one DoF, we represent the movement of every DoF with its own equation system (1)–(2), but with the common phase (3) to synchronize them.

The classical DMP formulation described in Section 2.1.1 applies to single DoF motions. Multidimensional motions are generated independently and synchronized with a common phase. In other words, equations (1) and (2) are repeated for each DoF while the phase variable in (3) is shared. This works when the evolution of different DoFs is independent, like for joint space or Cartesian position trajectories. Unlike Cartesian position, the elements of orientation representations like unit quaternion or rotation matrix are constrained. In this section, we present approaches that extend the classical DMP formulation to represent Cartesian orientations.

Abu-Dakka and Kyrki (2020) generalized DMP formulation in order to encode robotic manipulation data profiles encapsulated in the form of SPD matrices. The importance of SPD matrices comes from the fact that many robotics data are encapsulated in such matrices, for example, full stiffness matrices, manipulability ellipsoids, and inertia matrices among others. By defining $\mathbf{X}\in {\mathcal{S}}_{++}^m$ as an arbitrary SPD matrix and $\mathbf{\Xi }={\left\{t_j,{\mathbf{X}}_j\right\}}_{j=1}^{\mathfrak{T}}$ as the set of SPD matrices in one demonstration, where ${\mathcal{S}}_{++}^m$ defines the set of m × m SPD matrices. Afterward, we can rewrite (1) and (2) as followswhere σ = vec(Σ) is the Mandel representation of the symmetric matrix Σ and Σ is the time derivative of Ξ so that $\mathbf{\Sigma }\equiv \dot{\mathbf{\Xi }}=\left({\mathrm{Log}}_{{\mathbf{X}}_{j-1}}^{+}\left({\mathbf{X}}_j\right)\right)/\delta t$. The function ${\mathrm{Log}}_{{\mathbf{X}}_{j-1}}\left({\mathbf{X}}_j\right):\mathcal{M}\mapsto {\mathcal{T}}_{{\mathbf{X}}_{j-1}}\mathcal{M}$ maps a point X_ȷ in the manifold $\mathcal{M}$ to a point in the tangent space $\mathbf{\Delta }\in {\mathcal{T}}_{{\mathbf{X}}_{j-1}}\mathcal{M}$. vec (·) is a function that transforms a symmetric matrix into a vector using Mandel’s notation, for example, a vectorization of a 2 × 2 symmetric matrix is as follows

ξ is the vectorization of Ξ. ${\mathbf{X}}_g\in {\mathcal{S}}_{++}^m$ represents the goal of SPD matrix. $vec\left({\mathbb{B}}_{{\mathbf{X}}_{\mathbf{j}}\mapsto {\mathbf{X}}_1}\left({\mathrm{Log}}_{{\mathbf{X}}_{\mathbf{j}}}^{+}\left({\mathbf{X}}_g\right)\right)\right)$ is the vectorization of the transported symmetric matrix ${\mathrm{Log}}_{{\mathbf{X}}_{\mathbf{j}}}^{+}\left({\mathbf{X}}_g\right)$ over the geodesic from X_ȷ to X₁. Then we integrate (30) as followswhere the function mat(·) is the inverse of vec(·) and denotes to the matricization using Mandel’s notation. $\widehat{\mathbf{X}}\in {\mathcal{S}}_{++}^m$ represents the new SPD-matrices-based robot skills. The function ${\mathrm{Exp}}_{{\mathbf{X}}_{j-1}}^{+}\left(\mathbf{\Delta }\right):{\mathcal{T}}_{{\mathbf{X}}_{j-1}}\mathcal{M}\mapsto \mathcal{M}$ maps a point $\mathbf{\Delta }\in {\mathcal{T}}_{{\mathbf{X}}_{j-1}}\mathcal{M}$ to a point ${\mathbf{X}}_j\in \mathcal{M}$, so that it lies on the geodesic starting from ${\mathbf{X}}_{j-1}\in {\mathcal{S}}_{++}^m$ in the direction of Δ. An exemplar SPD DMP is shown in Figure 6.

Moreover, Abu-Dakka and Kyrki (2020) rewrote (13) for smooth goal adaptation in case of sudden goal switching as followsso g now is updated continually.

The classical periodic (or rhythmic) DMPs were first introduced by Ijspeert et al. (2002b), where they redefined the second-order differential equation system described in (1) and (2) as followswhere Ω is the frequency and y is the desired periodic trajectory that we want to encode with a DMP. The main difference between periodic DMPs and point-to-point DMPs is that the time constant related to trajectory duration is replaced by the frequency of trajectory execution (refer to Ijspeert et al., 2013 and 2002b for details). In addition, the periodic DMPs must ensure that the initial phase (ϕ = 0) and the final one (ϕ = 2π) coincide in order to achieve smooth transition during the repetitions.

Similar to (4), f(ϕ) is defined with N Gaussian kernels according to the following equationwhere the weights are uniformly distributed along the phase space and r is used to modulate the amplitude of the periodic signal (Gams et al., 2009; Ijspeert et al., 2002b) (if not used, it can be set to r = 1 [Peternel et al. 2016]).

Similar to discrete DMPs, LWR (Schaal and Atkeson, 1998) can be used to update the weight to learn a desired trajectory. In a standard periodic DMP setting (Gams et al., 2009; Ijspeert et al., 2002b), the desired shape f_d is approximated by solvingwhere y_d is some demonstrated input trajectory that needs to be encoded. The weights w_i can be updated using the recursive least-squares method (Schaal and Atkeson, 1998) with forgetting factor λ based on the error between the desired trajectory shape and the currently learned shape

The initial value of the parameters is w_i (0) = 0 and P_i(0) = 1. The forgetting factor determines the rate of weight changes. Refer to Schaal and Atkeson (1998) for details on parameter setting. An exemplar rhythmic DMP is shown in Figure 7.

The classical periodic DMP described by (34)–(36) does not encode the transient motion needed to start the periodic one. Transients are important in several applications like humanoid robot walking where usually the first step made from a rest position is a transient needed to start the periodic motion. To overcome this limitation, Nakanishi et al. (2004) presented a formulation of rhythmic DMPs including transient to achieve limit cycle with an application to biped locomotion. Ernesti et al. (2012) modified the classical formulation of periodic DMPs to explicitly consider transients as a motion trajectory that converges toward the limit cycle (i.e., periodic) one.

The same argument used in Section 2.1.2 is valid here too. Unlike Cartesian position, the elements of orientation representations like unit quaternion or rotation matrix are constrained. In this section, we present approaches that extend the classical periodic DMP formulation to represent periodic Cartesian orientations.

Classical DMPs are time-invariant, meaning that time scaling ςτ with ς > 0 generates topologically equivalent trajectories (Ijspeert et al., 2013). Using a simple modification of the transformation system, namely, substituting (1) withwhere y₀ is the initial value of y, Ijspeert et al. (2013) show that DMPs are also scale-invariant, meaning that the scaling of the movement amplitude ς(g − y₀) with ς > 0 generates topologically equivalent trajectories. The purpose of the green color used in (48) is to highlight differences w.r.t. (1). Kulvicius et al. (2011) use a similar strategy and scale the forcing term by the ratio between the start- and end-points of the demonstrated and desired trajectories. Apart from generating scaled—in time and space—versions of the demonstrated motion trajectory, classical DMPs also generalize to different initial/target states. However, the classical formulation—and its extension in (48)—may exhibit dangerous behaviors like over-amplification of the trajectory when reaching a different target and high accelerations when switching to a different target on-line (Ijspeert et al. 2013; Pastor et al. 2009). To alleviate the second issue, Ijspeert et al. (2013) replaced hard goal switches with the smooth switching law as in (13). However, the over-amplification issue still remains. Moreover, a DMP that uses (48) fails to learn motions with the same initial and target states (i.e., g = y₀, z₀ = 0 → y(t) = y₀ = g ∀t).

In order to remedy those issues, Pastor et al. (2009) proposed to modify the transformation system as followswhere the green color is used to highlight differences between (49) and (1). The most important change in this formulation is the term (g − y₀)x that has several benefits. It prevents high accelerations at the beginning of the motion (g − y − (g − y₀)x = 0 for t = 0) or when the goal is close to the initial state. It allows to reproduce motions with the same initial and target states and prevents over-amplifications and trajectory mirroring effects¹ when changing the goal. Hoffmann et al. (2009) derived a multidimensional representation of (49) from the behavior of the spinal force fields in frogs.

The goal can also change over time and, in this case, the tracking performance of the DMP mostly depends on the gains α_z and β_z. As proposed by Koutras and Doulgeri (2020b), the tracking performance can be improved by adapting the temporal scaling τ.

Dragan et al. (2015) showed that DMPs solve a trajectory optimization problem in order to minimize a particular Hilbert norm between the demonstration and the new trajectory subject to start and goal constraints. In this light, DMP adaptation capabilities to different start and goals can be improved by choosing (or learning) a proper Hilbert norm that reduces the deformation in the retrieved trajectory.

A via-point can be defined as a point in the state space where the trajectory has to pass. Failing to pass a via-point may cause the robot to fail the task execution. Therefore, having a motion primitive representation with the capability of modulating the via-points is important in robotic scenarios. It is not surprising that researchers have extended the DMP formulation to consider intermediate via-points in the trajectory generation process.

Ning et al. (2011, 2012) extend the classical DMP to satisfy position and velocity constraints at the beginning and at the end of a sample trajectory. Their approach to traverse via-points consists of creating a sample trajectory by combining locally linear trajectories connecting the via-points. This sample trajectory is used to fit a DMP that is constrained to pass the via-points.

Weitschat and Aschemann (2018) considered each via-point as an intermediate goal (via-goal) g_v for v = 1, …, V to reach. The last via-goal g_V corresponded to the target state of the DMP. In their formulation, they defined a variable goal as followswhere Ψ_v(x) are the Gaussian basis functions centered at the time corresponding to the v − th via-goal. The effectiveness of the approach is demonstrated in a task where the robot has to reach a different target while preventing possible self-collisions of the end-effector with the robot body. To this end, the authors placed the via-goals along the trajectory used to learn the DMP, forcing the generated trajectory to stay close to the demonstration while reaching the new target.

The problem of generalizing to via-point close (interpolation) and far (extrapolation) from the demonstration is faced by Zhou et al. (2019). Their approach, namely, Via-points Movement Primitives (VMPs), combines the benefits of DMP and Probabilistic Movement Primitives (ProMPs) (Paraschos et al., 2013). Authors assumed that the motion trajectory is generated as followswhere x is the phase variable defined as in (3) and the elementary trajectory e(x) can be defined as the linear attractor e(x) = (y₀ − g)x. The shape modulation term f_vmp(x) is defined as followswhere the Gaussian kernels Ψ_i(x) are defined as in (5), w_i are learnable weights, and ϵ_f is the Gaussian noise. As detailed in Paraschos et al. (2013), learning the shape modulation term f_vmp(x) means Learning from Demonstrations (LfDs) the prior probability distribution of the weights w_i. Having separated the generated trajectory into two parts like in (51) allows to adopt different strategies to pass a via-point y_v at x_v. Zhou et al. (2019) proposed to modify the shape modulation term for interpolation cases—when the via-point is “close” to the demonstrations. In extrapolation cases, instead, the elementary trajectory e(x) is rewritten as the polygonal line connecting y₀, y_v, and g. This approach easily generalizes to the case of multiple via-points. VMPs are experimentally compared with ProMPs, showing better performance especially in extrapolation cases.

Reaching a different goal, or passing through via-points, may not be enough to successfully execute a task in a different context. Approaches presented in this section adapt the DMP motion to new situations by adjusting the weights w_i of the forcing term (4), which modifies the entire DMP trajectory.

Weitschat et al. (2013) considered that L demonstrations are given, each encoded in a different DMP. In order to generalize, for instance, to a new goal g_new, they proposed to interpolate the weights of nearby DMPs, that is., DMPs that reached points around g_new. In formulaswhere o represents the indices of the nearby DMPs for which it holds that d_o < d_max, d_o is the distance (or, more generally, a cost) between g_new and g_o, d_max is the maximum distance to consider 2 DMPs close, and w_new = [w_1,new, …, w_N,new]^⊤ and w_o = [w_1,o, …, w_N,o]^⊤ are the new weights and the weight of nearby DMPs, respectively.

The approach by Forte et al. (2011, 2012) also assumes that L demonstrations are given and that each demonstration is encoded in a different DMP. Further, the authors exploited GP (Rasmussen and Williams, 2006) to learn a mapping between the query points q_l for l = 1, …, L (e.g., the goal of each DMP) and the DMP parameters [w_l, g_l, τ_l]. Given the new query point q_new, Gaussian Process Regression (GPR) (Rasmussen and Williams, 2006) is used to retrieve the new set of parameters [w_new, g_new, τ_new], that can be used to generate a DMP motion. This approach builds on previous work (Gams and Ude, 2009; Ude et al., 2010) where raw data from the L demonstrations are stored in memory and LWR is used to generate new DMP weights. Alizadeh et al. (2016) extend the approach in Ude et al. (2010) to retrieve the DMP weights even when the task parameters are partially observable. Finally, Zhou and Asfour (2017) extend the approach in Ude et al. (2010) to consider task-specific costs while learning the mapping between query points and DMP weights.

The aforementioned approaches follow a 2-steps procedure where first the shape parameters w are estimated given new task parameters and then execute the DMP. Matsubara et al. (2010, 2011) augmented the forcing term with a style parameter used to capture human variability across multiple demonstrations. Stulp et al. (2013) proposed a 1-step procedure where the DMP forcing term (4) is reformulated to explicitly depend on the task parameters. Their experiments show that a 1-step approach gives more freedom w.r.t. the used regression technique and increases the generalization performance. Along the same line, Pervez and Lee (2018) embedded task parameters directly in the forcing term. The authors proposed to use a mixture of Gaussians (Cohn et al., 1996) to learn the mapping between the task parameters (e.g., new goal and the height of an obstacle) and the forcing term. Given a new query task parameter, regression over the mixture of Gaussians is used to retrieve the forcing term parameters and generate the DMP motion. The approach is tested on a variety of tasks including sweeping and striking and additionally compared with the approaches presented by Forte et al. (2012), Stulp et al. (2013), and Ude et al. (2010) showing better performance, especially in extrapolation.

A Mixture of Motor Primitives (MoMP) is proposed in Mülling et al. (2010, 2013) and used to generalize table tennis skills like hitting and batting a ball. MoMP uses an augmented state that contains robot position and velocity as well as the meta-parameters of the table tennis task like the expected hitting position and velocity. The adapted motion is generated by the weighted summation of L DMPs and the responsibility of each DMP, representing the probability that a particular DMP is the correct one for the sensed augmented state, is also learned from data.

In high DoF systems, like humanoid robots, it is non-trivial to find a relationship between the task and the DMP parameters. This is especially true when the DMPs are used to encode joint space trajectories. Bitzer and Vijayakumar (2009) showed that such a relationship is easier to find in a latent (lower-dimensional) space obtained from training data. Therefore, they used dimensionality reduction techniques to find the latent space where to fit a DMP and show that the interpolation of DMP weights in the latent space results in better generalization performance.

A properly designed DMP reaches the desired target with zero velocity and acceleration, that is, once a DMP is fully executed the robot comes to a full stop. This also implies that the velocity “close” to the target is continuously decreasing. Using this property, Pastor et al. (2009) propose to combine successive DMPs by simply terminating the current DMP when the velocity is below a certain threshold and then starting the following primitive. When executing a single DMP, it is a common practice to initialize its velocity to zero—the robot is assumed to be still. In principle, this initialization can be used to sequence multiple DMPs (Lioutikov et al., 2016; Xu and Wang, 2004), but it may generate discontinuities if the robot does not fully stop in between two consecutive primitives. To prevent these discontinuities, Pastor et al. initialized the state of the current DMP with that of the previous one.

The velocity threshold approach is simple and effective since it directly applies to the DMP formulations in Sections 2.1.1 and 2.1.2. For instance, Saveriano et al. (2019) showed how to join multiple quaternion DMPs² (see Section 2.1.2.1) with the velocity threshold approach.

Results in Figure 8 are obtained when the velocity threshold is applied to merge 2 DMPs separately trained to fit minimum jerk trajectories (black-dashed lines). Figures 8(a)–(e) show the position and Figures 8(f)–(j) the orientation (unit quaternion) parts of the motion. The merged trajectory is generated by following the first DMP until the distance from the via-point is below 0.01 [m] and 0.01 [rad]. As shown in Figures 8(d) and (i), the switch occurs after about 4.7 [s]. Figures 8(e) and (j) show that the desired trajectory is accurately reproduced. More or less accurate trajectories can be obtained by tuning the distance from the via-point. However, the value of this distance is the time duration of the generated trajectory—a bigger (smaller) distance results in a shorter (longer) trajectory. For instance, in the considered case, the total motion ends after 9.5 [s] while the demonstration lasts for 10 [s]. Depending on the application, the time difference may cause failures; therefore, it has to be taken into account. Finally, the velocity threshold approach may generate discontinuities if the target of the current DMP is far from the demonstrated initial point of the following primitive.

There exist movements like hitting or batting that are correctly executed only if the target is reached with a non-zero velocity. To this end, Kober et al. (2010b) extend the classical DMP formulation in Section 2.1.1 to let the DMP to track a target moving at a given velocity. In their approach, the DMP passes the target with a given velocity exactly after T seconds. To achieve this, the acceleration in (1) is re-written as followswhere ${\dot{\widehat{y}}}_m$ is the desired velocity of the moving target $\widehat{g}$, which is defined as follows

By inspecting (55) and (56), and considering that the term −τ ln(x)/α_x represents the elapsed time if x is the phase defined in (3), it is possible to show that the moving target $\widehat{g}$ is designed to reach the goal g after T seconds, that is, $\widehat{g}\left(T\right)=g$ (Figure 9-left). The initial position of the moving target $\widehat{g}\left(0\right)$ is obtained by moving the goal position g for T seconds at constant velocity $-\dot{\widehat{y}}$. High accelerations at the beginning of the movement are avoided by the pre-factor (1 − x) which is set to zero at the beginning of the motion (x (0) = 1). The approach by Nemec and Ude (2012) combines a moving target and a particular initialization of the subsequent DMP to ensure continuity of the movement up to second-order derivatives.

Saveriano et al. (2019) extended this idea to quaternion DMP. The angular acceleration in (16) is modified as followswhere $\widehat{\mathit{\omega }}$ is the angular velocity of the moving quaternion target ${\widehat{\mathbf{g}}}_q$ and $2\hspace{0.17em}{\text{Log}}^q\left({\widehat{\mathbf{g}}}_q*\overline{\mathbf{q}}\right)$ measures the error between the current orientation q and ${\widehat{\mathbf{g}}}_q$. The pre-factor (1 − x) is used to avoid high angular accelerations at the beginning of the motion. The moving target for the quaternion DMPs is defined as followswhere g_q is the goal quaternion, T is the time duration of the DMP, and the exponential map Exp^q (·) is defined in (20). As shown in Figure 9-right, the moving target ${\widehat{\mathbf{g}}}_q$ reaches the goal orientation after T seconds, that is, ${\widehat{\mathbf{g}}}_q\left(T\right)={\mathbf{g}}_q$. This can be easily verified by considering that the initial value of the moving target ${\widehat{\mathbf{g}}}_q\left(0\right)$ is computed by moving the goal orientation g_q for T seconds at the desired velocity $-\widehat{\mathit{\omega }}$.

The presented target crossing approach allows crossing the target after T seconds. Assuming to have two DMPs with time duration T¹ and T², respectively, one can join them by running the first DMP for T¹ seconds and then switching to the second one. As for the velocity threshold approach, possible discontinuities at the switching point are prevented by initializing the state of DMP₂ with the final state of DMP₁. This procedure can be repeated to join L ≥ 2 consecutive DMPs.

Results in Figure 10 are obtained when the velocity threshold is applied for merging 2 separately trained DMPs to fit the minimum jerk trajectories (black-dashed lines). Figures 10(a)–(e) show the position and Figures 10(f)–(j) the orientation (unit quaternion) parts of the motion. The merged trajectory is generated by following the first DMP for T¹ = 5s and then switching to the second one. The required intermediate velocity is set to 0.01m/s (rad/s for the orientation) in each direction. The generated trajectory reaches the goal in 10s, that is, demonstration and execution times are the same. As required, the via-point is crossed at T = 5s with the desired velocity (Figures 10(c) and (h)). However, the non-zero crossing velocity introduces a deformation in the first part of the trajectory (Figures 10(e) and (j)).

The approach by Kulvicius et al. (2011, 2012) combines multiple DMPs into a complex one, guaranteeing a smooth transition between the primitives by ensuring that the basis functions composing f(x) in (4) overlap at the switching instances. First of all, Kulvicius et al. adopted a sigmoidal phase variable in (14) instead of the exponentially decaying one (3). As discussed in Section 2.1.1.3, the sigmoidal phase is $\approx 1$ for the large part of the motion which makes it possible to use smaller forcing terms to reproduce the demonstrations. On the contrary, the exponential phase is close to zero already before Ts (Figure 3), which results in larger forcing terms.

The classical acceleration dynamics in (1) is modified as follows

Similar to target crossing, Kulvicius et al. used a moving target $\stackrel{\sim }{g}$ in the acceleration dynamics, but called it the delayed goal function. The $\stackrel{\sim }{g}$ term in (59) is obtained by integratingwith $\stackrel{\sim }{g}\left(0\right)=y_0$. The delayed goal function in Figure 9 moves linearly from y₀ to g in T seconds and then remains constant, that is, $\stackrel{\sim }{g}\left(t\ge T\right)=g$.

The nonlinear forcing term f(s) is in green in (59) because it slightly differs from the classical one in (4). f(s) is defined as followswhere σ_i is the width and c_i is the center of the i-th basis function, and s is obtained by integrating (14). The term t/τT is used in (61) instead of the phase variable x. Being 0 ≤ t/τT ≤ 1, the basis functions are equally spaced between 0 and 1. Finally, σ_i are the widths of each kernel. They are constant and depend on the number of kernels.

Having presented the main differences with the canonical approach, it is possible to focus on how Kulvicius et al. (2012) solved the problem of joining L ≥ 2 DMPs. In general, each of the L DMPs has a different time duration T^l, desired target g^l, and initial position $y_0^l$, from which it is possible to compute the delayed goal functions by integrating

Note that, being ${\stackrel{\sim }{g}}^l\left(0\right)=y_0$, the acceleration (59) is s. For this reason, the term (1 − x) used in (54) is not needed in (59).

Assuming that L DMPs have been trained and that each DMP has N kernels, we can merge them into one DMP as follows. The centers of the joined DMP are computed as followswhere T^l is the duration of the l-th DMP and $T_{join}={\displaystyle \sum }_{l=1}^L{T}^l$ (duration of the joined motion). The widths of the joined DMP are computed as follows

The centers and widths computed in (63) and (64), respectively, overlap at the transition points allowing for smooth transitions between consecutive DMPs. The weights of the joined DMP are obtained by stacking the N weights of the L DMPs. Therefore, the joined DMP has N*L kernels and N*L weights. The phase variable (14) is modified to run for the duration T_join of the joint motion.

Saveriano et al. (2019) extended the basis functions overlay approach to quaternion DMPs. Assuming that a sequence of L quaternion DMPs is given. The angular acceleration in (16) is reformulated for each DMP as followswhere l indicates the l-th quaternion DMP and ${\mathbf{f}}_q^l\left(s\right)$ is defined as in (61). The term ${\stackrel{\sim }{\mathbf{g}}}_q^l$ is the quaternion delayed goal function, and it ranges from q^l (0) to ${\mathbf{g}}_q^l$ in T^l seconds (see Figure 9 [right]). To generate this moving target while preserving the geometry of ${\mathcal{S}}^3$, it is needed that ${\stackrel{\sim }{\mathbf{g}}}_q^l$ moves along the geodesic connecting q^l (0) to ${\mathbf{g}}_q^l$. Therefore, ${\stackrel{\sim }{\mathbf{g}}}_q^l$ is defined as followswhere

The angular velocity in (67) is computed for each l. The term $2{\text{Log}}^q\left({\mathbf{g}}_q^l*{\overline{\mathbf{q}}}^l\left(0\right)\right)$ represents the angular velocity that rotates q^l (0) into ${\mathbf{g}}_q^l$ in a unit time. Note, that the mappings Log^q(·) and Exp^q(·) are defined in (18) and (20), respectively. The delayed goal ${\stackrel{\sim }{\mathbf{g}}}_q^l$ crosses all the via-goals ${\mathbf{g}}_q^l,l=1,\dots ,L-1$ and then reaches the goal ${\mathbf{g}}_q^L$.

Results in Figure 11 are obtained when the velocity threshold is applied to merge 2 DMPs separately trained to fit the minimum jerk trajectories (black-dashed lines). Figure 11(a)–(e) show the position and Figure 11(f)–(j) the orientation (unit quaternion) parts of the motion. This approach does not require a switching rule and automatically generates a smooth trajectory—with continuous velocity as shown in Figure 11(c) and (h)—that passes close to the via-point which favors the overall reproduction accuracy (Figure 11(e) and (j)). However, the distance from the via-point depends on the weights of the joined primitives and cannot be separately decided. The trajectory generated with this approach tends to last longer than the demonstrations. This is due to the sigmoidal phase that vanishes after T + δ_ss (Figure 3). Depending on the application, the time difference may cause failures and has to be taken into account.

In Hoffmann et al. (2009), Park et al. (2008), and Tan et al. (2011), the detected obstacle was fitted with a potential field function to change the shape of the DMP to avoid it. More in detail, Tan et al. (2011) used the potential field to compute a time-varying goal and modified the resulting DMP trajectory, while Hoffmann et al. (2009), Park et al. (2008), and Zhai et al. (2022) added an extra forcing term to the DMP. Similarly in Gams et al. (2016), the human arm was fitted with a potential field function, which was used to reshape the DMP to perform coaching. The potential field was coupled to the position of the human hand to make pointing gestures and indicate the direction in which the robot arm position trajectory should change:

The added coupling term $C_{\mathcal{O}}$ is the obstacle avoidance term that contains the potential field and is given in a simplified form for the sake of explanation as follows:where $\mathcal{O}$ is the obstacle (or human pointing gesture) and y is the robot position. Exponential and ζ functions determine the potential field, while function d_s controls the distance at which the perturbation field should start affecting the DMP. For the full formulation of $C_{\mathcal{O}}$ and its parameters, see Gams et al. (2016). In Rai et al. (2017), the method was extended to include generalization of the obstacle avoidance formulation in (68). In a recent study, Ginesi et al. (2021a) proposed a novel approach for implementing volumetric obstacle avoidance by utilizing the theory of super quadratic potential functions.

Alternatively, the faulty segment of collision DMP trajectory can also be directly adjusted online by the human demonstrator (Karlsson et al., 2017b). On the other hand, the method in Kim et al. (2015) considers obstacle avoidance as a constraint of an optimization problem, which modifies the DMP trajectory to prevent collisions.

Similarly, as for obstacle avoidance, task dynamics can also be incorporated into DMP as coupling terms. In Gams et al. (2014), task dynamics were coupled on the acceleration and velocity level of the DMP. The presented method was utilized for interaction tasks, where the human changed the behavior of the robot based on the exerted dynamics on the manipulatorwhereas the force coupling term C_f = ςF is defined as a virtual or measured force F and ς is a scaling factor, which essentially changes the dynamic behavior of the DMP, enabling the motion primitive to instantly react to the coupled force. Later, Zhou et al. (2016b) introduced a PD controller-based coupling term formulation $C_{PD}=\varsigma \left(K^{\mathcal{P}}\left(F_d-F^e\right)-D^{\mathcal{V}}{\dot{F}}^e\right)$ coupled to the velocity part of the DMP (71). In the formulation F_d represents the desired force, F^e is the measured force, ς is a scaling factor, and $K^{\mathcal{P}}$ and $D^{\mathcal{V}}$ are the proportional and derivative gains of the Proportional Derivative (PD) controller, respectively. The coupling term formulation allows for a controlled adaptation of robot motion to changes in the environment.

In Kramberger et al. (2018) this approach was extended, with a force feedback loop coupled to the velocity (2) and the goal g of the DMP. The outcome of this approach is a similar behavior as an admittance controller (Villani and De Schutter, 2008), with a difference that the execution is directly on the trajectory generation level.

Here ${\dot{C}}_a=\varsigma \left(F_d-F^e\right)$ is the first time-derivative of the admittance coupling term, which changes the velocity and consequently the integrated coupling term, the position output of the DMP. The described approach can be used for Cartesian space motion, where the forces have to be substituted for desired and measured torques. This approach can be implemented in robot tasks involving contact with the environment as well as contact with humans. The constrained DMP formulation in Lu et al. (2021) unifies the formulations described in Sections 3.3.1 and 3.3.2 while ensuring stability of the generated motion.

In Peternel et al. (2016), human effort was used to provide information about the direction in which the assistive exoskeleton joint torque DMP should change in order to minimize it. The human was included into the robot control loop by replacing the error calculation in (41) with the human effort feedback term U(E):where E(t) is the current effort measured by human muscle activity through Electromyography (EMG) signals.³ Equations (34)–(38) and (42) are used in the original form. Equations (39)–(41) are not used, since (74) is used to modulate the weights in (37) instead.

The effort feedback term U(E) closes the loop and acts as feedback for adapting the weights of Gaussian kernels that define the shape of the trajectory. A positive U(E) increases, while a negative U(E) decreases the values of weights at a given section of the periodic DMP that encodes joint torque. If the shape of the DMP does not provide enough assistive power, the human has to exert effort (i.e., muscle activity) to produce the rest of the power required to achieve the desired task under given dynamics. In turn, muscle activity feedback then increases the magnitude of the DMP until the human effort term U(E) is minimized. Note that each joint has its own torque DMP and U(E) term (Peternel et al., 2016). After that point, the DMPs do not change unless the task, dynamics, or conditions change. If they change, the human has to compensate for the change by an additional muscle activity, which in turn adapts the DMPs to the new required joint torques.

In many LfD scenarios, it is desired to modify both the spatial motion and the speed of the learned motion at any stage of the execution. Speed-scaled dynamic motion primitives first presented in Nemec et al. (2013a) are applied for the underlying task representation. The original DMP formulations from (1) and (2) were extended by adding a temporal scaling factor υ on the velocity level of the DMP

From (75) and (76), it is evident that the velocity term is a function of phase and therefore encoded with a set of RBFs similarly as in (4). This method allows for modification of the spatial motion as well as the speed of the execution at any stage of the trajectory execution. The authors demonstrated the proposed method in a learning scenario, where after every learning cycle (using Iterative Learning Control [ILC]) a new velocity profile was encoded based on the wrench feedback and thus converged to an optimal velocity for the specific task. Vuga et al. (2016) extended the approach by incorporating a compact representation for non-uniformly accelerated motion as well as simple modulation of the movement parameters.

Later on, in Nemec et al. (2018) the authors extended the previous approach to also incorporate velocity scaling of the encoded orientation trajectories represented with unit quaternions. The outcome of the presented work is a unified approach to velocity scaling for tasks executed in Cartesian space. Furthermore, a reformulation of the velocity approach called Arc-Length-DMPs (AL-DMPs) was presented by Gašpar et al. (2018). In this work, they present a method, where the spatial and temporal components of the motion are separated, by means of the arc-length based on the time-parameterized trajectory. Arc-length, based on the differential geometry of curves, is related to the speed of the movement, given as the time derivative of the demonstrated trajectory. The approach is well suited when multiple demonstrations are compared for the extraction of relevant information for learning. Following the AL-DMPs idea, Simonič et al. (2021) introduced a constant speed DMPs to fully decouple the spatial and temporal part of the task. Pahic et al. (2021) used deep NN to map images into spatial paths represented by AL-DMPs. Weitschat and Aschemann (2018) added an extra forcing term to keep the velocity within a certain predefined limit. The aim of this work is to guarantee a safe execution of the robot task when interacting with humans, as well as provide a framework for safe interaction in a changing environment where the robot position and velocity have to change over time. For a full formulation of the coupling term, see Weitschat and Aschemann (2018).

To maintain consistency along the trajectory while ensuring a bounded velocity for each DoF, it is essential to decrease the velocities uniformly across all dimensions. This rationale prompts the consideration of temporal coupling, which involves modifying the DMP time constant, equivalent to the duration of path traversal, rather than spatial coupling. By increasing the time constant, the temporal evolution of all DoFs can be simultaneously reduced by the same factor. A temporal coupling approach based on tracking error was introduced in Ijspeert et al. (2013) and an enhanced version was proposed in Karlsson et al. (2017a). However, this method requires distorting the path in order to slow down the trajectory, and therefore, it cannot be directly utilized to limit the velocity or acceleration. One year later, the authors provided stability analysis of temporally coupled DMPs in Karlsson et al. (2018). Dahlin and Karayiannidis (2020) in their work proposed a temporal coupling based on a repulsive potential, keeping the DMP velocity within the predefined velocity limits while ensuring the path shape invariance. Subsequently, Dahlin and Karayiannidis (2021) introduced a temporal coupling method for DMPs that enables control over the velocity and acceleration constraints of the generated trajectory. This approach incorporates a filtering mechanism that proactively decelerates the trajectory as the acceleration constraints are approached, thereby mitigating the potential risk of infeasibility.

Successfully grasping an object is the first step toward robotic manipulation. Grasping necessitates the perception of the environment, particularly visual perception, to locate the object and determine suitable grasping points based on its geometric characteristics. In this context, even slight uncertainties can lead to object slippage and failed grasps. To improve the robustness of vision-driven grasping, Krömer et al. (2010a) augmented DMPs with a potential field based on visual descriptors to adapt hand and finger trajectories according to the local geometry of the object. This grasping strategy was integrated within a hierarchical control architecture where the upper level determines the object’s grasp location and the lower level locally adjusts the motion to achieve a robust grasp of the object (Krömer et al., 2010b). Stein et al. (2014) proposed a point cloud segmentation approach based on the convexity and concavity of surfaces. This approach is particularly well-suited for recognizing object handles, and DMPs are employed to execute grasping with a real robot.

The ability to grasp and use tools is also desirable to perform daily-life manipulation. In this respect, Guerin et al. (2014) proposed the so-called tool movement primitives that transform the demonstrations in a tool affordance frame. The result is a motion that generalizes to different tool poses and to tools that share the same affordance(s). Li and Fritz (2015) considered tool usage with low-cost, non-dexterous grippers and propose a framework to learn bi-manual strategies for tool usage and compensate for the lack of dexterity. Bi-manual robotic manipulation is a challenging task that requires precise coordination between hand movements and adherence to spatial constraints. Thota et al. (2016) developed a DMP-based control framework for bi-manual manipulation that ensures time synchronization of the two hands while being robust to spatial perturbations and goal changes.

Beyond object grasping, everyday manipulation requires a precise execution of complex movements. Often such complex movements are hard to encode into a single motion primitive, but they can be conveniently split into simpler motions (e.g., reach and grasp) that can be properly sequenced and executed (Figure 12).

The possibility of exploiting DMPs as the building blocks of complex tasks was investigated in Caccavale et al. (2018, 2019) and Ramirez-Amaro et al. (2015). In these works, a human teacher demonstrated a relatively complex task consisting of several actions performed on different objects. The demonstration was then automatically segmented into M basic motions used to fit M DMPs. While Ramirez-Amaro et al. (2015) exploit semantic rules (e.g., reach an object with a knife means cut) to infer high-level human activities, Caccavale et al. built a hierarchical structure to schedule the execution of the complex task by selecting the proper DMP for the current executive context. They used kinesthetic teaching and verbal cues (open/close gripper commands) to provide task demonstrations. Schwaner et al. (2021) used DMPs to build a library of surgical skills which used to autonomously execute part of a suturing task. Lemme et al. (2014) organize segmented task demonstrations into a motion primitives’ library learned from self-generated trajectory patches. They also introduced a mechanism to remove unused skills and update the library. Kinesthetic teaching and haptic feedback were also used by Eiband et al. (2019) to segment and recognize basic motions or skills and to build a tree describing geometric relationships—like reference frames and goal poses—between consecutive skills. At run time, the robot performed haptic exploration to locate objects in the scene and update the skill tree. The transformations in the skill tree were then used to define the initial and goal pose of the DMPs and execute the task. Finally, Wu et al. (2018) integrated DMPs into a dialogue system with speech and ontology to learn or re-learn a task using natural interaction modalities.

Collecting demonstrations becomes an issue of kinesthetic teaching or marker-based motion trackers cannot be used. The latter requires an expensive sensor infrastructure that is hard to build in real-world scenarios like factory floors. Kinesthetic teaching needs torque-controlled/collaborative robots that are still uncommon in industrial scenarios. To remedy this issue, Mao et al. (2015) exploited a low-cost RGB-D camera and track the human hand using the markerless approach proposed by Oikonomidis et al. (2011). Collected data were then segmented into basic motions and used to fit DMPs. Also, the approach in Yang et al. (2022) does not require tracking markers or manual annotations. Instead, authors exploit videos of random unpaired interactions with objects by the robot and human for unsupervised learning of a keypoint model of visual correspondences. Bayesian optimization is then used to find the parameters of rhythmic DMPs from a single human video demonstration within a few robot trials.

The described approaches assume that human teachers always provide consistent and noiseless task demonstrations. Ghalamzan E. et al. (2015) encoded noisy demonstrations into a GMM and computed a noise-free trajectory using GMR. The noise-free trajectory was then used to fit a DMP that generalized to different start, goal, and obstacle configurations. Dong et al. (2023) propose to fit a DMP from correct (positive) and incorrect (negative) demonstrations to increase the representation and generalization capabilities of the model. Niekum et al. (2012, 2015) designed a framework that learns from unstructured demonstrations by segmenting the task demonstrations, recognizing similar skills, and generalizing the task execution. Interestingly, a user study on 10 volunteers conducted by Gutzeit et al. (2018) showed that existing strategies for segmentation and learning are sufficiently robust to enable automatic transfer of manipulation skills from humans to robots in a reasonable time.

Some work (Deniša and Ude, 2013a, 2013b, 2015; Denisa et al., 2021) exploited transition graphs and trees to embed parts of a trajectory and search algorithms to discover a sequence of partial parts and generate motions that have not been demonstrated. Approaches that rely on a hierarchical, tree-like structure to represent the task have limited task generalization capabilities. Lee and Suh (2013) used probabilistic inference and object affordances to infer the adequate skill that can handle uncertainties in the executive context. Beetz et al. (2010) learned stereotypical task solutions from observation and used task planning and symbolic reasoning to execute novel mobile manipulation tasks. A generative learning framework was proposed by Wörgötter et al. (2015) to augment the robot’s knowledge base with missing information at different levels of the cognitive architecture, including symbolic planning as well as object and action properties. Paxton et al. (2016) used task and motion planning to generalize the execution of complex assembly tasks and proposed a learning by demonstration approach to ground symbolic actions. Agostini et al. (2020) performed task and motion planning by combining an object-centric description of geometric relations between objects in the scene, a symbol to motion hierarchical decomposition depending on three consecutive actions in the plan, and the LfD approach developed in Caccavale et al. (2019) (Figure 12). A manipulation task was described at three different levels by Aein et al. (2013). The top level provides symbolic descriptions of actions, objects, and their relationships. The mid-level uses a finite state machine to generate a sequence of action primitives grounded by the lower level. A common point among these approaches is that they use DMP to execute the task on real robots.