Actin®  Version 5.1.0
Software for Robotics Simulation and Control
Using the Manipulation Director

Using the Manipulation Director

The Manipulation Director coordinates the motion of multiple robots and objects simultaneously to perform tasks. The Manipulation Director can be opened by clicking on the toolbutton

sceneToolbarButton.png
Manipulation Director Toolbar Button

The Manipulation Director can be thought of as the director of a play, and the individual robots, tools, and objects can be thought of as the actors in the play. The Director consists of a Script, and Cast. The Script consists of an ordered list of Scenes. A Scene consists of roles that will take part in the Scene and each Role is assigned to a set of Directions or tasks thay they need to execute/perform as part of the Scene. The casting defines the mapping between the actors and the roles. A Script can contain multiple Scenes, each with different actors. This document describes the architecture and usage of the Manipulation Director to control robots in Actin. The animation below shows an example Script that moves two arms, each with a gripper, to manipulate and hand off an object.

Example Director Script in Actin

The "Manipulation Director" object is the highest level, and contains the director object directory, the active Script, and the cast. the .ecd files that you can open and save are referred to as a "Manipulation Director File". This file can be saved and loaded as .ecd (compressed) or .xml.

Example Director Script For UR3

The default Manipulation Director interface is shown below. The director is initially populated with a default cast containing all the manipulators (robots or objects) in the model at the time the plugin was loaded. This cast can be edited if you aren’t planning on interacting (attaching, moving, etc) with certain actors (manipulators or objects). The Roles in the cast can and should be renamed to more general names, such as "Arm" or "Gripper" or "Environment". This makes it more clear if manipulators are later exchanged.

initInterface.png
Starting Interface

After importing or creating manipulation Directions, you can begin to populate the Script.

Typically, the steps involved in populating Manipulation Director are briefly mentioned below. They are described in detail in section Simple Director Tutorial.

  • Add/Import Poses in the Poses tab.
  • Add/Import Directions in the Directions tab.
  • Create Scenes, and populate the Scene by dragging and dropping the Directions from "Directions" tab or by inserting Directions inside the Role columns in the scene
  • Create a Script, and populate the Script by dragging and dropping the scenes into the Script
  • Play the Script.
  • We can have more than one Script created for accomplishing different tasks

The Figure below shows a poulated Manipulation Director

populated.png
A Populated Manipulation Director Interface

The figure below shows the main window. Notice that there are four primary regions of the window: tool bar, Director object library, play controls, and the tabbed editor accessed through the navigation bar. The object library has different tabs for the supported director objects (Scripts, Scenes, Directions, Poses). The tabbed editor changes based on what objects are opened, and what type of object is being edited. We can keep the editors for 6 items open at a time and these items can be accessed through the navigation bar.

directorComponents.png
Director Components

Manipulation Director Toolbar

The Manipulation Director tool bar provides toolbuttons for performing various operations in the director such as creating a new Manipulation Director file, save etc. explained below.

fileMenu.png
Manipulation Director Tool bar

From left to right:

  • The "New" button creates a new Manipulation Director file which is empty and does not have any objects or items.
  • The "Open" button opens a previously-saved Manipulation Director file.
  • The "Save" button saves the current Manipulation Director to file.
  • "Save As" saves allows user to save the current Manipulation Director to a different file.
  • "Configure Path Planner" allows user to configure the path planner options for the Manipulation Director. The path planner options configured here are used by certain types of Directions for calculating the path and executing the Direction such as "End Effector Path Planning Direction", "Joint Path Planning Direction" etc.
  • The default extension for Manipulation Director files is ".ecd". = The "Add Role" button gives you the option to add a Role to the Scene that is being edited.
  • The "Edit Cast" button allows user to edit the Cast and map the manipulators or Roles to actors.

Play Controls

The play controls include the following: play button, pause button, stop button, track check box, and gravity check box. These can be seen below. the play controls are used to control execution of whatever library item is open in the tabbed editor, which could be Scripts, Scenes, or Directions.

playControls.png
Play Controls

Play Button: If the execution is currently stopped, pressing the play button will start the execution of the item opened in the tabbed editor. This button will only be enabled if the item is not currently playing.

Step Button: Pressing the step buton will step through the Directions in the Script or scene.

Pause Button: Pressing the pause button will pause the execution of the Script, Scene or direction. This button will only be enabled if the Script, Scene or Direction is currently playing.

Stop Button: Pressing the stop button will stop the execution of the Script, Scene or Direction. This button will be enabled if the Script, Scene or Direction is either playing or paused.

Track Check Box: This check box indicates whether the Script view will attempt to track the current position of a Script or Scene that is playing. When the check box is checked, the Script view will always try to vertically position the Script so that the currently-executing Directions in the Script are centered. In order to make runtime changes to the Script, it is often necessary to uncheck this box.

Gravity Check Box: This check box will turn gravity on and off for the Script or Scene view. When gravity is off, the Directions will raise to the level of their dependencies. While editing a Script or Scene, it is often useful to turn gravity off in order to see the implicit and explicit dependencies between Directions. At runtime gravity on provides more animation of the runtime process. The difference between disabled and enabled can be seen below.

gravDisabled.png
Gravity disabled

gravEnabled.png
Gravity enabled

Breakpoint Check Box: This check box will turn on the ability to set a breakpoint in the Scene or Script. When we turn it on, the Script or Scene is in the "Breakpoint mode" and all the blocks in the Scene or Script will have a green dot at the bottom right corner as shown in the Figure below. If you click on the green dot in any of the blocks, it turns red, indicating that breakpoint is set on that block. This is shown in the Figure Set Breakpoint below. It is a toggle mode, if you click again on that, the breakpoint is removed. When we play the Script/Scene and the control reaches the block with red dot, the execution is paused allowing us to debug the parameters, which could be simulation state, joint positions etc. The execution can be resumed by clicking on the red dot. This is shown in the Figure Execution paused in Breakpoint mode.

In the Script tab, when we check the Breakpoint option, initially the Scene block will not display anything, but when we start execution, all the blocks in the Scene will display the green dot and we can set the breakpoint in any of the block while execution is in progress.

breakpointMode.png
Breakpoint Mode

setBreakpoint.png
Set Breakpoint

waitAtBreakpoint.png
Execution paused in Breakpoint mode

Trace Check Box: This check box enables user to trace the execution of individual component of a Scene or a Script. This is shown in the following video.

Execution of Script with Trace enabled

Director Object Library

The Director Object Library section displays a library of Scripts, Scenes, Directions, and Poses. It provides the user interface to logically organize the library. It allows Direction objects in the library to be dragged into the Scene view to compose Scenes, and drag and drop Scenes to compose Scripts. It also allows Poses to be dragged into the editors of certain Directions, such as waypoint sequences and tool offset directions.

dodTabs.png
Director Object Library

The Director Object Library provides a hierarchical directory structure for storing director objects. It is essentially a database of reusable objects, such as Poses, Directions, Scenes and Scripts. Users interact with the Director Object Library using context menus as well as drag-and-drop operations.

dod.png
Director Object Library

You can create objects in the library using the context menus, or by using the buttons below the library widget shown below. Clicking the [+] button will create and object depending on which tab is selected.

dodAddDelete.png
Library Add/Delete

if the "Directions" tab is selected, then you can create a Direction using the toolbox by clicking on which type you want to create.

directionToolbox.png
Directions tool box

After selecting to create an object, you will be prompted for a name for the object. If its a Scene, then you can select a set of Roles to add to it. If its a Direction, you can select the intial Role. This Role is not permanent, because the Role is determined based on Role column in the Scene to which you add the Direction as you will see later in tutorial. The initial Role is for prototyping and creating the direction.

Multiple objects/folders can be selected within a tab and can be dragged to a different location by dropping into a destination folder item. A context menu will prompt the user whether to "Move" or "Copy" the selected items to the new destination.

Context Menus

Context menus appear when right-clicking in the director object library. The contents of the menu depend on if the mouse click is on the background, folder item, or object item.

Background Context Menu The background context menu provides the following options: "New Folder", "New Scene" (New item depending on the tab selected in DOD), "Import". This is shown in the Figure below.

backgroundContextMenu.png
Background Context Menu

New Folder: This option prompts the user for the name of a new top-level folder to add to the director object library. The prompt will ensure that the new directory name is unique.

New Scene: This option prompts the user for the name of a new Scene object to add to the director object library. The prompt will ensure that the new Scene name is unique. The new Scene will be empty; however, it can be inserted into the Script view and modified.

Import: This option prompts the user for the name of a file to import or a folder to import. We can import a DirectorObjectDirectory file or import all supported objects stored within a folder by selecting the "Directory" option.

Folder Item Context Menu

The folder item context menu provides the following options: "New Folder", "New Scene" (New item depending on the tab selected in DOD), "Import", "Export", "Rename", and "Delete". The New Folder, New Scene work the same as above, but use the currently selected folder instead. This is shown in the Figure below.

folderContextMenu.png
Folder Context Menu

Export: This option allows user to export the objects stored under the folder (including the sub folders) as a single "DirectorObjectDirectory" file or as a equivalent file system folder itself with the "Directory" option. The "DirectorObjectDirectory" file can be saved in ".ecz", ".ecx", ".xml" format. When it is exported as "Directory" all objects are saved in ".xml" format under the folders.

Rename: This option prompts the user for a new name for the currently-selected folder item. The prompt will ensure that the new directory name is unique.

Delete: This option will remove the currently-selected folder item from the director object library.

Object Item Context Menu

The object item context menu provides the following options: "Export", "Rename", "Delete" and "Copy". This is shown in the Figure below.

itemContextMenu.png
Item Context Menu

Export: This option prompts the user for a file format to export and exports the currently-selected object item to the selected format. The object can be exported to ".ecz". ".ecx", ".xml" or ".xml.gz" format.

Rename: This option prompts the user for a new name for the currently-selected object item. The prompt will ensure that the new name is unique.

Delete: This option will remove the currently-selected object item from the director object library.

Copy: This option allows the user to copy the object to clipboard and it could be pasted in another location in the director object library.

Scripts

Scripts are composed of an ordered list of scenes. While a Scene encapsulates a lower-level task, a Script combines the lower-level tasks in order to accomplish a higher-level task. For example one Scene could involve attaching a tool to a robot, and another Scene could involve using the tool to pick up an object. The Script can execute the two scenes in sequence. The Script editor displays the Scene contents of a Script. When the Script isnt being run, It visually displays the scenes in the order they will be run, and allows for reordering.

script.png
Script with 2 scenes before execution
scriptExecute.png
Script with 2 scenes during execution
scriptView.png
Script Editor View

To add scenes to Scripts, drag them from the scenes tab in the library.

Scenes

A Scene is a ordered set of Directions assigned to different Roles that take part in the Scene to accomplish a task. Thus, first step to compose a Scene would be, to add the roles that will take part in the execution of the scene. Then we can drag and drop Directions from the "Directions" tab into the Role columns of the scene. The order in which these roles execute the Directions are determined by their positions in the column and also by the cues. Typically all the roles will start simultaneously and execute the Directions bottom up. If we want to synchronize the tasks performed by these roles, then we can add cues between the various blocks of the Scene and control the order in which different roles perform different tasks/directions.

scene.png
Single Scene

The Scene editor visually displays the roles, actors, Scenes, Directions, and cues. The view allows you to add Directions to invidual roles, order them within the Role, and synchonize Directions across roles by creating cues.

Steps to create a Scene-

  • Select the Scene tab, create a "New Scene" in the tree
  • Provide the name for the Scene, select the roles that will participate in the Scene and click "Ok"
  • Double click on the newly created Scene item, this will open the Scene editor
  • Now change the DOD Tab to Directions, so you will have the left hand tab showing Directions and right hand side showing the Scene Editor
  • Right click on the Scene header column add Add roles if they have been missed earlier to the Scene
  • Drag and drop Directions from the tree view to the respective Role columns in the Scene editor.
  • Add cues between Directions of different roles to control the order in which they will be executed
  • Save the Scene for later use

This is shown in the Figure below.

sceneView.png
Scene View

Scene Context Menu Options

We can do the following operations in a Scene through a context menu as shown in the Figure below.

sceneContextMenu.png
Scene Context Menu
  • Add Role to the Scene: While creating a Scene, if we have missed selecting a Role, we can use this option to add additional roles to the Scene
  • Edit Cast: This allows us to edit the cast that has been set for the Manipulation Director
  • Toggle loop: This options allows us to toggle the "Loop" property for the Scene. If this property is set, it is indicated in the "Scene" column with a "LOOP" text. Now if we play the Scene, it will keep playing the Scene infinitely till the user stops the Scene by clicking on the Stop button in the Play Controls bar.
  • Set Loop Count: If the Loop property is set, and we select this option, we can set the number of times we want to play the Scene
  • Clear Script: When this option is clicked, all the Directions and cues from the Scene are cleared.

Cues

Cues provide a synchronization point between Directions for multiple roles. Each cue can either be a simultaneous cue or a following cue. A simultaneous cue implies that specific Directions for multiple roles should be executed simultaneously. A following cue implies that a specific Direction for a Role must wait until a specific Direction for another Role completes before it can begin execution.

followingCue.png
Following Cue
simultaneousCue.png
Simultaneous Cue

Create Cues If you have 2 Directions in a Script that are used by different roles, you can create a cue by dragging one Direction after (above) the other.

beforeQ.png
Before creating cue
createQ.png
Drag direction
afterQ.png
After creating cue

After creating a cue, you can right click on the Direction and edit any cues by selecting the "Edit Cues" menu as shown below.

editQContext.png
Edit Direction cues

This will bring up the dialog used to edit the cue type (following or simultaneous), or delete the cue.

editQDlg.png
Edit cues

Cast

The Cast defines the mapping between the actors and the roles. The actors are the manipulator’s themselves (stored as the index of the manipulator in the simulation), which can be robots, grippers, tools, or primitive shapes. The Role is the column that contains the Directions to perform (stored as a text string).

cast.png
Cast

Editing the Cast

To edit the cast of a Scene or director Script, right click on the scenes tab in the Script view, and select "Edit Cast".

editCastContext.png
Edit Cast

or click the button in the menu bar.

editCastButton.png

this will show the edit cast dialog, where you can add and remove roles, and change which actor is fulfilling each Role.

editCastDlg.png
Edit Cast Dialog

Directions

Directions can be thought of as either "stage" Directions or manipulation Directions. Stage Directions occur instantaneously when executed. Some examples would be Tool Attachment Directions, Tool Offset Directions, and Collision Exclusion Directions. Manipulation Directions differ in that they require time to execute. Some examples are waypoint sequences, or joint frame sequences, as well as any Directions that perform tracking. Directions are classified into different categories based on their function which is described below. This is shown in the Figure below.

directions_categories.png
Direction Categories

Direction Types

Motions

• Waypoint Sequence Direction: Used to move a Motion Constraint of the actor through a sequence of poses or waypoints

• Joint Frame Sequence Direction: Used to move an actor through a sequence of joint positions

• End Effector Path Planning Direction: Used to move an actor to an end effector position while avoiding obstacles

• Alignment Direction: Used to align an actor’s end effector orientation

• Gripper Direction: used to move an actor’s EcOneDofGripperEndEffector

• Multi-Arm Waypoint Sequence Direction: A waypoint sequence that moves multiple end effectors through waypoint sequences simultaneously

• Path Direction: Used to move the Motion Constraint of an actor along a path

• Joint Path Planning Sequence Direction: Used to move an actor through a sequence of joint positions while avoiding obstacles

• Motion Script Direction: Used to move or control motions of a manipulator using "Motion Script". Please refer to Motion Script Library for detailed description of this.

• Change Control Description Direction: This Direction should be used within a Scene to set the ControlDescription to be used for executing the scene.

• Approach Attach Finding Solution Direction: This Direction is used to find a valid joint solution to a Pose (normally an approach pose) while also checking to ensure that the robot will be able to move linearly from that joint configuration to a second Pose called as the attach pose. This will however not move the robot. The joint solution returned by ApproachAttachFindingSolutionDirection is saved in the data map and can then be extracted by a JointPathPlanningDirection that is used to move the robot to that joint solution. Thus user would need to create the ApproachAttachFindingSolutionDirection, JointPathPlanningDirection and then use both of them in a Scene to execute them.

• Joint Path Planning Direction: This Direction allows us to specify the ApproachAttachFindingSolutionDirection to be used for moving the manipulator or the actor. This Direction can also be used independently, by specifying a joint position as a Goal which should be reached when we execute the direction.

Controls

• Conditional Direction: Used to execute Directions if sets of conditions are satisfied

• If Direction: Used to execute a simple if, then, else control structure

• Continue Direction: Used to continue execution

• Stop Direction: Used to stop execution

• Exception Direction: Used to execute Directions if another Direction fails for any reason

• For Direction: Used to execute a sequence of Directions a number of times

• Wait Direction: Used to wait for a set of conditions to be true

• While Direction: Used to execute a sequence of Directions while a set of conditions is true

Stage

• Assign Object Direction: Used to assign an object to a manipulation sequence (not used often)

• Attach to Direction: Used to attach the active Role actor to another.

• Attach to Named Frame Direction: Used to attach the named frame associated with the Active Role to named frame of another role/link called as the Base Role.

• Set Attachment Direction: Used to attach another actor to the active Role.

• Collision Exclusion Direction: Used to enable and disable collision detection in the dynamic collision exclusion map

• Tool Offset Direction: Used to apply a tool offset to the actor’s end effector

Time

• Motion Settle Direction: Used to wait until the joint motion has completed in the event that the control system is still optimizing at the end of the previous direction

• Manual Pause Direction: Used to wait until the user responds

• Timed Wait Direction: used to wait for a set period of time

I/O

• Data Map Set Direction: Used to set variables in the data map, which can be used for conditions

• Data Map Read Direction: Used to read the value of a variable in the data map, or to check if the variable exists

• Print Direction: Used to print information to the results.txt file

• Send Signal Direction: Used to send out signals to other plugins when a specific point has been reached in the Script

Condition Types

Some Directions use conditions to determine the flow of events during execution.

• Has Attachment: Used to check if the active Role has an attachment to anything, or to a specific Role, or a specific link on a role.

• Is Attached To: Used to check if the any Role has an attachment to anything, or to a specific Role, or a specific link on a role.

• Role Location: Use to check the Role position

• Distance From: Use to check the Role's distance from a specific link

• Data Map Value: Used to check the value of a data map variable

• EE to Pose Distance: Used to check the distance of a Motion Constraint of a particular Role from a pose

ApproachAttachFindSolution

Poses

Poses are a low level building block, which stores a position and orientation in the simulation. Poses can be stored in the system (global) coordinate system, as well as relative to a manipulator or link, or a named frame on the link. Poses can be used in Waypoint Sequences, End Effector Path Planning Direction, Joint Path Planning Direction and Tool Offset Direction.

To create a Pose in the library, open the Pose tab and use the context menu or click the [+] button. You will then be prompted to enter a unique name for the pose. This creates a Pose that is stored relative to the system coordinate frame, at the current position of the end effector dragger.

contextNewPose.png
Create a new pose
poseNonRelative.png
A system relative pose

To create a new Pose that is stored relative to the primary frame of a link or manipulator, right click on the link in the simulation, and select "Pose -> Add Pose at this link". You will then be prompted to enter a unique name for the pose.

Create Relative Pose

You can create a relative Pose by the following steps also:

  • Create a new Pose in the clicking Pose DOD tab
  • Click on the "Select a frame" button in the Pose editor. This will bring a "Select a Frame" dialog
  • Select the option "Manipulator-Link" in the "Select a Frame" dialog
  • Select the Manipulator relative to which you want to create the link
  • Select the Link of the manipulator
  • Select the frame to be used for the pose
  • Click Ok, This will create a relative Pose to the manipulator-link that has been selected. Now if we move this link or the manipulator, the Pose would also be relative to the new position of the link.
selectLinkForRelativePose.png
Select link relative to which Pose is being created

The other way for creating a relative Pose to manipulator-link is, right click at the link in the viewer and select "Pose -> Add Pose at this link" option as shown in the figure below.

addPoseAtLink.png
Add Pose Relative to a Link

After choosing the link, the Pose is created relative to link as shown in the Figure below. Now if the link moves, the Pose moves relative to it.

poseRelative.png
A link relative pose

Edit a Pose

Opening a Pose will bring up the Pose editor in a new tab. The Pose editor widget allows you to move the Pose using a dragger dialog, as well as select which manipulator and link the Pose should be stored relative to.

Pose Placement

We can position the Pose using the dragger. To edit the position, click on the "Show Dragger and Enable Pose Editor" checkbox. you can move the dragger in the simulation or you can enter values numerically in the Dragger UI below.

setPosePosition.png
Edit Pose coordinates using Dragger

We can position the Pose from the Active Motion Constraint also. To update a pose's position to the current end effector (motion constraint) dragger position, click the "Set From Active Constraint" button.

setPoseToEEPosition.png
Set Pose From Active Constraint

Pose Position

To move the active constraint to the Pose, click on the "Move Active Motion Constraint To Pose" button.

setEEToPose.png
Move to pose

To set a Pose position by clicking in the simulation, select the "Click on Surface" option under Placement Mode. Then use the mouse cursor to click on the surface of an object in the render window. the position is set from where the cursor is clicked, but the orientation can be set from either the active EE dragger, the link frame of the object that was clicked, the surface normal from where on the surface was clicked, or from the system frame.

clickOnSurface.png
Click on surface

Waypoint sequences can use poses as waypoints. To add a Pose into a waypoint sequence, open the waypoint sequence in the editor, and go to the Pose library tab. drag the Pose from the library and into the waypoint sequence list widget.

Simple Manipulation Director Tutorial

This section is a tutorial that shows how to use the director to train a UR10 arm motions to perform a pick and place motion.

Open Director

to open the director click the "Manage Manipulation Director" option in the Manipulation menu. This will open the empty director interface in a docked widget. If the model was saved with a Manipulation Director active, you may need to click the "New" button in the upper left corner of the director dock widget.

manageManipDir.png

Set the Cast for the Manipulation Director

First set the the cast for the Manipulation Director. Click on the Edit Cast button. This will bring up the Cast Dialog. Now give Role names for the manipulators. These Role names will now be used for all Directions, Scenes and Scripts in the Manipulation Director. This is shown in the figure below. We will map the manipulator "UR10" as "Arm" and "Robotiq_2F85" as "Gripper".

editCast.png

Create a New Script

We will create a new Script that will ultimately contain the scenes to complete our task. To create a new Script, navigate to the Scripts tab in the library (the left most tab), then click the [+] button near the bottom, or right click in the tab and select "New Script".

newScript.png

You will then be prompted to select a name for the Script. For this tutorial, we will call our top level Script "Sample Script".

scriptName.png

Create a New Scene

Next we need to create a new Scene that will contain the Directions to perform the first section of our Script that will approch and grab the object. To create a new Scene, navigate to the scenes tab in the library, then click the [+] button near the bottom, or right click in the tab and select "New Scene".

newScene.png

You will then be prompted to select a name for the Scene, and the roles that will perform in the scene. For this we name it "Sample Scene" and add the Arm Role to the scene. This Scene will move the arm to a home position, then move to some imaginary object on the right side or the +y side.

sceneName.png
emptyScene.png

Create a Joint Frame Sequence

This joint frame sequence will move the arm to a joint position that we will consider our "home".

To create a joint frame sequence, navigate to the Directions tab in the library. Then you can either cllick the "Jointframe Sequence Direction" button in the add Directions widget (found under "Motions"), or you can right click in the tab and select Motions -> Jointframe Sequence Direction. You will then be prompted for a name and an intitial "temporary" Role for which this sequence is being created. In this case we name it "Home Joint Position" and select the Arm.

newJFSeq.png
jfSeqName.png

Once you have created the Direction, and it has been opened in the tabbed editor, we can add a joint frame to our sequence by moving the arm to a desired joint position, and then clicking the [+] button in the joint frame widget, or by right clicking in the widget and selecting "Add Joint Frame".

addJF.png

After the frame has been added, we can edit the desired joint positions in the table widget.

editJF.png

Create a Waypoint Sequence

Next we want our arm&aposs end effector to approach some imaginary object. to do this we will create a waypoint sequence, which moves the desired motion constraint through a series of waypoints in space. To create a waypoint seqence, either cllick the "Waypoint Sequence Direction" button in the add Directions widget (found under "Motions"), or you can right click in the tab and select Motions -> Waypoint Sequence Direction. You will then be prompted for a name and an intitial "temporary" role. In this case we name it "Approach +y" and select the Active Role as "Arm".

newWPSeq.png
wpSeqName.png

Once you have created the Direction, and it has been opened in the tabbed editor, we can add a waypoint to our sequence by moving the arm to a desired position, and then clicking the [+] button in the waypoint widget, or by right clicking in the widget and selecting "Add Waypoint in this case we are using our Frame EE constraint, so we select it from the "Motion Constraint Set" drop down.

After waypoints are created, their properties can be edited, as well as the placement in 3D. To make a waypoint a "stopPoint", select the waypoint and go to the "Properties" tab. then click on the current stopPoint value in the table to toggle it. To change the position of the waypoint, go to the dragger tab and enable the 3D dragger with the check box. You can edit the value numerically or by dragging the dragger in the simulation.

Populate the Scene

To execute a Direction, open it in the tabbed editor, and click the play button. If we want to execute a number of Directions in order, we add them to a scene. To add the Directions to the "Approach" Scene, open it in the editor. Then we drag the Directions from the library into the arm column (the Role we want to execute the scene). In this case we drag "Home Joint Position" and then "Approach +y". When we execute this Scene, the arm will move to the home joint position, then it will execute the waypoint sequence "Approach +y".

Create Gripper Directions

If we want to actuate the gripper on this arm, we can create gripper Direction. Since we have two roles, we organize the Directions related to the two roles in folders. We create a folder called "Gripper" in the "Directions" tab and store the Gripper Directions under that. To create a gripper Direction, we perform the same method as the last two motions. We need to create 2 Gripper Directions, one for open and one for close.

newGripperDirection.png

In the gripper Direction editors, we select the "Motion Constraint Set 0" and set the desired percentages to be 5% for the "Close Gripper" Direction and 99% for the "Open Gripper" direction.

gripperClose.png

Adding to the Scene

We can add these Directions into the Scene by dragging them from the library into the scene. In this case, we want to open the gripper before moving home (bottom most direction), and close the gripper after approaching (top most direction).

Create a New Scene

For this example, we will create one more Scene that will move the imaginary object that we have grasped to the left of the Scene or the -y direction. Create the Scene using the same method as before, but name the Scene "Approach -y".

newScene2.png

Create a Waypoint Sequence

Next we will create another waypoint sequence that will move the imaginary object to the left side or the -y Direction of the UR10 workspace. Follow the same steps as before to create a sequence of waypoints towards the negative Y Direction of the system.

waypoints2.png

Create a Timed Wait Direction

To create a "pause" in the sequence, create a timed wait Direction and specify the time duration to be 1 second. We name it "Pause 1 second" in this case.

newTimedWait.png
timedWait1sec.png

Populate the Scene

After we have created our Directions to move the object, we can populate our scene. Follow the same steps as the last Scene, making the arm execute "Home Joint Position", "Approach -y", "Open Gripper", "Pause 1 second", and then lastly "Home Joint Position".

moveScene.png

Add Scenes to the Script

To execute a Scene, open it in the tabbed editor, and click the play button. If we want to execute a number of scenes in order, we add them to a Script. To add the scenes to the "Sample Script" Script, open it in the editor. Then we drag the scenes from the library into the Script. This Script will execute the "Sample Scene" Scene followed by the "MoveToHome" scene.

Direction Tutorials

This section will go into more detail on each Direction type.

A manipulation Direction can be used in a Scene in two different ways:

  • As a Reference | First we need to add the Direction in the Object Library prior to use it in a Scene as a reference. The advantage of using it as a reference is, any modification that happens to the Direction will be into effect in the Scene object as well.| To add a Direction,
    • Go to Directions tab
    • Select Motions Group or any other group
    • Select the desired Direction type, like "waypoint sequence direction" or "joint frame sequence direction"
    • Other wise, Right clicking on any folder in Direction tab will bring you the menu option
      directions_categories.png
  • As an Object
    • Select the Scene where the Direction is to be added
    • Right click and select "Insert Direction" option
    • Select the desired Direction type here.
      insertDirection.png

Path Direction

Selecting a Path Direction will bring up the interface to edit a path direction.

pathDirectionWidget.png
Path Direction UI

This type of Direction allows you to record a path simply by moving the arm. Any delay while reorienting the camera or reorienting or moving the arm will also be recorded. To record a path, click the "Start Recording" button, then move the arm in the simulation. To stop recording, click the "Stop Recording" button where the start button was. You can choose to playback the exact joint states that were recorded in the path, or the end effector position that was recorded in the path. If you choose to playback in end effector mode, the software may choose a different inverse kinematics solution that will achieve the end effector positions. To Record a path using assistive mode and the UR10, stop the simulation and enter assistive mode. Be sure that the UR10 will not fall uncontrollably when torque is disabled. While in assistive mode, start the recording, move the robot around the desired path. When completed, stop the recording, and then exit assistive mode. To record an assistive mode path, create a new path Direction, switch to it, then stop the simulation, enter assistive mode, press record, move it around, press stop record, then exit assistive mode, start the simulation, and play it back To create a path Direction from .csv data, select the "Load State Path button". For this Direction to work properly, the csv data must be structured properly. The structure is as follows in the table below.

Time (ms)Joint 0 (rad) Joint 1 (rad)Joint 2 (rad)Joint 3 (rad) Joint 4 (rad)Joint 5 (rad)Joint 6 (rad) Joint 7 (rad) Joint 8 (rad)
4972.15 1.86402 -0.34084 -1.19069 -0.83333 -0.96459 -0.76886 2.11575 4.52E-05 4.52E-05
4972.18 1.86402 -0.34084 -1.19069 -0.83333 -0.96459 -0.76886 2.11575 4.52E-05 4.52E-05
4972.2 1.86402 -0.34084 -1.19069 -0.83333 -0.96459 -0.76886 2.11575 4.52E-05 4.52E-05
4972.23 1.86402 -0.34084 -1.19069 -0.83333 -0.96459 -0.76886 2.11575 4.52E-05 4.52E-05
CSV data to import as path direction

Waypoint Sequence Direction

Selecting a Waypoint Sequence Direction will bring up the Waypoint Sequence Editor. The Waypoint box will be empty if no waypoints have been added. To add a waypoint, move the robot end effector to the desired location and then right click in the white box and click "Add Waypoint", or press the 'w' key. A waypoint titled "Waypoint 1" should appear in the box. The sequence of waypoints will use the motion constraint set of the first point added.

waypointSequenceEditor.png
Waypoint Sequence Editor

The waypoint stores the position and orientation of the motion constraint, and if you playback this Direction the motion constraint will move to the waypoint that was just created. If you right click on the waypoint in the list, you can choose to delete it, move to it, show it, overwrite its position, or add another waypoint after it.

The waypoint sequence editor has three tabs at the bottom, "Display", "Properties", "Dragger" If you click on the "Display" tab, it provides us options to show all waypoints, and render the zoneRadius of all the points. If we do this here a guide frame will appear at the location in space of each waypoint, along with a sphere representing the zoneRadius or tolerance of the point. If we select a particular waypoint (by clicking the mouse on it), the "Properties" tab will display the waypoint properties explained a little later in the section. The Dragger tab allows us to view and modify the coordinates of the waypoint through a dragger

If you select a waypoint from the list and then click on the Waypoint Properties tab in the Direction manager, you can edit the parameters of the waypoint, including whether or not it is a stop point, if its position is relative to the previous waypoint, its zone radius, the acceleration and speed the end effector will apply to approach that waypoint, whether or not to interpolate, and the angular zone tolerance.

waypointProperties.png
Waypoint Properties

If you select a waypoint from the list and then click on the Waypoint Dragger tab, you can edit the position and orientation of the waypoint. Click the "Dragger Pose" check box, and a dragger will appear where the selected waypoint is. You can click and drag the red, green, and blue axes and rings to adjust the coordinates and orientation of the waypoint. You can also type in the position and orientation in the fields provided in the "Dragger" property tab.

dragger.png
Waypoint Dragger

Using Poses as Waypoints Poses can also be used as waypoints. Poses can be dragged from the Pose tab. Poses can be stored relative to other links or manipulators, and are useful in this case if there is a position that may be used multiple times in multiple sequences. This means you only need to edit it once if the program changes.

Dragging poses into waypoint sequence

waypointUsePose.png

Joint Frame Sequence Direction

Selecting a Joint Frame Sequence Direction will bring up the joint frame sequence editor. The joint frame box will be empty if no joint frames have been added. To add a joint frame, move the robot end effector to the desired joint state and then right click in the white box and click "Add Joint Frame", or press the 'w' key. A joint frame titled "Joint Frame 1" should appear in the box.

jointFrameEditor.png
Adding Joint Frame

The joint frame stores the position of each joint, and if you playback this Direction the joints will move to where they were in the joint frame that was created. If you right click on the joint frame in the list, you can choose to delete it, move to it, overwrite it, or add another joint frame after it. If you select a joint frame from the list and then click on the "Joint Frame Properties" tab, a menu will appear where you can edit the joint level tolerance of the joint frame. If you select a joint frame from the list and then click on the Joint Values tab, you can view and edit the joint positions of the selected joint frame.

jointValues.png
Joint Values

End effector path planning direction

Creating an end effector path planning Direction behaves like a waypoint sequence that only contains one waypoint, and the robot plans a path to the waypoint around obstacles. In order to run this type of Direction, it is required to load the pathPlanningPlugin from the Plugins -> load plugin menu, which allows Actin to perform RRT path planning. After creating a new end effector path planning Direction, the dialog as shown below will appear.

eePathPlanDlg.png
End effector path planning Direction UI

The desired end effector set and end effector should be chosen from the combo boxes. The joint tolerance is the tolerance that the joints need to achieve in order for the Direction to succeed. The Goal end effector position can be set by dragging a Pose into the "Pose Name:" box, and checking the "Use Pose" radio button. Alternatively you can use the "Use Current" option, and click "Add Current" to set the goal position to the current end effector position.

Gripper Direction

A gripper Direction allows you to move the gripper constraint, for example the "Driven" constraint in the "Motion Constraint Set 0" of UR10. To add a gripper Direction, click the "Gripper Direction" button under Motions category below or right click within the DOD tree and select "Motions -> Gripper Direction" option. In this example we label it Open Gripper.

In the Gripper Direction UI select the motion constraint set desired, and make sure the motion constraint is a gripper. The maximum and minimum values for the motion constraint should be populated automatically. Next select the desired percentage of openness desired for this Direction. 100% is fully opened, 0% is fully closed.

editGripperAction.png
Edit gripper direction

In this example the gripper will open. To make the gripper close, create another gripperAction, with the same parameters except make the desired percentage small, small enough to grasp the object without over exerting the gripper.

Multi Arm Waypoint Sequence Direction

The multi arm waypoint sequence Direction works just like the waypoint sequence Direction, but allows for the control of multiple end effectors. This is mostly used for bi handed models. First make sure the number of arms (really end effectors) is correct. For this example it is 2. This field is used to indicate the number of arms that we are controlling or configuring for movement through this particular sequence. It can have a value more than 1 and less than or equal to the maximum arms that the simulation has.

multiArmSeq.png
Multi Arm Waypoint Sequence Direction

First we can create a sequence for the "0" end effector, as shown here following the same procedure as a waypoint sequence. Select the Motion Constraint that you want to move through this and add waypoints for it by moving it.

multiArmSeqEE0.png
Sequence for EE 0

Next to create a sequence for the second end effector, we switch from arm 0 to 1 by setting the "Arm" combobox to 1, as shown below. Select the other motion constraint that we want to move through this. Note that, this motion constraint cannot be same as the one selected for the previous one.

multiArmSeqEE0to1.png
Switch from EE 0 to 1
multiArmSeqEE1.png
After switching to EE 1

Next we create a sequence for EE 1, following the same way.

Playing the Direction will move both end effectors to their desired waypoints in parallel. This Direction is useful for reducing cycle times if you have the arms doing different tasks, as well as grasping the same object with both grippers.

Alignment Direction

The alignment Direction is used to align the orientation of the desired end effector with the system frame or manipulator frame (when using the Manipulation Director). The interface for this action is shown below.

alignmentAction.png
Alignment Direction

Choose the alignment frame that is desired. This can align with the system frame, the base of the robot, or the base of another manipulator in the Manipulation Director. Also select which end effector set and end effector is desired to reorient, as well as the tolerance that needs to be achieved in order for the Direction to succeed.

Tool Path Direction

Coming Soon!

See here for how to import tool paths.

Created by Energid Technologies www.energid.com
Copyright © 2016 Energid. All trademarks mentioned in this document are property of their respective owners.