Bin Picking Studio 1.4.0: Build Robot Environments Without CAD Files

Environment builder

Have you ever found yourself in a situation where you needed to adjust the robot environment but no CAD engineer was around to help you out? From now on you can easily and all by yourself draw simple collision objects directly in the Bin Picking Studio! 

How?

We’ve prepared a comprehensive yet easily understandable guide for you in the following article.

The Bin Picking Studio 1.4.0 comes with a new, user-friendly tool “Environment builder”. It allows you to build a 3D model of the working cell, define the robot’s workspace and test its movement capabilities. The tool also allows you to verify the model of the environment by comparing it with a point cloud from the calibrated scanner.

The “Environment builder” consists of a 3D visualizer and a control panel divided into 3 tabs with different functionalities:

• Scene: here you can build a 3D model of the robot environment
• Robot: here you can test the robot movements
• Vision: here you can connect to the calibrated vision systems in order to compare the virtual environment with the real world

Scene

Prevent the robot from leaving its working space and enclose it in a virtual box – quickly and effortlessly! The easy nature of the job is boosted by displaying the robot in the environment configuration and by added features for resizing and scaling the environment objects.

To enjoy collision-free bin picking, you need to manage a precise 3D model of the real robotic cell. The virtual environment created in Bin Picking Studio must match with the real world at least within the reach of the robot. Although the trajectories are checked for collisions, the scanner never sees every object in the scene so you need to define static collision objects matching the reality. 

Object list

The “Scene” tab contains an “Object list”, supporting a hierarchical structure. This means that each object can have child objects linked to it in order to manipulate with a group of objects more easily. You can use the +/- buttons next to the parent object to expand/collapse the list of child objects.

To change the hierarchy of already defined objects you can grab an object in the “Object list” and move it to a desired position in the list (note the + icon that appears when the object is moved over another object to make it its child).

Example:

Consider a pallet with several objects. It is beneficial to define each object as a child to the parental object. This way you can manipulate (move, rotate etc…) all objects by simply changing position properties of the parent object.

Every object in the “Object list” offers 3 options listed next to it (from left to right):

• Sync icon: says whether the object is saved (synced) or unsaved
• Visibility toggle: enables to show/hide the object
• Remove button: enables to remove the object (please note that removing a parent object will also remove all its child objects)

The “Scene” tab with a highlighted “Object list” and the “+ Add object” button

You can add a new object to the scene by clicking the “+ Add object” button in the bottom left corner of the 3D visualizer. You can either define a collision object using one of the available primitive shapes (box/sphere/cylinder) or upload an STL file. 

When selecting a primitive shape, you can define its properties in the “Edit form” and “3D object tools panel”. 

If you choose an STL file, a pop-up window will appear, asking you to enter a name of the new object, choose the file to upload and select a type of the object. There are 3 available object types:

• Bin: meant for any CAD model that represents a bin (rendered in blue)
• Environment: other collision objects that too correspond with real objects in the robot’s vicinity (rendered in grey)
• Virtual collision object: used for objects that may not necessarily exist in the robot’s real environment – it is mainly used for limiting the robot’s workspace during bin picking but also for defining the cell walls to allow see-through inspection of the (virtual) robot inside (rendered in red and semi-transparent, as the only object type)

Each of these object types represent a “collision object” in the sense that the picked object will not collide with it.

After selecting an object from the “Object list”, it is highlighted in blue in the list and in red in the visualizer. Besides that, the “Edit form” and “3D object tools” panels appear. You can also select an object by directly clicking on it in the 3D visualizer. To deselect an object, click on it in the “Object list” once again or choose the “Close” option from the “3D object tools” panel.

Edit form & 3D object tools

After selecting an object either from the “Object list” or in the 3D visualizer, the “Edit form” and “3D object tools” panel will appear.

The “Scene” tab with a highlighted “Edit form”, “3D object tools” panel and “Save scene” button

The “Edit form” allows you to enter the following object properties:

• Name (of the object)
• Type (of the object as described above)
• CAD file (mesh): you can download the STL object
• Scale: you can scale the STL object
• Dimensions: you can resize the object (only for primitive shapes): width/height/depth for the box, radius for the sphere and radius/height for the cylinder
• Position: you can change the position of the object origin relative to the parent object origin
• Rotation: you can rotate the object around its own origin

You can also dynamically manipulate the objects with the “3D object tools”: 

• Move: this option shows the marker arrows so that you can change the position of the object along a desired axis by dragging the marker
• Rotate: this option shows the marker circles so that you can change the rotation of the object in the desired axis by dragging the marker
• Scale: this option shows the marker arrow so that you can scale the object
• Resize: this option shows the marker arrows so that you can change the object dimensions in the desired axis by dragging the marker (only for the box and cylinder)
• Self/Parent toggle: you can change the origin of the rotation when rotating using the marker; select “self” for rotating around its own origin; select “parent” for rotating around the parent’s origin

The markers also allow you to add increments to the position/rotation values for each axis. Instead of dragging the marker in the desired direction, just click on the arrow/circle for a specific axis, enter the desired value and apply by clicking the “Check” button.

The “+ Add child object” button allows you to create a child object for the currently selected object. Use the “x Close” button to deselect the object (end editing).

Complete the action by clicking the “Save scene” button. The virtual environment will synchronize with the robot. Only after that can you use the “Robot” tab with robot controls.

Robot

Before accessing the “Robot” tab, you first need to select a robot and gripper from the database and define the toolpoint. You can then virtually jog the robot and define its workspace.

Robot controls

You can choose from three motion modes:

• Joint
  Here you can virtually jog the individual robot joints in two ways:

   

  • By moving the slider for the corresponding joint
  • By manually entering the joint position in the input field

• Linear (Tool coordinate system / Robot-base coordinate system)
  You can move the robot’s toolpoint either in the coordinate system of the tool or of the robot base. There are three ways to move the robot:

   

  • By moving the marker in the robot’s toolpoint, dragging it to the desired position
  • By dragging the arrows to move it in each respective axis or by dragging the circles to rotate it
  • By manually entering the TCP position and orientation in the “Toolpoint position” panel

If you click the “Home pose” button, the robot will automatically be moved to its default position. To enable the collision checking feature, use the “Highlight collisions ON/OFF” toggle button.

When maneuvering the robot, joint limit restrictions are applied.

Example of a setup of joint limits

Joint limits allow you to define the path planning workspace of the robot.

The robot usually needs to move only in a fraction of its joint range. We strongly recommend you to set up the joint limits so that the path planning takes place only in that subspace of the robot’s full workspace. Correctly set up joint limits have a positive effect on the path planning computation time and success rate.

If you leave a joint limit field empty, the system will use the robot’s default hardware.

Please note that too restrictive joint limits can lead to a path planning failure. Always allow for some leeway. To verify that the reach of the robot is sufficient, use the jogging options of the “Robot controls” panel. Make sure the robot can reach every place in the bin model with various gripper orientations.

 

Vision

You may also have encountered difficulties stemming from discrepancies between the real bin picking cell and its CAD version. This will be no issue anymore with the new option to trigger a scan and check whether the scanned point cloud corresponds to the modeled virtual space. Given that now you can also check the scanning volumes of the employed vision systems, it has never been easier for you to set up the field of view.

The “Vision” tab enables you to manipulate the configured Vision Systems to validate the placement of the collision objects.

If scanners are mounted on the robotic arm, you first need to select a robot model for that Vision System to appear in the list. Otherwise you get a warning message.

The “Vision System list” with 3 Vision Systems, the fourth one cannot be displayed as no robot has been selected

A Vision System can have the following statuses:

• Available: the scanner is ready to connect and the Vision System is calibrated
• Not available: the scanner is currently not ready to connect; check the power, connection and network configuration
• Not calibrated: the Vision System is not yet calibrated – you first need to perform a successful calibration to use this Vision System in the “Environment builder”

Get detailed information about a Vision System (scanner’s ID, mount position and model type) by clicking on it . The bottom fields reflect the calibration of the Vision System – translation and orientation of the scanner’s camera relative to the robot base (extrinsic calibration) or robot flange (hand-eye calibration).

The Vision System overview window contains several buttons:

• Visibility: you can turn on/off the visibility of each scanner model, scan volume and the origin of the scanner (more specifically its camera)
• Connect: you can click the “Connect” button once the scanner is ready to connect and the Vision System has been calibrated
• Disconnect: once connected,  you can disconnect from the scanner using this button
• Trigger: while connected to the scanner, you can trigger scans

Detailed information about a Vision System

Please note that only one scanner can be connected at a time. In case the same scanner is used in multiple Vision Systems, it gets connected in all of them.

The “Environment builder” shares scanner controls with the PhoXi Interface – once connected in the “Environment builder”, the same scanner is connected in the PhoXi Interface as well.

Validation

Connecting a scanner and triggering scans helps you place collision objects (mainly the bin) so that the model matches the reality precisely.

To ensure a precise object placement, a calibrated Vision System must be available to connect. You can then trigger a scan and use it to position the CAD model so that the model and the point cloud overlap.