Usage

dope4j-app is a CLI application which can be used to run inference and detect object poses.

Actions

runInference

Required options:

• -modelUrl=<path> - path to DOPE model in ONNX format. Each model can detect single object. Where to download models and how to covert them to ONNX can be found in Quickstart to isaac_ros_pose_estimation.
• -objectSize=<width>,<height>,<length> - size of object`s 3D cuboid which DOPE model is trained to detect. See list of DOPE original object sizes
• -imagePath=<path> - path to input image file or folder. In case of folder dope4j will search it for image files inside
• -cameraInfo=<path> - path to file with camera information in camera_calibration_parsers format. The images should be rectified before feeding them to DOPE. It means that if you want to use your camera you need to calibrate it first. This can be done using camera_calibration_parsers as described in DOPE Camera Tutorial. For testing with DOPE original datasets use original camera_info.yaml (copy of it can be found inside dope4j "config" folder).

Optional:

• -showVerticesBeliefs=<true|false> - for each image show all candidate vertices detected during inference step. Candidate vertices belong to one of the 8 vertices of cuboid which surrounds the object. They all part of Belief Maps. There is one Belief Map per each vertex of the cuboid. Default is "false".
• -showCenterPointBeliefs=<true|false> - for each image show all center points detected during inference step. They all part of single Belief Map. Default is "false".
• -showAffinityFields=<true|false> - for each image show all Affinity Fields detected during inference step. There are Affinity Fields for all 8 vertices of cuboid which surrounds the object. Every Affinity Field is a vector field where each vector points towards center point of the cuboid. Default is "false".
• -showMatchedVertices=<true|false> - for each vertex show with which center point it matches. Default is "false".
• -showCuboids2D=<true|false> - for each detected object show a cuboid. The cuboid is rendered directly on the vertices which are returned by the DOPE network (no projection is done). Default is "false".
• -showProjectedCuboids2D=<true|false> - for each detected object show a projection of its cuboid 3D model. Default is "false".
• -lineThickness=<int> - set thickness of cuboid lines.
• -cache=<true|false> - enable caching of preprocessed input images and output tensors. Default is "false".
• -cacheFolder=<path> - file system location where cache will be stored. Default is "_cache_dope4j" inside system temporary folder.
• -recursiveScan=<true|false> - when -imagePath points to a folder then this option will control if dope4j will look for images in subfolders or not. Default is "false".
• -imageFileRegexp=<regexp> - when -imagePath points to a folder then this regular expression will be used to filter images. Default is ".*\.(png|jpg)" which means that only "png" and "jpg" images will be used to run inference.
• -threshold=<double> - set threshold value which is used to filter the keypoints. Default is "0.01".
• -debug=<true|false> - print debug information and log it to dope4j-debug.log inside system temporary folder. Default is "false".
• -exportMetricsToCsv=<path> - path to local folder where emitted metrics will be exported in CSV format.
• -exportMetricsToElastic=<elasticsearch_url> - address of ElasticSearch where to emit metrics. Credentials can be part of the URL. Example http://user:password@localhost:9200
• -totalRunTime=<true|false> - print total execution time when command finishes.

showResults

Read inference results from JSON file and for each image show cuboids of all detected objects.

Required options:

• -resultsJson=<path> - path to JSON file with inference results.

Optional:

• -imagesRoot=<path> - path which will be prepended to all image files which are read from results file. Default is path to current folder where command is executed.

Environment variables

It is possible to specify additional configuration through following environment variables:

• MAVEN_REPOSITORY - by default dope4j-app resolves dependencies using standard Maven repository location at home folder ~/.m2. This helps to avoid packaging all Deep Java Library dependencies into dope4j-app and keep its size small (all PyTorch engine dependencies are more than 1GB). If dependencies already present locally they will be used directly, otherwise they will be downloaded and added to local Maven repository.

Examples

First make sure that all native libraries available globally in the system or provided separately as follows:

Supported versions see in Requirements section.

Command to run inference for all images inside "/tmp/dataset" using ChocolatePudding model and:

• show all beliefs for object vertices and center points
• save the results to results.json

All the results are added to "/tmp/results.json" as follows:

{
  "imagePath": "/tmp/dataset/scene_0002/0004_rgb.jpg",
  "detectedPoses": {
    "objectCuboidModel": {
      "center": {
        "x": 0,
        "y": 0,
        "z": 0
      },
      "vertices": [
        {
          "x": 2.4735999107,
          "y": -1.4961500168,
          "z": 4.1749000549
        },
...

Command to show detected poses for images inside "/tmp/dataset" without running any inference but using results previously saved in "results.json"