Skip to main content

Bridge Remote DDS Networks With a DDS Router

· 13 min read
Dominik Nowak

DDS Router is a fresh solution by eProsima. It allows you to bridge different DDS networks with providing built-in ROS 2 topics filtering in the same time. You can define which ROS 2 topics from one remote host are visible in others. That's a powerful tool for challenges like creating secure fleets of robots or just specifying a ROS 2 interface for your ROS 2 robot for the outside world.

DDS Router and Husarnet fit perfectly, making it a great foundation for any ROS 2 project that needs to run on multiple devices operating both in the same LAN and over the Internet.

It is not an alternative to previously described Fast DDS Discovery Server but an additional building block that makes your ROS 2 network more secure, easy to maintain and easy to configure.

DDS Router for ROS 2 robot fleets

TL;DR​

In the following sections I will describe how DDS Router & Husarnet work together, but here is a quick summary of how to run and test it on your own. I have created a full, open source reference project utilizing DDS Router, Fast DDS Discovery Server and Husarnet in a single system with a demo ROS 2 nodes:

Open Source GitHub Example

Run the following Docker Compose deployments on one or on multiple hosts (in the same or different LANs) to test how DDS Router works:

  • compose.ds.yaml - DDS Discovery Server
  • compose.listener.yaml - 2 x listener node from demo_nodes_cpp
  • compose.listener.yaml - 2 x talker node from demo_nodes_cpp

The default DDS Router config file shows how to enable communication of only one pair of talker <-> listener.

The full source code is here:

https://github.com/dominikn/fastdds-router-demo

Test DDS Router

How does a DDS Router Work?​

Let's demonstrate it on a simple example system that looks like that (the same one as in example code above):

DDS router demo architecture for ROS 2 system

The problem we are solving here is:

"how to bridge ROS 2 nodes running on remote hosts but only for chosen ROS 2 topics?"

In this demo:

  • HOST 1 is running 2 x listener ROS 2 nodes, but each of them subscribing to different ROS 2 topics: /chatter1 or /chatter2. Each of the nodes runs in a separate Docker Container working in the same Docker Network (meaning these two listener nodes see both /chatter1 and /chatter2 if you run ros2 topic list on them).

  • HOST 2 is running 2 x talker ROS 2 nodes, publishing to /chatter1 or /chatter2. Also running in the same Docker Network.

In the previous blog post I presented how to connect any number of hosts running ROS 2 nodes both in LAN and over the Internet, but the solution had one important drawback - all nodes, running on all hosts, see all ROS 2 topics. It was due to the fact that ROS 2 nodes can not use the DDS Discovery Server and Simple Discovery at the same time.

In most cases it's overkill, and possible security risk - one hacked robot can give read/write the access to all ROS 2 communication in the fleet!

It would be great to be able to share only a preselected list of topics to remote hosts.

... and basically this is what DDS Router does!

Data & Discovery Traffic​

If we do not connect Docker Containers running ROS 2 nodes using Husarnet, and they communicate by default only locally, how can they talk with the nodes running on remote hosts then?

DDS Router data and discovery traffic

Only a DDS Router Container need to be Husarnet connected because it uses two discovery mechanism at once, by using two embedded participant profiles:

  1. Internal Participant - uses the default local multicasting based Simple Discovery Protocol so it sees what is going on in the local DDS Domain (like a regular ROS 2 node you launch here).

  2. External Participant - uses centralized Fast DDS Discovery Server for a service discovery, but the user data traffic goes between remote DDS routers peer-to-peer over the Internet. Of course if these DDS Routers are Husarnet connected.

Between the internal and external participant DDS Router decides which messages should be filtered, based on the allowlist: in YAML configuration file.

DDS Router Deployment​

DDS Router & Docker

You can use both Husarnet and DDS Router without Docker. With Docker it was easier to provide you with a demo system that "just works", but you can run everything directly on your host OS as well.

Using a DDS Router with your ROS 2 robots is straightforward - you don't need to touch your existing ROS 2 nodes at all (like changing the .xml DDS participant config file). Just add dds_router service together with a husarnet VPN sidecar container to the existing Docker Network where your ROS 2 nodes are running.

docker-compose.yaml
  ...
  dds_router:
    image: ghcr.io/dominikn/ros-galactic-fastdds:latest
    network_mode: service:husarnet
    volumes:
      - ./router-config.listener.yaml:/config.yaml
    command: ddsrouter -c /config.yaml -r 10

 husarnet:
    image: husarnet/husarnet
    devices:
      - /dev/net/tun
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=0
    cap_add:
      - NET_ADMIN
    volumes:
      - /var/lib/husarnet
    environment:
      - HOSTNAME=listener
    env_file:
      - ./.env  # contains Husarnet Join Code

Description:

  • image: ghcr.io/dominikn/ros-galactic-fastdds:latest: DDS Router is a fresh project introduced by eProsima recently, so I created a custom ROS 2 base image (for arm64 and x64 architectures) with Fast DDS, Discovery Server and DDS Router preinstalled. Find the source code for this Docker image here.
  • - ./router-config.listener.yaml:/config.yaml: DDS Router YAML config file is attached as a bind mount volume. You specify here a DDS Discovery Server configuration together with a list of topics you want to bridge with other hosts.
  • command: ddsrouter -c /config.yaml -r 10: launching DDS Router with reloading of the configuration (from .yaml file) every 10 seconds. That means you can modify the file when the container is already running!
  • network_mode: service:husarnet: using network namespace of Husarnet p2p VPN container. Thanks to that, DDS Router instances running on different hosts in remote networks act like they were in the same LAN (VLAN provided by Husarnet). Please note that only a DDS Router container will share network namespace with Husarnet service. Your ROS 2 nodes don't need to be VPN connected and work with their default DDS config!

DDS Router Configuration​

The DDS Router config file for talker and listener looks like that:

router-config.talker.yaml
allowlist:
  - name: "rt/chatter1"
    type: "std_msgs::msg::dds_::String_"

internal_partipant:
  type: local
  domain: 0

external_partipant:
  type: local-discovery-server
  id: 201

  connection-addresses:
    - id: 200
      addresses:
        - domain: "dds-discovery-server" # using hostname instead of Husarnet IPv6 addr
          port: 11811
          transport: "udp"

Description:

  • allowlist: list of ROS 2 topics that we want to share with other DDS Routers (and ROS 2 nodes from their Docker Network) in the VPN network.
  • internal_partipant: the agent for interfacing with local ROS 2 nodes (talker/listener) using a Simple Discovery Protocol (the default one for DDS).
  • external_partipant: in which type: local-discovery-server means it uses Fast DDS Discovery Server for discovering ROS 2 services running on remote ROS 2 nodes (on hosts connected over the Internet using Husarnet VPN).

You can launch Fast DDS Discovery Server directly in the DDS Router! Just use the following config:

router-config.ds.yaml
allowlist:

echo_participant:
  type: echo

external_partipant:
  type: "local-discovery-server"
  id: 200

  listening-addresses:
    - domain: "dds-discovery-server"  # using hostname instead of Husarnet IPv6 addr
      port: 11811
      transport: "udp"

So there is no need for using additional .xml DDS participant config files.

Running a Demo​

I have created this GitHub repository to demonstrate how to use DDS Router on multiple hosts with Docker:

https://github.com/dominikn/fastdds-router-demo

In the next section I will show each step needed to launch it.

danger

Prerequisites

I use the latest compose file spec (eg. no version: "3.7" on top of. .yaml file and using compose.yaml instead of docker-compose.yaml). Therefore make sure you use Docker Compose V2.

I have tested the system on the following host:

$ docker --version
Docker version 20.10.10, build b485636
$ docker compose version
Docker Compose version v2.2.3

In the repo there is also a GitHub Actions workflow showing each step needed to run the system, and testing on 3 GitHub runners whether the system works as expected.

GitHub Actions workflow for testing a DDS Router

Clone​

Clone this repo on all hosts. They don't need to be in the same LAN (but of course they can).

git clone git@github.com:DominikN/fastdds-router-demo.git
cd fastdds-router-demo

Husarnet Join Code​

Your hosts don't need to be connected to Husarnet. It's enough to connect just a DDS Router container by running a Husarnet container in the same Docker Network. You need to use the same Husarnet Join Code on each host.

info

You will find your Husarnet Join Code on your account at Husarnet Dashboard:

  1. Log in to https://app.husarnet.com/
  2. Select or create a network
  3. Click [Add element] button and select a Join Code tab:

Husarnet Join Code

Create .env file with your Join Code:

JOINCODE=fc94:b01d:1803:8dd8:b293:5c7d:7639:932a/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Deploy​

Depending on the role of each host, launch one compose.*.yaml file with Docker Compose (you can run them all on the same host if you want):

Discovery-Server and DDS Router​

docker compose -f compose.ds.yaml up

Talker and DDS Router​

docker compose -f compose.talker.yaml up

Listener and DDS Router​

docker compose -f compose.listener.yaml up

Log​

After you launch these three Docker Compose deployments, you should see the similar output logs:

docker compose -f compose.ds.yaml up
...
Container fastdds-router-demo-husarnet_ds-1  Created
Container fastdds-router-demo-dds_router_ds-1  Created
Attaching to fastdds-router-demo-dds_router_ds-1, fastdds-router-demo-husarnet_ds-1
fastdds-router-demo-husarnet_ds-1    | [step 1/3] Waiting for Husarnet daemon to start
fastdds-router-demo-husarnet_ds-1    | ...
fastdds-router-demo-husarnet_ds-1    | done
fastdds-router-demo-husarnet_ds-1    |
fastdds-router-demo-husarnet_ds-1    | [step 2/3] Waiting for Base Server connection
fastdds-router-demo-husarnet_ds-1    | ...
fastdds-router-demo-dds_router_ds-1  | Waiting for "dds-discovery-server" host to be available in /etc/hosts
fastdds-router-demo-husarnet_ds-1    | ...
fastdds-router-demo-husarnet_ds-1    | ...
fastdds-router-demo-husarnet_ds-1    | done
fastdds-router-demo-husarnet_ds-1    |
fastdds-router-demo-husarnet_ds-1    | [step 3/3] Joining to Husarnet network
fastdds-router-demo-husarnet_ds-1    | [101439] joining...
fastdds-router-demo-husarnet_ds-1    | [103440] joining...
fastdds-router-demo-husarnet_ds-1    | [105441] done.
fastdds-router-demo-husarnet_ds-1    | Husarnet IP address: fc94:7b64:3ca6:8d3b:297b:4620:3e42:dfcf
fastdds-router-demo-dds_router_ds-1  | "dds-discovery-server" present in /etc/hosts:
fastdds-router-demo-dds_router_ds-1  | fc94:7b64:3ca6:8d3b:297b:4620:3e42:dfcf dds-discovery-server # managed by Husarnet
fastdds-router-demo-dds_router_ds-1  | Ready to launch ROS 2 nodes
fastdds-router-demo-dds_router_ds-1  | Starting DDS Router execution.
fastdds-router-demo-dds_router_ds-1  | Starting DDS Router.
fastdds-router-demo-dds_router_ds-1  | Periodic event raised. Reloading configuration.
...

As you can see we run 2 x talker and 2 x listener nodes, but only the one talker-listener pair can talk to each other over /chatter1 topic.

Summary​

Thanks to DDS Router you can choose which ROS 2 topics you want to share on your host, making them (and only them) available for other ROS 2 hosts. By combining it with a Husarnet p2p VPN it works over the Internet even if hosts are behind NATs, with no public IP.

The provided example on GitHub is a reference project showing you how to configure DDS Router and Husarnet to work properly.

COMMENTS

If you have any questions or want to discuss something related to this blog post, let's do it at Husarnet Community Forum.

Read also ...​