The main premise of this paper is that, rather than looking only in the communication, one has to consider, in an integrated way, three operations: sampling, communication, and prediction. This is plausible to improve the performance indicators of synchronization. In doing this, a key observation is that a good prediction decreases the data quantity that is required to be communicated. The authors introduce a co-design framework that includes sampling, communication, and prediction. The framework includes an algorithm that combines Deep Reinforcement Learning (DRL) techniques with expert knowledge on sampling, communication, and prediction. The approach is evaluated through an actual prototype of a robotic arm, which brings an additional credibility to the proposed approach.    

JSAC: In the paper you make certain assumptions in the communication model about synchronization, slotted structure, or similar. Which practical scenarios would challenge this model? How would the model change if you assume bandwidth-intensive transmissions, such as video streaming along the trajectory?

Authors: In this paper, we took a standard orthogonal frequency division multiplexing (OFDM) communication model to conduct the cross-system design among sampling, communication and prediction. An ideal assumption in our work is that the communication delay is bounded and known by the system.

In practical scenarios, the upper bound of communication delay may not be available to the system. The wireless link is part of the whole communication system, and the latency and jitter in backhaul and core networks heavily rely on the specific scenario. Although we showed in the paper that our system can be easily extended to some other communication models, we need to fine-tune the pre-trained deep reinforcement learning algorithm in a new scenario.

For bandwidth-intensive transmissions, our approach can reduce the communication load and thus can save bandwidth if the state of the system is predictable. Let’s take virtual reality (VR) as an example. In VR video streaming, a head-mounted device (e.g., VR glasses) needs to transmit the trajectory of human user to the access point, and the access point sends the video within the field-of-view to be requested by the user. Since the trajectory of the user is predictable, our framework can be used to predict the future trajectory and the corresponding field-of-view. In this way, we can reduce the required bandwidth for VR streaming.

JSAC: What was criticized by the reviewers and how did you address it?

Authors: The reviewers’ comments were generally positive. The reviewers paid particular attention to how the expert knowledge is adopted to assist the constrained deep reinforcement learning algorithm. Thanks to this comment, we were able to better sort out the role of expert knowledge as an aid for training. To address this comment, we discussed the impacts of different expert knowledge on the training performance of the proposed knowledge-assisted constrained twin-delayed deep deterministic algorithm (KT-TD3). In addition, we showed that the proposed strategy with full expert knowledge has the best performance in terms of stability and can meet the average tracking error constraint.

JSAC: What are the main communication challenges, not necessarily wireless, you are seeing on the way forward towards enabling fusion of digital with the physical world, such as in the Metaverse?

Authors: There are several communication challenges in enabling the fusion of the digital and physical worlds. One of them is to define new key performance indicators (KPIs) (e.g., motion-to-photon (MTP) latency in human-avatar interaction) for Metaverse applications. Existing communication KPIs are not sufficient to support emerging applications in the Metaverse.

Scalability is another issue to both wireless and wired communications. As the communication, computing, and storage resources are limited, it is difficult to support a large number of users or devices with diverse applications and human-computer interfaces in one communication network. The integration of our sampling, communication, and prediction framework and different types of human-computer interfaces or devices is challenging.

Another challenge is the design approach. Most of the existing communication design and optimization methods are independent of sensing, computing, and robotics systems. Without the domain knowledge and dynamics of these applications, we can only obtain sub-optimal solutions with high communication and computing overhead. Thus, the task-oriented cross-system design would be the approach to tailor the communication system for specific use cases and applications.