Diagnosing Non-Intermittent Anomalies in Reinforcement Learning Policy Executions (Short Paper)

# Diagnosing Non-Intermittent Anomalies in Reinforcement Learning Policy Executions (Short Paper) > OpenAlex Metadata Hub · https://openalex.org/W2736601468 ## Bibliographic - **DOI:** 10.4230/oasics.dx.2024.16 - **Year:** 2017 - **Citations:** 11334 - **Open Access:** Yes (green) - **License:** — - **Source:** https://arxiv.org/pdf/1707.06347 ## Authors - Coursey, Austin - Quinones-Grueiro, Marcos - Biswas, Gautam ## Abstract Due to the safety risks and training sample inefficiency, it is often preferred to develop controllers in simulation. However, minor differences between the simulation and the real world can cause a significant sim-to-real gap. This gap can reduce the effectiveness of the developed controller. In this paper, we examine a case study of transferring an octorotor reinforcement learning controller from simulation to the real world. First, we quantify the effectiveness of the real-world transfer by examining safety metrics. We find that although there is a noticeable (around 100%) increase in deviation in real flights, this deviation may not be considered unsafe, as it will be within > 2m safety corridors. Then, we estimate the densities of the measurement distributions and compare the Jensen-Shannon divergences of simulated and real measurements. From this, we show that the vehicle’s orientation is significantly different between simulated and real flights. We attribute this to a different flight mode in real flights where the vehicle turns to face the next waypoint. We also find that the reinforcement learning controller actions appear to correctly counteract disturbance forces. Then, we analyze the errors of a measurement autoencoder and state transition model neural network applied to real data. We find that these models further reinforce the difference between the simulated and real attitude control, showing the errors directly on the flight paths. Finally, we discuss important lessons learned in the sim-to-real transfer of our controller. ## Keywords Computer science, Optimization algorithm, Algorithm, Mathematical optimization, Mathematics ## Concepts - Computer science - Optimization algorithm - Algorithm - Mathematical optimization - Mathematics --- *Metadata only — full text not imported unless Open Access license permits.*

Bài “Diagnosing Non-Intermittent Anomalies in Reinforcement Learning Policy Executions (Short Paper)” được TradingBase chuyển thành Knowledge Product cho trader — không phải trang đọc abstract OpenAlex. Tóm lược học thuật (đã diễn giải): Due to the safety risks and training sample inefficiency, it is often preferred to develop controllers in simulation. However, minor differences between the simulation and the real world can cause a significant sim-to-real gap. This gap can reduce the effectiveness of the developed controller. In this paper, we examine a case study of transferring an octorotor reinforcement learning controller from simulation to the real world. First, we quantify the effectiveness of the real-world transfer by examining safety metrics. We find that although there is a noticeable (around 100%) increase in deviation in real flights, this deviation may not be considered unsafe, as it will be within > 2m safety corridors. Then, we estimate the densities of the measurement distributions and compare the Jensen-Shannon divergences of simulated and real measurements. From this, we show that the vehicle’s orienta… Phần Trading Insights bên dưới nối nghiên cứu với Forex, vàng, USD, lãi suất và risk regime — để bạn đưa vào journal và playbook. Metadata DOI/OA chỉ là rail tham chiếu; nội dung chính là summary, takeaways và ứng dụng thị trường do Content Factory sinh.