Profile
| Keywords: | Control, Power electronics, Distributed systems |
FES Funded ProjectsOutputs
| Title |
Category |
Date |
Authors |
| Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in RoboticsAlthough Deep Reinforcement Learning (DRL) has achieved notable success in numerous robotic applications, designing a high-performing reward function remains a challenging task that often requires substantial manual input. Recently, Large Language Models (LLMs) have been extensively adopted to address tasks demanding in-depth common-sense knowledge, such as reasoning and planning. Recognizing that reward function design is also inherently linked to such knowledge, LLM offers a promising potential in this context. Motivated by this, we propose in this work a novel LLM framework with a self-refinement mechanism for automated reward function design. The framework commences with the LLM formulating an initial reward function based on natural language inputs. Then, the performance of the reward function is assessed, and the results are presented back to the LLM for guiding its self-refinement process. We examine the performance of our proposed framework through a variety of continuous robotic control tasks across three diverse robotic systems. The results indicate that our LLM-designed reward functions are able to rival or even surpass manually designed reward functions, highlighting the efficacy and applicability of our approach.T13-Q02 University of Alberta | Publication | 2024-02-19 | Jiayang Song, Zhehua Zhou, "Jiawei Liu ", "Chunrong Fang ", Shu, Z., Lei Ma | | ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task PlanningMotivated by the substantial achievements observed in Large Language Models (LLMs) in the field of natural language processing, recent research has commenced investigations into the application of LLMs for complex, long-horizon sequential task planning challenges in robotics. LLMs are advantageous in offering the potential to enhance the generalizability as task-agnostic planners and facilitate flexible interaction between human instructors and planning systems. However, task plans generated by LLMs often lack feasibility and correctness. To address this challenge, we introduce ISR-LLM, a novel framework that improves LLM-based planning through an iterative self-refinement process. The framework operates through three sequential steps: preprocessing, planning, and iterative self-refinement. During preprocessing, an LLM translator is employed to convert natural language input into a Planning Domain Definition Language (PDDL) formulation. In the planning phase, an LLM planner formulates an initial plan, which is then assessed and refined in the iterative self-refinement step by using a validator. We examine the performance of ISR-LLM across three distinct planning domains. The results show that ISR-LLM is able to achieve markedly higher success rates in task accomplishments compared to state-of-the-art LLM-based planners. Moreover, it also preserves the broad applicability and generalizability of working with natural language instructions.T13-Q02 University of Alberta | Publication | 2024-01-29 | Zhehua Zhou, Jiayang Song, "Kunpeng Yao ", Shu, Z., Lei Ma | | Towards Building AI-CPS with NVIDIA Isaac Sim: An Industrial Benchmark and Case Study for Robotics ManipulationAs a representative cyber-physical system (CPS), robotic manipulator has been widely adopted in various academic research and industrial processes, indicating its potential to act as a universal interface between the cyber and the physical worlds. Recent studies in robotics manipulation have started employing artificial intelligence (AI) approaches as controllers to achieve better adaptability and performance. However, the inherent challenge of explaining AI components introduces uncertainty and unreliability to these AI-enabled robotics systems, necessitating a reliable development platform for system design and performance assessment. As a foundational step towards building reliable AI-enabled robotics systems, we propose a public industrial benchmark for robotics manipulation in this paper. It leverages NVIDIA Omniverse Isaac Sim as the simulation platform, encompassing eight representative manipulation tasks and multiple AI software controllers. An extensive evaluation is conducted to analyze the performance of AI controllers in solving robotics manipulation tasks, enabling a thorough understanding of their effectiveness. To further demonstrate the applicability of our benchmark, we develop a falsification framework that is compatible with physical simulators and OpenAI Gym environments. This framework bridges the gap between traditional testing methods and modern physics engine-based simulations. The effectiveness of different optimization methods in falsifying AI-enabled robotics manipulation with physical simulators is examined via a falsification test. Our work not only establishes a foundation for the design and development of AI-enabled robotics systems but also provides practical experience and guidance to practitioners in this field, promoting further research in this critical academic and industrial domain.T13-Q02 University of Alberta | Publication | 2024-01-04 | Zhehua Zhou, Jiayang Song, Xuan Xie, Shu, Z., Lei Ma, "Dikai Liu ", "Jianxiong Yin ", "Simon See " | | Distributed control of DC microgrids: A relaxed upper bound for constant power loadsMotivated by the increasing interest in DC microgrids, we study the distributed secondary control problem of DC microgrids which aims to simultaneously guarantee load current sharing and DC bus voltage restoration. In particular, we analyze in-depth the effects of nonlinear constant power loads (CPLs), and successfully establish an upper bound for CPLs to guarantee the stability of DC microgrids. This established bound is less conservative than existing counterparts, allowing the connected power of CPLs to be much larger than that of linear resistive loads. Moreover, this bound explicitly reveals the connections between CPLs and droop gains, i.e., the upper bound of CPLs can be increased by decreasing droop gains, and vice versa. Three case studies are provided to verify the established results.T06-Q07 University of Alberta | Publication | 2025-03-25 | "Lantao Xing ", Shu, Z., "Jingya Fang ", "Changyun Wen ", "Chenghui Zhang " | | Distributed Secondary Control for DC Microgrids with Near-Infinite Constant Power Load AccommodationIn DC microgrids, constant power loads (CPLs) inherently exhibit negative impedance characteristics, which are widely believed to degrade system stability as their penetration level increases. Consequently, extensive research has aimed to establish safe upper bounds for CPL penetration. However, these upper bounds are typically derived as sufficient conditions, making them overly conservative. Moreover, when multiple DC sources are connected in parallel to a common DC bus, the simultaneous need for current sharing and DC bus voltage regulation further complicates system control. To address these challenges, this paper proposes a novel distributed secondary control method based on the dynamic averaging of virtual voltage drops (VVDs). The proposed method offers two key advantages: 1) It ensures both precise current sharing and voltage regulation in single-bus DC microgrids, even in the presence of mixed ZIP loads, i.e., constant impedance loads (Z), constant current loads (I), and constant power loads (P). 2) Unlike existing approaches that impose conservative limits on CPL penetration, the proposed method theoretically demonstrates that the safe upper bound for CPLs can be arbitrarily large, enabling the DC microgrid to accommodate an almost infinite number of CPLs without compromising stability. Both Simulation and experiment studies are conducted to validate the effectiveness of the proposed method.T06-Q07 University of Alberta | Publication | 2026-06-02 | "Zhiyong Liu ", "Lantao Xing ", "Jingyang Fang ", Shu, Z., "Hongye Su " |
|
|
