Engineering Commons^™

Cooperative Deep Q -Learning Framework For Environments Providing Image Feedback, Krishnan Raghavan, Vignesh Narayanan, Sarangapani Jagannathan

Publications

In this article, we address two key challenges in deep reinforcement learning (DRL) setting, sample inefficiency, and slow learning, with a dual-neural network (NN)-driven learning approach. In the proposed approach, we use two deep NNs with independent initialization to robustly approximate the action-value function in the presence of image inputs. In particular, we develop a temporal difference (TD) error-driven learning (EDL) approach, where we introduce a set of linear transformations of the TD error to directly update the parameters of each layer in the deep NN. We demonstrate theoretically that the cost minimized by the EDL regime is an approximation …

A New Economic Dispatch For Coupled Transmission And Active Distribution Networks Via Hierarchical Communication Structure, Wael T. El-Sayed, Ahmad Sa Awad, Maher Azzouz, Mostafa F. Shaaban

Electrical and Computer Engineering Publications

Traditionally, the economic dispatch problem (EDP) of the bulk generators connected to transmission networks (TNs) is solved in a centralized dispatching center (CDC) while modeling distribution networks as passive loads. With the increasing penetration levels of distributed generation, coordinating the economic dispatch between TNs and active distribution networks (ADNs) became vital to maximizing system efficiency. This article proposes a hierarchical communication structure, which requires minimal upgrades to the CDC, for solving the EDP of coupled TNs and ADNs. Based on the minimal data transfer between the CDC and distribution network operators, the problem is formulated and solved while considering the …

Multi-Layered Energy Management Framework For Extreme Fast Charging Stations Considering Demand Charges, Battery Degradation, And Forecast Uncertainties, Waqas Ur Rehman, Jonathan W. Kimball, Rui Bo

Electrical and Computer Engineering Faculty Research & Creative Works

To achieve a cost-effective and expeditious charging experience for extreme fast charging station (XFCS) owners and electric vehicle (EV) users, the optimal operation of XFCS is crucial. It is however challenging to simultaneously manage the profit from energy arbitrage, the cost of demand charges, and the degradation of a battery energy storage system (BESS) under uncertainties. This paper, therefore, proposes a multi-layered multi-time scale energy flow management framework for an XFCS by considering long- and short-term forecast uncertainties, monthly demand charges reduction, and BESS life degradation. In the proposed approach, an upper scheduling layer (USL) ensures the overall operation economy …

Energy-Efficient Multi-Rate Opportunistic Routing In Wireless Mesh Networks, Mohammad Ali Mansouri Khah, Neda Moghim, Nasrin Gholami, Sachin Shetty

VMASC Publications

Opportunistic or anypath routing protocols are focused on improving the performance of traditional routing in wireless mesh networks. They do so by leveraging the broadcast nature of the wireless medium and the spatial diversity of the network. Using a set of neighboring nodes, instead of a single specific node, as the next hop forwarder is a crucial aspect of opportunistic routing protocols, and the selection of the forwarder set plays a vital role in their performance. However, most opportunistic routing protocols consider a single transmission rate and power for the nodes, which limits their potential. To address this limitation, this …

Cooperative Deep $Q$ -Learning Framework For Environments Providing Image Feedback, Krishnan Raghavan, Vignesh Narayanan, Sarangapani Jagannathan

Electrical and Computer Engineering Faculty Research & Creative Works

In This Article, We Address Two Key Challenges in Deep Reinforcement Learning (DRL) Setting, Sample Inefficiency and Slow Learning, with a Dual-Neural Network (NN)-Driven Learning Approach. in the Proposed Approach, We Use Two Deep NNs with Independent Initialization to Robustly Approximate the Action-Value Function in the Presence of Image Inputs. in Particular, We Develop a Temporal Difference (TD) Error-Driven Learning (EDL) Approach, Where We Introduce a Set of Linear Transformations of the TD Error to Directly Update the Parameters of Each Layer in the Deep NN. We Demonstrate Theoretically that the Cost Minimized by the EDL Regime is an Approximation …

Continual Reinforcement Learning Formulation For Zero-Sum Game-Based Constrained Optimal Tracking, Behzad Farzanegan, Sarangapani Jagannathan

Electrical and Computer Engineering Faculty Research & Creative Works

This study provides a novel reinforcement learning-based optimal tracking control of partially uncertain nonlinear discrete-time (DT) systems with state constraints using zero-sum game (ZSG) formulation. To address optimal tracking, a novel augmented system consisting of tracking error and its integral value, along with an uncertain desired trajectory, is constructed. A barrier function (BF) with a tradeoff factor is incorporated into the cost function to keep the state trajectories to remain within a compact set and to balance safety with optimality. Next, by using the modified value functional, the ZSG formulation is introduced wherein an actor–critic neural network (NN) framework is …

Virtual Inertia Scheduling (Vis) For Real-Time Economic Dispatch Of Ibrs-Penetrated Power Systems, Buxin She, Fangxing Li, Hantao Cui, Jinning Wang, Qiwei Zhang, Rui Bo

Electrical and Computer Engineering Faculty Research & Creative Works

A New Concept Called Virtual Inertia Scheduling (VIS) is Proposed to Efficiently Handle the Increasing Penetration of Inverter-Based Resources (IBRs) in Power Systems. VIS is an Inertia Management Framework that Targets Security-Constrained and Economy-Oriented Inertia Scheduling and Generation Dispatch with a Large Scale of Renewable Generations. Specifically, It Determines the Appropriate Power Setting Points and Reserved Capacities of Synchronous Generators and IBRs, as Well as the Control Modes and Control Parameters of IBRs to Provide Secure and Cost-Effective Inertia Support. First, a Uniform System Model is Employed to Quantify the Frequency Dynamics of the IBRs-Penetrated Power Systems after Disturbances. Leveraging …

Generation Expansion Planning Considering Discrete Storage Model And Renewable Energy Uncertainty: A Bi-Interval Optimization Approach, Siyuan Wang, Guangchao Geng, Qy Jiang, Rui Bo

Electrical and Computer Engineering Faculty Research & Creative Works

Both discrete storage model (DSM) and continuous storage model (CSM) have been used in the power system planning literature. In this work, we conduct a sizing-error analysis for the use of CSM in generation expansion planning (GEP), which shows more reasonable storage sizing decisions are offered by the DSM in comparison to the CSM. However, when the DSM is considered in the context of interval optimization, the discrete status variables in mutually exclusive constraints and the strong temporal coupling in state-of-charge (SOC) constraints create significant challenges. To tackle this, a tailored interval optimization approach is proposed to consider both DSM …

Slides: The Green Climate Fund: Challenges And Opportunities: Some Thoughts On How The Green Climate Fund Could Close The Energy Justice Gap, Martin Hiller

2012 Energy Justice Conference and Technology Exposition (September 17-18)

Presenter: Martin Hiller, Director‐General, Renewable Energy and Energy Efficiency Partnership (REEEP), Vienna, Austria

22 slides

Parameter Optimization For Image Denoising Based On Block Matching And 3d Collaborative Filtering, Ramu Pedada, Emin Kugu, Jiang Li, Zhanfeng Yue, Yuzhong Shen, Josien P.W. Pluim (Ed.), Benoit M. Dawant (Ed.)

Electrical & Computer Engineering Faculty Publications

Clinical MRI images are generally corrupted by random noise during acquisition with blurred subtle structure features. Many denoising methods have been proposed to remove noise from corrupted images at the expense of distorted structure features. Therefore, there is always compromise between removing noise and preserving structure information for denoising methods. For a specific denoising method, it is crucial to tune it so that the best tradeoff can be obtained. In this paper, we define several cost functions to assess the quality of noise removal and that of structure information preserved in the denoised image. Strength Pareto Evolutionary Algorithm 2 (SPEA2) …

An Autonomous And Adaptable Wireless Device For Flood Monitoring, Valerio Plessi, Filippo Bastianini, Sahra Sedigh

Electrical and Computer Engineering Faculty Research & Creative Works

Wireless devices can be used to monitor and record a broad range of phenomena. Their advantages include ease of installation and maintenance and considerable reduction in wiring costs. The addition of battery power and radio communication to such wireless devices can result in a completely The operating environment of monitoring systems is often hostile, due to temperature fluctuations, humidity, electromagnetic noise, and other interfering phenomena. The system should be able to adapt to changing conditions to maintain dependability in its operations This paper presents the case study of adapting a flood detection device to the environmental threat of submersion.

A General Purpose Framework For Wireless Sensor Network Applications, Ayman Z. Faza, Sahra Sedigh

Electrical and Computer Engineering Faculty Research & Creative Works

Wireless sensor networks are becoming a basis for a rapidly increasing range of applications. Habitat, flood, and wildfire monitoring are interesting examples of such applications. Each application has different requirements in terms of node functionalities, network size, complexity and cost; therefore, it is worthwhile time investment to design and implement a general purpose framework for wireless sensor networks that would be adaptable to any monitoring application of interest with a minimum amount of effort. In this manuscript, we propose a basic structure for such a framework and highlight a number of challenges anticipated during the course of this doctoral research.

Non-Unity Active Pfc Methods For Filter Size Optimization, Yongxiang Chen, Jonathan W. Kimball, Philip T. Krein

Electrical and Computer Engineering Faculty Research & Creative Works

Active power factor correction seeks to obtain unity power factor and sinusoidal line currents. Optimized nonsinusoidal line currents reduce filter capacitor requirements with a nonunity target power factor. Implementation methods are presented that permit reduced power factor to be traded off against filter size in a nearly optimum manner. A simple waveform shape can reduce filter component size by about 40% in active PFC converters at the same level of complexity as in conventional PFC designs while yielding power factor as high as 0.9. Two approximate methods to generate appropriate shapes are presented. They offer direct practical implementation of nonunity …

Machine Design Considerations For The Future Energy Challenge, Jonathan W. Kimball, Marco Amrhein

Electrical and Computer Engineering Faculty Research & Creative Works

Motors consume a significant fraction of electricity in the United States and in the world. As part of the International Future Energy Challenge, student teams are endeavoring to improve the efficiency of fractional-horsepower machines. The present work summarizes the motor design and construction process for a 500 W prototype induction machine targeting efficiency above 80%. Analytical and finite-element results are shown.

A Graph-Based Model For Component-Based Software Development, Sahra Sedigh, Arif Ghafoor

Electrical and Computer Engineering Faculty Research & Creative Works

Software metrics can be used to objectively quantify the quality of software components and systems, alleviating quality and risk concerns and raising assurance in component-based systems. In this paper, we present a graph-based model for component-based software development. We assume that a number of components have been characterized in terms of non-functional metrics of importance to the software system being developed, and that the interfaces connecting various components have been similarly characterized. The emphasis of this work is on cost and quality of the system under development, and reaching an acceptable compromise between the two.

A Fast Full-Search Adaptive Vector Quantizer For Video Coding, Scott E. Budge, Christian B. Peel

Electrical and Computer Engineering Faculty Publications

This paper presents a novel VQ structure which provides very good quality encoding for video sequences and exploits the computational savings gained from a fast-search algorithm. It uses an adaptive-search, variable-length encoding method which allows for very fast matching of a wide range of transmission rates. Both the encoding quality and the computational benefits from the fast-search algorithm are presented. Simulations show that full-search tree residual VQ (FTRVQ) can provide up to 3 dB improvement over a similar RVQ encoder on video sequences.

Locally Optimal, Buffer-Constrained Motion Estimation And Mode Selection For Video Sequences, C. B. Peel, Scott E. Budge, K. Liang, C.-M. Huang

Electrical and Computer Engineering Faculty Publications

We describe a method of using a Lagrange multiplier to make a locally optimal trade off between rate and distortion in the motion search for video sequences, while maintaining a constant bit rate channel. Simulation of this method shows that it gives up to 3.5 dB PSNR improvement in a high motion sequence. A locally rate-distortion (R-D) optimal mode selection mechanism is also described. This method also gives significant quality benefit over the nominal method. Though the benefit of these techniques is significant when used separately, when the optimal mode selection is combined with the R-D optimal motion search, it …

Variable-Complexity Trellis Decoding Of Binary Convolutional Codes, David W. Matolak, S. G. Wilson

Faculty Publications

Considers trellis decoding of convolutional codes with selectable effort, as measured by decoder complexity. Decoding is described for single parent codes with a variety of complexities, with performance "near" that of the optimal fixed receiver complexity coding system. Effective free distance is examined. Criteria are proposed for ranking parent codes, and some codes found to be best according to the criteria are tabulated, Several codes with effective free distance better than the best code of comparable complexity were found. Asymptotic (high SNR) performance analysis and error propagation are discussed. Simulation results are also provided.

Fast Thermal Generation Rescheduling, F. Eugenio Villaseca, B. Fardanesh

Electrical and Computer Engineering Faculty Publications

A new dynamic programming algorithm for fast rescheduling thermal generation is presented. The savings in computational times are brought about by the introduction of two new techniques: the variable truncation dynamic programming and the limitation of the solution space to be searched. Several examples on a 20 machine system are used to illustrate the application of the algorithm and to show that optimal solutions are obtained at significantly reduced computational times.