Summary

In model based meta-reinforcement learning, the agent first learns to learn from a distribution of tasks, then adapts to a new task with a as few steps as possible. During this adaptation phase, vanilla algorithms typically implement random exploration which can be inefficient in determining an accurate world model. Additionally, in highly random environments, accurately differentiating between different states in the latent space can be challenging. To address the issues, I am working on Hypothesis Network Planned Exploration with Latent Distributions (HyPE-LD) with Dr. YeXiang Xue and Ph.D. student Maxwell Jacobson. HyPE-LD is an action-planning algorithm which models states as distributions in the latent space. The algorithm gathers multiple sequences of state distributions by following a set of actions on multiple available models. By comparing these sequences, HyPE-LD aims to more efficiently deduce the most accurate world model, thereby improving the agent’s ability to quickly adapt to new tasks in highly random environments.

Existing Work

My contributions build upon the existing Hypothesis Network Planned Exploration (HyPE) algorithm which is detailed in Hypothesis Network Planned Exploration for Rapid Meta-Reinforcement Learning Adaptation. HyPE takes a pool of models and strategically plans an exploration path to deduce which model is most accurate for the current task. This is very similar to how the scientific method works: by forming a hypothesis, testing it, and then refining the hypothesis based on the results. However, HyPE’s exploration was tested on an entirely deterministic environment. My method extends the idea of HyPE to stochastic environments.

Note: Because the project is still in development, the repository is not available to the public per my employer’s instruction.

Summary

At Purdue’s Digital Enterprise Center, we are developing a manufacturing workflow which integrates with IoT tools, computer vision, and Solumina MES to streamline the assembly process. To showcase this workflow, we are assembling a commercial oil pump over three workstations that will each be responsible for a subset of the assembly steps. As a part of this project, I am building software to perform computer vision aided foreign object debris detection, assembly verification and comprehensive data collection which promotes efficiency, consistency, and traceability of the assembly process.

Notable Work

• For a faster training pipeline, I leveraged image augmentation to increase our dataset by up to 64 times, allowing us to efficiently add new objects of interest to our set of foreign objects. I also streamlined the development and new-hire integration process by creating a CI/CD pipeline using the pdoc library alongside GitHub Workflow, enabling automated building and deployment of a documentation page for the data wrangling and model training scripts.
• Using Rust, Tauri, and Svelte, I developed a desktop application prototype for multi-camera selection and real-time object detection. Starting with example object detection code from the official YOLOv8 repository, I improved performance 5x by parallelizing pre and post-processing using Rayon.
• Re-wrote legacy Python code, correcting major issues like unblocked concurrent file access. Replaced inefficient system calls that launched new command-line processes with lightweight threading for improved performance and resource management.
• Currently working on integrating the computer vision pipeline with Atlas Copco’s PF6000 controller for real-time feedback on the assembly process. This will allow us to detect and correct errors in real-time, reducing the need for manual inspection and rework. Assembly data is stored in Solumina MES via its native REST API for enhanced traceability and analysis.