Summary
In model based meta-reinforcement learning, the agent first learns to learn from a distribution of tasks, then adapts to a new task with a as few steps as possible. During this adaptation phase, vanilla algorithms typically implement random exploration which can be inefficient in determining an accurate world model. Additionally, in highly random environments, accurately differentiating between different states in the latent space can be challenging. To address the issues, I am working on HyPE-LD with Dr. YeXiang Xue and Ph.D. student Maxwell Jacobson. HyPE-LD is an action-planning algorithm which models states as distributions in the latent space. By comparing the distributions of different states in different models, HyPE-LD aims to more efficiently reduce the uncertainty in the world model and improve the agent’s ability to adapt to new tasks in highly random environments.
Note: Because the project is still in development, the repository is not available to the public per my employer’s instruction.
Summary
At Purdue’s Digital Enterprise Center, we are developing a manufacturing workflow which integrates with IoT tools, computer vision, and Solumina MES to streamline the assembly process. To showcase this workflow, we are assembling a commercial oil pump over three workstations that will each be responsible for a subset of the assembly steps. As a part of this project, I am building software to perform computer vision aided foreign object debris detection, assembly verification and comprehensive data collection which promotes efficiency, consistency, and traceability of the assembly process.
Notable Work
- For a faster training pipeline, I leveraged image augmentation to increase our dataset by up to 64 times, allowing us to efficiently add new objects of interest to our set of foreign objects. I also streamlined the development and new-hire integration process by creating a CI/CD pipeline using the pdoc library alongside GitHub Workflow, enabling automated building and deployment of a documentation page for the data wrangling and model training scripts.
- Using Rust, Tauri, and Svelte, I developed a desktop application prototype for multi-camera selection and real-time object detection. Starting with example object detection code from the official YOLOv8 repository, I improved performance 5x by parallelizing pre and post-processing using Rayon.
- Re-wrote legacy Python code, correcting major issues like unblocked concurrent file access. Replaced inefficient system calls that launched new command-line processes with lightweight threading for improved performance and resource management.
- Currently working on integrating the computer vision pipeline with Atlas Copco’s PF6000 controller for real-time feedback on the assembly process. This will allow us to detect and correct errors in real-time, reducing the need for manual inspection and rework. Assembly data is stored in Solumina MES via its native REST API for enhanced traceability and analysis.