Ruo Yu (David) Tao
Computer Science Ph.D. Candidate @ Brown
Email: ruoyutao "at" brown "dot" edu
CV: here
I'm a PhD candidate researching reinforcement learning in the Intelligent Robot Lab at Brown University advised by George Konidaris. My current research interests and work are on agent-state and representation learning in partially observable environments for reinforcement learning. I also think eligibility traces are super neat.
Previously, I was a Computing Science M.Sc. (Thesis) student at the University of Alberta, co-advised by Adam White and Marlos C. Machado in the RLAI lab. Even further in the distant past, I was a First Class Honours and Dean's Honour List student studying Computer Science at McGill University for my B.Sc. I was previously a research intern at both Mila and Microsoft Research.
If you're interested in collaborating, feel free to send me an email.
Publications
Benchmarking Partial Observability in RL with a Suite of Memory-Improvable Domains
*joint first author
To appear in The Second Annual Reinforcement Learning Conference (RLC), August 2025
Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy
*joint first author
Neural Information Processing Systems (NeurIPS), 2024
Auxiliary Inputs for Agent-State Construction
In Transactions on Machine Learning Research (TMLR), 2023
Measuring and Mitigating Interference in Reinforcement Learning
Conference on Lifelong Learning Agents (CoLLAs), 2023
Novelty Search in Representational Space for Sample Efficient Exploration
Selected for oral presentation. In Advances in Neural Information Processing Systems (NeurIPS), 2020
Towards Solving Text-Based Games by Producing Adaptive Action Spaces
Presented at the WordPlay Workshop at Neural Information Processing Systems (NeurIPS), 2018
TextWorld: A Learning Environment for Text-Based Games
Presented at the Computer Games Workshop at International Joint Conferences on Artificial Intelligence (IJCAI), 2018
Education
McGill University
Hons. B.Sc., 2016 - 2020
Undergraduate research advisor: Joelle Pineau.
First Class Honors, Dean's List.
Experience
TikTok
In the summer of 2025, I was part of the Responsible Recommender Systems team at TikTok as a research intern, working on improving reinforcement learning algorithms for large language models.
National University of Singapore (NUS)
Over the summer of 2020 I was working on a research project advised by Lee Wee Sun on neural-based memory for RL and SLAM.
Mila (Quebec AI Institute)
At Mila, I worked on exploration in a model-based reinforcement learning setting. This was during my final year of my undergraduate degree at McGill. The project was mentored by Vincent François-Lavet and advised by Joelle Pineau. Our paper on this was accepted to NeurIPS 2020 for an oral presentation.
Microsoft Research Montreal
During my time at Microsoft Research, I worked on reinforcement learning and natural language problems. I was advised by Marc-Alexandre Côté on projects including:
-
Adaptive text action spaces: see the above paper.
-
TextWorld: a reinforcement learning framework for agents to learn text-based games.
Articles
-
After my tumultuous time applying for and deciding on a PhD program, I wrote a short guide on how to successfully apply for a CS PhD.
-
I wrote a short piece on the dangers of reshaping in tensor manipulation libraries (PyTorch in this case).
Miscellaneous things about me
-
I have a cooking blog you should check out!
-
In a past life (and technically still), I was a lieutenant in the Singapore Armed Forces and a Combat Engineer by vocation.
-
I spent my formative years in Beijing, and still call it home.
-
I love to read.
-
I love climbing: especially outdoor sport climbing.
Contact
GithubGoogle Scholar
Come see me out people on my Twitter
Need a book recommendation? Check out my Goodreads
