Dhruvesh Patel
dhruveshpate@umass.edu
I am a Computer Science PhD Researcher at UMass Amherst, advised by Prof. Andrew McCallum at the Information Extraction and Synthesis Laboratory, and a Visiting Researcher at IBM Research. My research focuses on generative modeling for discrete sequences, especially alternatives to left-to-right language modeling. Before UMass, I completed my undergraduate and first master’s degree at IIT Madras, where I worked on robotics research with Prof. Sandipan Bandyopadhyay.
I have also been fortunate to work with collaborators across industry research labs, including Meta Reality Labs and Abridge AI. Before graduate school, I spent two years as a software engineer at MathWorks and a year collaborating with Prof. Partha Talukdar on applied NLP problems.
CV available at the bottom of this page.
research
Most language models generate text one token at a time, from left to right. I am interested in models that can draft, revise, infill, and reason over text in more flexible ways. My current work focuses on probabilistic models for non-autoregressive sequence generation, with an emphasis on making generation faster and more controllable.
I am especially interested in how to make these alternatives practical at scale: adapting pre-trained autoregressive LLMs, designing efficient non-autoregressive pre-training objectives, and improving sampling for discrete diffusion models.
Much of my earlier work studies the same question from a more fundamental angle: how should neural models represent, score, and search over structured discrete spaces? This includes structured prediction with energy-based models, geometric representations such as box embeddings, and models for label spaces, hierarchies, and relational structure.
Together with Benjamin Rozonoyer, I run dIESL, a reading and working group on non-autoregressive LLMs at IESL.
affiliations and internships
news
| Jun 23, 2026 | The new dIESL page is up — our reading and working group on non-autoregressive LLMs at IESL. |
|---|---|
| May 15, 2026 | Learned Relay Representations for Forward-Thinking Discrete Diffusion Models was accepted at the ICML 2026 Workshop on Foundations of Deep Generative Models! |
| May 1, 2026 | Insertion Based Sequence Generation with Learnable Order Dynamics was accepted at ICML 2026! |
| Mar 10, 2026 | xLM: A Python Package for Non-Autoregressive Language Models was accepted at the EACL 2026 System Demonstrations track! |
| Jan 15, 2026 | A Continuous-Time Markov Chain Framework for Insertion Language Models was accepted as a spotlight paper (top 6%) at AISTATS 2026! |
| Oct 1, 2025 | I will be presenting Improved Sampling from Masked Diffusion Models with Position Contrastive Guidance at the Structured Probabilistic Inference and Generative Models workshop at NeurIPS 2025. |
| Jun 1, 2025 | Work on Insertion Language Models (ILMs) is out on arXiv! It will be presented at the Structured Probabilistic Inference and Generative Models workshop at NeurIPS 2025. |
| Oct 1, 2024 | Learning Representations for Hierarchies with Minimal Support was accepted at NeurIPS 2024! |
| Apr 1, 2024 | Language Guided Exploration for RL Agents in Text Environments was accepted at NAACL (findings) 2024. |
| Aug 1, 2023 | My work on Pre-trained language models for Visual Planning for Human Assistance, done as a research intern at Meta Reality Labs., has been accepted at ICCV 2023. |
| Sep 7, 2022 | Super excited to start my internship at Meta Reality Labs! |
| Apr 25, 2022 | Excited to present our work on multi-label classification using box embeddings at ICLR 2022! |
| Nov 1, 2020 | Happy to announce that I will be starting my Ph.D. in Spring (January) 2021 at UMass Amherst with Prof. Andrew McCallum as my advisor. |
| Oct 1, 2020 | Internship work done at Abridge AI is accepted at Clinical NLP workshop 2020. |
| Sep 30, 2020 | Paper titled Reading Comprehension as Natural Language Inference: A Semantic Analysis is accepted at *SEM 2020. |
| May 10, 2020 | Excited to start research internship at Abridge AI. |
| Jan 15, 2020 | Paper titled “Representing Joint Hierarchies using Box Embeddings” is accepted at AKBC 2020. |
mentors and collaborators
I have been fortunate to work with many amazing people over the years. Here are my mentors and collaborators.
