Dhruvesh Patel

I am a Computer Science PhD Researcher at UMass Amherst, advised by Prof. Andrew McCallum at the Information Extraction and Synthesis Laboratory, and an External Visiting Researcher at IBM Research. My research focuses on generative modeling for discrete sequences, especially alternatives to left-to-right language modeling. Before UMass, I completed my undergraduate and first master's degree at IIT Madras, where I worked on robotics research with Prof. Sandipan Bandyopadhyay.

I have also been fortunate to work with collaborators across industry research labs, including IBM Research, Meta Reality Labs and Abridge AI. Before graduate school, I spent two years as a software engineer at MathWorks and a year collaborating with Prof. Partha Talukdar on applied NLP problems.

Download CV Google Scholar GitHub Twitter LinkedIn

Research

Most language models generate text one token at a time, from left to right. I am interested in models that can draft, revise, infill, and reason over text in more flexible ways. My current work focuses on probabilistic models for non-autoregressive sequence generation, with an emphasis on making generation faster and more controllable.

I am especially interested in how to make these alternatives practical at scale: adapting pre-trained autoregressive LLMs, designing efficient non-autoregressive pre-training objectives, and improving sampling for discrete diffusion models.

Much of my earlier work studies the same question from a more fundamental angle: how should neural models represent, score, and search over structured discrete spaces? This includes structured prediction with energy-based models, geometric representations such as box embeddings, and models for label spaces, hierarchies, and relational structure.

Together with Benjamin Rozonoyer, I host dIESL, a reading and working group on non-autoregressive LLMs at IESL.

Selected Publications

View all publications →

Affiliations & Internships

Sep 2025–present IBM Research — External Visiting Researcher
May 2025–Aug 2025 IBM Research — Research Intern
Sep 2022–Dec 2022 Meta Reality Labs — Research Intern
May 2020–Dec 2020 Abridge AI — Research Intern
May 2019–present IESL, UMass Amherst — Graduate Researcher
Jan 2021–present CICS, UMass Amherst — PhD Student
Jan 2019–Dec 2020 CICS, UMass Amherst — MS Student
Jun 2016–Jan 2018 MathWorks — Software Engineer
Jun 2011–Jan 2016 IIT Madras — Undergraduate + Master's

News

Jul 8, 2026 Presenting xLM at the Bridging Research and Open Source social at ICML 2026 in Seoul!
Jun 23, 2026 The new dIESL page is up — our reading and working group on non-autoregressive LLMs at IESL.
May 15, 2026 Learned Relay Representations for Forward-Thinking Discrete Diffusion Models was accepted at the ICML 2026 Workshop on Foundations of Deep Generative Models!
May 1, 2026 Insertion Based Sequence Generation with Learnable Order Dynamics was accepted at ICML 2026!
Mar 10, 2026 xLM: A Python Package for Non-Autoregressive Language Models was accepted at the EACL 2026 System Demonstrations track!
Jan 15, 2026 A Continuous-Time Markov Chain Framework for Insertion Language Models was accepted as a spotlight paper (top 6%) at AISTATS 2026!
Oct 1, 2025 Presenting Improved Sampling from Masked Diffusion Models with Position Contrastive Guidance at the SPIGM workshop at NeurIPS 2025.
Jun 1, 2025 Work on Insertion Language Models (ILMs) is out on arXiv! It will be presented at the SPIGM workshop at NeurIPS 2025.
Oct 1, 2024 Learning Representations for Hierarchies with Minimal Support was accepted at NeurIPS 2024!
Apr 1, 2024 Language Guided Exploration for RL Agents in Text Environments was accepted at NAACL (findings) 2024.
Aug 1, 2023 My work on Pre-trained language models for Visual Planning for Human Assistance, done as a research intern at Meta Reality Labs, has been accepted at ICCV 2023.
Sep 7, 2022 Super excited to start my internship at Meta Reality Labs!
Apr 25, 2022 Excited to present our work on multi-label classification using box embeddings at ICLR 2022!
Nov 1, 2020 Happy to announce that I will be starting my Ph.D. in Spring (January) 2021 at UMass Amherst with Prof. Andrew McCallum as my advisor.
Oct 1, 2020 Internship work done at Abridge AI is accepted at the Clinical NLP workshop 2020.
Sep 30, 2020 Reading Comprehension as Natural Language Inference: A Semantic Analysis is accepted at *SEM 2020.
May 10, 2020 Excited to start a research internship at Abridge AI.
Jan 15, 2020 Paper titled Representing Joint Hierarchies using Box Embeddings is accepted at AKBC 2020.

Talks

Aug 21, 2026 Invited Towards More Flexible and Efficient Non-Autoregressive Language Models — Cerebras Seminar Series
Aug 17, 2026 Invited Non-Autoregressive Sequence Generation — IBM AI Foundations Technical Forum
Jul 8, 2026 Invited xLM: A Python Package for Non-Autoregressive Language Models — Bridging Research and Open Source social, ICML 2026, Seoul
May 3, 2026 Spotlight A Continuous Time Markov Chain Framework for Insertion Language Models — AISTATS 2026, Tangier, Morocco
Oct 17, 2025 Lightning Insertion Language Models — UMass Theory Day, Student Lightning Talk

Mentors & Collaborators

Current

Previous

Services

VenueRoleYears
NeurIPSReviewer; Top Reviewer (2023, 2024)2023, 2024, 2025
ICLRReviewer2024, 2025
ICMLReviewer; Silver Reviewer (2026)2024, 2025, 2026
ARRReviewer2023
AAAIReviewer2025
TMLRReviewer2026