Dhruvesh Patel

I am a Computer Science PhD Researcher at UMass Amherst, advised by Prof. Andrew McCallum at the Information Extraction and Synthesis Laboratory, and an External Visiting Researcher at IBM Research. My research focuses on generative modeling for discrete sequences, especially alternatives to left-to-right language modeling. Before UMass, I completed my undergraduate and first master's degree at IIT Madras, where I worked on robotics research with Prof. Sandipan Bandyopadhyay.

I have also been fortunate to work with collaborators across industry research labs, including IBM Research, Meta Reality Labs and Abridge AI. Before graduate school, I spent two years as a software engineer at MathWorks and a year collaborating with Prof. Partha Talukdar on applied NLP problems.

Download CV Google Scholar GitHub Twitter LinkedIn

Research

Most language models generate text one token at a time, from left to right. I am interested in models that can draft, revise, infill, and reason over text in more flexible ways. My current work focuses on probabilistic models for non-autoregressive sequence generation, with an emphasis on making generation faster and more controllable.

I am especially interested in how to make these alternatives practical at scale: adapting pre-trained autoregressive LLMs, designing efficient non-autoregressive pre-training objectives, and improving sampling for discrete diffusion models.

Much of my earlier work studies the same question from a more fundamental angle: how should neural models represent, score, and search over structured discrete spaces? This includes structured prediction with energy-based models, geometric representations such as box embeddings, and models for label spaces, hierarchies, and relational structure.

Together with Benjamin Rozonoyer, I host dIESL, a reading and working group on non-autoregressive LLMs at IESL.

Selected Publications

View all publications →

Affiliations & Internships

Sep 2025–present IBM Research — External Visiting Researcher
May 2025–Aug 2025 IBM Research — Research Intern
Sep 2022–Dec 2022 Meta Reality Labs — Research Intern
May 2020–Dec 2020 Abridge AI — Research Intern
May 2019–present IESL, UMass Amherst — Graduate Researcher
Jan 2021–present CICS, UMass Amherst — PhD Student
Jan 2019–Dec 2020 CICS, UMass Amherst — MS Student
Jun 2016–Jan 2018 MathWorks — Software Engineer
Jun 2011–Jan 2016 IIT Madras — Undergraduate + Master's

News

Jul 8, 2026 Presenting xLM at the Bridging Research and Open Source social at ICML 2026 in Seoul!
Jun 23, 2026 The new dIESL page is up — our reading and working group on non-autoregressive LLMs at IESL.
May 15, 2026 Learned Relay Representations for Forward-Thinking Discrete Diffusion Models was accepted at the ICML 2026 Workshop on Foundations of Deep Generative Models!
May 1, 2026 Insertion Based Sequence Generation with Learnable Order Dynamics was accepted at ICML 2026!
Mar 10, 2026 xLM: A Python Package for Non-Autoregressive Language Models was accepted at the EACL 2026 System Demonstrations track!
Jan 15, 2026 A Continuous-Time Markov Chain Framework for Insertion Language Models was accepted as a spotlight paper (top 6%) at AISTATS 2026!
Oct 1, 2025 Presenting Improved Sampling from Masked Diffusion Models with Position Contrastive Guidance at the SPIGM workshop at NeurIPS 2025.
Jun 1, 2025 Work on Insertion Language Models (ILMs) is out on arXiv! It will be presented at the SPIGM workshop at NeurIPS 2025.

Talks

Aug 21, 2026 Invited Towards More Flexible and Efficient Non-Autoregressive Language Models — Cerebras Seminar Series
Aug 17, 2026 Invited Non-Autoregressive Sequence Generation — IBM AI Foundations Technical Forum
Jul 8, 2026 Invited xLM: A Python Package for Non-Autoregressive Language Models — Bridging Research and Open Source social, ICML 2026, Seoul
May 3, 2026 Spotlight A Continuous Time Markov Chain Framework for Insertion Language Models — AISTATS 2026, Tangier, Morocco
Oct 17, 2025 Lightning Insertion Language Models — UMass Theory Day, Student Lightning Talk

Mentors & Collaborators

Current

Previous

Services

VenueRoleYears
NeurIPSReviewer2023 (Top Reviewer), 2024 (Top Reviewer), 2025
ICLRReviewer2024, 2025
ICMLReviewer2024, 2025, 2026 (Silver Reviewer)
ARRReviewer2023
AAAIReviewer2025
TMLRReviewer2026
AISTATSReviewer2026, 2027