AI/ML Lecture & Networking

A monthly ML research colloquium by the New York Machine Learning Research Guild — this session in collaboration with our partner, Intuit.

Long Context and Retrieval

Rohun Agrawal · PhD Candidate, Computer Science, Columbia University · NSF Graduate Research Fellow

Leon Li · PhD Candidate, Computer Science, NYU Courant · Modal

Topic: Advances in Architectures for Long Context and Retrieval

Long-context inference is bottlenecked by the growing KV cache, in both memory and latency. However, existing KV cache compression methods either lose accuracy or must prefill the entire context before they can compress it. We introduce Latent Context Language Models (LCLMs), a family of encoder-decoder compressors trained end-to-end at scale that map long inputs to a short sequence of latent tokens before the decoder consumes them. LCLMs establish a new Pareto frontier in accuracy versus latency and memory over KV cache compression, while unlocking new capabilities such as agents that skim a compressed context and expand relevant segments on demand, and nearly lossless online compaction of the model's own outputs during generation. For grounding language models in large document corpora, Retrieval-Augmented Generation (RAG) is the ubiquitous solution, yet it relies on a brittle lookup that can't be trained end-to-end and slows generation by prefilling long contexts. We introduce Retrieval Layers, learnable sparse cross-attention layers that retrieve external text in latent space and Pareto-dominate RAG on accuracy versus latency.

About Rohun

Rohun Agrawal is a PhD candidate in computer science at Columbia University, co-advised by Micah Goldblum and Pavel Izmailov, and an NSF Graduate Research Fellow. Rohun's research pushes the temporal horizons of machine learning models: extending the future horizon through planning with world models, and the past horizon through efficient large-scale memory retrieval.

Recent work includes

• Closing the train–test gap in world models for gradient-based planning
• Visual agentic AI for spatial reasoning with a dynamic API (CVPR 2025)

Before the PhD, Rohun earned a BS in applied and computational mathematics at Caltech, working with Georgia Gkioxari on visual reasoning and in Katie Bouman's lab on imaging inverse problems. Rohun has also interned at Apple on large video model training and at NASA's Jet Propulsion Laboratory.

About Leon

Leon Li is a second-year PhD candidate in computer science at NYU Courant, advised by Pavel Izmailov and Micah Goldblum, and a researcher at Modal. Leon studies how to build AI systems that learn, improve themselves, and operate safely in the real world, with interests spanning automated AI research, continual learning, post-training science, and the alignment of superintelligence.

Recent work includes

• End-to-End Context Compression at Scale (2026)
• PersonalLLM, for tailoring LLMs to individual preferences (ICLR 2025)
• LLM Generated Persona is a Promise with a Catch (NeurIPS 2025)

Before the PhD, Leon earned a BS and MS in computer science at Columbia, graduating magna cum laude.

Who this is for

NY-MLR colloquia are small by design. The room is capped, and we keep it that way so the discussion stays substantive and everyone in it can contribute.

We'd love to see you if you're

• A practitioner working seriously on ML, in industry or in a lab
• A researcher or graduate student in ML, mathematics, or an adjacent field
• A highly motivated student or self-directed learner of any age or background, with real depth of interest

Sincere interest is enough to belong — that's one of our core principles. What we ask is that you come ready to engage. More on the Guild and our ethos at nymlr.com.

Schedule

• 5:00 – 5:30 PM · Doors open, arrivals, mingling
• 5:30 – 6:30 PM · Talks
• 6:30 – 7:00 PM · Moderated Q&A
• 7:00 – 8:00 PM · After hours, mingling, wind-down
• 8:00 PM · Lights out

Venue: Intuit NYC, 51 Astor Pl, New York, NY 10003

A note on no-shows

Seats are strictly limited by room capacity, and every in-person RSVP takes one from someone else who wanted it.

If you register to attend in person and your plans change, please amend your registration or cancel on Luma. It takes ten seconds and frees the seat.

Registering in person and not showing up without updating your RSVP will forfeit your access to future NY-MLR events. Please mark your attendance mode accurately.