Bay Area Frontier Research Club #29 | Learning Human Preferences (dinner + paper discussion)

What should a model learn when human preferences and measured outcomes disagree?

A creative work can win votes from other creators, satisfy a brand’s brief, and produce a different result once it reaches an audience. These signals reflect different judgments and objectives. Turning them into useful training data requires deciding what a model should learn—and how to evaluate whether it has learned it.

This session explores “Learning Human Preferences: Reward Models, Evaluation, and Creative Judgment.” We’ll examine how subjective feedback becomes a learning signal: preference modeling and ranking, reward design, post-training, and evaluation for tasks where quality is difficult to define and people may reasonably disagree.

The questions extend beyond predicting which output people prefer. How do we distinguish differences in taste from annotation noise? What happens when a reward model learns an easy proxy for quality? And when real-world outcomes are delayed or observed only for selected outputs, how do we determine whether a creative decision actually improved performance?

Join us for dinner, two research talks, and an extended discussion of the methods, assumptions, and evidence behind learning from human judgment.

Hosted at Array Ventures, with General Creative as a sponsoring partner.

The Frontier Research Club is a curated forum for rigorous technical discussion at the frontier of AI. We bring together researchers from frontier labs, Stanford, Berkeley, and teams building in production to examine concrete work — papers, systems, methods, and results — with particular attention to assumptions, evaluation methodology, failure modes, and what would count as convincing evidence.

Each session features two talks selected for rigor and discussion value. Presentations are intentionally brief so most of the time is reserved for questions, critique, and technical discussion. Papers and supporting materials are shared in advance to establish a strong baseline for the room.

Agenda

• 5:30pm: Doors open
• 5:30pm – 6:30pm: Networking + light dinner
• 6:30pm – 8:00pm: Research presentations + discussion
• 8:00pm – 8:30pm: Networking

The evening will include a brief introduction from General Creative to its research agenda and research cofounder/CTO opportunity.

Presenters & topics

Talk 1: TBA

Presenter, abstract, and pre-read to be announced.

Talk 2: TBA

Presenter, abstract, and pre-read to be announced.

Want to present your work?

If you have a research paper you’d like to discuss at one of our next sessions, please submit it for consideration.

Submit your paper here!

Who should attend

• Researchers and engineers working on reward models, RLHF, preference optimization, and post-training
• Researchers studying multimodal generation, creative evaluation, and human feedback
• Applied scientists working on ranking, recommendations, advertising, causal inference, and experimentation
• Founders and technical leaders building creative AI and systems that learn from human judgment

Capacity is limited. Registration is subject to approval.

We will take photos and short video clips for event recap and promotion. By attending, you consent to being photographed and recorded, and to the use of those images and clips by the organizers on social media and other event marketing channels.

🌐 Connect with Frontier Research Club

• Luma Calendar: luma.com/frontiersyndicate
• YouTube: youtube.com/@FrontierResearchClub
• LinkedIn: linkedin.com/company/frontier-research-club
• Instagram: @frontierresearchclub
• Email: kristopher@frontiersyndicate.vc

Hosted by

The Frontier Syndicate is a venture community connecting frontier tech researchers, builders, and investors through curated convenings and early-stage capital. Across the Bay Area, we host recurring research forums and intimate investor dinners—and back exceptional companies emerging from the technical networks we convene.

Array Ventures is an early-stage venture capital firm backing technical founders building AI infrastructure, developer tools, and data platforms. Array pairs capital with hands-on support connecting founders to customers and industry experts—and is hosting this session at its San Francisco office.

General Creative is developing AI systems that learn from creator feedback, brand preferences, and live campaign outcomes to evaluate and improve creative work. As this session’s sponsoring partner, General Creative brings a research agenda spanning reward modeling, preference learning, and creative evaluation.