(This was re-posted from a blog post I recently wrote on Orbital's Tech Blog)
---
Background
I was at the AI Engineer Summit in NYC in February 2025, listening to engineers trade war stories about building agentic systems. Sitting in the crowd, it hit me: my team and I had our own scars, lessons, and hacks from shipping at the frontier and they were worth sharing. By the time the next conference rolled around in San Francisco, my talk had been selected.
Title: “Buy Now, Maybe Pay Later: Dealing with Prompt-Tax While Staying at the Frontier”
The core idea? Frontier LLMs now drop at warp speed. Each upgrade hits teams with a Prompt‑Tax: busted prompts, cranky domain experts, and evals that show up fashionably late. In my talk, I share 18 months of bruises (and wins) from shipping an agentic product for real‑estate lawyers:
• The challenge of an evolving prompt library that breaks every time the model jumps
• The bare‑bones tactics that actually work for faster migrations
• Our “betting on the model” mantra: ship the newest frontier model even when it’s rough around the edges, then race to close the gaps before anyone else does
I wanted listeners to walk away with a playbook to stay frontier‑fresh without blowing up their roadmap or their team’s sanity.
Talk
Here is the full 25 minute video of my talk for the AI Engineer World’s Fair held in San Francisco on June 3-5, 2025:
Slide Deck
Here is the complete slide deck I presented for my talk with references below:
No comments:
Post a Comment