AI Experiments Are Fun And Fruitful

I’m shutting down one of my favorite things I've ever built.

agent.ai had a good run. Over the past two years, 3 million people used it, and thousands of them built their own AI agents.

Watching that community experiment, create, and share reinforced something important: the best agents are the ones with the best context. And for businesses, a lot of that critical customer context already lives in the HubSpot CRM.

I’m proud to say we’ve taken our learnings from agent.ai and brought them into this new tool that thousands of HubSpot customers can use. You can learn more about it at AgentBuilder.com (yes, I love great, simple domain names).

There’s a lesson in this that may not be obvious to folks who have been newly introduced to the wonder of building things thanks to AI. And that lesson is the reason why I love building things in the first place: ideas often don't end up where you first imagined.

That used to frustrate me. Now it's genuinely the part I look forward to most.

The most useful way I've found to think about my own AI work is to treat it like a lab. Not the tidy kind, with white coats and spotless benches -- mine is a lot messier than that.

But a place where you run small experiments, watch what actually happens, and hold the results loosely. (I try to re-test my AI assumptions at least every 6 months).

You start with a plan, of course. But the plan is only your starting hypothesis.

So today I want to break down:

• How an experiment turned into a real HubSpot product

• Why my smallest, silliest projects taught me a ton

• How you can run one AI experiment of your own this week

—@dharmesh

The Upside to Tinkering

Rewind to 2023, which in AI years feels like a couple of decades ago.

As part of HubSpot Labs, I built something called ChatSpot. The thesis was simple: talking to your CRM in plain language should feel as normal as typing a question into a search box.

ChatSpot the product ultimately got folded into HubSpot, and the original idea behind it lives on there today as Breeze Assistant, which is doing swimmingly well.

Back in 2024 I thought that agent.ai would probably follow a similar arc: build it, learn from it, and when the time was right, let a team at HubSpot take it over, just like we did with ChatSpot.

So when I say ideas don't end up where you first imagined, I don't mean it as a consolation prize for experiments that flop. This is a pattern I’ve personally experienced time and time again: play can lead to learnings, insights, or even products that are then parlayed into something meaningful.

ChatSpot the product didn't have to survive for ChatSpot the idea to win. The experiment did its job the moment it taught us what people actually wanted.

If there's one mental move I'd love more people to make with AI, it's this one. The attempts that teach you something are usually worth just as much as the ones that go exactly to plan.

It all starts with experiments.

The Smaller The Better

Most of my experiments are a lot smaller, and a lot sillier, than a CRM assistant.

Take my silliest one. I spent a few weekends I'll never get back (but totally don’t regret) building an AI that writes dad jokes. I tweaked the prompts, tinkered with context engineering, and (tastefully) fed it training data. When new frontier models came out, I would try them to see how they did with one of humanity’s toughest things to replicate — dad jokes. The result, which you can try for free at dadjoke.ai, is a generator that every so often produces a dad joke that isn't fully awful.

On paper, that's a spectacular waste of time. But it actually gave me a learned feel for a handful of different models and how they differ. My silly little joke bot ran headfirst into the same walls the serious apps do: a limited context window, a tendency to hallucinate, and a modest lack of comedic instincts. OK, fine. It wasn’t a modest lack of comedic instincts, it was a spectacular lack.

Learning those limits on something with zero stakes made me much better at building things that matter.

Then, there’s also the “meta” lesson. How do I build a system that gets better over time? How will users react to a blind taste test of frontier models to see which ones generate the better jokes? All ideas I’m trying in dadjoke.ai — and ideas that I’ll be sure to use in future projects.

A lot of what I build starts as a personal itch. I wanted to know what a domain name was actually worth, so I built a little agent to estimate it (domainvalue.com) . I got tired of opening ten browser tabs to research a company, so I built a company-research agent (companyresearch.ai) for myself long before it was for anyone else.

None of these started life as "products." They started as experiments I ran because they were fun, or because I needed the thing and nobody had built it yet.

To find the right experiment for you, consider:

• Trying a newly released AI feature. Run an old task through a newer model, or open a feature you've been ignoring (voice mode, projects, scheduled tasks). I recently highlighted several AI features that have significantly improved.

• Building something tiny. Take a prompt you keep pasting over and over and turn it into a custom GPT or a Claude Project. Or vibe-code a simple one-page tool for something you still do by hand.

• Piloting one work task. Pick a single recurring thing at work, like meeting prep or triaging your email inbox, and let AI take a real swing at it for a week.

Resist the urge to over-plan it. As with any new habit you’re trying to make progress on, it can help to prioritize momentum over specific outcomes. Have some fun with it!

A mental model you may find handy for internalizing this concept is that of the Minimum Viable Agent -- a simple piece of AI-powered software that goes through multiple steps to accomplish a goal.

Most of my experiments stayed small. But one of them grew into something far bigger than I ever planned. I could not have told you in advance which would be which, and that unpredictability is the part I love most.

Tinkering is a ton of fun. Don’t forget that part: messing around with AI should be fun!

Follow Your Curiosity

Following my curiosity led me to building agent.ai.

And now that agent.ai is becoming AgentBuilder.com, thousands of HubSpot customers will be able to create AI agents using the full customer context that already lives in their HubSpot accounts.

But you don’t need a 1-2 word domain name or thousands of customers for your experiments to lead to powerful insights. This is about embracing experimentation as a means of improving your personal abilities with AI.

To this day, I’m up most nights past 2am building with AI. That magic feeling of building something -- until very recently -- was gated behind years of coding experience. But that’s no longer the case: with the strength of today’s models, anybody with the right amount of knowledge, patience, and creativity can bring their ideas to life.

Your experimentation may not lead you to where you initially intended it to. But after enough years of doing this, I've learned to treat that as the best possible outcome.

What’s keeping you from experimenting more?

Hit reply and let me know. I read every response, and reply to a few emails here and there when I can. Your feedback is always a gift — and just the act of writing that email might spark an idea (happens to me all the time). 🙂

Cheers.

—Dharmesh (@dharmesh)