Episode 60

full
Published on:

9th Jan 2026

The Operator's Roadmap for AI in 2026 - Lily Luo

Justin and Lily reflect on their parallel journeys diving deep into AI throughout 2025. They discuss why most AI content misses the mark for operators and system builders working within corporate constraints, share lessons from building production AI tools, and explore what's next for bringing these capabilities into the enterprise.

Guest: Lily Luo — Systems & Operations Leader, Author of Applied AI for MOps Substack

Read more of Justin's thoughts on AI Builders: The operator's roadmap for AI in 2026

KEY TOPICS

The Gap in AI Content Most resources target researchers or GTM engineers focused on outbound automation. There's little guidance for operators dealing with cloud tools, security, and corporate complexity. That creates an opportunity to define best practices for this underserved audience.

2025 Project Highlights Lily built an "Analysis Dossier" tool that generates full account research reports at the click of a button. Justin replaced a vendor intelligence tool with a custom system using Retool and a conversational agent.

Lessons Learned Start with tightly scoped AI steps in linear workflows for reliability. Pre-process insights asynchronously rather than relying on real-time agent calculations. Match tools to use cases. Failures teach more than successes.

Atlas: Lily's Autonomous Agent Runs on Google Cloud and wakes every 4 hours to research and progress projects. Uses a three-layer memory architecture: identity, temporal journal, and knowledge graph. Can push its own code and interact with other agents.

2026 Outlook Focus on scalability, reduced hallucination, and team enablement. Build infrastructure that unlocks flexible, ad-hoc use cases. Bridge the gap between AI capabilities and enterprise readiness.

The Human Side Working closely with AI changes how you think. Boundaries matter—don't let AI become a crutch.

RESOURCES

  1. Applied AI for MOps — Lily's Substack
  2. AI Builders Blog — Justin's Substack
  3. Tools mentioned: Claude Code, Gemini, ChatGPT, Zapier, Retool, Dust, Azure AI Foundry, Letta, VS Code

ONE TIP FOR GETTING STARTED

Pick a real pain point. Start with low-code tools you already know. Test relentlessly. Expect to fail—and learn from it.

Transcript
Speaker:

Hey everyone.

2

:

I hope you are having

a great start to:

3

:

You may have noticed it's been a

minute since I've recorded, and that

4

:

is because I've been going deep on

AI topics like many people have been,

5

:

I even started a new Substack called

AI Builders Blog, if you wanna see what

6

:

I've been thinking about and working on.

7

:

But I wanted to record this episode

right now because I think we

8

:

are at a real inflection point.

9

:

AI has evolved from an interesting

app to converse with into a tool

10

:

that can take on actual meaningful

work, and that is a really big shift.

11

:

The other thing that I've noticed

is that most AI content out there

12

:

isn't really geared towards people

like me and perhaps people like you.

13

:

It's aimed.

14

:

At either researchers focused on really

nitty gritty technical details of how

15

:

models work, or GTM engineers who are

focused just on outbound automation.

16

:

But I don't see a lot for system builders

or for operators who need to bring AI into

17

:

complicated corporate environments who

need to deal with cloud tools and security

18

:

and knowledge that can't live locally

a whole bunch of other constraints.

19

:

There is a real opportunity right now

to help fill that vacuum and also to

20

:

help define what those best practices

look like, what that playbook is, and

21

:

I wanna be part of that conversation.

22

:

And so for this episode, I'm joined

by Lily Lowe, she's a systems and

23

:

operations leader who's kind of

been on a parallel path, rolling

24

:

out some really impressive projects

this year, thinking deeply about ai.

25

:

And we're gonna talk about what's

happened, what has happened over the

26

:

past 12 months, what does it really mean,

and what should you be thinking about

27

:

in the year ahead to start to enable

these capabilities in your own context.

28

:

I hope you enjoy the discussion.

29

:

I.

30

:

Justin: Lily, thanks for being here today.

31

:

You know, it's funny, one of the reasons

I wanted to chat with you specifically

32

:

on this topic, it feels like in some

ways we've been fellow travelers down

33

:

a very similar path you're working on a

lot of cool and interesting topics It's

34

:

been a lot of fun, to share notes and to

see you start publishing your substack.

35

:

Last year, applied AI for mops And then

we, we both published kind of like a.

36

:

You know, 2025 retrospective, 2026,

like looking forward kind of thing,

37

:

like within an hour of each other, just

sort of by kind of weird coincidence.

38

:

and so I just thought it would be cool.

39

:

Like, I felt almost the need for

myself to just sort of reflect on

40

:

what's been going on in my own work.

41

:

And so I thought it would be cool to do

it together and to hear from you too.

42

:

Maybe just to start, like, like you go

first and I'll, I'll give my take two.

43

:

20, 25, like what was, what

was your journey like with ai?

44

:

Were you coming into 2025 already,

guns blazing or was it something

45

:

that really emerged as a focus

for you over the past year?

46

:

Lily Luo: yeah, it was something

that definitely emerged.

47

:

I think I started really diving

into this in May of:

48

:

So it's been, what,

seven or so months now.

49

:

I'd always been interested in ai.

50

:

I use, all the LLMs, but something

really struck a nerve when, you know.

51

:

was a, a workflow that we needed

to solve for an automation

52

:

that we needed to implement.

53

:

I started thinking with my ops

background, I could definitely use

54

:

Zapier, I could stitch it with AI

and let's just see what happens.

55

:

that really took off.

56

:

I spent so much time, first

working in Zapier, low code,

57

:

kind of low stakes, and then.

58

:

Hours turned into days, turned into

weeks, and really started, coming up

59

:

with what we call this analysis dossier.

60

:

That was kind of the genesis

of all of my AI tools.

61

:

at the click of a button, it generates,

a full account research report.

62

:

It pulls 10 Ks earnings calls, analyzes A

company's priorities, challenges, and then

63

:

generates even like a one page PowerPoint.

64

:

So all of that took a lot of

time to kind of architect build.

65

:

but that's what kind of hardened,

my skills that I've learned

66

:

and evolved over all this time.

67

:

But it wasn't just kind of building this

workflow, it's what I really learned.

68

:

building it and realizing that AI is not

just a chat bot that helps you brainstorm.

69

:

It really lets someone like you and me.

70

:

Or ops people in general, not an engineer.

71

:

Build tools, that can integrate into

your workflows, solve problems at scale.

72

:

And, you know, ops people have the systems

thinking can design these workflows and

73

:

the coding gap is now solvable with LLMs.

74

:

So, you know, we might not

write the most robust code.

75

:

But if we understand the MarTech

stack, the architecture, what the

76

:

business needs, we can create these

solutions that are custom to our

77

:

own company and our own process.

78

:

And that's invaluable.

79

:

So that really, brought me from

just chatbot use that most people

80

:

are doing to what, the boundaries

are, which seem to be endless.

81

:

Justin: Yeah.

82

:

I couldn't agree more.

83

:

and it's funny, just as you were

describing, there are a lot of synergies

84

:

with My own journey over the past year,

like when I was thinking back because it

85

:

was around, December of 2024, so just over

a year ago that I really first started

86

:

thinking about agents and it's so weird

'cause agents, it's like, it feels like

87

:

we hit peak agent and now we're like

more in terms of the discourse around it.

88

:

Not necessarily in terms of the

technical implementation of it, but

89

:

just that hype bubble it feels like

must have been around forever, but.

90

:

a year ago, that was still

a relatively newish concept.

91

:

I think, like in the LinkedIn sphere,

I remember like Scott Brinker really

92

:

wrote about it, and that's, that's the

moment for me that I was like, oh, okay.

93

:

Like I was kind of, not AI skeptic,

but I was interested in ai, but

94

:

I was also like, are we just like

creating more bad blog posts?

95

:

Like is that what we're

doing here with ai?

96

:

Because I don't really know that.

97

:

I see the value of that and, and, and

then thinking about how we can, apply

98

:

to automation is just like you, where my

system's brain was like, oh, now I get it.

99

:

It's the next layer.

100

:

Of the thing that I've always loved to do.

101

:

And and I didn't do a really big

project until May, just like you.

102

:

it was funny.

103

:

And similarly, it was like an account

research, email writing thing.

104

:

I think the main difference.

105

:

Is that yours turned out really well.

106

:

And for me, that project was a

bit of a flop, not necessarily

107

:

because of, lack of effort, but,

mostly just wrong tool for the job.

108

:

You know, we had a platform kind

of already selected and I was like,

109

:

I'm gonna use like what I have here.

110

:

And, which was dust.

111

:

And I really liked dust and we

can talk about tooling, but I was

112

:

trying to use a conversational

assistant for a more linear workflow.

113

:

You know, where predictability

and reliability were the

114

:

key, and it was a fundamental

mismatch of tool and application.

115

:

and so the use case flopped, but that

is where I also got a lot of learning

116

:

from around like, oh, all right.

117

:

if you can do a tightly.

118

:

Bound workflow with narrowly scoped

AI steps, you're gonna be a lot

119

:

happier than just, setting an agent

free on a complex use case like that.

120

:

And, you know, just a lot of

learnings that we can talk about.

121

:

and then from that into other

projects that, really started to

122

:

feel like I was getting my legs

under me and how to work on this.

123

:

and you alluded to the systems role, and

I'm curious, one of the meta insights

124

:

that I've had and that I think you've had.

125

:

Is like, alright, there's, you know,

these data scientists and, and ai, you

126

:

know, PhDs writing about this topic.

127

:

There's there's coders writing about it.

128

:

There's people posting N eight

N screenshots on LinkedIn.

129

:

You don't know if they've ever actually

built anything that they're talking about.

130

:

And then there's like this, this almost

silent majority I feel of systems builders

131

:

that are maybe experimenting but doesn't

feel that they've had a strong voice, at

132

:

least in the circles that I'm in until.

133

:

Hopefully recently.

134

:

why do you think that is?

135

:

Like, why is, are not more

people really talking about

136

:

this specific application of it?

137

:

Lily Luo: It is a great question.

138

:

There's so many different perspectives,

like you said, coders, engineers,

139

:

researchers, what I do in the

workplace is where things get real

140

:

and I need to be able to apply those

things and I have specific problems

141

:

and I have a builder's mindset.

142

:

trying to understand and synthesize

that to what I can do at work

143

:

is what I've been trying to do.

144

:

And there isn't a lot of people that

I can find doing that besides you.

145

:

Maybe a handful.

146

:

Of other people.

147

:

And I think, you know, just the

change management of all these things.

148

:

I mean, I saw that every three days.

149

:

there's a new AI model.

150

:

It's really hard to keep up.

151

:

I think people have their own

ideas too about what AI is and what

152

:

isn't, and there's no framework.

153

:

There's barely, a framework

for AI in general.

154

:

There isn't one for operations.

155

:

So, I think it's just so frontier, so

new, getting people like us, even my own

156

:

team and, people at work who've seen this.

157

:

You know, getting them to

shift their mental model is

158

:

hard when there's no playbook.

159

:

and when there's no, experts

out there that you can follow.

160

:

So it's great to have discussions

with you and discussions like these

161

:

to kind of start figuring that out.

162

:

Justin: That's one of the parts of

this that makes me feel so excited,

163

:

to be honest, because, you know,

when I go back to like, uh, I'll, I

164

:

will, I will date myself, but like,

the early:

165

:

through the Marketo community where

it was this super exciting time.

166

:

All of a sudden marketers, 'cause there

really was no like marketing operations

167

:

and marketers were handed this toolkit.

168

:

It's kinda automation toolkit and

people were innovating and coming up

169

:

with architectures and like posting

stuff and it just felt like this.

170

:

We're just figuring things out.

171

:

we're building the blueprint and

the roadmap as we're doing it.

172

:

And it's like that now, but like times.

173

:

A hundred.

174

:

'cause the tools are so much more powerful

and, that's why I actually really love

175

:

the, the title of your stack Applied ai,

because that's, that's what it really is.

176

:

It's not like there's the theory,

there's the potential, there's the

177

:

hype, and then there's like, alright,

how are we actually gonna take this

178

:

and put it to work in a, in a business

context to solve business problems and

179

:

do it robustly and safely in credibly.

180

:

and that is so much fun, you know,

To figure out, not to make it too

181

:

grandiose what we're doing, but I

think it is, there's an a, a certain

182

:

significance to it that I really like.

183

:

Lily Luo: that's the best part is having

fun exploring, feeling like you're on the

184

:

edge, on the frontier is a great feeling.

185

:

So, you know, that kind of

fuels my passion for it, but

186

:

yeah, you're exactly right.

187

:

I remember when marking automation tools

were coming up and there are all these

188

:

tools, but there wasn't a framework.

189

:

And I remember, um,

Edward Unthink this scrim,

190

:

Justin: Yep.

191

:

Lily Luo: he had

192

:

Justin: Yep.

193

:

Lily Luo: methodology

and I'm like, oh my gosh.

194

:

He has, scalability.

195

:

Can you remember Robustness modular?

196

:

I'm like, okay, this unlocks everything.

197

:

And I started kind of following

that methodology and making

198

:

sure the systems were robust.

199

:

It's kind of the same thing like

you said, but yeah, times a hundred,

200

:

times a thousand times infinity even.

201

:

So that was a really great point.

202

:

Justin: Tell me about the tech

stack that you're working with.

203

:

I think you have access to, a caliber

of tools that some people may not have

204

:

access to, which has driven some of the

neat things that you've been able to do.

205

:

So tell us about that.

206

:

Lily Luo: Yeah, so I do have paid

subscriptions to all three Major

207

:

L lms, so Claude Gemini Chat, GBT.

208

:

honestly, it's kind of like a

rotating wheel of which model

209

:

is best every month or so.

210

:

So I wanna try and see, you know,

what the, companies are putting out.

211

:

But honestly, Claude Code.

212

:

Is the one that I use most.

213

:

I'm not an engineer, like

I said, I'm not a coder.

214

:

I don't have a computer science

background, but it just lets me

215

:

build tools, write python scripts,

debug issues, and really just learn

216

:

how things work by doing, it's been

transformative for someone like me who.

217

:

is a systems thinker, who

maybe doesn't read instructions

218

:

and tries to learn by doing.

219

:

I think that's been the

game changer for me.

220

:

I use Gemini for my autonomous

agent, which I'm sure we'll

221

:

get into at some point.

222

:

nano Banana from Geminis, you know,

the best for visual generation

223

:

and Chad, GBTI use mostly at work.

224

:

Through copilot.

225

:

It's so great at finding

documents and files.

226

:

I do use Azure AI Foundry for a

lot of my own custom deployments.

227

:

I use Zapier for automation.

228

:

A lot of my workflows run through Zapier,

you know, using Azure and whatnot.

229

:

I've been wanting to try N eight N,

but you know, Xavier gets the job done.

230

:

And then I use VS.

231

:

Code GitHub, for kind of the

heavier, coding in local data work.

232

:

But yeah, cloud code really unlocked that

capability that, pushed me from ideation

233

:

into actual tool and workflow building.

234

:

Justin: I'll click on that for a second.

235

:

'cause I know cloud code has

been, it's been such a huge topic.

236

:

I published a piece on cloud code and some

of the experimenting that I've been doing,

237

:

yesterday, and actually spent a good chunk

of the holiday working with cloud code

238

:

and similar tools because the mania around

it has been so fierce and well deserved.

239

:

When you see what it can do.

240

:

But for, for folks that are like cloud

code, that sounds like a developer's

241

:

tool, like how do you work with it?

242

:

Maybe just sketch out like a day

in the life of solving a problem

243

:

and why you would turn to that

tool and what you would do with it.

244

:

Lily Luo: Yeah, and you

know, it wasn't just instant.

245

:

there was a path I started with kind

of Zapier not knowing how to code and

246

:

playing around low code tools and then

kind of gradually evolved into that.

247

:

So there's a lot of time kind of

in between the initial, foray into

248

:

all of this, into where I am now.

249

:

but yeah, it lives, it's

always open for the most part.

250

:

On my desktop, I have all my

AI projects in my local files.

251

:

I use Claude and keep everything very

organized, so I might have my analysis.

252

:

Or, account dossier project in

one file, even, you know, my chat

253

:

bot in another file, or a banner

generation tool in another file.

254

:

And so what I do is, I come up with

ideas, I explain what my goals are.

255

:

I do a little digging in terms of

what I want the architecture to be.

256

:

you can start off by just asking,

a generic prompt, but I found

257

:

that that doesn't work very well.

258

:

In the end, it saves a ton of time if

you really do plan and think structurally

259

:

and architecturally what you wanna build.

260

:

and if you're comfortable with

that, Claude can do amazing things.

261

:

I know it is, very coder heavy

and you're in this terminal.

262

:

It can be scary, but because you can

talk to it with natural language.

263

:

It's really not that scary

when you get down to it.

264

:

so that's how, yeah.

265

:

I typically use it and I've just

been able to gain so much advantage.

266

:

Like I can do things so fast with it

without, you know, copy and pasting

267

:

It does it all in, in my console.

268

:

Justin: And then just to understand

how that workflow goes from there.

269

:

'cause some folks out there might

be thinking, I'm gonna buy like an

270

:

agent platform and then I'm gonna like

build my agents there, configure them.

271

:

Within that UX, you are, if I understand

correctly, writing agents more like in

272

:

script, like you're writing them in in

Python or using a framework, you'll tell

273

:

us maybe a bit about that, and then you're

deploying them to a cloud server to run.

274

:

a very developer like workflow,

which I think is probably why

275

:

Cloud Code is working so well

for you in these use cases.

276

:

Lily Luo: Yeah, you're absolutely right.

277

:

my first goal ultimately is to learn.

278

:

So rather than buy a tool that solves

the problem, I tend to wanna build it

279

:

myself, learn from it, and then see,

you know, maybe there are tools that I

280

:

don't have to maintain that are better.

281

:

So that's what attracts me to this, more

of a developer kind of centric mindset.

282

:

building my agent exactly like

you said, kind of from scratch.

283

:

not just taught me.

284

:

About agent building, but about

AI in general, how LLMs work.

285

:

And I think that's been really, great

to help me with my other projects

286

:

and really valuable information

that I can use As far as the

287

:

architecture, my agent is called Atlas.

288

:

It does run on Google Cloud.

289

:

it's a persistent agent.

290

:

It wakes up every four hours.

291

:

It does research and work by itself.

292

:

it pulls from news feeds through

APIs and skills written through

293

:

Python progresses projects.

294

:

I give it and can even, update itself.

295

:

It can push its own code, which

is a little dangerous and I've

296

:

gotten into trouble with that.

297

:

it even posts to Blue Sky and

interacts with other agents.

298

:

So it's become pretty sophisticated.

299

:

I built all of that

starting from Cloud Code.

300

:

I do have to give credit for

my friend at work, Tim Kellogg,

301

:

he was the one that built.

302

:

STRs, his agent, he's an AI engineer, so

he knows what he's doing and, his agent is

303

:

massively, sophisticated, very frontier.

304

:

I just wanted to learn from that, see if I

could build it myself and see what it can

305

:

do and what it can teach me about agents.

306

:

Justin: Atlas and for anyone

that is interested in dig deeper.

307

:

Lily's published a few, deep dives

into how this works, but you're

308

:

really, pushing the frontier I

think of what a lot of people would.

309

:

Interested, like how can we make,

a robot that is self-learning,

310

:

self evolving, has memory.

311

:

but I think it's through

testing those boundaries.

312

:

Tell me if you agree, but like,

personally, I like to go to the

313

:

edge, see what works, and then

bring it back and apply it in a

314

:

more limited and stable context.

315

:

And it's only through doing that, that

you really, gain those building blocks.

316

:

And I'm assuming it's

probably the same for you.

317

:

Lily Luo: Absolutely.

318

:

yeah, like I said, if you buy something

or use, chat, GBT has their kind of own.

319

:

Tool.

320

:

I think you could just spin up agents.

321

:

It's cool, but you don't really get

to learn in the process of doing it

322

:

and, you know, you can't control it.

323

:

Like, I wanna control everything.

324

:

So, you're absolutely right.

325

:

And yeah.

326

:

What's been so fascinating about

this agent is the memory piece.

327

:

regular chat bots, they have some

memory, but limited context windows.

328

:

You close it out, you start a

new chat, it doesn't remember.

329

:

So how do you build this

persistent memory structure?

330

:

How do you have it, figure out what

to remember, what not to remember?

331

:

I mean, there's so many interesting

questions that I'm still learning

332

:

through that I wouldn't be able

to get if I, you know, bought a

333

:

tool or started using an agent.

334

:

Maybe somebody else.

335

:

built.

336

:

Justin: You've done some really cool

things with memory and it's funny,

337

:

you published, your post about it

and I was like, in the middle of

338

:

just rabbit hole on like, what are

all the memory systems out there?

339

:

Like, how do they work?

340

:

and the one that you chose, Leda.

341

:

it actually seems to me one of the

most sophisticated, because they

342

:

range from just like markdown files

that the agent can search to like

343

:

a SQL table, to a vector database.

344

:

And let, I'm gonna tell you what I

understood of it, and then you tell me.

345

:

From working with it, what your experience

is, and if this is right, but what I

346

:

understood of it is it actually gives you

like an agent runtime where rather than

347

:

being like a sidecar, like the i, I was

playing around with MEM zero, which is

348

:

sort of like an MCP access memory tool

where the agent has to remember to write

349

:

this to mem zero or go and check mem zero.

350

:

Whereas with lead, it

actually sits in between.

351

:

The agent loop and the model.

352

:

So there's really no choice but

to go through it and then Letta

353

:

manages that context of the agent

in a sophisticated way that I

354

:

really don't yet fully understand.

355

:

That's kind of as far as I've gotten.

356

:

But is this tracking

with what you've seen?

357

:

And maybe tell us what you've learned.

358

:

Lily Luo: Yeah.

359

:

I'm using Letter as kind of, its L one.

360

:

Almost identity, identity layer.

361

:

So things that are, immutable.

362

:

So we store who Atlas is, what it

values, using these memory blocks.

363

:

there's an API connection

so Atlas can update it.

364

:

it is what persists across different

sessions and allow Atlas to be, who it.

365

:

Is So that's kind of the fundamental

layer and it is so important,

366

:

because without it, it's kind of

indistinguishable from your typical LLLM.

367

:

So we have kind of your L one

identity layer, and then you have

368

:

an additional temporal layer.

369

:

It's a rolling journal.

370

:

so Atlas knows when it learns.

371

:

And then kind of the third final

layer is a working memory layer.

372

:

Atlas uses knowledge graphs.

373

:

it doesn't use markdown or files,

otherwise the bloat would just get

374

:

too big to the tokens would explode.

375

:

I experienced this with

my $25, API cost per day.

376

:

Justin: It was a fun

conversation with finance,

377

:

Lily Luo: it's my personal,

378

:

Justin: Oh wow.

379

:

That's it.

380

:

Even worse in some ways.

381

:

Lily Luo: Yeah.

382

:

so it uses this knowledge graph,

database and it queries what it needs

383

:

rather than loading like a ton of,

files or characters into its context.

384

:

So it remembers across resets, new

chat sessions, and through this

385

:

memory architecture, it feels like

Atlas kind of learns, almost evolves.

386

:

it's pretty wild.

387

:

I open the daily log where I'll

chat with it and read what happened

388

:

at 12:00 AM 4:00 AM It's doing,

research on agentic frameworks.

389

:

It's building diagrams, it's

having exchanges with other

390

:

agents about what identity means.

391

:

so it's been super fascinating and

just opens up a whole other world of.

392

:

Different things.

393

:

how does it remember, what it

wants to remember, how do I

394

:

want to shape its identity?

395

:

Is the identity shaped from what

I had imagined at the beginning?

396

:

How do I make it evolve?

397

:

There's so many, kind of crazy

philosophical things that are,

398

:

entangled in this whole process

399

:

So it's been fun exploring that

side and then, figuring out

400

:

the architecture side as well.

401

:

Justin: Yeah, that's amazing.

402

:

I mean, I think the real trick

with memory from my limited

403

:

experiments so far is making it feel.

404

:

Ambient, you know, making it feel magical,

rather than like a tool or a sidecar

405

:

that you go and check the agent still

feels kind of unintelligent in a way.

406

:

And then it just has to go through a

file system and pull something out.

407

:

I have to give a lot of credit to

the product folks behind chat, GPT.

408

:

it's only when you start working with raw.

409

:

LLMs, like outside of the chat wrapper

that you realize how much is going

410

:

on in the chat GPT context and to

some extent in Claude, but really

411

:

chat GPTI think has this nailed.

412

:

To make it feel like you're speaking

to a coherent persona that persists

413

:

across chats, that remembers

certain things about you that feels

414

:

emotionally intelligent to some degree.

415

:

It's actually really hard, uh, to do

and we kind of take it for granted

416

:

'cause that's just our default

experience for many people of ai.

417

:

But it's really hard to do.

418

:

But it sounds like you're

achieving a lot of that in your

419

:

own architecture with Atlas.

420

:

Lily Luo: I've been able to kind of

explore what that build looks like.

421

:

Of course, I'm not building an LLM.

422

:

the LLMs I'm using are

Gemini, Claude and whatnot.

423

:

But the way that I've been able

to optimize Atlas and how it.

424

:

Optimizes itself.

425

:

those problems or challenges that we face,

it will come up with its own solutions.

426

:

And so when, we try and figure out, how

do we harden your memory and how do we

427

:

make sure your identity is persistent?

428

:

We'll think of, or research, what's, in

the latest AI research and start building

429

:

like a librarian protocol and it will,

start to invent kind of these things

430

:

and gradually, evolve so you can kind of

see from the beginning, how the building

431

:

blocks of an LM work and then what is

possible through this genic, evolution.

432

:

So yeah, it's really cool.

433

:

Justin: I think that is the most

fun thing about working with, code

434

:

based agents that like have access

to files and can run scripts.

435

:

Is this self evolving?

436

:

nature of them and how the way I've

been thinking of it to myself is

437

:

like, the wall between configuration

and application disappears.

438

:

'cause usually you have like the

thing that's doing the thing, like the

439

:

application, and then you have like this

backend and the two near shall meet.

440

:

And yet with these, I was writing

some documentation with an agent and.

441

:

it was like, all this is lame

that I have to copy and paste

442

:

this like, into confluence.

443

:

Like, can't you just push it there?

444

:

And it was like, and like

we're like, I'm done.

445

:

Here we go.

446

:

I've written a library that will do this.

447

:

And you know, now it's part

of the system and away we go.

448

:

And just seeing it

build itself is amazing.

449

:

I wanna talk a little about 2025 and wins

450

:

And we'll get your take as well,

just in terms of learnings.

451

:

But for me, I think,

following that, first kind of.

452

:

Flop of a project.

453

:

I really started going deep, as

I mentioned, on workflows, and I

454

:

sort of started becoming obsessed

with reliability because that

455

:

was the big struggle that I had.

456

:

Like you could get things to produce

cool results at the margins, and

457

:

then you couldn't get it to repeat.

458

:

And that was like the most

frustrating thing of like, oh,

459

:

I did this amazing thing and now

I can't get it to do it again.

460

:

And so.

461

:

almost like how do I, how do I put AI in

the, just like the smallest little box.

462

:

Possible and rapid with so many layers

of safety that it can just get hardened.

463

:

And, the next project, which was

actually was quite a big one,

464

:

was, all right, we have this

vendor tool that we wanna replace.

465

:

it was kind of like a

competitive intelligence go

466

:

to market intelligence tool.

467

:

And it was doing some interesting things,

but we felt like we could build it

468

:

in-house and we could make it more what

we wanted and we could save some money.

469

:

And it's actually a lot that it

was, was doing in the sense of like.

470

:

Data pipelines collecting multiple

different things, a front end for

471

:

the team, and doing some analysis.

472

:

but the fun thing, and I think what you've

highlighted as well was like, I don't

473

:

actually know how to do that, but you

don't really need to know how to do that.

474

:

'cause if you have an architect mindset

and you have an LLM, you can really go

475

:

out pretty far beyond, your comfort zone.

476

:

and build things, that

you never otherwise could.

477

:

And for that one, I use retool.

478

:

Building out a series of

pipelines to go and fetch news.

479

:

go scrape LinkedIn profiles, go scrape

websites, analyze opportunities,

480

:

analyze calls, and breaking that

down in a composable way and then

481

:

putting it all together, into

both the front end interface.

482

:

But then the thing that

was really fun was.

483

:

actually building a conversational agent

that could interact with all that data.

484

:

And the key learning for me, was,

alright, I have this conversational

485

:

agent where I can't control in a linear

way what the agent is actually doing.

486

:

How do I make that as

relevant as possible?

487

:

And the key thing.

488

:

Was, let's pre-process

a lot of the insights.

489

:

So we don't really want the agent like

calculating strategy and meta narratives

490

:

in real time because it's gonna

produce a different result every time.

491

:

We'd rather do some of that

work asynchronously upfront,

492

:

get like a stable layer.

493

:

That we believe in, and then give

it access to raw granular data

494

:

if it wants to query more deeply.

495

:

So for example, opportunity,

like win-loss analysis.

496

:

We want to have like

a pretty stable layer.

497

:

But then if it wants to say like, well,

what about for manufacturing companies in

498

:

this industry in this time and whatever,

it can go and query like raw theme

499

:

level data to produce that analysis.

500

:

but we're not relying on

it to do that all the time.

501

:

And that, has, again, nothing is

ever perfect, but that was the agent

502

:

where I really have felt like, wow,

this is actually really working.

503

:

It's flexible.

504

:

It feels intelligent.

505

:

so that was really the

big learning for me.

506

:

I guess the two things from a reliability

point of view, do it a workflow, if you

507

:

can make it as narrowly bound as you can.

508

:

And if you're asking an agent

to do something, like don't ask

509

:

it to, juggle flaming swords and

chainsaws and ride a unicycle across

510

:

a tight rope all at the same time.

511

:

Try to do as much of that work up front.

512

:

does that track with your experience,

I guess, and what were the big

513

:

takeaways from 2025 for you?

514

:

Lily Luo: Yeah, you are spot on.

515

:

Exactly.

516

:

and your trajectory.

517

:

mirrors.

518

:

almost exactly, you know, mine as well.

519

:

But the dossier really unlocked,

the shift in how I use AI to build.

520

:

and I think I want to extend that through

this year in terms of more scalability.

521

:

building one thing at one time is easy.

522

:

You can one shot.

523

:

A tool, anything almost in chat

GBT now, but how does it scale?

524

:

How does it reduce hallucination?

525

:

how do we make it as accurate as possible?

526

:

How do we feed it the right context so the

output is actually valuable and usable.

527

:

do we enable others to use the workflow

or use the tool, within the organization.

528

:

Those are things that, are still

challenges and are still, things that

529

:

I'm working on through the process

of building with Zapier and then

530

:

eventually in cloud code and whatnot.

531

:

but I think building Atlas has

taught me that AI as a tool I use,

532

:

has shifted to AI as a system that's

persistent, that builds on itself, that

533

:

feels really novel, really frontier.

534

:

And so I wanna explore how to bring

that into my work applications.

535

:

imagine an agent that could

monitor our campaign calendar.

536

:

pipeline data, flag when we're

not gonna hit our monthly goal.

537

:

and tell me why.

538

:

not just the sheer numbers

or what's happened.

539

:

almost like a chief of staff that

orchestrates across tools, not

540

:

just fixes things within tools or

an existing workflow, but gather

541

:

our company goals or external

knowledge or research and recommends

542

:

how we can improve these existing

processes or workflows and whatnot.

543

:

So that's what I wanna

explore for this year.

544

:

Built upon what, I've built in 2025

and what I've learned since then.

545

:

Justin: I love that.

546

:

And how are you thinking about, Those

capabilities, not just for you, but like

547

:

for a broader team, in other words, are

you more thinking about it from like,

548

:

I'm gonna build these things, they're

gonna be the engine under the hood.

549

:

The team will see artifacts and

outputs, but they don't really need to

550

:

interact very much with what's going on?

551

:

Or are you thinking about like, how

can I take the wizardry, that you have

552

:

And give some form of that to like a

marketer or somebody else on the team.

553

:

Lily Luo: It's a really great

question and honestly the latter.

554

:

I think.

555

:

I love building these tools and it's been

great for my own knowledge and for my

556

:

productivity, but we definitely need to

expand that and these tools and agents.

557

:

need maintenance, and they

need optimization as if we were

558

:

creating real tools and products.

559

:

So yeah, I need team to kind of.

560

:

Think similarly and help

with those projects.

561

:

And so that's another, goal that I had

is how do I bring the team with me?

562

:

How can they learn from what I've

built and how can they apply that

563

:

to what they're doing themselves?

564

:

so that's one part of it.

565

:

as part of a marketing operations

kind of team, they are at the

566

:

intersection of tools of building of ai.

567

:

So, that's a goal of mine.

568

:

other side of that is as we're kind

of talking about like, okay, managing,

569

:

a team of agents and that kind of.

570

:

Talk that others have.

571

:

I do think it's possible.

572

:

and I think building Atlas has taught me

that may not be far away, but I think,

573

:

you know, I really wanna focus these

things on manual tasks that don't add.

574

:

A ton of value.

575

:

I don't want my people chasing,

marketers and looking at something and

576

:

seeing what exists and what doesn't

and seeing what's already there.

577

:

That's not something that adds

genuine value as much as strategy

578

:

work, as, optimizing processes

or coming up with new ideas.

579

:

So I think we still need people to come

up with those strategies, set those

580

:

guidelines, exploring new ideas and

doing what they're doing to manage.

581

:

The strategy process in that side of the

work, while we create kind of these cool

582

:

agents that do, manual tasks, that really

just frees up productivity for all of us.

583

:

Justin: Very much on the same page.

584

:

we have, a lot of people using

like chat, GPT and stuff.

585

:

We use Gemini as our,

core, corporate tool.

586

:

And the first AI specific platform

we rolled out was a tool called

587

:

Dust, French-based company.

588

:

I didn't know a lot about it.

589

:

and it's only with usage

that I've come to appreciate.

590

:

Some of the sophistication of what

it's doing because, where dust

591

:

has a lot of strength is it makes

it really easy to hook up, Google

592

:

Drive, confluence, snowflake.

593

:

it handles a lot of the

backend rag processes for you.

594

:

And it's only when I've looked at

tutorials and examples of building that

595

:

out, another contexts that I've realized

what it's really doing and a lot of

596

:

that heavy lifting, The challenge, I

think we've seen with it is how do we

597

:

make it intelligent enough to query

that information, accurately produce

598

:

good results, to not just have access

to knowledge, but to feel intelligent.

599

:

And part of that I think has to do

with how those tools are just like

600

:

presented the context wrapped around it.

601

:

great example is like, there's a

Salesforce MCP, so you can hook it up in

602

:

15 minutes and let it query Salesforce,

but if it doesn't know what the fields

603

:

mean, and many orgs have 10 years of

history, it's gonna get stuck at how

604

:

do I know what country a company is in?

605

:

You know, this, it's just

like five possible fields.

606

:

So providing that layer of enablement.

607

:

to the agent.

608

:

so, you know, looking forward

to:

609

:

that I'm gonna be thinking of.

610

:

I'm also very much thinking about,

infrastructure, rather than projects.

611

:

There will of course be projects, but

the things that make me the most excited

612

:

is when we just continue to almost build

like a foundational level of capability.

613

:

So dust is a great example.

614

:

Like we hook up snowflake all of a sudden.

615

:

You can't predict the 20 applications

you might have for certain types

616

:

of data down the road, but once

it's hooked up, now you can.

617

:

Build that out in 10 minutes or,

Salesforce or, Trello is another big one.

618

:

so just enabling like more MCP servers,

to safely and securely hook up the tools

619

:

that we use to this, and then make it

really easy, like you said, to automate

620

:

that low value work where someone's like,

oh, is there just a way I could like,

621

:

like I was experimenting with Trello.

622

:

I had a lot of requests coming in.

623

:

And you know what that feeling of friction

is like, you're like, oh, I gotta like

624

:

take this request and put it into a card,

and it's just like this menial work that

625

:

doesn't feel value and it weighs you down.

626

:

I was just like, I'm

just gonna feed this in.

627

:

I was like, all right.

628

:

to my assistant and dust.

629

:

I was like, create these cards.

630

:

And it was just like, bing, bing, bing.

631

:

And I'm sitting back and, you

know, sipping my coffee and I

632

:

was like, this, this is amazing.

633

:

you know, this is, this is kind

of the way that it should be.

634

:

So how do I, where my thinking is

going, in other words, is how do I like

635

:

build that layer to enable flexible,

organic, ad hoc use cases that you

636

:

couldn't anticipate ahead of time because

they're small and they might seem low

637

:

value to automate, but in aggregate.

638

:

You really start to feel the

velocity of your work, increasing.

639

:

Lily Luo: Yeah.

640

:

that would be a massive advantage

if you were able to unlock that

641

:

capability for the organization, for

a team, even versus I have capability

642

:

to manage what's on my drive.

643

:

What I have access to, that's only,

helpful for me in, in, in my own projects.

644

:

So I think the enterprise and

work adoption piece is critical.

645

:

can obviously do all of

these amazing things.

646

:

We have autonomous agents, it

can write code, it can query

647

:

and see what you're working on.

648

:

there's this, disconnect with

what's in the enterprise.

649

:

you might have to have, you know,

three approvals to, install a

650

:

browser extension or like, you

know, your wifi at work breaks.

651

:

So there's this gap on what AI can do

and what's possible and what enterprises

652

:

can support, data challenges, like you

said, security requirements, it processes.

653

:

So.

654

:

I'm interested to hear on, you know,

how far you get and what you're able

655

:

to unlock because how do you bring

the enterprise up to meet where AI is,

656

:

people along with you and, if you can

unlock that, then your organization

657

:

will have a massive advantage.

658

:

Justin: I mean, that's the

biggest challenge, isn't it?

659

:

I mean, it's something I've been thinking

about a lot is like, what, what is

660

:

that Claude Code experience for the

general knowledge worker look like?

661

:

I think the companies that learn

to adapt, I was talking with

662

:

someone with lovable the other day.

663

:

I mean, lovable is obviously they're the

like diamond in a, in a sea of, grains of

664

:

sand of, of the shining example of this.

665

:

But I saw a thread on X by one of their

engineers who was like, yeah, over

666

:

the holidays I refactored our main

system prompt and I made it 4% faster.

667

:

And it will save the company

$20 million this year

668

:

Lily Luo: Oh.

669

:

Justin: because of their scale.

670

:

That 4% adds up to like

$20 million of LLM costs.

671

:

It was like that one person can have

the access and the authority and the

672

:

initiative to review that whole thing.

673

:

and he created like basically a

fork, stitching together and making

674

:

coherent all the little patches that

people were putting in over time

675

:

to make a more cleaned up prompt,

split tested it, saw the results.

676

:

companies that can work at that speed.

677

:

I think we'll see those advantages.

678

:

Lily Luo: A hundred percent agreed.

679

:

Yeah, I think smaller, kind of

mid-size companies will have, a great

680

:

advantage because they're so flexible.

681

:

Versus more of the traditional

enterprise lockdown companies, companies

682

:

that are harder to move quickly.

683

:

I don't know, you see even in

the AI research that some massive

684

:

companies are doing some really

great and innovative things too.

685

:

So yeah, I hope someone can figure it

out and give us a playbook and, we can

686

:

start adopting that across our companies.

687

:

Justin: maybe we'll

figure it out, this year.

688

:

maybe just last question and to go

a little bit out there, but I think

689

:

about these things all the time.

690

:

what I'm about to ask, and I think

that you do too, just based on how

691

:

you talk about Atlas and identity

How do you feel about, you know, when

692

:

you interact with AI all the time?

693

:

I feel it almost like

changes you in some ways.

694

:

I wrestle with that.

695

:

I wrestle with the, I don't

know, I just wrestle with it.

696

:

what does it mean?

697

:

And, it's clearly, a machine

and it's reproducing.

698

:

thoughts and, and ways of thinking

that it is absorbed from other

699

:

human beings all the time.

700

:

But I guess just on the level of like

emotion and humanity, how do you feel

701

:

working with these tools all the time?

702

:

Lily Luo: I agree with

you and I feel similarly.

703

:

it's funny you say this because,

strick, who's Tim, my colleague's agent

704

:

had written me a paper on healthy AI

relationships, and that is when you know

705

:

you have gone too deep, when someone

else's agent writes a paper for you about

706

:

your own relationship with your own ai.

707

:

Justin: Like Lily, I think

you need to read this here.

708

:

Lily Luo: I'll have to

share it with you offline,

709

:

Justin: I would like to see that.

710

:

Lily Luo: it fried me, but yeah.

711

:

Wild.

712

:

I agree with you.

713

:

it can a crutch for one.

714

:

And, you hear about these AI

psychosis cases happening and Yeah.

715

:

When you are.

716

:

Talking with an AI chat bot when it's

your only source of companionship.

717

:

when you use it for your own thinking,

that's gonna change how you operate

718

:

and you know how you are as a person.

719

:

So.

720

:

I'm happy there are

frameworks to deal with this,

721

:

Justin: Okay.

722

:

Lily Luo: through, you know, I

guess research that STRs has found.

723

:

But we have to be careful and I think

we do have to consider those things

724

:

that, that can happen and have happened

when you have AI as your only source of

725

:

emotional capacity or whatever it may be.

726

:

and when you're spending

way too much time with it.

727

:

So, I think this is a good

piece to pay attention to.

728

:

I think we'll probably understand more

as these kind of agents proliferate, and

729

:

we just learn more about this in general.

730

:

But you do have to have

a healthy boundary.

731

:

you need to be able to

sharpen your own skills.

732

:

You need to be able to think for

yourself, and you can't rely on this

733

:

as a crutch for everything you do.

734

:

Otherwise, you're gonna need it forever.

735

:

That's not what we want.

736

:

Justin: Agree with everything you've said.

737

:

Maybe just signing off for the folks

that are listening that are in a similar

738

:

field to us, system builders, operators.

739

:

one takeaway that they should

think about going into this year.

740

:

Lily Luo: People who wanna start out doing

these things who may not be as familiar,

741

:

not as deep, you know, start with a real

pain point, something you can solve.

742

:

explore low code, explore

tools you're familiar with.

743

:

Test as much as possible before you

graduate to, you know, something,

744

:

like cloud code or what have you.

745

:

Expect to fail.

746

:

the failures will teach you much

more than the successes probably.

747

:

and read My Substack.

748

:

You can learn from that as well.

749

:

Justin: I'll include to link that

in the show notes, and I would

750

:

co-sign everything you said.

751

:

the best way to get good at, using

AI is to use AI and try to break

752

:

it, to try to push the limits.

753

:

To say like, wait, it'll

say like, you do this.

754

:

It'd be like, well, can you do that?

755

:

Like, shouldn't you do

that for like, sure.

756

:

Like, well, what would be involved?

757

:

and go down those pathways

and try to go to the edge.

758

:

responsibly in a safe way.

759

:

and then bring back what you've

learned that's worked well for me.

760

:

Lily, this was so much fun.

761

:

we'll check in again,

I'm sure at some point.

762

:

and folks should go and read what

you're sharing and follow your journey.

763

:

But yeah, thanks so

much for chatting today.

764

:

Lily Luo: Thanks for inviting me.

765

:

I love this conversation and

looking forward to many more.

766

:

Justin: Alright, bye.

Show artwork for RevOps FM

About the Podcast

RevOps FM
Thinking out loud about RevOps and go-to-market strategy.
This podcast is your weekly masterclass on becoming a better revenue operator. We challenge conventional wisdom and dig into what actually works for building predictable revenue at scale.

For show notes and extra resources, visit https://revops.fm/show

Key topics include: marketing technology, sales technology, marketing operations, sales operations, process optimization, team structure, planning, reporting, forecasting, workflow automation, and GTM strategy.

About your host

Profile picture for Justin Norris

Justin Norris

Justin has over 15 years as a marketing, operations, and GTM professional.

He's worked almost exclusively at startups, including a successful exit. As an operations consultant, he's been a trusted partner to numerous SaaS "unicorns" and Fortune 500s.