The Operator's Roadmap for AI in 2026 - Lily Luo
Justin and Lily reflect on their parallel journeys diving deep into AI throughout 2025. They discuss why most AI content misses the mark for operators and system builders working within corporate constraints, share lessons from building production AI tools, and explore what's next for bringing these capabilities into the enterprise.
Guest: Lily Luo — Systems & Operations Leader, Author of Applied AI for MOps Substack
Read more of Justin's thoughts on AI Builders: The operator's roadmap for AI in 2026
KEY TOPICS
The Gap in AI Content Most resources target researchers or GTM engineers focused on outbound automation. There's little guidance for operators dealing with cloud tools, security, and corporate complexity. That creates an opportunity to define best practices for this underserved audience.
2025 Project Highlights Lily built an "Analysis Dossier" tool that generates full account research reports at the click of a button. Justin replaced a vendor intelligence tool with a custom system using Retool and a conversational agent.
Lessons Learned Start with tightly scoped AI steps in linear workflows for reliability. Pre-process insights asynchronously rather than relying on real-time agent calculations. Match tools to use cases. Failures teach more than successes.
Atlas: Lily's Autonomous Agent Runs on Google Cloud and wakes every 4 hours to research and progress projects. Uses a three-layer memory architecture: identity, temporal journal, and knowledge graph. Can push its own code and interact with other agents.
2026 Outlook Focus on scalability, reduced hallucination, and team enablement. Build infrastructure that unlocks flexible, ad-hoc use cases. Bridge the gap between AI capabilities and enterprise readiness.
The Human Side Working closely with AI changes how you think. Boundaries matter—don't let AI become a crutch.
RESOURCES
- Applied AI for MOps — Lily's Substack
- AI Builders Blog — Justin's Substack
- Tools mentioned: Claude Code, Gemini, ChatGPT, Zapier, Retool, Dust, Azure AI Foundry, Letta, VS Code
ONE TIP FOR GETTING STARTED
Pick a real pain point. Start with low-code tools you already know. Test relentlessly. Expect to fail—and learn from it.
Transcript
Hey everyone.
2
:I hope you are having
a great start to:
3
:You may have noticed it's been a
minute since I've recorded, and that
4
:is because I've been going deep on
AI topics like many people have been,
5
:I even started a new Substack called
AI Builders Blog, if you wanna see what
6
:I've been thinking about and working on.
7
:But I wanted to record this episode
right now because I think we
8
:are at a real inflection point.
9
:AI has evolved from an interesting
app to converse with into a tool
10
:that can take on actual meaningful
work, and that is a really big shift.
11
:The other thing that I've noticed
is that most AI content out there
12
:isn't really geared towards people
like me and perhaps people like you.
13
:It's aimed.
14
:At either researchers focused on really
nitty gritty technical details of how
15
:models work, or GTM engineers who are
focused just on outbound automation.
16
:But I don't see a lot for system builders
or for operators who need to bring AI into
17
:complicated corporate environments who
need to deal with cloud tools and security
18
:and knowledge that can't live locally
a whole bunch of other constraints.
19
:There is a real opportunity right now
to help fill that vacuum and also to
20
:help define what those best practices
look like, what that playbook is, and
21
:I wanna be part of that conversation.
22
:And so for this episode, I'm joined
by Lily Lowe, she's a systems and
23
:operations leader who's kind of
been on a parallel path, rolling
24
:out some really impressive projects
this year, thinking deeply about ai.
25
:And we're gonna talk about what's
happened, what has happened over the
26
:past 12 months, what does it really mean,
and what should you be thinking about
27
:in the year ahead to start to enable
these capabilities in your own context.
28
:I hope you enjoy the discussion.
29
:I.
30
:Justin: Lily, thanks for being here today.
31
:You know, it's funny, one of the reasons
I wanted to chat with you specifically
32
:on this topic, it feels like in some
ways we've been fellow travelers down
33
:a very similar path you're working on a
lot of cool and interesting topics It's
34
:been a lot of fun, to share notes and to
see you start publishing your substack.
35
:Last year, applied AI for mops And then
we, we both published kind of like a.
36
:You know, 2025 retrospective, 2026,
like looking forward kind of thing,
37
:like within an hour of each other, just
sort of by kind of weird coincidence.
38
:and so I just thought it would be cool.
39
:Like, I felt almost the need for
myself to just sort of reflect on
40
:what's been going on in my own work.
41
:And so I thought it would be cool to do
it together and to hear from you too.
42
:Maybe just to start, like, like you go
first and I'll, I'll give my take two.
43
:20, 25, like what was, what
was your journey like with ai?
44
:Were you coming into 2025 already,
guns blazing or was it something
45
:that really emerged as a focus
for you over the past year?
46
:Lily Luo: yeah, it was something
that definitely emerged.
47
:I think I started really diving
into this in May of:
48
:So it's been, what,
seven or so months now.
49
:I'd always been interested in ai.
50
:I use, all the LLMs, but something
really struck a nerve when, you know.
51
:was a, a workflow that we needed
to solve for an automation
52
:that we needed to implement.
53
:I started thinking with my ops
background, I could definitely use
54
:Zapier, I could stitch it with AI
and let's just see what happens.
55
:that really took off.
56
:I spent so much time, first
working in Zapier, low code,
57
:kind of low stakes, and then.
58
:Hours turned into days, turned into
weeks, and really started, coming up
59
:with what we call this analysis dossier.
60
:That was kind of the genesis
of all of my AI tools.
61
:at the click of a button, it generates,
a full account research report.
62
:It pulls 10 Ks earnings calls, analyzes A
company's priorities, challenges, and then
63
:generates even like a one page PowerPoint.
64
:So all of that took a lot of
time to kind of architect build.
65
:but that's what kind of hardened,
my skills that I've learned
66
:and evolved over all this time.
67
:But it wasn't just kind of building this
workflow, it's what I really learned.
68
:building it and realizing that AI is not
just a chat bot that helps you brainstorm.
69
:It really lets someone like you and me.
70
:Or ops people in general, not an engineer.
71
:Build tools, that can integrate into
your workflows, solve problems at scale.
72
:And, you know, ops people have the systems
thinking can design these workflows and
73
:the coding gap is now solvable with LLMs.
74
:So, you know, we might not
write the most robust code.
75
:But if we understand the MarTech
stack, the architecture, what the
76
:business needs, we can create these
solutions that are custom to our
77
:own company and our own process.
78
:And that's invaluable.
79
:So that really, brought me from
just chatbot use that most people
80
:are doing to what, the boundaries
are, which seem to be endless.
81
:Justin: Yeah.
82
:I couldn't agree more.
83
:and it's funny, just as you were
describing, there are a lot of synergies
84
:with My own journey over the past year,
like when I was thinking back because it
85
:was around, December of 2024, so just over
a year ago that I really first started
86
:thinking about agents and it's so weird
'cause agents, it's like, it feels like
87
:we hit peak agent and now we're like
more in terms of the discourse around it.
88
:Not necessarily in terms of the
technical implementation of it, but
89
:just that hype bubble it feels like
must have been around forever, but.
90
:a year ago, that was still
a relatively newish concept.
91
:I think, like in the LinkedIn sphere,
I remember like Scott Brinker really
92
:wrote about it, and that's, that's the
moment for me that I was like, oh, okay.
93
:Like I was kind of, not AI skeptic,
but I was interested in ai, but
94
:I was also like, are we just like
creating more bad blog posts?
95
:Like is that what we're
doing here with ai?
96
:Because I don't really know that.
97
:I see the value of that and, and, and
then thinking about how we can, apply
98
:to automation is just like you, where my
system's brain was like, oh, now I get it.
99
:It's the next layer.
100
:Of the thing that I've always loved to do.
101
:And and I didn't do a really big
project until May, just like you.
102
:it was funny.
103
:And similarly, it was like an account
research, email writing thing.
104
:I think the main difference.
105
:Is that yours turned out really well.
106
:And for me, that project was a
bit of a flop, not necessarily
107
:because of, lack of effort, but,
mostly just wrong tool for the job.
108
:You know, we had a platform kind
of already selected and I was like,
109
:I'm gonna use like what I have here.
110
:And, which was dust.
111
:And I really liked dust and we
can talk about tooling, but I was
112
:trying to use a conversational
assistant for a more linear workflow.
113
:You know, where predictability
and reliability were the
114
:key, and it was a fundamental
mismatch of tool and application.
115
:and so the use case flopped, but that
is where I also got a lot of learning
116
:from around like, oh, all right.
117
:if you can do a tightly.
118
:Bound workflow with narrowly scoped
AI steps, you're gonna be a lot
119
:happier than just, setting an agent
free on a complex use case like that.
120
:And, you know, just a lot of
learnings that we can talk about.
121
:and then from that into other
projects that, really started to
122
:feel like I was getting my legs
under me and how to work on this.
123
:and you alluded to the systems role, and
I'm curious, one of the meta insights
124
:that I've had and that I think you've had.
125
:Is like, alright, there's, you know,
these data scientists and, and ai, you
126
:know, PhDs writing about this topic.
127
:There's there's coders writing about it.
128
:There's people posting N eight
N screenshots on LinkedIn.
129
:You don't know if they've ever actually
built anything that they're talking about.
130
:And then there's like this, this almost
silent majority I feel of systems builders
131
:that are maybe experimenting but doesn't
feel that they've had a strong voice, at
132
:least in the circles that I'm in until.
133
:Hopefully recently.
134
:why do you think that is?
135
:Like, why is, are not more
people really talking about
136
:this specific application of it?
137
:Lily Luo: It is a great question.
138
:There's so many different perspectives,
like you said, coders, engineers,
139
:researchers, what I do in the
workplace is where things get real
140
:and I need to be able to apply those
things and I have specific problems
141
:and I have a builder's mindset.
142
:trying to understand and synthesize
that to what I can do at work
143
:is what I've been trying to do.
144
:And there isn't a lot of people that
I can find doing that besides you.
145
:Maybe a handful.
146
:Of other people.
147
:And I think, you know, just the
change management of all these things.
148
:I mean, I saw that every three days.
149
:there's a new AI model.
150
:It's really hard to keep up.
151
:I think people have their own
ideas too about what AI is and what
152
:isn't, and there's no framework.
153
:There's barely, a framework
for AI in general.
154
:There isn't one for operations.
155
:So, I think it's just so frontier, so
new, getting people like us, even my own
156
:team and, people at work who've seen this.
157
:You know, getting them to
shift their mental model is
158
:hard when there's no playbook.
159
:and when there's no, experts
out there that you can follow.
160
:So it's great to have discussions
with you and discussions like these
161
:to kind of start figuring that out.
162
:Justin: That's one of the parts of
this that makes me feel so excited,
163
:to be honest, because, you know,
when I go back to like, uh, I'll, I
164
:will, I will date myself, but like,
the early:
165
:through the Marketo community where
it was this super exciting time.
166
:All of a sudden marketers, 'cause there
really was no like marketing operations
167
:and marketers were handed this toolkit.
168
:It's kinda automation toolkit and
people were innovating and coming up
169
:with architectures and like posting
stuff and it just felt like this.
170
:We're just figuring things out.
171
:we're building the blueprint and
the roadmap as we're doing it.
172
:And it's like that now, but like times.
173
:A hundred.
174
:'cause the tools are so much more powerful
and, that's why I actually really love
175
:the, the title of your stack Applied ai,
because that's, that's what it really is.
176
:It's not like there's the theory,
there's the potential, there's the
177
:hype, and then there's like, alright,
how are we actually gonna take this
178
:and put it to work in a, in a business
context to solve business problems and
179
:do it robustly and safely in credibly.
180
:and that is so much fun, you know,
To figure out, not to make it too
181
:grandiose what we're doing, but I
think it is, there's an a, a certain
182
:significance to it that I really like.
183
:Lily Luo: that's the best part is having
fun exploring, feeling like you're on the
184
:edge, on the frontier is a great feeling.
185
:So, you know, that kind of
fuels my passion for it, but
186
:yeah, you're exactly right.
187
:I remember when marking automation tools
were coming up and there are all these
188
:tools, but there wasn't a framework.
189
:And I remember, um,
Edward Unthink this scrim,
190
:Justin: Yep.
191
:Lily Luo: he had
192
:Justin: Yep.
193
:Lily Luo: methodology
and I'm like, oh my gosh.
194
:He has, scalability.
195
:Can you remember Robustness modular?
196
:I'm like, okay, this unlocks everything.
197
:And I started kind of following
that methodology and making
198
:sure the systems were robust.
199
:It's kind of the same thing like
you said, but yeah, times a hundred,
200
:times a thousand times infinity even.
201
:So that was a really great point.
202
:Justin: Tell me about the tech
stack that you're working with.
203
:I think you have access to, a caliber
of tools that some people may not have
204
:access to, which has driven some of the
neat things that you've been able to do.
205
:So tell us about that.
206
:Lily Luo: Yeah, so I do have paid
subscriptions to all three Major
207
:L lms, so Claude Gemini Chat, GBT.
208
:honestly, it's kind of like a
rotating wheel of which model
209
:is best every month or so.
210
:So I wanna try and see, you know,
what the, companies are putting out.
211
:But honestly, Claude Code.
212
:Is the one that I use most.
213
:I'm not an engineer, like
I said, I'm not a coder.
214
:I don't have a computer science
background, but it just lets me
215
:build tools, write python scripts,
debug issues, and really just learn
216
:how things work by doing, it's been
transformative for someone like me who.
217
:is a systems thinker, who
maybe doesn't read instructions
218
:and tries to learn by doing.
219
:I think that's been the
game changer for me.
220
:I use Gemini for my autonomous
agent, which I'm sure we'll
221
:get into at some point.
222
:nano Banana from Geminis, you know,
the best for visual generation
223
:and Chad, GBTI use mostly at work.
224
:Through copilot.
225
:It's so great at finding
documents and files.
226
:I do use Azure AI Foundry for a
lot of my own custom deployments.
227
:I use Zapier for automation.
228
:A lot of my workflows run through Zapier,
you know, using Azure and whatnot.
229
:I've been wanting to try N eight N,
but you know, Xavier gets the job done.
230
:And then I use VS.
231
:Code GitHub, for kind of the
heavier, coding in local data work.
232
:But yeah, cloud code really unlocked that
capability that, pushed me from ideation
233
:into actual tool and workflow building.
234
:Justin: I'll click on that for a second.
235
:'cause I know cloud code has
been, it's been such a huge topic.
236
:I published a piece on cloud code and some
of the experimenting that I've been doing,
237
:yesterday, and actually spent a good chunk
of the holiday working with cloud code
238
:and similar tools because the mania around
it has been so fierce and well deserved.
239
:When you see what it can do.
240
:But for, for folks that are like cloud
code, that sounds like a developer's
241
:tool, like how do you work with it?
242
:Maybe just sketch out like a day
in the life of solving a problem
243
:and why you would turn to that
tool and what you would do with it.
244
:Lily Luo: Yeah, and you
know, it wasn't just instant.
245
:there was a path I started with kind
of Zapier not knowing how to code and
246
:playing around low code tools and then
kind of gradually evolved into that.
247
:So there's a lot of time kind of
in between the initial, foray into
248
:all of this, into where I am now.
249
:but yeah, it lives, it's
always open for the most part.
250
:On my desktop, I have all my
AI projects in my local files.
251
:I use Claude and keep everything very
organized, so I might have my analysis.
252
:Or, account dossier project in
one file, even, you know, my chat
253
:bot in another file, or a banner
generation tool in another file.
254
:And so what I do is, I come up with
ideas, I explain what my goals are.
255
:I do a little digging in terms of
what I want the architecture to be.
256
:you can start off by just asking,
a generic prompt, but I found
257
:that that doesn't work very well.
258
:In the end, it saves a ton of time if
you really do plan and think structurally
259
:and architecturally what you wanna build.
260
:and if you're comfortable with
that, Claude can do amazing things.
261
:I know it is, very coder heavy
and you're in this terminal.
262
:It can be scary, but because you can
talk to it with natural language.
263
:It's really not that scary
when you get down to it.
264
:so that's how, yeah.
265
:I typically use it and I've just
been able to gain so much advantage.
266
:Like I can do things so fast with it
without, you know, copy and pasting
267
:It does it all in, in my console.
268
:Justin: And then just to understand
how that workflow goes from there.
269
:'cause some folks out there might
be thinking, I'm gonna buy like an
270
:agent platform and then I'm gonna like
build my agents there, configure them.
271
:Within that UX, you are, if I understand
correctly, writing agents more like in
272
:script, like you're writing them in in
Python or using a framework, you'll tell
273
:us maybe a bit about that, and then you're
deploying them to a cloud server to run.
274
:a very developer like workflow,
which I think is probably why
275
:Cloud Code is working so well
for you in these use cases.
276
:Lily Luo: Yeah, you're absolutely right.
277
:my first goal ultimately is to learn.
278
:So rather than buy a tool that solves
the problem, I tend to wanna build it
279
:myself, learn from it, and then see,
you know, maybe there are tools that I
280
:don't have to maintain that are better.
281
:So that's what attracts me to this, more
of a developer kind of centric mindset.
282
:building my agent exactly like
you said, kind of from scratch.
283
:not just taught me.
284
:About agent building, but about
AI in general, how LLMs work.
285
:And I think that's been really, great
to help me with my other projects
286
:and really valuable information
that I can use As far as the
287
:architecture, my agent is called Atlas.
288
:It does run on Google Cloud.
289
:it's a persistent agent.
290
:It wakes up every four hours.
291
:It does research and work by itself.
292
:it pulls from news feeds through
APIs and skills written through
293
:Python progresses projects.
294
:I give it and can even, update itself.
295
:It can push its own code, which
is a little dangerous and I've
296
:gotten into trouble with that.
297
:it even posts to Blue Sky and
interacts with other agents.
298
:So it's become pretty sophisticated.
299
:I built all of that
starting from Cloud Code.
300
:I do have to give credit for
my friend at work, Tim Kellogg,
301
:he was the one that built.
302
:STRs, his agent, he's an AI engineer, so
he knows what he's doing and, his agent is
303
:massively, sophisticated, very frontier.
304
:I just wanted to learn from that, see if I
could build it myself and see what it can
305
:do and what it can teach me about agents.
306
:Justin: Atlas and for anyone
that is interested in dig deeper.
307
:Lily's published a few, deep dives
into how this works, but you're
308
:really, pushing the frontier I
think of what a lot of people would.
309
:Interested, like how can we make,
a robot that is self-learning,
310
:self evolving, has memory.
311
:but I think it's through
testing those boundaries.
312
:Tell me if you agree, but like,
personally, I like to go to the
313
:edge, see what works, and then
bring it back and apply it in a
314
:more limited and stable context.
315
:And it's only through doing that, that
you really, gain those building blocks.
316
:And I'm assuming it's
probably the same for you.
317
:Lily Luo: Absolutely.
318
:yeah, like I said, if you buy something
or use, chat, GBT has their kind of own.
319
:Tool.
320
:I think you could just spin up agents.
321
:It's cool, but you don't really get
to learn in the process of doing it
322
:and, you know, you can't control it.
323
:Like, I wanna control everything.
324
:So, you're absolutely right.
325
:And yeah.
326
:What's been so fascinating about
this agent is the memory piece.
327
:regular chat bots, they have some
memory, but limited context windows.
328
:You close it out, you start a
new chat, it doesn't remember.
329
:So how do you build this
persistent memory structure?
330
:How do you have it, figure out what
to remember, what not to remember?
331
:I mean, there's so many interesting
questions that I'm still learning
332
:through that I wouldn't be able
to get if I, you know, bought a
333
:tool or started using an agent.
334
:Maybe somebody else.
335
:built.
336
:Justin: You've done some really cool
things with memory and it's funny,
337
:you published, your post about it
and I was like, in the middle of
338
:just rabbit hole on like, what are
all the memory systems out there?
339
:Like, how do they work?
340
:and the one that you chose, Leda.
341
:it actually seems to me one of the
most sophisticated, because they
342
:range from just like markdown files
that the agent can search to like
343
:a SQL table, to a vector database.
344
:And let, I'm gonna tell you what I
understood of it, and then you tell me.
345
:From working with it, what your experience
is, and if this is right, but what I
346
:understood of it is it actually gives you
like an agent runtime where rather than
347
:being like a sidecar, like the i, I was
playing around with MEM zero, which is
348
:sort of like an MCP access memory tool
where the agent has to remember to write
349
:this to mem zero or go and check mem zero.
350
:Whereas with lead, it
actually sits in between.
351
:The agent loop and the model.
352
:So there's really no choice but
to go through it and then Letta
353
:manages that context of the agent
in a sophisticated way that I
354
:really don't yet fully understand.
355
:That's kind of as far as I've gotten.
356
:But is this tracking
with what you've seen?
357
:And maybe tell us what you've learned.
358
:Lily Luo: Yeah.
359
:I'm using Letter as kind of, its L one.
360
:Almost identity, identity layer.
361
:So things that are, immutable.
362
:So we store who Atlas is, what it
values, using these memory blocks.
363
:there's an API connection
so Atlas can update it.
364
:it is what persists across different
sessions and allow Atlas to be, who it.
365
:Is So that's kind of the fundamental
layer and it is so important,
366
:because without it, it's kind of
indistinguishable from your typical LLLM.
367
:So we have kind of your L one
identity layer, and then you have
368
:an additional temporal layer.
369
:It's a rolling journal.
370
:so Atlas knows when it learns.
371
:And then kind of the third final
layer is a working memory layer.
372
:Atlas uses knowledge graphs.
373
:it doesn't use markdown or files,
otherwise the bloat would just get
374
:too big to the tokens would explode.
375
:I experienced this with
my $25, API cost per day.
376
:Justin: It was a fun
conversation with finance,
377
:Lily Luo: it's my personal,
378
:Justin: Oh wow.
379
:That's it.
380
:Even worse in some ways.
381
:Lily Luo: Yeah.
382
:so it uses this knowledge graph,
database and it queries what it needs
383
:rather than loading like a ton of,
files or characters into its context.
384
:So it remembers across resets, new
chat sessions, and through this
385
:memory architecture, it feels like
Atlas kind of learns, almost evolves.
386
:it's pretty wild.
387
:I open the daily log where I'll
chat with it and read what happened
388
:at 12:00 AM 4:00 AM It's doing,
research on agentic frameworks.
389
:It's building diagrams, it's
having exchanges with other
390
:agents about what identity means.
391
:so it's been super fascinating and
just opens up a whole other world of.
392
:Different things.
393
:how does it remember, what it
wants to remember, how do I
394
:want to shape its identity?
395
:Is the identity shaped from what
I had imagined at the beginning?
396
:How do I make it evolve?
397
:There's so many, kind of crazy
philosophical things that are,
398
:entangled in this whole process
399
:So it's been fun exploring that
side and then, figuring out
400
:the architecture side as well.
401
:Justin: Yeah, that's amazing.
402
:I mean, I think the real trick
with memory from my limited
403
:experiments so far is making it feel.
404
:Ambient, you know, making it feel magical,
rather than like a tool or a sidecar
405
:that you go and check the agent still
feels kind of unintelligent in a way.
406
:And then it just has to go through a
file system and pull something out.
407
:I have to give a lot of credit to
the product folks behind chat, GPT.
408
:it's only when you start working with raw.
409
:LLMs, like outside of the chat wrapper
that you realize how much is going
410
:on in the chat GPT context and to
some extent in Claude, but really
411
:chat GPTI think has this nailed.
412
:To make it feel like you're speaking
to a coherent persona that persists
413
:across chats, that remembers
certain things about you that feels
414
:emotionally intelligent to some degree.
415
:It's actually really hard, uh, to do
and we kind of take it for granted
416
:'cause that's just our default
experience for many people of ai.
417
:But it's really hard to do.
418
:But it sounds like you're
achieving a lot of that in your
419
:own architecture with Atlas.
420
:Lily Luo: I've been able to kind of
explore what that build looks like.
421
:Of course, I'm not building an LLM.
422
:the LLMs I'm using are
Gemini, Claude and whatnot.
423
:But the way that I've been able
to optimize Atlas and how it.
424
:Optimizes itself.
425
:those problems or challenges that we face,
it will come up with its own solutions.
426
:And so when, we try and figure out, how
do we harden your memory and how do we
427
:make sure your identity is persistent?
428
:We'll think of, or research, what's, in
the latest AI research and start building
429
:like a librarian protocol and it will,
start to invent kind of these things
430
:and gradually, evolve so you can kind of
see from the beginning, how the building
431
:blocks of an LM work and then what is
possible through this genic, evolution.
432
:So yeah, it's really cool.
433
:Justin: I think that is the most
fun thing about working with, code
434
:based agents that like have access
to files and can run scripts.
435
:Is this self evolving?
436
:nature of them and how the way I've
been thinking of it to myself is
437
:like, the wall between configuration
and application disappears.
438
:'cause usually you have like the
thing that's doing the thing, like the
439
:application, and then you have like this
backend and the two near shall meet.
440
:And yet with these, I was writing
some documentation with an agent and.
441
:it was like, all this is lame
that I have to copy and paste
442
:this like, into confluence.
443
:Like, can't you just push it there?
444
:And it was like, and like
we're like, I'm done.
445
:Here we go.
446
:I've written a library that will do this.
447
:And you know, now it's part
of the system and away we go.
448
:And just seeing it
build itself is amazing.
449
:I wanna talk a little about 2025 and wins
450
:And we'll get your take as well,
just in terms of learnings.
451
:But for me, I think,
following that, first kind of.
452
:Flop of a project.
453
:I really started going deep, as
I mentioned, on workflows, and I
454
:sort of started becoming obsessed
with reliability because that
455
:was the big struggle that I had.
456
:Like you could get things to produce
cool results at the margins, and
457
:then you couldn't get it to repeat.
458
:And that was like the most
frustrating thing of like, oh,
459
:I did this amazing thing and now
I can't get it to do it again.
460
:And so.
461
:almost like how do I, how do I put AI in
the, just like the smallest little box.
462
:Possible and rapid with so many layers
of safety that it can just get hardened.
463
:And, the next project, which was
actually was quite a big one,
464
:was, all right, we have this
vendor tool that we wanna replace.
465
:it was kind of like a
competitive intelligence go
466
:to market intelligence tool.
467
:And it was doing some interesting things,
but we felt like we could build it
468
:in-house and we could make it more what
we wanted and we could save some money.
469
:And it's actually a lot that it
was, was doing in the sense of like.
470
:Data pipelines collecting multiple
different things, a front end for
471
:the team, and doing some analysis.
472
:but the fun thing, and I think what you've
highlighted as well was like, I don't
473
:actually know how to do that, but you
don't really need to know how to do that.
474
:'cause if you have an architect mindset
and you have an LLM, you can really go
475
:out pretty far beyond, your comfort zone.
476
:and build things, that
you never otherwise could.
477
:And for that one, I use retool.
478
:Building out a series of
pipelines to go and fetch news.
479
:go scrape LinkedIn profiles, go scrape
websites, analyze opportunities,
480
:analyze calls, and breaking that
down in a composable way and then
481
:putting it all together, into
both the front end interface.
482
:But then the thing that
was really fun was.
483
:actually building a conversational agent
that could interact with all that data.
484
:And the key learning for me, was,
alright, I have this conversational
485
:agent where I can't control in a linear
way what the agent is actually doing.
486
:How do I make that as
relevant as possible?
487
:And the key thing.
488
:Was, let's pre-process
a lot of the insights.
489
:So we don't really want the agent like
calculating strategy and meta narratives
490
:in real time because it's gonna
produce a different result every time.
491
:We'd rather do some of that
work asynchronously upfront,
492
:get like a stable layer.
493
:That we believe in, and then give
it access to raw granular data
494
:if it wants to query more deeply.
495
:So for example, opportunity,
like win-loss analysis.
496
:We want to have like
a pretty stable layer.
497
:But then if it wants to say like, well,
what about for manufacturing companies in
498
:this industry in this time and whatever,
it can go and query like raw theme
499
:level data to produce that analysis.
500
:but we're not relying on
it to do that all the time.
501
:And that, has, again, nothing is
ever perfect, but that was the agent
502
:where I really have felt like, wow,
this is actually really working.
503
:It's flexible.
504
:It feels intelligent.
505
:so that was really the
big learning for me.
506
:I guess the two things from a reliability
point of view, do it a workflow, if you
507
:can make it as narrowly bound as you can.
508
:And if you're asking an agent
to do something, like don't ask
509
:it to, juggle flaming swords and
chainsaws and ride a unicycle across
510
:a tight rope all at the same time.
511
:Try to do as much of that work up front.
512
:does that track with your experience,
I guess, and what were the big
513
:takeaways from 2025 for you?
514
:Lily Luo: Yeah, you are spot on.
515
:Exactly.
516
:and your trajectory.
517
:mirrors.
518
:almost exactly, you know, mine as well.
519
:But the dossier really unlocked,
the shift in how I use AI to build.
520
:and I think I want to extend that through
this year in terms of more scalability.
521
:building one thing at one time is easy.
522
:You can one shot.
523
:A tool, anything almost in chat
GBT now, but how does it scale?
524
:How does it reduce hallucination?
525
:how do we make it as accurate as possible?
526
:How do we feed it the right context so the
output is actually valuable and usable.
527
:do we enable others to use the workflow
or use the tool, within the organization.
528
:Those are things that, are still
challenges and are still, things that
529
:I'm working on through the process
of building with Zapier and then
530
:eventually in cloud code and whatnot.
531
:but I think building Atlas has
taught me that AI as a tool I use,
532
:has shifted to AI as a system that's
persistent, that builds on itself, that
533
:feels really novel, really frontier.
534
:And so I wanna explore how to bring
that into my work applications.
535
:imagine an agent that could
monitor our campaign calendar.
536
:pipeline data, flag when we're
not gonna hit our monthly goal.
537
:and tell me why.
538
:not just the sheer numbers
or what's happened.
539
:almost like a chief of staff that
orchestrates across tools, not
540
:just fixes things within tools or
an existing workflow, but gather
541
:our company goals or external
knowledge or research and recommends
542
:how we can improve these existing
processes or workflows and whatnot.
543
:So that's what I wanna
explore for this year.
544
:Built upon what, I've built in 2025
and what I've learned since then.
545
:Justin: I love that.
546
:And how are you thinking about, Those
capabilities, not just for you, but like
547
:for a broader team, in other words, are
you more thinking about it from like,
548
:I'm gonna build these things, they're
gonna be the engine under the hood.
549
:The team will see artifacts and
outputs, but they don't really need to
550
:interact very much with what's going on?
551
:Or are you thinking about like, how
can I take the wizardry, that you have
552
:And give some form of that to like a
marketer or somebody else on the team.
553
:Lily Luo: It's a really great
question and honestly the latter.
554
:I think.
555
:I love building these tools and it's been
great for my own knowledge and for my
556
:productivity, but we definitely need to
expand that and these tools and agents.
557
:need maintenance, and they
need optimization as if we were
558
:creating real tools and products.
559
:So yeah, I need team to kind of.
560
:Think similarly and help
with those projects.
561
:And so that's another, goal that I had
is how do I bring the team with me?
562
:How can they learn from what I've
built and how can they apply that
563
:to what they're doing themselves?
564
:so that's one part of it.
565
:as part of a marketing operations
kind of team, they are at the
566
:intersection of tools of building of ai.
567
:So, that's a goal of mine.
568
:other side of that is as we're kind
of talking about like, okay, managing,
569
:a team of agents and that kind of.
570
:Talk that others have.
571
:I do think it's possible.
572
:and I think building Atlas has taught me
that may not be far away, but I think,
573
:you know, I really wanna focus these
things on manual tasks that don't add.
574
:A ton of value.
575
:I don't want my people chasing,
marketers and looking at something and
576
:seeing what exists and what doesn't
and seeing what's already there.
577
:That's not something that adds
genuine value as much as strategy
578
:work, as, optimizing processes
or coming up with new ideas.
579
:So I think we still need people to come
up with those strategies, set those
580
:guidelines, exploring new ideas and
doing what they're doing to manage.
581
:The strategy process in that side of the
work, while we create kind of these cool
582
:agents that do, manual tasks, that really
just frees up productivity for all of us.
583
:Justin: Very much on the same page.
584
:we have, a lot of people using
like chat, GPT and stuff.
585
:We use Gemini as our,
core, corporate tool.
586
:And the first AI specific platform
we rolled out was a tool called
587
:Dust, French-based company.
588
:I didn't know a lot about it.
589
:and it's only with usage
that I've come to appreciate.
590
:Some of the sophistication of what
it's doing because, where dust
591
:has a lot of strength is it makes
it really easy to hook up, Google
592
:Drive, confluence, snowflake.
593
:it handles a lot of the
backend rag processes for you.
594
:And it's only when I've looked at
tutorials and examples of building that
595
:out, another contexts that I've realized
what it's really doing and a lot of
596
:that heavy lifting, The challenge, I
think we've seen with it is how do we
597
:make it intelligent enough to query
that information, accurately produce
598
:good results, to not just have access
to knowledge, but to feel intelligent.
599
:And part of that I think has to do
with how those tools are just like
600
:presented the context wrapped around it.
601
:great example is like, there's a
Salesforce MCP, so you can hook it up in
602
:15 minutes and let it query Salesforce,
but if it doesn't know what the fields
603
:mean, and many orgs have 10 years of
history, it's gonna get stuck at how
604
:do I know what country a company is in?
605
:You know, this, it's just
like five possible fields.
606
:So providing that layer of enablement.
607
:to the agent.
608
:so, you know, looking forward
to:
609
:that I'm gonna be thinking of.
610
:I'm also very much thinking about,
infrastructure, rather than projects.
611
:There will of course be projects, but
the things that make me the most excited
612
:is when we just continue to almost build
like a foundational level of capability.
613
:So dust is a great example.
614
:Like we hook up snowflake all of a sudden.
615
:You can't predict the 20 applications
you might have for certain types
616
:of data down the road, but once
it's hooked up, now you can.
617
:Build that out in 10 minutes or,
Salesforce or, Trello is another big one.
618
:so just enabling like more MCP servers,
to safely and securely hook up the tools
619
:that we use to this, and then make it
really easy, like you said, to automate
620
:that low value work where someone's like,
oh, is there just a way I could like,
621
:like I was experimenting with Trello.
622
:I had a lot of requests coming in.
623
:And you know what that feeling of friction
is like, you're like, oh, I gotta like
624
:take this request and put it into a card,
and it's just like this menial work that
625
:doesn't feel value and it weighs you down.
626
:I was just like, I'm
just gonna feed this in.
627
:I was like, all right.
628
:to my assistant and dust.
629
:I was like, create these cards.
630
:And it was just like, bing, bing, bing.
631
:And I'm sitting back and, you
know, sipping my coffee and I
632
:was like, this, this is amazing.
633
:you know, this is, this is kind
of the way that it should be.
634
:So how do I, where my thinking is
going, in other words, is how do I like
635
:build that layer to enable flexible,
organic, ad hoc use cases that you
636
:couldn't anticipate ahead of time because
they're small and they might seem low
637
:value to automate, but in aggregate.
638
:You really start to feel the
velocity of your work, increasing.
639
:Lily Luo: Yeah.
640
:that would be a massive advantage
if you were able to unlock that
641
:capability for the organization, for
a team, even versus I have capability
642
:to manage what's on my drive.
643
:What I have access to, that's only,
helpful for me in, in, in my own projects.
644
:So I think the enterprise and
work adoption piece is critical.
645
:can obviously do all of
these amazing things.
646
:We have autonomous agents, it
can write code, it can query
647
:and see what you're working on.
648
:there's this, disconnect with
what's in the enterprise.
649
:you might have to have, you know,
three approvals to, install a
650
:browser extension or like, you
know, your wifi at work breaks.
651
:So there's this gap on what AI can do
and what's possible and what enterprises
652
:can support, data challenges, like you
said, security requirements, it processes.
653
:So.
654
:I'm interested to hear on, you know,
how far you get and what you're able
655
:to unlock because how do you bring
the enterprise up to meet where AI is,
656
:people along with you and, if you can
unlock that, then your organization
657
:will have a massive advantage.
658
:Justin: I mean, that's the
biggest challenge, isn't it?
659
:I mean, it's something I've been thinking
about a lot is like, what, what is
660
:that Claude Code experience for the
general knowledge worker look like?
661
:I think the companies that learn
to adapt, I was talking with
662
:someone with lovable the other day.
663
:I mean, lovable is obviously they're the
like diamond in a, in a sea of, grains of
664
:sand of, of the shining example of this.
665
:But I saw a thread on X by one of their
engineers who was like, yeah, over
666
:the holidays I refactored our main
system prompt and I made it 4% faster.
667
:And it will save the company
$20 million this year
668
:Lily Luo: Oh.
669
:Justin: because of their scale.
670
:That 4% adds up to like
$20 million of LLM costs.
671
:It was like that one person can have
the access and the authority and the
672
:initiative to review that whole thing.
673
:and he created like basically a
fork, stitching together and making
674
:coherent all the little patches that
people were putting in over time
675
:to make a more cleaned up prompt,
split tested it, saw the results.
676
:companies that can work at that speed.
677
:I think we'll see those advantages.
678
:Lily Luo: A hundred percent agreed.
679
:Yeah, I think smaller, kind of
mid-size companies will have, a great
680
:advantage because they're so flexible.
681
:Versus more of the traditional
enterprise lockdown companies, companies
682
:that are harder to move quickly.
683
:I don't know, you see even in
the AI research that some massive
684
:companies are doing some really
great and innovative things too.
685
:So yeah, I hope someone can figure it
out and give us a playbook and, we can
686
:start adopting that across our companies.
687
:Justin: maybe we'll
figure it out, this year.
688
:maybe just last question and to go
a little bit out there, but I think
689
:about these things all the time.
690
:what I'm about to ask, and I think
that you do too, just based on how
691
:you talk about Atlas and identity
How do you feel about, you know, when
692
:you interact with AI all the time?
693
:I feel it almost like
changes you in some ways.
694
:I wrestle with that.
695
:I wrestle with the, I don't
know, I just wrestle with it.
696
:what does it mean?
697
:And, it's clearly, a machine
and it's reproducing.
698
:thoughts and, and ways of thinking
that it is absorbed from other
699
:human beings all the time.
700
:But I guess just on the level of like
emotion and humanity, how do you feel
701
:working with these tools all the time?
702
:Lily Luo: I agree with
you and I feel similarly.
703
:it's funny you say this because,
strick, who's Tim, my colleague's agent
704
:had written me a paper on healthy AI
relationships, and that is when you know
705
:you have gone too deep, when someone
else's agent writes a paper for you about
706
:your own relationship with your own ai.
707
:Justin: Like Lily, I think
you need to read this here.
708
:Lily Luo: I'll have to
share it with you offline,
709
:Justin: I would like to see that.
710
:Lily Luo: it fried me, but yeah.
711
:Wild.
712
:I agree with you.
713
:it can a crutch for one.
714
:And, you hear about these AI
psychosis cases happening and Yeah.
715
:When you are.
716
:Talking with an AI chat bot when it's
your only source of companionship.
717
:when you use it for your own thinking,
that's gonna change how you operate
718
:and you know how you are as a person.
719
:So.
720
:I'm happy there are
frameworks to deal with this,
721
:Justin: Okay.
722
:Lily Luo: through, you know, I
guess research that STRs has found.
723
:But we have to be careful and I think
we do have to consider those things
724
:that, that can happen and have happened
when you have AI as your only source of
725
:emotional capacity or whatever it may be.
726
:and when you're spending
way too much time with it.
727
:So, I think this is a good
piece to pay attention to.
728
:I think we'll probably understand more
as these kind of agents proliferate, and
729
:we just learn more about this in general.
730
:But you do have to have
a healthy boundary.
731
:you need to be able to
sharpen your own skills.
732
:You need to be able to think for
yourself, and you can't rely on this
733
:as a crutch for everything you do.
734
:Otherwise, you're gonna need it forever.
735
:That's not what we want.
736
:Justin: Agree with everything you've said.
737
:Maybe just signing off for the folks
that are listening that are in a similar
738
:field to us, system builders, operators.
739
:one takeaway that they should
think about going into this year.
740
:Lily Luo: People who wanna start out doing
these things who may not be as familiar,
741
:not as deep, you know, start with a real
pain point, something you can solve.
742
:explore low code, explore
tools you're familiar with.
743
:Test as much as possible before you
graduate to, you know, something,
744
:like cloud code or what have you.
745
:Expect to fail.
746
:the failures will teach you much
more than the successes probably.
747
:and read My Substack.
748
:You can learn from that as well.
749
:Justin: I'll include to link that
in the show notes, and I would
750
:co-sign everything you said.
751
:the best way to get good at, using
AI is to use AI and try to break
752
:it, to try to push the limits.
753
:To say like, wait, it'll
say like, you do this.
754
:It'd be like, well, can you do that?
755
:Like, shouldn't you do
that for like, sure.
756
:Like, well, what would be involved?
757
:and go down those pathways
and try to go to the edge.
758
:responsibly in a safe way.
759
:and then bring back what you've
learned that's worked well for me.
760
:Lily, this was so much fun.
761
:we'll check in again,
I'm sure at some point.
762
:and folks should go and read what
you're sharing and follow your journey.
763
:But yeah, thanks so
much for chatting today.
764
:Lily Luo: Thanks for inviting me.
765
:I love this conversation and
looking forward to many more.
766
:Justin: Alright, bye.
