Name: Blueprint Roundtable: B2B support leaders who made AI work
Uploaded: 2026-10-28T18:19:08.028Z
Duration: 1 h 28 s
Description: Blueprint Roundtable: B2B support leaders who made AI work

Transcript for "Blueprint Roundtable: B2B support leaders who made AI work": Hello, everybody. Welcome to the first blueprint round table in our series. So excited to be chatting with you all. My name is Janan Pang. I'm the SVP of solutions, deployment and success here at Fin. My teams help customers, get their Fin AI agent live and then push towards the highest levels of performance and impact for their businesses. So, quick background for anyone that is new to the blueprint. It is our guide to deploying an AI agent for customer service, but then getting past that launch phase and into scaling. And so we have cocreated this with many of our customers as we have gone through this journey ourselves and with them. And it's a living, breathing, asset to help CX leaders, think about launching and scaling an AI agent for customer service. It's vendor agnostic. I am from Fin, but this is not a sales pitch. And so what you learn today should apply to, you know, anything you do in CX with AI. So, the the focus for today, is a b to b focus. And so the blueprint that we've created, it applies everywhere across any industry, but we wanted to bring it to life specifically for the b to b space for today. And so I am joined by two incredible pioneers and leaders in this space, Victoria Lowney, who is the VP of Customer Support at Inside Real Estate, and Alex Rivas, who is the Manager of Knowledge and Content and Instructional, Instructional so couple of housekeeping items as we get started today. The session is being recorded, so we will share it afterwards. We encourage you to ask questions throughout. Drop them in the chat, as we go, and we will dedicate time at the end to answer your questions. Alright? Welcome, everybody. I'm seeing a lot of engagement in chat already, so let's get into it. Victoria, would love for you to kick off by introducing yourself, and then, Alex, you can chime in afterwards. Hi, everyone. Victoria Lowney, here from Inside Real Estate. As Junan mentioned, I'm VP of support, and I've actually been with the company almost ten years. And so seen seen customer support through through the journey over the past decade. For those of you who don't know, just a little background. Inside Real Estate is a real estate software. It's kind of a one stop solution. So CRM website lead gen, all of that for customers and and, that include agents from small teams, brokerages, all the way through large brands such as REMAX. So thank you guys for having me on today. Alex, you wanna do a quick intro? Yeah. Absolutely. Thanks for having me. Hey, everyone. My name is Alex Rivas. I'm from HoneyBook. I've been at HoneyBook for a little over a year and a half now. My role at HoneyBook is the manager of our knowledge and instructional content programs, and I've... My team has also been leading Fin, our support chatbot. HoneyBook, just a little bit about it, is a CRM that focuses on, small independent businesses and helping them run more autonomously and efficiently. We've helped over a 100,000 businesses run through our platform, and use our services. Yeah. And that's a little bit about me. Alright. Let's get into it. I'm so excited for this. I have... I feel like I've heard so much about your stories, from your CSMs. They talk about you all the time, so I'm excited to dig in. And so the conversation today will actually kind of follow the structure of the blueprint. And so let's start at the very beginning. You know, one of the things that we talk about is, you know, leadership support and buy in as you are getting into this journey. And I know both of you, Victoria and Alex, you both had leadership buy in from the very start. Would love maybe, Victoria, if you start, share a little bit about what that looked like and, what was that... What did that mean in terms of, you know, your journey to have that backing? Yeah. We... We've always taken the approach of continuous improvement, and so whether it's AI, whether it's some other new technology, we want to be there first. We want to explore and see, could this be a benefit for our business and for our customers? The two two big drivers, you know, is efficiency and and experience. How can we drive a better experience for our customers but also work more efficiently, work smarter with with our our our resources that we already have? And so, yeah, it wasn't it wasn't so much a question of if it was when and really more how how can we do this faster? How can we move faster? And so no no hesitancy really from from our leadership. If anyone was hesitant, it was probably b of, hey. We should probably look through these things and and, and and, you know, it was... Nope. Let's let's let's blaze the frontier and and figure this out and and see what we can... How we can leverage it for for everybody. So, yeah, it was, not a not a hard decision at all. Alex? Yeah. That's a good question. Much similar to Vic, we've had, you know, leadership backing, in leveraging AI and Fin in particular. You know, we started this off a few years ago with Fin as kind of more of a a testing ground for, for this chatbot and what it could actually do. And so, being able to have that leadership involvement and that leadership backing around, like, being able to move quickly, test things out, see if they're going to see if they're going to work and to win. It really helps us be able to move fast, really scale it, and understand whether or not it's actually working and what we can do to continue to improve that process as well. So it really emphasizes that ability to move fast for us to have leadership leadership backing and just making sure that we're looking at things from a lens of, you know, if it's if it's working we continue and if not we pivot and figure something else out. it sounds like it was kind of like an across the board, across the company, not just a CX specific thing, but across the company, there was a push towards AI innovation and and also most like a a culture of experimentation, with an understanding of, like, what are the business goals you're you're you're trying to drive towards. Is that right for both of you? Yeah. Yeah. We... I think the the biggest... You know, we we we... Our efforts really centered around support to start, but as we have quickly seen results, we've seen that expand through the organization. And even though support is one piece of it, it is a huge part of the customer journey. So you have to have buy in across the org, and and not just work in a vacuum as... You know, still move fast, but but but kind of keep everybody in the loop on, hey. Here's what we're doing. Here's here's what we're finding is working, and here's where we need to pivot. Awesome. Okay. So, alright. You have the leadership buy in and you're like, alright, we should go. Victoria, you're like, we need to think through some things. Where did you start? Take us through that journey. Victoria, let's start with you. I know you you you thought about this as, like, a phased approach, almost more structured. Would would love you to take us through that. Yeah. We... So even though we're moving fast, we did... I I did break it out, with the team on where do we start and how can we limit risk and and impact if if it doesn't go the way we think it will. And so we actually looked at our volume, kind of by driver first. What what are... Where's the most opportunity to get the biggest kind of, or or or view the biggest kind of return on this? And so a lot of our... Probably about 50% of our inquiries are just related to how to. Our system is quite robust, and so there's a lot of different areas that we just get questions on. There's there's a learning curve. And so rather than kind of trying to figure out the big... Bigger rocks, we kind of said, hey. We have this information. It lives in the help desk. It lives in our our reps', you know, brains. And how can we design our, our FIN content or use existing content to to feed FIN, to to just see, hey. Can we get these customers' answers faster and accurately? And so we chose to start with our our how to, and I should say now, non surprisingly, it it was a success. Was able to give customers information faster, which means they're not waiting around. They're they're getting what they need and moving on with their day, which is better for everybody because it also creates a better experience for customers. They get what they need, and they can continue on with their business. So started with the how to, and then from there, kind of went, okay. Well, what's next? We also started by channel. We started with our chat first. We've we've historically kind of always, like I said, used a continuous improvement approach, and so we'd had chatbots running. It wasn't anything brand new to our customers to all of a sudden have more, let's say, automated, answers. And so once we started to see, okay, this is working, we're refining, We then expanded to email and now actually voice as well. So we've been able to kind of take a rinse and repeat approach by channel, based on the the nature content of of the questions being asked. Yeah. Alex, would love to to move to you to talk about kind of what the journey has looked like. I know you kinda joined, a little bit after I had gotten started, but Victoria actually mentioned something that was very interesting, which is, like, sometimes the content that you need is is locked in the support team's head. And as as somebody that owns content for HoneyBook, like, I also would love you to maybe take, take a pass. So, like, how do you how do you get that content out of their heads? Yeah. That's a that's a really good question. You know, and I think it's it's one of the main challenges that we face with, you know, just with teams in general, with companies in general. One of the approaches that we had taken is actually partnering with our support team, building really close and steady relationships with our product management team, our our engineering teams, etcetera, to make sure that we're really understanding what our products and features are, understanding where the gaps are within our current documentation, things that are, you know, that that are being done that may not be accurately reflected within our content, as well as being able to constantly solicit feedback from our agents, from our support line, from, our engineering teams, making sure we're leveraging, AI as well to also kinda help us spot check things if we're saying something, then our, our content should reflect that. And so being able to put those, those different channels of of feedback in place as well as spot checking it with, like, AI functionalities as well really helps us make sure that we're, we're keeping our content as up to date as possible as our product changes, and also making sure that we're auditing our content on a more regular cadence. As everybody in the content realm knows, content has a has a tendency to become stale if we're not paying attention to it. So, putting together a mechanism so we can audit that content is is one of the bigger, initiatives that helps helps us to drive improvements and accuracy with our content which subsequently feeds Fin as well. I guess while we're on that topic, I know I'm I'm going off script a little bit, but, like, what is the cadence? Like, as you as you've started to kinda scale and and realize that content is effectively infrastructure, for AI, you know, what is the what is the cadence that you actually are auditing your content? Yeah. It's a really good question. So right now, we, we audit we audit on, like, more of an ad hoc basis. We take a look at, like, where our high volume, conversation drivers are, where search leads us to as well, where things that Fin isn't resolving that get to our humans or end up in, you know, the engineering queue, etcetera. And we take a look at those areas and begin auditing that. What we're working on right now is building a more, robust and, a more kind of strict in a sense scheduling for auditing our our holistic, repository of content within our help center as well as, like, our internal documentation for our support agents. So we're working on building that into a more structured program, that gives us a more regular cadence. And with the use of AI at this point, it's it's making it a heck of a lot easier to put that together. Yeah. Awesome. So you're so you're looking at, like, the biggest kinda areas of opportunity, you know, any unresolved areas, things like that, and targeting that and inspecting kind of the content underneath. Absolutely. Alright. Let's move on. Early results, as you as you both launched, I would love to hear, you know, what surprised you as you... As your programs kicked off? And then maybe part two to that question is something that's very important when you're driving change within an organization is, like, how you communicate this change across the board, right? You have leadership backing, you have leadership vision to say, hey, like we wanna lean in here. I would love to hear, like, how did you socialize the results, especially because just like, you know, calling a spade to spade, like, support teams are never really going to be the most outspoken teams to kind of puff up their chest and say, hey, like, look at what we did. So I would love to hear, you know, how you did how you did that. Victoria, sorry. Let's let's start with you. I'll I'll I'll. separate... No. No. That's okay. I know you've got a visual, so we'll pull, that up in a second. yeah, yeah. I was genuinely surprised with how early we saw engagement with our our AI assistant, and and, honestly, early resolution rates. You know, I mentioned we started with our our how to. And so if I think back, okay, if we have accurate content, the answers are are being delivered effectively, I shouldn't be surprised by by the engagement and the resolution because... And really one drives the other resolution if we're delivering, the, you know, right results. It's effective. Like I said, it's creating a better experience, but it's also creating that engagement and trust so customers know, hey. I can come back and ask other questions and get on my way and get what I need. We weren't sure initially how our customers would respond. Just we have very outspoken customers. We love them. They tell us what they want and, and and communicate effectively with us. So we weren't quite sure, but it was actually... It's it's been quite well received where it's very effective and and like I said, 50 of that volume we see is how to and it's just... It's learning. And so, it it can be... Yeah. It was... Just just with the the the speed at which customers engaged was what kind of caught me off guard. And then and then also as we went through and expanded across our different channels, even more surprised. You know, chat's very interactive. I I don't think it was very surprising to customers to see, okay. I've got an AI assistant now in in support. But once we enabled it on email and then specifically voice, I was blown away by how effective voice was. We we monitor those monitor those calls very closely early on because voice and real time interactions are so critical to adding value to customers, especially within support, that it can be a make or break experience. And so we were almost listening in as as as real time as we could, and there's one, very, very early on, transcript I actually read, and it was so natural back and forth. Customer got exactly what they needed and left a five star review right after the call. I mean, it was the most perfect experience you could ask for, and so we built on that. And to your second question, one, how did we share and really celebrate the wins? We had, very early on, a lot of buy in, you know, from our our executive team. We also, had from our senior leaders, you know, this was something new no other team really had kind of dived in and started to explore yet. And so we, it was every day. It was here's our win from yesterday, here's the next thing, or here's what we're focusing on, via Slack. You know, we're we're... Got some folks in office, some some remote, and I'm sure everybody is aware of that work dynamic. And so we leverage Slack quite a bit to just share wins both in real time and then, kind of a a day before recap. As we've continued to scale over the last year, since our initial launch, we now share weekly updates with our executive team. Here's here's our our resolution rates. Here's our involvement rate. Here's our customer feedback. And then any big milestones, or new initiatives that are either underway or or just completed and then how those are going. So we've continued the conversation, and, Eugenia, you said, you know, we're not the ones to puff up our chest, but maybe we should be because this is really cool stuff and it's it's been exciting to to live in real time and and share out with the org. And we we have all hands as well, and so those come up of, look what we're doing. Look what our customers are experiencing, and look what they're actually giving us feedback on telling us. Thank... You know, this was very effective. I got on my day. I mean, even sharing when they've reached out at 3AM, where traditionally we wouldn't have anybody online, they would have to wait, you know, till the next day to get help, and now they're just able to kind of come and go on their schedule, get what they need, and and it's been very effective. And, I know you guys have... The the visual pulled up, this is kind of a year over year. One thing in particular, I mean, very obvious their involvement rate and deflection rates are significantly higher year over year just as we continue to invest in AI for support. But the other thing is what we've actually been able to do is reverse that customer satisfaction, experience because we've been able to actually take then those live experiences with our resources and add more value. We're not necessarily, focused on the transactional, support interactions where if AI wasn't able to solve, it's generally more complex, and then we connect with a human, and there... We we can spend time actually diagnosing, digging in, and creating more of those value added moments with our live team, which has gone a long way. The other thing too is, on the far right, probably most... Maybe less talked about, but most impactful, I think to support is, what's your running backlog? We've been able to decrease our running backlog, so any any case open at any given time by 17% year over year because we're not carrying the load of cases that AI is not able to solve. So, yeah, we were consistently sharing. I know I've digressed a little bit, but consistently sharing these wins internally and, because they they they impact support, but more importantly, they impact the entire customer experience, which is what we're all here I love that. And actually if we could pull that visual back up, what... What's cool is, you know, you talked in the very beginning in terms of, like, the leadership buy in, and there were kinda, like, two major goals. Right? Efficiency for. and then experience. And so you're reporting back on those, like, high level, like, hey, I would love to be able to, like, go after these things. And so you're actually showing the numbers. The other thing that stands out is it sounds like you're also sharing, maybe, like, anecdotes or, like, qualitative, like, snippets and and, like, real experiences. Like, that doesn't show up in a dashboard like that. How do you how do you find a way to show that type of color and, like, emotion, you know, that's coming from your customer base? Yeah. It's a it's a good question. We... So in our weekly readouts, internally, those generally tend to be a little bit more data driven. Here's, you know, quick quick snippet. Where we take advantage is, I mentioned, we use Slack quite a bit, and we've got a lot of company channels or team channels. And so when we come across, these really wow customer moments, we'll drop them internally, just throughout the day when we come across them of here's here's what so and so had to say or shout out to, you know, this individual or, you know, we... Monty is our AI assistant. Look what Monty was able to do. And, and then on our our larger kind of all hands, department updates, we'll just take screenshots and and pop them in and reinforce, hey. This is why we're doing what we're doing. This is why AI can be so pivotal in an organization, especially with support, is it's not just this tool to to leverage for efficiency, you know, and and I... That's why I don't like to say efficiency on its own because experience is just as important, and the two go hand in hand. And so when we can say, yes. We we have, you know, result 50% of of our customer inquiries with AI. But, look, we've also driven our experience up significantly because, like I said, we're giving... We're delivering faster, we're cutting out a lot of the back and forth or wait times that customers have historically had to to to work with and support, and we're able to to really create those those value added moments. And and so, yeah, every time we we come across something, we're we're showcasing it, posting it internally, wherever we can, really. Because, again, it is... You know, the the... There's data, but then to your point, there's the story... The the full story, really. Yeah. Awesome. Alex, I I know when we chatted last week, you talked about how, like, your executive team was really impressed, by the early results. And, you know, this is something that you also, you know, have a... Also have a weekly conversation with, like, the c level and reporting that out. Let's bring up your visual as well, but, you know, take us through kind of how you, the results you've seen and also how you've kind of socialized that across the organization too. Yeah. Absolutely. Some of the results that we've seen with this, one, it's it's really impressive to see what Fin is capable of, tailor... You know, partnering it with some some strong content as well to back its knowledge and and to back its foundation. Some of the early signs of success that we began seeing was much similar to to Victoria's as well like how quickly a member was able to receive an answer and receive a correct answer. On top of that our CSAT, our customer experience component of it as well, how we measured the customer experience remained high and it still continues to remain pretty stable. Roughly around 9092% is somewhere in that range. So those are some of the early wins that we started seeing is that an AI chatbot can actually answer so... A lot of these questions, answer them accurately and answer them fast. And in some time... In some cases and probably most cases, faster than a human might be able to. So if we're taking a look at this, like, you see that we started, we started... It's a year... Years long graph, just a very high level picture of it. But we've realistically gained about a 15 increase in our resolution rate over this past year. And a lot of that work stemmed from the underlying content improvements, but also building out more agentic solutions and actions that go along with it. And from a reporting perspective, I'll dig into the reporting part of it right now. What we've often seen is we will... We used to have these on more of, like, a semi monthly, cadence of where we would report out, to our leadership, help them understand exactly where we are, some of the the challenges that we're facing, things that we're doing to, to continue progressing this while not sacrificing our customer experience. Our customer experience is is, you know, at the core of of this along with the resolution rate. So those two aren't independent of one another. They have to pair together where we retain a high customer satisfaction, and also continuing to drive the increase in resolution. Excuse me. And so from a reporting perspective, the way that that's changed is we typically will, we now send a weekly report out that goes to our our executive team, to all of our leadership. And what it really showcases is the work that has happened over the past week, the improvement to our resolution rate, indicators for our customer satisfaction as well, work that's coming down the pipeline and, like, projections of how that's supposed to impact things. And this gets shared out on a regular on a regular cadence. The whole company has visibility into a lot of this information. And so that's usually how we're reporting out at at the moment is doing it on a weekly cadence to where the entire company has visibility. And from the perspective of, like, we aren't the ones that typically shout this out, I think this has really empowered us to be able to be those folks that that are shouting about, the wins that we're having within Fin from a content perspective and how that impacts Fin as well. And this really gives us an opportunity to highlight the impact that individual things that are happening within Fin and and, its capabilities that we're building into it and that y'all are building into it as well, have on our customer journey and the customer experience itself. Yeah. And it's... I mean, it's not even just Finn. Right? It's the it's the team that's behind, you know, it's the team that... It's your guys' teams that are driving this program. I think the other thing that stands out when I look at both of these visuals is there's, like, there's trending. There is a, it shows a path and a journey. And, so I think that's maybe, like, a good segue into, like, the next phase of, like, scaling. Right? Like, you've you've gotten off to a really fast start, and now it's, like, okay, how do we take this to the next level? Alex, maybe let's let's stay with you. What did the next phase of scaling look like, once you got past kinda like the early wins? Yeah. A lot of those early wins were the how to... The the more foundational questions about how the product functions, what they can do with it, etcetera. As we started digging in more and looking at what those gaps are that that still need to be filled, what we started looking at is essentially, like, are there things that we could be doing on behalf of a member? Are there things that we could be building out within Fin that takes an action on behalf of the member that our human team generally does? So part of our, like, high level guiding principle is if our human team is doing it and we're comfortable with, what this... The use case is, FIN should be able to do it. And I'll caveat that within reason. There are areas and opportunities that we we also still, take a deeper look at and and make sure that we're doing what's right, not only for the customer experience, but also from, like, a data perspective, a privacy perspective as well. And just making sure that we're not, we're not exposing information that shouldn't be that the right members of, you know, the right member of the account sees the information that they're, that is relevant to them. But largely, we take that that high level approach of if our our human agents are doing it, then Fin should be able to do it. And what that means in practice is we're... We've built out, agentic actions for Fin to be able to handle things like adding team members to their account, changing the, you know, the team member status so they can do different things within the account, removing, removing their accounts, deleting an account, etcetera. Many other things that have been implemented as well. Status checks on, you know, pieces of information about it. Your ten ninety nine reporting as an example. We can more proactively give you that information, tell you whether or not you... You've qualified for, you know, the tax reporting etcetera. But more information about your account that would generally take somebody to go in and dig into our systems to figure out what that answer is. We've been able to really tie into those systems and start being more proactive with that and being able to provide that that information and make Fin, actually do that work. I love that. Really kind of expanding the aperture in in which, you apply FIN, but I think it's also taking a hard look at, like, what are the jobs, that you should be looking at. Victoria, would love to hear from you, like, how are you... How did you think about kind of hitting that next phase? Yeah. Sim... Similarly to to Alex and looking at, okay, what else can can FIn or can AI do for our customers now that we've kind of solved that how to? And so next, we kind of looked at... There's early on troubleshooting when, you know, as mentioned, we've got a more complex system that's quite robust that maybe something presents differently than than it than it should. And so what we've done is created, several MCPs that that work with our our product connected to Fin to actually go in and and look at, you know, settings, for for websites. You know? My listing isn't showing. Okay. Fin is now looking at the website settings to see what may be causing the listing to to not be hidden. Is it a coverage area, setting? Is it, you know, something else? And and to be able to do that upfront troubleshooting is the next step we took to how do we further scale. Now when it does reach a live human, we have all that initial troubleshooting upfront. You know? Again, if enable... Isn't able to solve, in most cases, we've we've equipped it. So it's here's what you need to do to fix it, and, again, customer's on their way. And in the event that that there's additional troubleshooting required, we now have that all teed up to our live specialist. So even even though they've gotta connect, the the team member doesn't have to spend time doing all those same things because Finn's already done that. And from there, they can go take the steps they need to to solve. The other component that we're looking at is is acting on behalf of our customers, but as Alex mentioned, there are there are certain elements where we may entrust the team to do something, but we're not necessarily sure if AI is there yet. And and it's generally around security, data, you know, data protection, things like that. So depending on on the product area, we're we're we're spending a bit more time. We're spending a bit more time cross functionally as well. We have an an AI council internally that we've we've assembled that has members from across the organization because as we continue to scale it, there might be things that we... You know, again, we don't wanna operate in a vacuum, and so there might be things that we think are fine to go enable, you know, acting on behalf, but then there might be implications in other areas. So taking that very, those steps very carefully, so maybe not able to move as quickly, but still with that momentum to continue moving forward. We've, you know, part of scaling means new roles. And so red red team testing. You know? Hey. If we were to put put this type of, functionality or skill live, you know, let's put it through the ringer before we actually, before we really, you know, make it widely available so that we're a 100% confident in what we're what we're releasing. So still moving fast, but but being more, methodical in in what we're doing because of the nature of of the requests and and inquiry types. And and then, again, of course, making sure that our customers' data, is secure. You you touched on... Both of you touched on kind of, like, you know, setting up MCPs, getting, you know, access to data and systems, and, you talked about, from, like, a security compliance perspective, making sure that, like, that partnership and that awareness and transparency is there. Was there... You know, one of the challenges sometimes customers may have that I've that I've worked with is, getting engineering buy in to say, hey, like, we're gonna spend some time to expose these endpoints to create the MCP to allow you to connect in with these systems. Was that a challenge for you and, like, how did you navigate that? Victoria, let's say on you and Alex, I'm sure you have some some thoughts on this too. It wasn't so much a challenge. No. We actually stood up a a CX ops team that consisted of dedicated resourcing from product and engineering to power and drive this initiative. And and so, you know, again, we had buy in very early on, and so once we started to see the results, there there wasn't necessarily, again, a question of, okay. Well, how do we fit this into the road map? It was, nope. We need these dedicated resources to just focus on this so that we can really make more of an impact and scale, long long term. And then, again, where can we... You know, with with MCPs, how... You know, it's not just about, okay, adding to to the to to the, resolution rate. It's how do we make our customers' lives easier. And so, yeah, there... It was it was actually pretty easy to get the the resourcing. Going forward now, though, we've got, you know, there's always there's always more initiatives. So we are working very closely with the CX ops team, with our our our SVP of product on saying, here's what we would like to do next. Here's where the next big kind of gets are, and how do we how do we coordinate together to go continue to plan and and and really be successful with this, you know, 2027? What's our next twelve months look like? I'd argue that like a lot of the stuff we talked about maybe compounds to allow it to... For allowed that conversation to be easy, right? It's like the leadership buy in, the understanding of, like, the impact to the overall business, and then you reporting back out on the early wins, and then having, you know, that counsel as well as, like, the close relationship with the SVP of product. Like, it allows that transparency, that understanding the impact to kinda carry forward. So that way, when you do have an ask, hopefully, it's like, you know, it's like, of course, I understand the impact of this. Alex, Yeah. Yeah. It's... oh, Victoria. Go ahead. Oh, sorry. I was gonna. say real quick too. I mean, it it it does tie into overall strategy too. And so, you know, right now, we're in a we're in a... An interesting time. AI is definitely... I mean, it's new to everybody, and I... We're... You know, I mentioned take take that approach of continuous improvement, but also, like, really blazing the frontier. And so I think everybody has been kind of on the same page of let's be first. Let's not wait around for somebody else to prove something out and and and get after it. And so that... That's been kind of part of that that as well. So depending on, obviously, your org, the the strategy and approach is a little bit different, but does does impact, you know, the buy in overall. Yeah. Absolutely. Alex, how about you? Yeah. I'll say we, we didn't have a dedicate... We... Like, we didn't have dedicated engineering resources tied to Fin and tied to the initiative, explicitly. What we did have, though, was, a number of engineers throughout the company that were very excited about this and willing to dedicate their time, to helping us build out a lot of these agentic actions, the API endpoints, the coding that goes behind the scenes that makes these things come to life. And so early on, we had the opportunity to work and partner with our engineering teams. A lot of those who still... They all still partner with us to to this day. So, it's it's still an ongoing relationship, thankfully. But what... The way that we started was essentially, like, we would build out our our use cases. We would build out the process flows, etcetera, and and kinda map out all the little what if scenarios that could happen. That way as we handed that over to our engineering team to build, they knew exactly what the structure should look like, what the, you know, kinda choose your own adventure style, would lead to and what needs to be built out. Over time, we explored the opportunity of nontechnical folks, myself and my team, actually beginning to test out and build API endpoints and build the coding into our database, ourselves with the help of our engineering, kind of ramping ramping us up and getting us familiar with, like, how coding actually functions, some of the language terminology, things to think about as more of, like, a a software engineer. And so we were able to actually build that out and prove the success of that type of structure. And so what we do now is we actually build out all the agentic actions, that we're going... That that we wanna put into FIM, the data endpoints that we wanna surface, etcetera. And then we submit them as PRs, and they go to our engineering team for, like, a final pass and final review. And it gets published, you know, according to our publishing schedules, etcetera. And then we push it live within within Fin and continue monitoring, immediately afterwards and just closely monitor those type of actions to make sure that they're doing what we expected them to do. They're handling the conversations the way that we wanted them to, saying the things, surfacing the right information, etcetera. So we've had the... We've been blessed with having, a lot of our engineering team and members really excited about Fin and really excited about the goals that we have for Fin, and wanting and continuing to, dedicate their time to supporting this. That's incredible. I mean, I think same same thing. Right? Like, I think the the communication, the visibility helps to drive excitement and understanding of the impact that, and how they can get involved. But also, just I think, like, the credibility and trust that you've built with engineering, is really cool to hear. You know, as as Fin has continued to scale across your organizations, and it started to take on more and more work, I'm sure your organizations have changed around it and you've been able to do more. I mean, the the thing you just talked about, Alex, is is pretty amazing. So, can you give us an inside look into... Maybe let's start with, like, what does the org structure look like to... That, like, supports kind of the AI automation kinda arm of customer experience and then, any processes that you've had to adjust or change, to kinda go along with that. Alex, let's stay with you and then we'll come back to you. Absolutely. That's a good question. So the way that our our organization has shifted and changed, I'll preface with we actually just brought in somebody to the team, a few weeks ago who's got a bit more background and experience with chatbots to help us take Fin to the next level, hit our goals, and do things beyond as well. So what this actually means for our team is we focus a bit more heavily on, you know, our content, making sure our content remains accurate, up to date, etcetera, because that is a prominent dataset that Fint feeds off of. So, again, I I mentioned earlier, auditing our content is becoming a bit more, important to ensure that Fin's knowledge base, and the information that Fin is actually, giving to our members remains accurate. What that does for the rest of our organization as well. So, our... It opens up our support team to really focus on those complex cases, the complexities of our product, the the unique, situations that come in from our members that are far more complex for Fin and the knowledge base that we've created thus far. And so it gives us an opportunity to really hone that area and make sure that we're doing our due diligence as the human support team as well to dig into those and make sure that the, you know, situation is handled and resolved. But also this opens the door to... For us to have an even more, relationship focused experience with our members versus that transactional experience of here's your problem, here's your solution, thanks. Gives us that opportunity to really dive in and understand that person, their business, etcetera, that's using our product, and help them in that moment, but also, also think about what the future holds as well. So I think this goes into a bit more of the future state, but thinking about, like, how do we help them understand more of the product? How do we help them understand and, adopt more features potentially that would benefit them and their business and the way that they're using, our platform currently? So that's kind of how we're, shifting and structuring our our team right now. I can speak for for my team. Our support team is handled by, our support org, but that's largely kind of like the the ideas that are, that are spinning up. Amazing. Vic, how about you? Yeah. You know, I mentioned we... We've set up a CX ops team, and so there's there's a lot of roles that exist now that didn't even exist, you know, six months ago or twelve months ago because we have these new needs. And what we've been able to do is take, you know, a lot of our top performers and and allow them to have a growth path within the org that... That's beyond just support. We have some members of our team who who actually moved over to the CX ops team as AI, solutions architects. We have AI QA. There's obviously a a... We... We've had a content role before, but that role has evolved because it's now not just curating or generating content for our help desk, but it's also ensuring that does this actually... Is it digestible for for AI to effectively deliver answers to customers. We have also created new consultant type roles that some of our senior live specialists now now have taken on where, you know, as Alex mentioned and we've talked about is is it's not the transactional interactions. It's really more the value add. It's how do I do this with my account, or this is my business outcome that I'm trying to get to. How can I how can I do this? And some more complex inquiries. What we've also been able to do is roll out live phone support for all of our customers, which up until six months ago, we we weren't able to do. So it's actually freed up some of our resourcing to have more of those value, value added touch points. You're connecting over the phone where before it was, I'm just trying to do multiple chats. I've got multiple emails, and so now... No. It's... I've got actually capacity to answer the phone where where Fin, you know, Fin kind of intercepts all of our our calls first, but as soon as it's identified that a a human is needed, we transfer and we're able to capture more calls. We're able to to solve things more in real time, and what we've actually seen as kind of a a different outcome is our phone volume now actually has surpassed our email volume, which historically has been... Was not even something we would have even thought of. And what what we're also seeing, and if you're a support leader listening in, is historically, you know, you're thinking, okay. Well, to answer the phone, it's one to one. It... My my rep can only take so many calls in a day where they might be able to do more chats in a day. Well, what we're seeing is we're able to solve things much faster over the phone with a live rep much quicker than a chat or email historically would have been because they're... They have the time to do so, and and it's just creating... You know, again, I've I've mentioned this a few times, but but but far more value in those interactions than we've ever had before. So it's it's forced us. AI has forced us to get more creative and innovative with our resourcing and then develop these new roles to fill gaps that that, again, didn't exist before because we didn't leverage AI in that way before. So so it's been exciting, and and there's no doubt in my mind that we'll... We will need to develop additional roles as we continue to scale going forward. And we've got great resourcing, and so, you know, I think Alex mentioned this, like, the knowledge lives lives in their head, and so we wanna we wanna leverage that. Like, it's it's great for AI, but when humans are needed, we need we need really valuable skilled individuals to step in and and create those good experiences. So nice balance between between the two, AI and humans. I I did notice in the dashboard that, like, the the teammate CSAT, which you measure alongside kinda AI agencies that has also been going up. And so, like, it it kinda speaks to this idea of, like, the human support org continues to evolve and that you're... You have new offerings. You can be more proactive. You can take better care and deliver better experiences for your customers. I think both of you talked about that. Maybe one thing that's a it's a bit off script I'm I'm curious on is, like, you know, both of your dashboards showed effectively an efficiency and an experience style metric. Who wakes up every single day on your teams thinking, like, how do I continue to improve those metrics? Is it the whole team or is there as a person accountable for that? Can you take us through that? Vic, maybe we'll start with you. It's, probably me. Me and and and the few members who are really dialed in on on our AI operations. They're... You know, it's a daily conversation at this point. We're constantly looking at the data to see, one, where can we improve the experience of existing AI interactions, and two, where else can we expand this, across, you know, across the support org, but also where else is there opportunity in the business. And so probably our our AI architects, myself, and some of our other kind of more senior leaders across the org, it's like, where else can we do this? Where else can we do this? We know... We've been... Support is very reactive in nature and so by by proxy, so is our our AI support specialist. So now though that we've scaled, I mean, we're starting to think, like, where can we get proactive with this? Where can we anticipate the the needs of our customers better? And that's really then where you start, you know, engaging and and kind of consciously maybe forcing them to start thinking about this day to day is where can we leverage AI to to just create better experiences proactively as well. Yeah. One of the things I think about is, like, you almost have to treat it like a product, and you need to have a product road map and, like, a backlog. that you're executing against. Yeah. Alex, how about you? Yep. Yeah. I'll I'll take a a quick step back and talk about, the the AI component as well. The... Like, what this has opened up the doors for is is really just AI in general, allowing more more folks, more nontechnical folks like myself to really be able to build out and and optimize our own time and our own efforts, building out systems, tools, automations, things of that nature that can, as an example, go and scour our help center, our internal support documentations, you know, our internal docs within, you know, our our code base, etcetera, to really help us understand, like, when requests come in. It can... We've built out the the systems and processes and are continuing to evolve on these things of, like, it can go and do a lot of that legwork for us and go track down that information. So realistically, it can give us essentially, like, a suggested output and where it should live, etcetera. So we become the human in the loop. And I know there is a human in the loop, functionality and feature within Fin, which is awesome as well. But that's really where we, like, start spending our time. And I apologize. I I went back and... Can you remind me the question, Jen? Just who who wakes up every day thinking about how you continue to, yeah, improve? Yeah. Yeah. I think it's a... I I think it's a group effort, really. We have, you know, a program manager who, oversees our quality and CSAT goals. We have our support leaders who, you know, take a look at their CSAT metrics from their own teams as well. We have, our our person, who just recently joined and and our previous that was actually owning Fin as well, that wake up, take a look at our CSAT, take a look at, what resolution rates are, constantly talking about how we can make sure that we're continuing to improve or at least hold them steady at where they are if they're, you know, well above goal, and opportunities to just stretch them a little bit more. So I think from from my perspective and within our company, it's it's a it's a group effort. Yeah. There's there's multiple people that take a look at these different metrics that, that come in daily and are are checking them, and we have conversations around them every so often just to kind of, compare. So speaking of, like, inspecting metrics, let's let's talk about metrics. How have the... Kind of call it, like, the... Your metrics evolved, and there's probably some metrics that you used pre AI, that maybe change and and, they look different these days. But I would love, you know, Vic, if you could share how have metrics for your support org evolved over time? Yeah. We... You know, because we still have live support, I mean, AI is not a... Not gonna replace everything, for our customers, but we still have our our primary metrics for our live team in that we're looking at, you know, first response time, we're looking at resolution time, and we're looking at that customer satisfaction feedback. The calculations have changed a bit. You know, if we're looking at first response time, it's not necessarily first response time from creation because we know AI is actually taking up some of the time. So it's first response from that human pass off. How quickly are we, once once... How how quickly are we responding once it's actually ready to be, interacted with? And and then resolution time similarly, you know, again, what... The calculation is different. We're not necessarily looking at... Or holding reps accountable, I should say, actually, to the full resolution time when they've only maybe had the opportunity to resolve since a certain point in time. Now when we look at, AI metrics, which is almost treated as as a team slash an individual that is responsible for for these metrics similar to our live team in that we're looking at our resolution rate versus resolution time, and then we're looking at AI CSAT. So with Fin, you know... And, again, I know this is... You can do this with with with all sorts of different AI tools. But with Fin, what I love is there is a CX score that's generated by AI, and that that actually looks at all of the interactions that, have occurred within that that that that that case thread already and kind of scores it. So it's a little bit more inclusive versus just maybe the last moment that that there's feedback. What we've also done, though, it has... We've enabled your traditional CSAT survey for our AI assistant as well because sometimes customers, you know, aren't maybe expressing everything in their... In in the thread, good or bad, and we do want to initiate and ask for that feedback. So we look at a combination of that, but it's really been kind of taking what we've measured our live reps on and and that experience as a team and applied it to AI to make sure that AI is also accountable and the team that's running AI operations is accountable to the same expectations that we would have for our live staff, if not, you know, more. How about you, Alex? On the metrics side, how have metrics evolved? Yeah. It's, similar to Vic as well. Like, it... Our core metrics were, and our resolution rate and CSAT for FIM. For our human side, it it is also CSAT, you know, the customer satisfaction, scoring as well as, you know, first response times. The way that these have evolved and what we actually measure with them is really understanding where those where those actually currently sit. One, we take a look at all of our CSAT and, you know, our our metrics holistically. But we're now also beginning to use monitors within within FIN to start taking a look at, like, our human and, FIN scoring to make sure that we're evaluating them fairly, amongst what FIN is and what our human agents are actually doing and, beginning to start leveraging that system. That way, we can actually score it and and close in on it. We informally take a look at some of the metrics like, speed or average handle time from from Fin's perspective as well. That's a more informal just kind of anecdotal of Fin is doing its job, it's doing its job well, etcetera. We don't necessarily report out on that type of information as of right now, but, largely, the the main focus has still remained around, resolution rate and customer satisfaction scoring for Fin and largely for our human support team as well. Amazing. Alright. Well, with that, I I tried to cover as many of the questions. I've been kinda keeping an eye on those, as they've come in. Let's move to some of the the q and a that we've gotten. One of the questions that came through was our customers expect to speak with a human in all of our channels, voice, chat, email, and it's vital to the relationship, specifically kind of to to their industry that they, you know, you know, meet them where they're at. And so the question is, how can we use AI in the most, human way, quote, given that we will continue to have human agents online during our normal business hours supporting customers? We primarily need AI for after hours or weekend coverage, not to improve success rate slash speed or support team, headcount. So, Vic, Alex, any any advice, you'd give to this person, given that question? I I could take a first stab. I mean, if it were me, and kind of the approach we've taken is we greet every... So so every time a customer reaches out, Monty or our AI support specialist greets them first. It's, hi. I'm Monty, your AI support specialist. I can, answer most questions, do, you know, troubleshooting. It's... It it... It's... It gives them an intro of who Monty is and, and reminds them though that if at any point in time you need to speak with a live member, I can always pass you off. So upfront lets them know that there is that connection that's not this, hey. You can no longer connect with a human. What we've done too is, you know, initially upfront, what we were, you know... And and and I think a lot of folks do this is you wanna see how much AI can actually solve. Once you start to analyze the con, conversations taking place, the interactions, keywords, phrases being used, You identify where pass off should happen that maybe isn't explicitly being asked for. So I would say make sure you... You're you're constantly reviewing the interactions over, you know, text channels or voice. I will say a thin voice is so natural. It's it's very easy when you're listening back. I mean, you test it out as well that you can ask questions. It's very conversational, so it's still that personal touch. It's not this very, like, dry interaction. But really design your prompts around this. Provide, you know, your AI with guidance that you're a very customer centric AI agent. If any sort of tension is is... Or frustration is is detected, immediately pass off. So you can you can put the guardrails in place, tighter or stricter depending on your business model. And and I would recommend just upfront having, you know, that information provided. This is who I am as an AI agent. This is what I can do. If you wanna talk to a human, let me know, especially if you aren't, trying to drive towards maybe stricter efficiency targets and really leaning more on on the experience. So where I would start. Yeah. I'd I'd echo everything that that Vicks talked about, and, the sentiment analysis components of that have been really, helpful to us in the past as well to be able to understand and I... So, like, self identify for Fin, at least when somebody is frustrated or when Fin is not answering a question over a certain period of time or certain amount of responses that it's given, to be able to really understand, like, okay. Is this the tipping point to where we should hand off versus continue this conversation when it may not be providing the right response or the right direction for this member? So... Yeah. Amazing. Well, thank you both. Thank you for taking the time to share this, with the community. It has been so helpful. I know I have learned a lot, really appreciate the partnership and, thank you for taking the time. Thanks everybody for joining us. If you want to learn more about the blueprint, you can go to fin.ai/blueprint to see the whole kind of end to end. We just covered a sliver of this focused on b two b. Thank you all for joining us. Talk to you all soon. Thank you. Thank you.