GSD Podcast - Taking the Stress Out of Go-Live Day: Tips and Tricks from Pat Muirragui

Pat Muirragui and Jeff discussed the importance of a successful go-live for deploying large enterprise applications.

The process begins in pre-sales with good discovery calls to capture information from stakeholders, followed by an internal handoff, external kickoff, planning session, development cycle, and testing cycle.

Pat outlined who is involved on their side: Solution Architect; Customer Onboarding Manager; Engagement Manager; Cloud Operations Team & App Ops Team; Customer Support team.

The discussion also includes important steps such as:

• holding a launch plan session and setting an agreed-upon day for going live.

• understanding the customer's rollout plan, which might involve turning on the spigot at 10%, 25%, or 50%.

• encouraging customers to do thorough user acceptance testing and provide a demo as well.

• taking care of provisions including licenses that are specific to certain environments, server-to-server communication certificates, app store approvals, etc.

• running a smoke test by sending through some transactions in order to check if everything looks good from both sides.

• opening a War Room and measuring success criteria.

They also discussed post-go-live activities such as sending surveys to customers and celebrating internally with team meetings.

Pat Muirragui and Jeff discussed the importance of keeping customers happy, as well as addressing any potential issues that arise.

Finally, Jeff and Pat also spoke about creating a Proserv SAS community to help pro-serve people with templates, questions, and other resources related to their work.

You can connect with Pat Muirragui via LinkedIn:

Transcript:

Jeff  00:05

All right, we are back here. I think our first round number two with Pat Muirragui. How's it going?

Pat Muirragui  00:12

Well, Jeff, good to see you. Oh, it's always great to chat

Jeff  00:16

Sunny California. I can you know, I can see sun beaming off your face right now, which is

Pat Muirragui  00:21

actually pretty sunny day.

Jeff  00:22

Yeah. So Pat was here a few months ago, I if I was a good podcast host, I would have numbers and everything like that, but I don't so. But we talked about the diagnostic thing to improve the implementation experience. That's what the thing was, it's fine, because that's the implementation experience today. I'm just laughing because we've been dying and record this one particular topic for a long time. And then we have a special announcement at the end of this to talk about but we talk a lot in my podcasts and articles online, LinkedIn and everything about like, the pant, the handoff and the pre sales and then the kickoff and then the actual implementation. We never talk about go live, which is just everything gets real. And and as Padnos diseases in it every day. Now, if there's some stuff that you don't do, it's hard, right? Engagements go along. You're supporting them for life? Like it's just yeah, there's all sorts of stuff. Yeah. So also,

Pat Muirragui  01:27

we're the opposite where the customer never goes live. And they keep telling you next month, next month,

Jeff  01:32

and the finance seems like we can bill them right. You know, like we did all our stuff. So shores, Bill. Yeah. So So let's take this from the top. I think we, we have a document that we put together we meeting Pat and me with like one or two live, but like, but there's a whole bunch of things to go through. And we should probably turn this into a checklist that we'll talk about in an environment that will roll out later. But why don't you walk me through this because I view this as something that you probably have internal and, and have your team's going through it. And it's pretty extensive. And in, a lot of stuff has to go wrong if you follow this checklist, and you still can't have a good go live. So

Pat Muirragui  02:21

yeah, absolutely. Thanks, Jeff. So yeah, the, you know, the sheet that we that I had, or the list that I had initially shared is really, it's really a collection of six years of taking customer live and having things go wrong.

Jeff  02:38

I learned everything from breaking it. Yeah,

Pat Muirragui  02:41

you learn the lessons, you know, those hard lessons are sometimes the best lessons learned so So really, it's a, it's a collection. And I think you said it right where, you know, go to have a great successful go live, I mean, you need the right team. And it's also a build up, right. So you need to have. And when I say a build up, but it really starts the momentum, that initial beginning of the wave, if you will, starts in pre sales. So in pre sales, right, having good pre sales, Discovery call really sets the tone, builds a relationship with a customer captures all that information from all these different stakeholders that they have on their side, product and engineering. And so that leads to a successful internal handoff, and then an external kickoff, a good planning session, a good development cycle, a great testing cycle. And then finally, you get to this deploy stage.

Jeff  03:42

Yeah. So you guys are deploying large enterprise applications.

Pat Muirragui  03:47

You talk to your banks, biggest banks in the world, we actually we take them live, and that's you're trying to coordinate, you know, with a handful of folks on our side, and I'll get to who those people are with multiple teams on their side. And at times, you know, we're on the call, and we hear them kind of, you know, kind of squabbling back and forth, to be doing what and it gets kind of interesting, but we kind of help them, you know, we help them too. And we give them suggestions. And you know, the list that we'll go through those are, those are things that we actually write down. And today we actually provide through some tooling. And we've talked about tools, you can use any tool, I think the biggest thing is actually being able to collaborate on the tooling and mark things off and check things off and add notes and add questions. And you can see where stuff has been.

Jeff  04:36

Yeah, on that note, too. And I usually try and take a pretty agnostic view on the tooling. But I think if you're doing standard implementations, you can skate by for a little while with Google Sheets and whatnot, doing what you're doing. I would definitely use a collaborative tool. You know, the tools that are out there. You know, even if it's Basecamp, right, which I don't think people are allowed to use anymore. That moves that they made. But you know, it's an environment where customers log in and they see what what they have to do and what they need to do next and make decisions. And you're not trapped in email and slack and all that other

Pat Muirragui  05:12

stuff. So yeah, that's so right. That's the worst that's not really searchable. It's not, you know, it starts to become pretty clunky. And that doesn't scale. Well. Yeah. So. But yeah, the tooling, you know, we use Smartsheet, that we turned, we turned to that this year, but we used to have, you know, in our developer hub, which is used a lot during the development cycle, we would post up a sheet, a list, and we would say, hey, you know, here's the list, go take a look. And it goes well. And then we started thinking, we started kind of getting a little bit smarter about it going, Okay, let we want to see them, check the boxes. Yeah, we know that that's been done. But, you know, so, so just kind of going back and just kind of going through the list here. Like who's involved on our side. And really, you know, in the beginning, it's primarily the solution architect. And we have a what we call customer onboarding Manager, which is a real a kind of a new role that we've had over the last year and a half, and it's served us, you know, it's worth its weight in gold. It's, it's it that that role really drives the project? Yeah,

Jeff  06:22

are they doing? I've seen this role do two separate things. One is more of the configuration management and going in and clicking some buttons and levers. And the other is kind of more on the project management type of side.

Pat Muirragui  06:35

Yeah, it's more project management. But I would say that it really depends on the tooling and how you're set up for us all the controls are actually owned by our support team and our cloud operations team. So to make sure that those get the to get that done or communicate those we have some tools that we use from our CRM or, or Yeah, and that will initiate requests, and they'll go there. So we need to make sure that those are done. And yeah, but the but the what the call, we call the comments also, you know, also a serve an engagement manager. But really driving the project, it point. Managing escalations, we had some hiccups along the way, and who's got to run back the product? And, you know, make sure they get that one answer who's in charge of it? It's, it's a calm, who's, who's who's really onpoint to make sure that, you know, they get to the launch pad, and they get live. I mean, that's really it's really up to that calm, you know, that's their role.

Jeff  07:33

Um, just quick question, because this always pops up when you have these types of larger engagement. Is the CSM involved? Do they get introduced post go live?

Pat Muirragui  07:41

Great question. So we transition from having a solution architect and a customer success manager to a solution architect and a calm after we hit a certain point. post go live, then the comm will switch out they'll do a handoff with the CSM. Yeah, and, and the success person comes in and now that, hey, this, this, this thing's in flight. Now we can go and we can, you know, make sure we can go around and we can say hey, how's everything going? Let's track the eyes. Upcoming maintenance coming heads up? So that's a great question. It depends on what your model is really, right. If you can get up and running and get live quickly, then I would say then the customer success manager might show up on on on go live. Or you might even have all three during the entire engagement

Jeff  08:33

strategy, strategy questions and things like that. And yeah,

Pat Muirragui  08:37

yeah, do you do it's fitting for us today that's it's, it's, we have a series of you know, it's, it's usually Solution Architect, customer onboard manager, and then post go live. And a little bit like I'm talking a couple months down the road, it ends up being then a, what we call a tam, a technical account manager, and then a CSM together and they they end up kind of running the rest of the of the, you know, like for the

Jeff  09:06

model, especially with these larger key enterprise types of customers that you have there. So absolutely,

Pat Muirragui  09:10

yeah. Yeah. Cool. So all right. So the Yeah, so the folks, so those are the primary folks. And then on the secondary side, we've got some other some other folks from the cloud operations team, we actually have another team called the app ops team, which is also part of the cloud, the cloud team, and then customer support. And these are the folks really they're super important as well. They're in the background. They've got their hands on the controls. Yeah. So as a solution architect and as a even a calm, nobody, nobody has any, any really hands on the dial for production on production, or even the sandbox environments. Okay, so and that's just the way that we've designed it and that's that works fine. So, yeah, so those are the folks went over kind of the prerequisites and you know, hopefully, you know, as you have gone through this process, you've what's really important is that you've developed a good relationship with this customer. Right? And then when I say customer, it's like it's the many folks that they have. And that could that could be a team of three. It could be teams of teams. Yeah. Right. So hopefully you've gone in, they've gone in there, you've established a single point of contact. And you know, and so you hopefully you have that on the right, because that really helps as you as you go all the way through and you know, obviously, the entire lifecycle of a customer, but that's a that's a, that's a bigger conversation. But getting back to the go live. When you get to we call it the deploy phase, we used to call the Go Live phase, or just semantics deploy, it was a little more accurate. Yeah, because it wasn't, it wasn't like don't celebrate yet. deployment, then you hit Go Live and go live is actually

Jeff  10:56

not until a rocket is launched to seller. Yeah,

Pat Muirragui  10:59

exactly. Yeah. So the, you know that and that's, that's funny. You mentioned the rocket launcher, because that's that that's that analogy really lives to me today. It's a silly one. But I like I use it with my team all the time. It's like, let's get the rocket to the launch pad. Yep, you can only get to the launch pad, if we've done all these great things before. Yeah, once we get to the launch pad, let's make sure we've got a plan. And then when the engines fire up, and they're going, right, and then the Rockets Launching, that's probably the most critical time during this go live, because a lot of things can go wrong. And you know, if expectations aren't set, and even in real life, like that's one of the most dangerous times that in critical times where you know, when a rockets trying to get up in orbit, getting out of the atmosphere, right. And so we think of that same thing, and it's like, let's get that get that rocking, hit up, get it into orbit. And, you know, now we're our

Jeff  11:57

true because I mean, just, you know, with so many, you know, commerce launches that that I had, it's it's getting into the atmosphere and maybe do a one on one run around because you can go live and then suddenly, like it crashes because you've got too much traffic, or Oh, my God, this one critical bug that nobody would you know, somebody took a path that was not, you know, all that stuff happens. So

Pat Muirragui  12:19

happens all the time. Yeah. And I think it's how do you how do you, you know, how do you best manage that. But not to get ahead of ourselves here just going. So the one of the first things as you're done as you exit the testing phase, is to, and you go into the deploy phase is hold a launch plan session, okay, we seem to gather and say, Okay, who's gonna be in charge, let's come together, let's plan. And that's really where we go. And we give them this template list. And we say, here's the everything. Let's start crossing things out. And let's start adding new things on that are relevant for you. And so a lot of those things are any, and I'll just kind of run through the list here. So sign ops that they need, right? There's other folks on their side that they need to check boxes to make sure that, hey, they're ready to go. Production credentials, for example, right? They've got to go through a provisioning process, they've got to have security, a lot of times we get held up because, oh, the security team didn't come in, you know, they didn't

Jeff  13:25

talk to compliance check.

Pat Muirragui  13:26

Right? Yeah, right. Right. All those checks that need to be done from typically that's from from any kind of security or compliance team, but they need to make sure that check off the box and know that you know, hey, this is a day that's another thing is agree on an actual day, like you Oh, we will be going live at this time, you know, during the day, or whatever, at this hour, your worst time to go live Friday. Learn that that Tuesday,

Jeff  13:54

for the morning is what I used to have to do for commerce launches is just, you know, get up in the morning and you know, start thing and see what happens.

Pat Muirragui  14:06

Watch me go. Yeah, I would say you know, early to midweek is probably best because then you've got a whole week to kind of, you know, get that thing in orbit and get it get it in or orbiting and smooth.

Jeff  14:19

It I'll just throw in because this has happened to me more times than I like to think like launching on a Friday something goes wrong in your critical devs or on like Sorry, I'm at a wedding this weekend. I can't do anything about this. Like,

Pat Muirragui  14:33

right, I mean, everybody, right? A lot of people and just being respectable and respectful of everybody's time. People got stuff going on during the weekend. And you know, nobody Yeah, nobody wants to be you know, stuck chasing bugs over the weekend, trying to get trying to get the project live. So we try to as best encourage, go lives, you know, earlier during the week, but unfortunately sometimes sometimes that does happen. Yeah. So but I think the important thing is understanding the day of if. So the the converse of this is, you know, you show up at Monday at the checkpoint and the customer says, oh, yeah, well, we went live last weekend, or we went live live last Friday? We Yeah, well, we didn't tell you

Jeff  15:20

that we were just deploying, and it looked good. So we just, we just switched all the traffic over here.

Pat Muirragui  15:25

Yeah, well, well, but at times, it might be it might be, Oh, we didn't even get you the production credentials. So Oh, so you're live on sandbox, okay. And that's happened, that's happened to so important, you know, when you go over this, this planning session to hit the day, and also, the rollout plan that they're going to do, right. So when I say rollout plan, like they might, you might have a contract with this customer that says we're gonna give you a million transactions this year. But day one will not be, you know, the million transactions, right. And so it's almost like the turning on the spigot, you know, for for like a you know, water faucet, first you do 10%, take a check, make sure everything's running smoothly, and then kind of open up the aperture. So usually, it's like a 10 25% 50%, you know, and hopefully 100% That might take that might take months, it might take weeks, it might take days, ideally, you want days, but you know, the customer to they're gaining confidence in the product, because even though that, you know, you've you've you've had your best, you're getting the best story. They've done some testing, they had a great proof of concept. There's still this, Hey, how well is it going to work? You know, does it work while you're going live? They're looking at, let's jump, let's

Jeff  16:46

take one thing there. Because the thing that I always have, continually will every time on services stuff problems with is getting the people to test, like, test the application? Oh, yeah, we went in and we looked at it, you know, like, what strategies do you put in place to make sure that there's been thorough UA t.

Pat Muirragui  17:08

So at the end, a UA t, we so we have some sample test plans that we've we provide the the customers, and because we have multiple points of integration, we'll provide them unit tests. And then we'll also give some suggestions for end to end testing, which requires like end users like real end users. So we'll ask them to do that we can actually look and see different transactions. So we can see in the backend, what they've run. And we can see how much testing that that they've actually done on their side. And so that's how we get kind of a feel we're getting a little better at this, one of the things that we want to do is actually provide them a module to say, hey, run this battery, run this thing on your side. And then we'll exercise a lot of different scenario testing tools. That's great. Yeah, so we're getting to that point. Right now. But yeah, it is important to get them to task to encourage them at the end of that. But here's here's another thing that we do is at the end of the the user acceptance, we ask for a demo, we're like, show us what you bought.

Jeff  18:12

This is a I've heard this from some other people recently, I loved it. I wish I had done it back in the day. You demo it off to us now, like you're showing your executives. Yeah,

Pat Muirragui  18:20

absolutely. Yeah. And so during that time, we're also looking in, and we're able to go in there and critique and add, you know, of course, positive feedback. And, you know, Hey, move this button over here. And, you know, you might want to change the verbiage there. And we'll give him some examples of what you know, I'm talking to like user, for us, what we're seeing is we're seeing user experience screens, right, the way that our product works, there's like a user journey, super important. And what does that what does that user journey look like? And, you know, how are they managing errors and things like that. So that really is, is kind of a good litmus test there to see what have they done, how much testing that they've done? And that usually gets us some confidence that, you know, the launch will hopefully go well, and they can get to a good life status.

Jeff  19:13

Yeah. Well, I know something on our list that we learned the hard place in like 2008. But the app approvals, right. That is difficult, right? Because who Oh, yeah. Like the keys and the dev certificates and all that.

Pat Muirragui  19:29

Yeah, so the app approval so so the laundry list continuing on the laundry list here? The provisioning for so first of all, there's provisioning on our side, right. And so we've got a couple you know, nuts and bolts and things like that, that go into the production on our on our software development kits, and even on the app. So like, we have licenses that sit on the client side SDK, so on the mobile apps themselves, we have licenses that are there, and those need to some of them need to be read. nude some of them have are specific to certain environments. And so we want to make sure that they actually, when they get those, they put those in the right environments and that those are, you know, organized and and, you know, inserted in the right place. Mutual law certificates, so that server to server communication, if they have message level encryption, they want to make sure that stuff in sandboxes is not copied over to production of something. Production. I mean, that's really important, like, make sure you got the production version on the production environment and not have, you know, the wires crossed there. Yeah, that's a, that's a common thing that happens. So yeah, and then getting to the App Store approval, because if we have different versions and different channels of our products, and the the app vert, the native version, of course, needs to be approved by those now they can get some customers can't go live until those are actually approved. They've got a launch day, and they're stuck in approval for whatever reason, that might take take a couple of days, it might, it might, it might delay the launch. Yeah. So which is which is not uncommon, I'll I'll, I'll touch on that in a little bit. Like a, you know, a push launch, not a not an uncommon thing. But so the provisioning, you know, getting that taken care of them promoting the code from their environments, environment, understanding how they operate, is really important. And as

Jeff  21:27

your architect kind of shepherding them along the way. Just,

Pat Muirragui  21:31

yeah, yeah, he's asking the questions going, Hey, is what environment are you going to be moving from your sandbox environment to their product production environment? And, you know, they might say, Yeah, but we call it staging or we call it you know, something else. But just to get an understanding how long you know, what they need to do and how long that takes? That's, that's the that's, that's really important. You know, there's only there's only so much that you can see, but by asking these questions, you understand how they're operating on on their side, right. And so once all that's done, what's really important to before launch is to run a smoke test, we call it a smoke test, but it's basically get on production, send through some transactions. Let's see if we can see him on the other side, let's take a look at the health of those. Right, like if the licensing is wrong, then on our side, you know, we might see data that's got some Asterix or something in it, right. So there's, there's little things that we look for that are particular to us to say, okay, smoke test looks good, nothing's nothing's burning up. And so that gets us that gets us launch ready. The other thing, too, is the support piece. So we've actually aggregated a bunch of information for Hey, once you go live, you know, what do you do? Who do you contact? Who are they going to be the, you know, what's going to happen next. So we we put together like a support flyer, it's like a PDF, a flyer that we send them it can be, it can be in any kind of format that you want, or any kind of communication, I mean, it can be it can be an email, but we send it to them, it's got the support numbers, it's got the, you know, the red alert, you know, red phone number and really goes wrong. Hopefully they don't use it, their customer success manager that's going to be up and coming. The common that in the end, the solution architect, are our folks hanging on there for a little while. They don't leave right away, but it's like, Hey, if you need to open up a ticket, here's how you do it. Here's the support hours of operation, we have a follow the sun model. So there's always there's always somebody, there's always somebody there at the helm,

Jeff  23:40

or they're not being introduced until the live, maybe different subject, but I like encouraging getting the customers start using customer support for customer support type requests instead of relying on your tech team for customer support, because then it's harder to move them over to using the support. Yeah, that's

Pat Muirragui  23:58

a good point. I think we've done that kind of on demand. Yeah. But at a minimum we do here the contacts, here's the number. Yeah, but it's good. If you know, if you've got a if you've got a customer support regional lead or somebody it's good to say, you know, here's your point person, you know, on on the on the, you know, on the customer side, yeah, is a really important, you know, another point is gathering their contacts. So as we've done the work, we're going to go live, we're going to launch, what happens a lot of times as you'll see transition on the customer side, the product team might move around, those developers are not necessarily assigned to this project always and forever, right. They'll move on to the next integration, the next next project, and they'll be gone. So what we like to encourage is we need content. We want to have contacts for support, any maintenance and product updates, like hey, our product is you know, we got a new version coming out or the endpoint is going to be revived If there's going to be changed on this endpoint heads up any of their, to let them know that like maintenance is beginning maintenance is ending, and certain windows. And so what we like to do not only get emails, but we liked we asked them to actually form email groups. That's the Yeah. So then because even on their side, right, you've got a costume you

Jeff  25:23

mean distribution was just for people you just email distribution was that they're creating So you send it to one group? Absolutely,

Pat Muirragui  25:29

absolutely because they're gonna cycle through that's an important thing who's there today is not going to guarantee the team is going to shift in six months. Oh yeah, even a year and every six months, people are going to leave and new people are going to come. So it's a guy's Manager, please manage your distribution list. Ideally, that's what we want. We don't always get it, but we encourage that it's a real important one. And for those, you know, for those buckets for maintenance, and also for product for product updates, yeah. Yeah,

Jeff  26:01

that's important to know, I think next is launched. It was a you that was telling me treat launch day like the doc like the surgery, like, Okay, this is what we're doing, we're operating on the left side of the left.

Pat Muirragui  26:12

So we have a pre launch checkpoint, which is like the last like the pre op, like, hey, Okay, you ready to go, we're gonna, we're gonna go over here, we got the little black marker. We got the Sharpie pen, we're gonna X out the leg that we're not doing.

Jeff  26:30

Yep, still, though, people still get wrong side surgeries all the time. It's similar.

Pat Muirragui  26:34

It's similar to that. The other the other thing on that day, too, is that we open up a war room, which today is probably not the best term, but we have a mission control room. Yeah, the mission control, we bring everybody this is

Jeff  26:48

a physical room like, or is this like a teams, Microsoft Teams or slack room or

Pat Muirragui  26:53

virtual, it could be it could be a Slack channel, it could also be a bridge, it could also be a zoom bridge. It's really what the customer is

Jeff  27:03

just envisioning you in a place with like 15 monitors, you know, it's like the show 24. And they're just kind of bank terminals. And

Pat Muirragui  27:11

that's really our our cloud ops and NOC teams, they're doing that they're looking at that they've got deeper level tools, like we can see some things come in at at certain levels. But our operations are operational teams, they're able to see like things at the component level, like this hit the API, this hit the back end, you know, service engine, and so they can see things at a real real low level. And so they're there, those are the guys with the 20 monitors that are sitting there in the room. But we have a we have a higher level Mission Control, where we'll have somebody from support, you know, the calm the PS, and we'll have folks on their side. And you know, and it'll be literally like, okay, we're about ready to turn it on. We've turned it on. And then we can look and we can confirm transactions actually come through? Yes. Oh, it's to that detail. And that's some customers don't, you know, they might, they don't want that. Or maybe they'll just do it, like, Hey, we're gonna start at this time, and we'll send you some messages over slack. You know, that works, too. And the important thing is have the right folks in there in case something goes wrong, something can go wrong, and you want somebody on the controls right away, you don't want to have to open tickets and and wait for others to you know, pick it up and notice it,

Jeff  28:33

I have to imagine that since you're working with like banks and security in their security teams I've been through, there's just a ridiculous amount of stuff that they appreciate this level of detail for for walking, because I've certainly have launched stuff where it's me and a developer in a room and we go Do you want to click the mouse? Right? So there's a lot

Pat Muirragui  28:59

they do. And we hear we hear about that? We hear a ton of feedback from that we get a lot of feedback on

Jeff  29:08

pre sales, right? Like pulling out your project plan, like this is how it happens to people like whoa, they get their act together. Right?

Pat Muirragui  29:14

Yeah, they love the white glove treatment that we provide for them. And, you know, again, it's from its from its learned experience over, you know, a number of years and just making it better, right, make better perfection by iteration. Every you know, we've had hundreds of go lives and every you know, as I tell the team, and we come together and we talk about them, it's Hey, what did we learn new and tell the team tell the rest of the team about it?

Jeff  29:42

Ya know, those are great those we used to have the bi weekly or the monthly meeting where somebody is the case study of walking through and retro. We never call them post mortems we call them after parties. And yeah, and we just like let's walk through what worked and didn't work and all that stuff. So yeah,

Pat Muirragui  29:58

yeah, so long. Going back to launch day then. So we've got the activities or, you know, we do the checkpoint, we open the War Room, we hit the you know, the buttons hit, we're watching the transactions, we're looking at the health. Now we take the we take to try to correlate also the success criteria, right. And that's what we're aiming for. So if the customer says, you know, hey, we want to make sure that we've got out of our out of our, you know, every, we want to make sure that we have got 80% approval rate, right, that's just a little something for our transact our our business and our transactions, when you send something you can get something approved, you can get some disapprove. So that success criteria is, hey, we want to see a 80 or 90% approval rate, or whatever it is, right. And so when we're looking at the health monitoring, you know, we're looking for those basic things that we always see. But also we're tying it in to what the customer is actually expecting on their side, right? And so at all, that's all that fun, success criteria conversation. And then after you know, that it's really, you know, Hey, how did we do at the end of the day, and then throughout the week, right, if you if you go live on that Monday or Tuesday, you got the whole week to actually go in there and check and check again, and look, and the frequency will become, you know, you know, it'll, it'll become not so frequent as things get mature and healthy. And, you know, you've got confidence. And, you know, that's how you're, you're inching towards that you get to that orbiting life state.

Jeff  31:25

You're right. And that's great. It's very thorough, too. Yeah. Right. So let's talk about some of these post go live activities, which is what were all the fun starts, right. So

Pat Muirragui  31:34

yeah, so so a lot of it is, you know, measuring both, you know, externally, you know, going back to this transaction, the health transaction, how are we doing? How are we measuring, is it is it up to snuff what we have and what our goals are on our side, and then also the, the external optimization, any chance to optimize at that point, I mean, it could be minor, it could be simple, you know, really kind of, you know, small controls there that that fine tune, but you want to you want to keep optimizing that. And if you got the, you know, there's really good precise KPIs, then you'll see the optimizations there, hopefully make some changes there. And

Jeff  32:18

your team's on for what do they burn in for about two weeks, three weeks,

Pat Muirragui  32:22

you know, the, the, it's really the solution architect is there for another 30 days, we want to get really stability, I mean, we're not actually counting, you know, exactly 30 days, but we want to be, we want to look for stability and what they're doing. And then at that point, then the solution architect can roll off, right, we want it to be 30 days, sometimes a little more, sometimes it's less. And then after that the customer onboarding manager, he'll roll off and are there or she will roll off and then it gets to CSM and Tim, yeah, so so that's good. We do here's another one, a good one that we do is send a survey. So we send a quick survey, not too long, but not too short. Just how do we do? What's one thing you like? What's one thing you didn't like? What can we improve? Would you recommend us to anybody? Right, that NPS question? And so

Jeff  33:18

are they rolling that into like, the regular customer baseline scores after that, and everything?

Pat Muirragui  33:22

Yeah, yeah. Yeah. Right. And so that's something that you know, I think, any any success or really has to do, that's the key, it's heart and soul to that answer the company, you know, and so, you know, I think it's important to, you know, when you're, when you're sampling that, because at go live, things could be great. Or not great, right? And if you take a you know, you take another measurement at three months, or six months, you can say, Well, gosh, we were great at live, what happened? Right? Or we weren't great. And we did awesome. And they turned it around, you know, now that now they're, you know, they can't we came in with an MP that you know, that recommendation a 10. And so those are, those are great learnings. And you can also, you know, when you when you ask them to when you're asking the surveys, and you can you have like open, you know, text boxes, and you can say hey, is it the team? Or is the product or is it the program?

Jeff  34:20

Because you can have great product bad team? Yeah, bad team and or sorry, sorry, bad product, great team. And you know, that has a big swing on how some of those questions get answered. So

Pat Muirragui  34:33

or any combination of that? No, yeah, so that's that's important. So the surveys really good. The the communication on the wind down obviously the transitioning the letting them know that hey, solution architects going to be rolling off, so they don't, you know, so they're developers later on, you know, three months down the road, they're not still sending mail. So the solution architect, no questions about that at all. The deployment that they did.

Jeff  35:02

Yeah, whenever we talked about to like, especially if you've got set durations on these things, it's like we're gonna go live because Solution Architect rolls off on this date. So

Pat Muirragui  35:11

pumpkin and he's gone. Yeah, absolutely. Actually, I think I got that one from you. Yeah, it's ingrained. So yeah, the wind down the roll off. And then, you know, the celebrating the celebration and celebrating internally. So we have a, you know, we have a, we have a series of meetings, we have team meetings, we have group meetings, and we talk about it, and we celebrate, and we talk about what went good, what went wrong. What What can we improve? We have a, we have an internal, we have like a, a, an internal, like, blog, kind of an entire company. So posting up the go live, we call go live announcement? Yep. So we put it up there. And we congratulate all the folks and don't

Jeff  36:00

forget somebody's name on that list. Yeah,

Pat Muirragui  36:02

know that. Yeah, they don't

Jeff  36:06

forget one day.

Pat Muirragui  36:08

Remember everyone? Yeah. But it takes a lot of hands, you know, and it's really, you know, this conveyor belt of, you know, getting to the launchpad and building that rocket along the way and, you know, having that thing take off. And

Jeff  36:25

so let me ask you this, because I know we're, we're getting along, we have some others have to chat about my worst nightmare is six months from now, you haven't heard from the customer, everything's going great. Suddenly, we've got this bug. It's actually pretty big. Your team goes in and looks at it. And they say it's going to take probably like a month to fix it. What's the scenario? Is your work? warrantied? Do you go in and fix it? Is there a certain time?

Pat Muirragui  36:54

This is a tough one. That is a real tough one. So if you go back and like, yeah, we made a mistake six months ago, or either we didn't, yeah. Or we didn't have line of sight into something obscure like that. We don't necessarily have a warranty, we have a you know, we have a master a lot of it's in our in our service agreements. And in the legal work that we have, I mean, we do everything to ensure success along the way at, you know, almost at any, any cost. Within reason, right. So we'll we'll do everything to provide a workaround. If we can't get that workaround through the professional services team, then we'll we'll see what product can do in terms of a hot fix. Got it? Maybe the hot fix, you know, maybe it's like, hey, not only is customer X seeing it, this other customer seeing it, and they don't know it yet, and we've got 10 other customers that don't know.

Jeff  37:53

Yeah, and you know, for my other tips from that I bumped into like, Hey, did you change something? Did you go in and mess something in the code? Did you update a library that maybe is not supported? Or, because that's where those things mostly happened. Either the product changes something which throws everything six ways to Sunday, or you're using some open source plugin or library and then next day, change something and then or there's a security breach, and you gotta go replace it? And then, yeah, it's tough stuff. And then I always usually try and abide by my golden rule of Don't f with the ARR. So try and you know, right, like, because it cost your team five grand to fix. You know, you're working on probably several $100,000 AR licenses and stuff like that. So you have to get a little bit of perspective there. So

Pat Muirragui  38:39

Correct. Yeah, you got, you know, you know, you want to keep the ARR stream going, you want to keep the customer happy, you want to get that, you know, you want to renewals and things like that all come into consideration, right? And customers remember, ugly things that happen. Yeah, that's one of the they'll they'll remember, you know, that is a difficult thing, even when you get into success. And they're successful. They remember, you know, they might forgive, but you know, a lot of a lot of them don't forget, you know, I say no, leave the company. So we do everything that we can hot fixes, PS, they're calling meetings, and just, you know, care, you know, care about the customer and do everything you can. But we do you know, bugs is just, you know, in software, like you said, things that we can control, but there's also things that we can't control. Like, like, for example, we have products that run on browsers. And so those browsers we have no control over, you know, what Apple just did or what Google you know, what the the Chrome team just changed for a security reason. And now our products not working. And so we've got to go and say, Okay, can we have what, what can we do and we triage it and we've got, you know, smart a lot of times it's looking smart people, you know, the developers and solution architects to figure out you know, what can be done,

Jeff  40:06

ya know, I'm preaching to the choir. I mean, my main product for a long time was flash, which now I think if you use your money goes over to Russia and stuff, but so, so I think we covered a lot here. And I, so this list here was talking about this, but you got a checklist. So Pat, and my good friend, Mark Holland, who I believe has been a guest as well to have been bugging me for a few years, there used to be something called the PS village, which to me it was like, the blue collar like people who were doing it asking all sorts of who's got an S O W template, what do you do about this? year's just what do you do about present just all the questions, right, and this community was awesome. I think it survived the cutover to LinkedIn, but then you know, based on some unfortunate scenarios with the people who are running it, health stuff that just kind of died wet, and, and so. So this is something that we're putting back out there, we're gonna get a Slack group going together, calling it Proserv. SAS, and it's just a community where we pro serve people can go in, you know, I'm laughing, I'm not gonna say, Well, I don't like you know, it's not other communities, it's not management consultants. Pretty much not going to have customer success people there because they have million other places where they can go and we love them. But there's places where we just want to talk some shop every once in a while,

Pat Muirragui  41:34

right? Yeah, absolutely. Some centric topics.

Jeff  41:39

See, if people aren't listening by now. They heard projects and

Pat Muirragui  41:43

boring stuff. For this Yes, group, which we love. Yeah. And then we'll have

Jeff  41:49

like templates, and I'm gonna get a little web hosting thing just to try and put some of these things in, because I've got just 20 years worth of templates and everything, right? So it's just for people who need help, you know, Mike, just like, you know, we talked and you know, you shouldn't be Brad one of these days, but you know, people who they might be running a CS team, and then suddenly, they're like, now you're on Prosser Right? Or, hey, that you're a manager of this one group. Now you're the manager of everything, right? And you've got now you've got pre sales, too. So if you can answer pre sales and that whole thing. And yeah, so super excited about this. And we'll have the links in the show notes. And we're going to start it organically and post a little bit about it, but have loved to have you join us and we'll put the links in this. And in you know, nobody's selling anything. And I'll make sure that except for my company, of course, but not just and we'll make sure that it's a just a place where people can go and go and answer questions and ask questions and, you know, get together and stuff like that, because that's what we had. And I actually got it went into the LinkedIn group, and it was checking it out again, I was like, yeah, there's just good to and then there was an old school forum like PHP hosted and things like that. But super excited about this. Oh, Pat and Mark, thanks so much for pushing me to do this. I think it's worth going on, like two years of me blowing it off. So So now, we're gonna do it. So yeah. Awesome. Awesome. So Pat, thanks again. I'm gonna I'm gonna end the recording in one second, and then we'll close out so thanks a lot, a lot of info. There

Pat Muirragui  43:21

was a great conversation. Loved it. Thanks, man. Talk to you again.