Dynamics 365 Sales Forecasting with AI with AJ Ezzour

[mark]: 3:42
today's episode we're going to focus on dynamics three six five sales forecasting and artificial intelligence or a i my guest is from seattle in the US he works at microsoph as a product manager in don three six five sales you can find his bio linked in twitter all the socials et cetera in the show notes for this episode welcome to the show AJ good to have you on the show i always like to get to know my guests background outside of what they do with microsof to start with so tell us a bit about food family and fun what do they mean to you so tell me a bit about what you do when you're not at microsof what do you do for food family and fun have you always been based in seattle what's about your history there

[aj_ezzour]: 8:42
actually i grew up in los angeles my family is out there and get to visit them at least a few times a year it's always nice to get the contrast of the heat and the rain so we ve had a pretty long summer out in seattle which has been wonderful taking the opportunity to do few hikes did hike to their lake this past weekend which was incredible and yeah as soon as long as it's been sunny out here been trying to get out to the restaurants something that hadn't been able to do or maybe a lot of people haven't been able to do much in the past few years so it's nice to see things opening up and kind of enjoy the city more

[mark]: 9:17
nice so what was your journey into into it how did you originally get started was back university or before

[aj_ezzour]: 9:27
yeah it's been quite a long meandering path i started an undergrad i studied nero science so on the pre med track decided i didn't want to do that and worked in higher education administrate and for a while i decided didn't want to do that and jumped into the world it start off that during my BA did some independent consulting and finally realized you know i love entrepneurship i do want to keep doing this at some point and but i need to put myself in a place where i can learn from the best and the brightest and i said you know what microsoft is a place where i can do that

[mark]: 10:01
well and so that took you into micrsoftdwere you working for microsoft originally in california and then moved up to seattle

[aj_ezzour]: 10:09
i was not working for micro so i moved up to seattle for microsoft actually

[mark]: 10:12
okay okay awesome awesome so tell us a bit about your role in microstof what's your day to day what are you do what are you responsible for

[aj_ezzour]: 10:21
sure i'm a product manager i was focused on the forecasting and productive scoring futures which i think will cover a little bit today more recently i am shifting to some of the v v sales ishitives but as a product manager i get to interface with customers on a weekly basis collect requirements from them understand problem space that we're solving for and then work with engineer team in the design team to build out the right features that can drive value for our customers

[mark]: 10:50
okay okay so tell me where i comes into the mix here for you oh

[aj_ezzour]: 10:55
sure so the two futures that i was working on forecasting and predictive scoring both have an a i element so we know very data centric tool the dynamics cr m and so we figured okay we had all this data in the c m is there a way to leverage this to benefit the the nduserin the customer and both of these tools utilized this concept of kind of looking at an objective view of your data meaning we can look at your historical data and tell you what what the trends have been for closing a deal successfully in the past and so do that with predictive scoring which allows you to bailly receive a score and then a bucket of grades eight d breach of your opportunities or leads so as a seller you can prioritize which one are the most potentially impactful and you know it's a it's a tool to use that we don't we don't like to say that it's creating the exact priority because you also want that the user to be a man in the loop so that they can make right decisions about you know what deal like this deal might be rated the same as this other deal but i have my intuition as a seller that i know this one is more likely to close because i have hands on knowledge that i met with this person last week and they told me they were going to close it or the seal is a higher value so if i close this now it would be more impactbul for my my team or maybe just tend to close a certain type of dealing in industry more often so that the i is there to support sellers not not replace them because we think that you know the people on the ground will have all i have an intuition that can't be replaced by a

[mark]: 12:35
so how you know with the predictive scoring is that really about probability is it about is it is it purely matching past behaviors or trends with the opportunity like this and therefore applying it to it what's your kind of thinking or what data where do you gather your data points to kind of inform the algorithm

[aj_ezzour]: 12:59
you kind of nailed it there so for opportunity scoring we have opportunity and lead scoring but for either the primary entity will be either the opportunity or lead entity and look at attributes on that and what the what's basically doing is drawing correlations for different attributes and example is the industry is a good one where we say in a specific industry let's say finance if finance is the industry of this offer we know that historically this is more likely to close if it's something else like retail it's less likely to close so that adds certain level of impact to the score either increasing or increasing now we take that across all of the attributes that are within the entity and consider if you have our entities as well but ultimately what it's doing is just drawing these correlations from the and then sort of extravilating into the future to say knowing that in the past having finances this budget amount is this contact is this the result is positive and we xpect that in similar cases yu'll get the same result so as you said it's it's very related to probability

[mark]: 14:05
how much how much you know as a sales manager looking at you know a diverse sales team there's a whole element of human behavior involved in sales as well and you know individuals might talk a big game but past behavior in forms future you know activity in your modeling are you doing anything around the individual so of dynamic three six five sales in other words the sales person how much of your scoring involves understanding this individual's behavior over time and therefore that in forming also the decision because as a sales manager that could help me a lot and that you know i'm responsible upline of me how much is that going to enable me to make sure that ll give me the confidence level in my pipe line and the date it's due to close quarter on quarter of territory that type of thing

[aj_ezzour]: 15:05
i know that's a very good question from the predictive scoring perspective we have heard this request my customers want to understand you know how i was a particular selling seller doing you know and i can i predict what their performance will be so i can serve them up the right deals may be things that they may be more successful as a present the predictive scoring tool doesn't have that capability to score sellers because it's a litt of a trickerous space around you know motivating sellers and how they'll feel if they feel like they have this sort of m surveilance on top of their work so i think it's a little bit trickier then not as simple as it might seem but beyond that just kind of keeping to the opportunities and leads being you know scoring being something to benefit the seller to provide an additional but not really feel like something that they have to fight against or for but i will tell you that sometimes helersee a score that's low on a deal but they're like you know challenge accepted we'll see we'll see about that and that kensky things but i think part of the challenge of an i model is you know can can we get folks to trust that it's just serving their data back to them and so that they can then actually utilize it to its full potential but in addition to that like we do have the forecasting tool which is more about keeping track of pipe lines and that allows managers to look into what a seller is doing there on and rather rather than you kind of call them out on where their failings are be a presence to coach them and how they might improve

[mark]: 16:37
yeah but like you used a key word there which was coach and so therefore let's move to a sporting model and take the sales you know from a sporting perspective in a game i'll use the reference to rugby because i come from new zealand and that's our national fort and so in there right when the whistle blows at full time we have a score and an outcome of the game we either one or we've lost and that's our you know the end of the quarter or the end of the year that's the but the game is over nobody can change it dates passed et cetera however a coach in a game has got people tracking a whole range of things they're tracking how many tackles were done how many yards or meters were gained on the on the field how many blocks how many scrubs how many penalties all those incremental activities attract because they know that if they if those numbers equal x or why it's most like they're going to win right if they dominate the territory the most if they get the most you know yardage that type of thing they're gonna win the game so therefore right back in the first quarter of the game of those states high it's a good predictor of where the game is going to go half times going to tell you more three quarters of full time that's the type of information that i think that you know a sales manager would find incredibly powerful and the thing is is that i don't think from a sales person's perspective that they need to see this as you know you're monitoring my behavior we want a coach we want to score just like a game and if we don't address it at the start of the quarter or mid quarter when it comes in of quarter the game is over we got nothing to play with anymore so so that's where i wonder where people have concerns with a i and and scoring is when it's purely black box in other words i don't dart a win and i have no idea what went on and so there was a model of showing how the dart is used i think you know the risk is and would people gain it in other words you know if i'm saying that the probability that sale is going to close is ninety percent yet there is no email activity with the car omer no phone activity with a customer no meeting activity with a customer for the last ninety days would be like what's the likelihood of that but if if the system could start kind of giving me that insight because once again the activity data is there right it's available to you can hook into it and you you know even now with marcrusoff teams if you've you know if you're using the voice channel on that there's a whole bunch of additional data that could feed a model sediment analysis a whole range of things right of that conversation key words all that type of thing are you thinking that way are you thinking that you know we really could enrich the so much more that would allow for real prirtized coaching allow you know sales managers to coach to those keys areas and really drive positive outcome for an organization

[aj_ezzour]: 19:46
yeah i mean those are all really great points we are increasingly including new attributes ways that the model can consider data like tivities um recent one we have is time spent in sales stage so you know score can be impacted whether seller has left it in a particular stage sale stage are not over time and that that will be very indicative of success particular opportunity or lead we actually even design this attribute insight spain which lets you goes into the attribute itself you know of all the potential selections for this attribute which ones are more likely to be successful and we will actually explain to you why an attribute is impactful or not and that the benefit of that is to say um if you thought you know the black fox component if you thought that a particular actrobute should be indicative of success or you know an increased score but you find that the model is telling you it's not well the adment can go in look at that and say there's a reason it's not the reason is either you know whenever there's a like cognitive distinance around that giving them that tool allows them to go in and be like actually this is what we're seing we're seeing that we know that you know anecdotally it's important our sellers have not been filling out this information in the model or in the in the car so there's no way the model could pick up that this was important that feature allows there to be kind of a coaching at a larger scale maybe at a sales team scale to say look guys this is very important because if you fill this out it will impact the score and so you know influences behavior in that way which is great there is more to be said anything more to be done about the points that you said that maybe this is potentially very useful information for sales managers to have to share with their sellers directly and i think you know in the future definitely something we would consider because i think that empowers the sellers to be more on top of the data entry because they see the direct impact of it right it will feed them out the information they need to know what to prioritize so yeah i mean very great point and i definitely think that the team is going forward will consider something like that

[mark]: 21:59
awesome tell us about forecasting what's the latest things happening in sales forecasting that you're either you know might be in the road map that's been announced you know in the upcoming wave releases or is it really just tweaking what's been there now for some time since i know about you know twenty twenty or or what what's changing what's moving what are you adapting

[aj_ezzour]: 22:27
yeah now were continuously iterating we're adding new features at a regular basis we've been working on you know data compliance is a huge thing for microsof so there's just a huge runs work that necessary and you know forecasting is a complex so that's taken up you know a lot of work to make sure that we're up to date a compliance on that we have been releasing a few feet there's one that allows you to filthrughout the hierarchy so that you know in the past we get these hierarchies that are just within the cremaare super fluttered because you'd have users that are not opportunity owning or maybe ron leave for whatever reason that they still show up in the hierarchy so sales managed like okay i don't know who i'm actually even looking at here that is relevant now you can use a simple filter to exclude those folks

[mark]: 23:13
awesome

[aj_ezzour]: 23:13
and other another feature that's come out recently which is really nice we've gotten really good feedback on this is this share as feature which is effectively if i'm a sales manager and i have a seller who is on leave or forever reason isn't active but i need someone to have visibility on their pipe line i can actually share their much subordinates record with someone at my level or even someone the other sellers so then we can have someone else having visibility on that pipe line being able to make adjustments on it so nothing gets left behind or falls by the wayside

[mark]: 23:49
a nice nice any any what feedback you're getting you're saying you spend a lot of time with customers and then translate that through the engineering what are the big asks that you're seeing just aggregate level that you're seeing from customers when it comes to dynamics three six five sales because of all the appsthat we have in dynamics on the customer engagement side this is the one that's been here the longest right so so at eighteen nineteen years i think now it's currently been in market i only know that figure because that's how long i've been working with it so so so tell me how you're hearing from customers

[aj_ezzour]: 24:28
yeah so are you asking about sales in its entirety or

[mark]: 24:31
yeah and even in the area that because i do want to touch on fever because i know that you're working in that and that's becoming a big deal fever sales but yeah what what are you hearing in those casts a meetings that you're having about what they're wanting or what they're wanting the tool to do or you know that would be more impactful to their businesses

[aj_ezzour]: 24:56
yeah i mean we continuously hear a push for it's kind of a weird push and pull that's around having more flexibility within the tool and then also having a simpler experienc um which seems kind of like okay how can you want both things but i think you think about it from perspective of

[mark]: 25:14
right

[aj_ezzour]: 25:14
like managers and sales leadership want to be able to get some sort of accuracy reporting just reporting in general on what you know their objectives are and what the sellers are doing sellers want to just get in and out and they want to work with their their customers right so it's a weird difficult balance to strike sometimes but i think that's that leads to some of the things the other initiatives like viva sales where we're saying maybe we can deliver deliver an experience to sellers where rather than having them to jump into the c r m if they don't have to yeah

[mark]: 25:48
okay so so that's cool that's cool and that's that's leading i assume you're thinking and development tell me about a bit about viva sales what's been your observation of its acceptance and me and i think it's it's you know you talk about simplicity and i feel that that's that's really what you're doing with vive sales but what are you hearing

[aj_ezzour]: 26:08

hm yeah i mean i'm fairly noted the sales team but it's really exciting initiative i know that microstuff is getting behind it very very well and there's a lot of opportunity there's a lot of opportunity to see how we can provide value to the sellers where they're at i think we're developing the of the team that i'm working on we're working specifically on the outlook side anne and also the teams teams side panel so between teams an outlook and maybe even the three sixty five set how can we you know derive some insight or some new knowledge for sellers that will make them more productive in the space they're at because we you know absolutely want it to be something that is seller centric and not something it feels like this is just crmyouknowanother you know cluttered cram box but in my outlook so there's a huge focus on that and how do we create a simple experience that driving value for sellers that doesn't like they need to go through a thousand clicks to you know to keep track of their customers

[mark]: 27:10
yeah i love it i love it back you know in the original days with dynamics carm is that it really their outlook experience i think was really what made the product phenomenal sometimes it was hard use the old com you know and and things like that so you know where are most people what application they have the most open on their desk top you know i don't know what the difference s now between macrsoff teams and out but traditionally it's always been outlook that's the application you have open that you're working with so i think it's a smart smart direction that the product teams going before i let you go is there anything so you'd like to add any resources where if people want to go deeper into understanding what you're doing with a i what you're doing and forecasting and predict of scoring any recommendations there

[aj_ezzour]: 28:01
um yeah i mean keep an eye out on our release plan you'll see some really cool futures coming out i think we're teasing a lot of new things sales is going to be launching into GA in october coming very soon so i think yeah keeping an eye out on that you'll see some really interesting things

[mark]: 28:21
AJ thanks very much for coming on the show

[aj_ezzour]: 28:24
thank you so much mark