Mixed Reality and Geospatial capabilities of the Power Platform with Anthony Uitz

[mark]: 1:49
today's guest is from washington in the united states working at microsophis a senior program manager as always links to things like linked in twitter bios et cetera will be in the show notes this episode so check them out on the app that you're listening with welcome to the show anthony

[anthony]: 2:07
hi thank you so much for having me excited to be here

[mark]: 2:10
good to have you on the show tell us a bit about what you do outside of microstoff when you're not working what does it look like for you there and are you in seattle redmond what particular area and

[anthony]: 2:21
yeah so i actually moved to san francisco with my fiance about three or four weeks ago and so it's been super exciting kind of readjusting to having sunlight and things like that that seattle is quite sunday in the summer and so i've been etting into actually discovering new hobbies figurin what to do in the area and i live next to a bouldering gym so i've been trying to check that out it more frequently like rock climbing stuff it's been super fun but i've destroyed my fingers a little bit but it's been super nice and so looking to discover any no new new social circles new hobby stuff like that i think one of the big ones for me is that gotten really into cooking recently i used to watch a lot of cooking youtube back in seattle i lived in capital hill which is like the central district between like so many restaurants and so it was always like yeah i can make food that tastes really bad or walk ten feet and get really good food my new era in san francisco has quite quite a few less restaurants and so i've been trying to like craving and recreating the taste of the restaurants i loved in seattle's haven't getten really into that recently

[mark]: 3:17
that's cool that's cool and i mean san fran is such an awesome city right i mean

[anthony]: 3:21
yeah for sure

[mark]: 3:22
it's epic i mean i love seattle and and i spent a lot of time there but boy i really enjoyed my time in san fran and there this is so much for you to do right

[anthony]: 3:32
ah for sure it's it's super awesome like there's like hikes right in the middle of the city there's a beautiful pier we're still kind of exploring all th nearby areas like you know santa cruz is near by and the beach and the boar walk down there there's that beautiful big surf drive slash hicratstuff like that so we're still new looking for things to do so excited to kind of discover more but it's definitely been a fantastic move i'm glad to be here

[mark]: 3:53
love it i love it so tell us a bit about your area of specialty inside microsopht what do you own what do you look after what's type you know what's the top of mind thing that you're focusing on

[anthony]: 4:04
for sure so within power ape which is our locoed application platform my team kind of lives within the group that owns the mobile experiences and the experiences right and so my team in particular within that group is focused on our kind of native capabilities and so that includes things like mixed reality include things like go special we also sort of kind of growing that into things like barcoe scanning and f c scanning and other device censors like that but right i focus historically has been the mixed reality stuff and so tha's kind of where my expert tease lies actually my career and even in college i kind of did mix reality and virtual reality reality stuff that then so it's kind of a it's an area of both passion and of course my work area which is nice kind of overlap there and so it's been really exciting to kind of it could of enable that mission of how do we democratize a super awesome super powerful technology like mixtureality for everyone to use through through it's like loco

[mark]: 4:53
like it i like it now one thing i want to clear up straight away and it's it's a misconception that i have had in the passes that when you hear that microsive talked about mixed reality we all of a sudden you know i definitely have done this i've defaulted over so hollow lens is what we're talking about we're obviously not talking about hollow lens here we're talking about everything outside of potentially hollow lens from a mixed reality geo special capabilities ex perspective are right

[anthony]: 5:20
yeah for sure i think there's some fantastic work in like yourself going on in hollands and you can actually there's some stuff about bringing power as to hall lands like in guides dynamics for guides you can now bring power apsis as a part of your your steps within guides on the hollands as well but my team focuses more on the idea of once again democratizing mixture out for everyone right through low code and there just aren't enough holland's devices right now for everyone to get their hands one right you know it's definitely limited hardware thing and so we're really focused on specifically mobile augmented to reality as a piece within within mix reality and so how can you use your phone and your phone censors and your tablet and your table censors to kind of get a better understand ing of physical space and these mix reality to kind of empower business workflows through loco

[mark]: 6:03
so to take this into the practical world because sometimes i know people like you know so what are the use cases so since you and i last talked i'll be responding to a big r f p round um waste management for a city and and the mobile experience around that right and so you take that that bin collection or rubbish collection locations are all over the city and so there's geospacial points and then the toughen device and he and the driver of the truck we can have now visually identify the bin where's the code on the bin that they there their actual truck scans at the n f c code so what was the exact time of pick did it happen so all that and of course this totally fits like a whole part of our response is this whole mixed reality barcoe scanning as you say um and and being able to um you know where where trash locations are in unique spots we can overlay that and say hey yeah it's actually down here around this alley down or that's where you'd find it we can provide now all that sort is not kind of car backs from the drivers to you know despatcher where is this how it can actually all come up now on their device

[anthony]: 7:23
yeah for sure it's one of those super cool things that we see around those kind of front line first line worker work flows that's pretty common is that combination of using your devices location and using its ability to detect bar codes and f c so on and eases the kind of overlay stuff in the real road from ixturyaut as well i would say the one particular part you tlkdabout from their way finding like routwayfindin like in door way finding is still technology that we're kind of working through you know those things that we can record a very spacially aware video as you mark up the space and walk through through kind of our mark up in mark control but the idea of like you know you take out your phone it says hey turn left and then right and then and right and there's little arrows in the physical space that we we don't quite do that ye it's like really cool out there and there's a bunch of technology being invested in by microsoff to make that more possible right if you're familiar with kind of the mess announcement or as your spacial anchors is a part of that right like that's that's the end goal of those technologies for sure to enable that kind of live mapping where you combined you know s coordinates with mixed reality overlays to make this super cool senario but it's once again you know a little bit future ttacknfor us we think about mixed reality as another tool in your tool bell really right so like measuring things visualizing content and real world scale stuff like that and so it's definitely gonna be wrong it's all things that we're super excited about and i'm really looking to bring to the space but i just want to caution that some of that stuff you can't do right in power rafts to day you know

[mark]: 8:44
and so and so when we look at the enablement surface that you're working with are you is power ape that tool or are you also seeing it being used in things like the field service you know part of dynamics where do you really see the era that you are seeing customers adopt right now in the mixed reality space what is that surface they're generally working with

[anthony]: 9:09
yeah so most of our cus tomers today are building canvas ops in particular so powerapsand canvas ps they're doing things like you know they're part of a major network that has to do building sections and they need to audit to make sure it's accessible or make sure that it follows specific quiteria compliance guide lines until they can make sure things like you know your doorways of a certain threshold size or it's high enough for the door knobs too far off the ground or your the total square potage you have in your bathroom stall even is big enough right and so they can use they use mixturreality to make that process much faster and much less painful from other workers right other than i mean to you know take out your tape measure and go crawl on the ground and make sure your doorways you know two feet at least or whatever you just take out your phone kind of wig it a little bit it's camera figures out where everything is and ten you can just measure on the ground automatically and because it's integrated within the power apse you can write business logic on top of these measured things you can you can automatically generate you know a check mark if it's if it follows compliance or an if it doesn't you can turn that into a compliance support that you send to your supervisor automatically you can even

[mark]: 10:11
i love it

[anthony]: 10:11
provide very specific guidance it's like hey like this didn't quite match the thing can you verify if you can do these steps to fix it in the moment and you can generate work quarters like that so that kind of integration with that business object through power absence is super powerful with mixture idea as well an there's a couple other snares that we see as well but that that's one of the kind of key pieces we see

[mark]: 10:29
so so you've mentioned that one area around building inspection what are the other type of use cases or even industries that you're seeing as the adoptors of this technology

[anthony]: 10:41
for sure i think one of the big ones is consumer package goods or retail kind of that same industry where it's the idea of how do we enable sellers to have better conversations if you're you know a person who distribute let's say chips and other snacks nd you have to go to a grocery store pharmacy and say hey we want to put this display stand in here and help you sell more chips and drive pass through and all that stuff right there's a lot of stuff you can't talk about to picture a video but you say hey look it's my display than it looks really cool right like that's great and all but what we found is that mixed reality helps sellers close more deals close deals faster and close larger deals gonna have a much higher fidelity conversation about the product they can say hey look here's exactly how this look in your space here's how it plays with lines of sight and egress here's how it looks next to this other thing here's the real world scale and these this ability to have conversation really un block on block stores right like we have we have people giving us feed back that's like yeah like if it wasn't for the ability to show my product in the customers space they were they were a hard know beforehand hey i don't think you know your big chain retailer thing look good in my tiny like maman pop shop and they say no ike hey just give me one chance to show me exactly the space you have available i'll show you hat that product looks righte like you know actually if you just made black and white i think you look perfect or like oh if you make this little change here i think look great

[mark]: 11:55
right

[anthony]: 11:55
and they can capture that all they can do that and kind of help drive sales so that that retail scenario super important i think also in part because of that post sales process as well right you have folks who you know if you've ever been a part of that work flow before you say okay you and i as let's say where the seller and the buyer the store manager we agree like let's put the cooler right here right but ten i have to go tell my my installation team yeah when she opened the shop in three weeks like the cooler goes you know two feet from the door and it's like a this thirty degree angle off the shelf and it's like it's like right here man like and it's all written down right there's no real record of that and so there's a lot of post sales installation mistakes that and a result of that that method of communication but reality you can literally be like yeah here's a picture of exactly where the product should be exactly where you and i as the shop the shop manager and the owner agree and the tansationalteam guysadthey can see that they can see it in context in world scale and install it right there and so it kind of helps with that post sales process driving an increase satisfaction is all just super nice

[mark]: 12:50
so are you creating three models that can then as you say turn you cameron go into a space and there would be constraints on those models for so far example let's say the space it was only put it in your terms four foot wide and your device was in use metric and let's say your device was six foot wide your shelving or your display et cetera there's no way that you could go just visually make that four foot wide and fit within the space i assume there's a whole bunch of constraints et cetera that you can make sure how this visually appears in a landscape in a field of view doesn't allow you to display something that looks good visually but actually physically is impossible

[anthony]: 13:40
yeah for sure that's like the benefit of misurethat we have this thing called inclusion which is where the camera can see when something is in the way and kind of handle that programmatically right and give you kind of notification of that and when you're inside of as well we do support different features for things like scaling so one of the things that conversation that do happen is like well what if this shelf was just a little bit smaller because people can customize their products in many cases and so they can use their typical pinch to zoom features to make it smaller or bigger and visualize it that way and then also you know around the day of three d content i think one of the misconceptions of a mixureauty an this particular work flow in general you need really high fidelity models that you've pre made that are now made by a custom third party house that specializes in that what you can do don't get me wrong like those are like when you talk about sales those are super helpful but yo don't ee to have that like we support this ability where and say hey i have an image of what the front of my shelf looks like and i know that my shelf is going to be two feet by one feet by four feet using our evil imperial system i guess and so when you do that like you can we generate that that back for you and so you can you can see what that image looks like with depth and still those same great conversation s around you know lines of site and egrasm does it fit here and how does it without having the need for the super high fidelity three d content you just ed an image right and then we also support any you know we supported from file for mats two so if you happen to have content to use for you know s t l which is very common in three d printing or manu ring or b j which is just coming across the board as i shared file for a right we support that as well as g l b and so we're pretty flexible with how we support content we also support connectors once we had great power of power as we have a building connect to what's called to three model cutlery called c g trader they're one of the largest kind of three d model galleries that exists that has both free and paid content we have a built in conector to them and so you can build a three d content that way or you can and you can of course build a connector to any service that you have to use for the content and use it as your contemprofiter that way and so or you can use data erse play your files that way you can use share point or any of the connectors and so that kind of three d content library is pretty easy to build support and to manage thanks to the power of the connectoreosystem that we have within power as and so it's all you know designed for that purpose built for that purpose and so it's hopefully a pretty nice experiential folks we think it's repritodbut feed back s of course i was welcome

[mark]: 15:53
nice um so we've looked at building inspections who have looked at retail as being great use cases any others at a kind of top of mind

[anthony]: 16:02
yeah i would say the last two are estimations and inspections so estimation is isn't really like an industry person but think it's a very common scenario so to give an example you might be doing some kind of kitchen renovation as like you know company that does that kind of thing and so as a part of hat you have to replace the flooring but you want to give a quote to the customer about how much i cost replete that flooring and so you can use you know the measurement control once again to go measure these kind of non regular surfaces like someone's floor and their room you cut out the eyelids cut out the different shapes all that stuff and then we do the math to be like yeah this is you know four hundred and twenty six are feet right or how many square

[mark]: 16:39
nice

[anthony]: 16:39
meters you want to have and so you can use that measurement to do area and volume as well i mean so you can you can do estimations that way like another commissnarios have to estimate how much it llit'll how much paint take to replace the wall or how much it will cost to renew a painting or something like that right these kind of different area estimations are very important right and that ties in a little bit to that first of building inspections as well which is that auditing that my idea is like i need to like i contract the obligated to this much shelf space let me go make sure that i'm getting that right you can measure that out as well and then for the the second star just mentioned which is inspections right there's there's clear examples of things like you're the technician repairing in eight m and you want to see what the guts of that eight look like without taking it apart you can drop in a three d model you can see that scale it will animate you can kind of see inside stuff like that right so that that's example and then recently we released a new control called mark up in m r let's you paint lines and three d arrows in the space and so you can use this to kind of you now let's say you're like like i'm inspecting this water cooler there's an issue with the way that it's con figured right now let me say hey go turn this gear and go right this pump here and so if you're familiar with assist as our product right they have that same capability we've brought that kind of off line parts of that not the calling bits but the drawing lines and drawing arrows into power wraps as a stand component so you can do that as well

[mark]: 18:00
nice and so so when we look at that those surfaces i talked about before so all these controls are available right now in power up studio

[anthony]: 18:08
that's correct they're part of the standard office license as well and so as long as you have access to office you have access to assume it's not of course blocked in your tenant son you have access to office you have access to powerapps and then you ave access to more controls and so if you go into the insert pay they're they're right there under the mixture reality section in the insert pain there's also a couple controls also available in the media one but we kind of duplicate them withdifferent names to make sure that people discover them find them but they're a their mixreality sections oh

[mark]: 18:33
so so because it's in there i take it that all the tal sets are available as part of the business output so for example when you gave that floor estimation in my mind right away i'm like hey i know you know how contractors work in the space they need to allow for wastage they need to allow if their tiles going on the floor and there's a wedge we're going to have to use you know maybe two tiles that where dimensionally it's only one tile but they're gonna do it so i take it with things like power automat is part of the story you could actually make sure all those business rules were honored based on what the application was going to be used for

[anthony]: 19:15
yeah for sure you can the controls themselves have sets of output properties that give you everything that we have in terms of control so you let's say let's say you measure twelve different lines that make up a polygon of sorts right we give you each of the x y coordinates of each of those lines as well as the total area that we've estimated calculated based off of the polygon itself as well the different shapes and all those things so you have those all his oupaproperties and so you can take those you can write automate flow or some power effect logic in your power up and use that to run a bunch of rules like hey like if it's you know not quite at the right angle to x y z otherwise do a b c otherwise do you know whatever and you can write these logic to make pretty complex you know i guess mathematical formula is in a sense to figure out exactly what to do here and to render it that way so we we fully support like it's all it's all to you right we'll give you all the numbers you need an you can do every wan with that data and so we give you literally everything that we have and so to the precision that we have it as well and so we do that

[mark]: 20:15

tell me have you seen anything out in the market where people have taken these tools and without mentioning the companies et cetera that are you like wow this is awesome this is what we intend it

[anthony]: 20:27

yeah i definitely think the first one i mentioned around the building audenceis one where it was it was surprising how fantastic that was like getting the workers to talk to us about that was it was fantastic right like i think the one of the big things that's part of you know what's called a d a compliance in america which is the american of disability act is like you have to make sure that toilet stalls and other bathrooms are up to compliance right this is not really the most fun job in the world to go take out a tape measure make sure that like your single style is up to date and so then being able to just take out their phone and just do it and audmatically generate the report they won't have one from like the super paper heavy process if you walk in there get a tape measure your in it down you go take that paper you written to like you know facts that a someone puts into an excel sheet who then tells you two weeks later that actually you mis measured it probably right had to go back and double check that right like the returned that into a single digitally transformed process or the disuse mixureality right and so it

[mark]: 21:19
nice

[anthony]: 21:19
was it was crazy to see you know like because we we actually didn't give the that much guidance they reach out one re like okay lie there's a bug nd we're like oh cool and we fixed it there like save one were like you got our p an ike i'm sorry like it was like this really beautiful integrate system that we saw and so i really love that one and then another on that raly liked in that same vein actually was this company that does repairs like restorations and so they go into a house building or whatever that's been effected by fire or by flooding or by other disasters right they have to go figure out how much will it cost repair your rugs and replace your carpeting and fix your walls a lot stuff and they built this lie beautiful like inventory and auditing system through that control as well which is super nice to see

[mark]: 22:02
i love it i love it tell me a bit more and we'll finish on this area as around-the-go special capabilities kind of what are the key points that people need to be aware of around the go special capability

[anthony]: 22:14
yeah it's it's a pretty rich set of capabilities that are also i would say verely table stakes in the market when you think about it right there's a very common address imp control you type in you know the first half of your address and it completes the rest of it and gives you the latitude and longitude of that and all the different split out pieces like which ones like the municipality in which one is the country and all that stuff on that one i would say fairly self explanatory and how it works and it's pretty nice it's very smooth it's simple it works you know the map gets a bit more feature rich in how it operates and that kind of makes a little bit more complex to use but the payoff is i think quite nice if you kind of use it for these complexenaris as well aware we support effectively all the things that as our map does to an extent right and anything that that we don't support that as our map does there's a if we can unblock you with something canary for that we're appy to bring that in right it's definitely built on top of it sure not capabilities and so it's things like you can drop a bunch of pens you can customize those pins colors and icons and so on and so if you're doing for example covid management back back and i guess currently but also back in the day um you can see things like you can colorize the icons by number of cases you can change the icon base stuff of what type of building it is what type of case it is you can do that kind of stuff right we also support in folk so if you click on a pen you can pull up detailed information about that including things like a u r l a hyper link to turn by turn directions that open up in your your our s s favorite application my pictures call phone type come number is full names et cetera you can put on information there we support routing so you can you can give yourself an optimized route between multiple way points to figure out the travel the classic traveling salesmen prob for example and you can of course combine that with the previous pictures o have like custom pins like oh yeah like here in my top five opportunities for the day that need to go visit what's the best order to visit them what's my estimated time of traffic all that kind of good stuff

[mark]: 24:03
love it

[anthony]: 24:04
and we support polygons so you can draw you know areas and radius and we'll give you back like the calculated circumference and distance and areas stuff and so if you're familiar with g j song right there's a bunch of publicly available g j on data sets for things like state lines you can bring that in and use that for territory management you can build your own right go jason and stuff like that and then something we added recently which is small but i think cool as we added different map stylings now so you can have a dark mode map which is pretty nice and so you can do like our theme and so lots of lots of great stuff there it's pretty like i said like feature in and table stakes and so it's one of those things that you know there's a little bit of nuance to getting data set c f controls to work in power so once you get past that kind of little nuance it's a super powerful control that we love to

[mark]: 24:45
nice

[anthony]: 24:45
see using

[mark]: 24:47
have you seen anyone light up a g s data or israel data inside inside it

[anthony]: 24:52
yeah there's been some people who have kind of taken arctisdata into gojsonhuse that on that they have like a separate service that converts that and converts back into that that they use i've seen that before for territory management in particular and then there's some conversations going on about how can we do like like global positioning data to do intelligence on top of like satelighte tiles which like people are looking at there i've never i haven't een something in production but things like can we use sad light images plus the azure mapcapabilities to give you a to give like a forest fire fighter an area to go forest fight right because we can tack that from the cloud right so there's really cool stuff happening in the in the special space that's like all going to eventually bring its powerful services into power as to make it accessible so looking forward to that but nothing nothing to to exist right now you know very future thinking

[mark]: 25:42
awesome anthony if people want to go and develop their skills in this area and and get up to speed what are their resources you recommend they go look at

[anthony]: 25:53
yeah there's a couple so there's of course our official documentation that exists in published it's very much technical knowledge right what are the properties on the exact input output hat does that look like we have a couple of how two guides as wel with another recommendation that's kind of a different section but still it's official recomntation that we've published some ms learned edtorials as well so if you look up building mobile applications with three d you'll see a guide there we also have blog posts that sample applications and future releases and stuff like that so if you look up like you know improving sales with mixed reality poweraps i think it shows up as one of the top results in search engines and so you can you can use that as well and then you know as always this super with power wraps in particular this is super vibrant community and so folks have at least really cool videos of how to leverage the m r n special controls to do really cool things and so that's always if you're if you're more of a visual video learner there's bunch of great stuff there as well

[mark]: 26:43
i like it anthony thanks so much for coming on the show

[anthony]: 26:47
no thank you so much for having me