Dataverse Connector with Matt Townsend

[mark]: 2:20
in this episode will be focusing on the data vers connector today's guest is from atlanta georgia in the u s a he works for microsopt as a senior product manager for power automated you can find links to his io any social media et cetera in the show notes for this episode matt welcome to the show

[matt_townsend]: 2:38
high markhetancs for having me

[mark]: 2:40
good to have you on the show i'm really keen to get a bit of your background now you're in seattle then move back to atlanta is that right

[matt_townsend]: 2:49
so i just moved back to atlanta after about six years working from microsoft in seattle great to have that flexibility to keep working from home to move back

[mark]: 2:58
nice

[matt_townsend]: 2:59
and yeah i just got all set up back in atlanta and closer to family and things closer to home

[mark]: 3:04
nice nice so just on that front what do you do outside of microscope what what do you do for entertainment you know what food do you into an atlanta and yeah think about family

[matt_townsend]: 3:18
definitely well for about the past year and a half i've been raising a many australian shepard her name is re taken care of most of my time outside of work as you can imagine they are quite a handful that's that's two or three hours a day of just activity and getting

[mark]: 3:31
okay

[matt_townsend]: 3:31
out there and and training and all of that i also have a cat named clover she is a small orange tabby she's getting up there she's aged nine

[mark]: 3:41
wow

[matt_townsend]: 3:42
also i'm caring for those guys besides that do some biking and running and um yeah definitely miss some of the bike trails and some of the cenery around seattle but it's great to be back in atlanta and back in range of everything familiar here

[mark]: 3:55
what's the best thing about atlanta

[matt_townsend]: 4:00
for me i'd say it's family and you know there's a lot o know at atlanta everything is well i mean depending on on traffic you know around a twenty minute drive away so there's

[mark]: 4:09
nice

[matt_townsend]: 4:09
any quzine you can imagine even a wider range than in seattle or some some parts out there is right near by and you now you can drive all the way up the east coast in a day you want you no i've done the drive to new york before it's about about thirteen hours so it feels like

[mark]: 4:26
okay

[matt_townsend]: 4:26
hm you know if you get if you get tired of anything in atlanta you can always just head up the coast or head into the you know the smoky mountains or other pat the dorterorthings it's all not too far out of reach

[mark]: 4:37
nice atlanta seems to be one of those places that magrashoft used historically in the bizet space for a lot of their events right

[matt_townsend]: 4:45
definitely some of the major business applications conferences have been here and the microsofh campus is growing there's now i believe a couple of thousand people

[mark]: 4:52
wow

[matt_townsend]: 4:52
based here with the new hub and atlantic station and they're planning a further expansion as well and i guess it all is very familiar with me i grew up in the area and went to georgia tech as well so it's all i feels very close to home to

[mark]: 5:07
nice

[matt_townsend]: 5:07
be back

[mark]: 5:08
at one of those conferences because i've attended many conferences in tlanta over over my career and i got to shoot the largest gun i've ever shot in my life so it was a fifty cell gun sniper rifle which i think every time you pulled the trigger you spent about twelve or thirteen dollars because you know each cartridge was it was amazing but just a phenomenal experience

[matt_townsend]: 5:34
oh well

[mark]: 5:35
in in a gun range there

[matt_townsend]: 5:38
it's gonna say at eily a conference experience that i have not that i have not had but i do remember one of the times i was going home from seattle it was i think around on the conference so the whole play and i was like i was ike am i flying away from work because everyone on the plane scene be from micro soft going right right

[mark]: 5:52
so true

[matt_townsend]: 5:52
back to the conference

[mark]: 5:53
so true i tell you in the early days of v p summit in seattle is that we used to pretty much every time we went to summit we would always go to a gun range because you somebody like me from my country guns are pretty restricted and so you know to get to with my american colleagues and get to go and fire some phenomenal hardware at the gun range quite entertaining which we did every time in seattle that we went so

[matt_townsend]: 6:19
nice

[mark]: 6:19
yeah kind of and associate with the micro of event anyhow how did you how did you end up a micro so what was your journey ah

[matt_townsend]: 6:30
okay so my journey i guess i'd say yeah i think fomeliniwas al ways involved in a sort of tech assistant roles or a v pieces or that type of hardware georgia tech i studied computer science and had kind of a first intern ship for company doing process automation and RP vendor it's since been acquired but kind of got got to experience kind of all those aspects of solution development right not just not just leveraging computer science concepts and stuff they're apt but building really targeted applications for you know for call center processes or other back office pieces weigh where you an kind of immediately see see the business value right and see how the work flows if you know just one thing is off you've messed up something for hundreds of people all of that right so got a moved on from m some of those pieces to another intern ship but micro soft followed on to the same the same team from there and spent about four years working on different internal teams and micro soft of right you have internal support tools part of bringing our support experiences to leverage dynamics to sixty five internally so got to travel all over you know microsopsapar is very broad footprint worldwide got to see different sides of you know almost every microsoph product it feels feels like an um so got a lot of exposure there and then about two and a half years ago made the switch to a more of a product role than than some of those internal pieces and switched to working on some of the the studio and now i phos experiences in power automat so that's since been rebranded to dust up flows as some folks may be familiar we've done a lot of work to make it possible not just to automate pieces in the cloud but things on your on your dust top as well so uh since working on some of those integration pieces of move back more into some of the dynamics and data verse pieces and currently managing our data verse connect some of our dynamic integration pieces classic work flows some of these other aspects that no power to me is wrapped up with in making the power platform the best i could experience it can be

[mark]: 8:45
yeah so tell us the Microsoft data verse connect what is it where does it sit in the echo

[matt_townsend]: 8:54
yeah right

[mark]: 8:54
system and what is it commonly used for

[matt_townsend]: 8:58
so the the microsoft data verse connectoris power automates inter face for automation of data verse and data verse backed apse so that's dynamic through sixty five industry cloud ps and any customer apse that you've made you know data verse for teams outside of data verse for teams where you're leveraging um you know any any data verse capability any data stored in data verse and you're trying to now leverage that to integrate with other applications through the power automat connect your eco system or simply build more advanced work flows or automated processes with those steps in data verse so it's customer facing in the sense that we know oh if you're listening depend on this very heavily and your cloud flows and integration with data verse and dynamic there sixty five across the platform but we also use it very heavily internally so a lot of the dynamic apse inari sixty five marketing and other pieces are are built themselves different features are built themselves on the state of verse connector and power automate to build different features and things that are backed by these automation capabilities so we we serve a lot of tay quotes from that respect but it's all in that spirit of improving data verse automation making more types of automation possible through through data verse

[mark]: 10:21
nice so there was here's two connectors right there's there's an old and a new um is the old one pretty much nobody's using it any more or is it still actively being used what's what's the position and why was there a transition between you know without just up reading the old why was there a decision

[matt_townsend]: 10:43
right

[mark]: 10:44
around why a new connector

[matt_townsend]: 10:47
yeah so you can you can think of it like with each major generation of some of the data verse pieces coming to be and moving from you know the older parts of dynamics and everything that as the platform has evolved there's basically been new connectors at different phases along the way so the dynamic three sixty five connector was much older piece of that the microsophdata vers legacy connector is you know a bit more recent with that integration and the current connector you see now microsofhdata verse connector um is looking to build on all those past aspects right expose new capabilities new actions and the major piece that sets it apart is that it's natively integrated with data verse so the other connectors follow some of the standard paths used by other onnectors in the platform but our current microsofdata is connector is that up to actually directly connect a data verso calls are going direct without extra overhead that means your flows run faster with less overhead than some of the past pieces so kind of doubling down on that native integration you may have seen power ups has a similar tables experience where they've natively integrated with data verse as well to optimize some of those experiences um that's where as we're looking to phase out the legacy connectors double down investment of this connector as the primary way that you can automate data verse through power automateum we'll get i think more some of the migration questions or pieces are on this later in the show but um the focus is on ensuring that you have control over moving your past flows to this connector we don't want to break your connections your business logic those other pieces so um currently we have migration stent that can help with the migration and can help create a new copy of your flow with you know the dynamic scheme and those actions and things migrated but we haven't set it up to be fully automatic fully behind the scenes which we know would be much much easier an experience but at this point we don't because new connections are required and some of the other pieces we want to make sure that you have full control over that migration experience at this point in time and so you're another question on just what is the usage of the legacy connectors today and definitely if if you're using legacy connector you are not alone we have many many thousands of users still using the legacy connectors and still facing migration and looking at what's needed to move from these pieces we do see more than ninety five percent of new flows are created with the native connector so we've kind of hit this milestone where for most scenarios the native connector is now the connector that that works that is adopted that is being used the body of migration is primarily on these past flows that have bee created that we want to support you in continuing to run and move forward without interruption and just calling out that the two major capabilities were working towards that are not yet supported in this connector are connecting to other environments besides the current environment and the winner did trigger that you use from model driven power ps and from your dynamic through sixty five api so both of those are things we're actively working on in the coming months to bring support to that migration utility um and you make sure everyone has a smooth experience of moving off of the legacy connectors without interruption to their current processes

[mark]: 14:24
so you mentioned ninety five percent of new flows using the new connector is any reason why you didn't make the old connector unselectable as on foot thing new so yes find a legacy but already in play but prevent people from using that going forward

[matt_townsend]: 14:44
definitely and i guess part of it is because of those two scenarios i just mentioned that there's not currently a work ground for those you need the legacy connector today

[mark]: 14:51
right

[matt_townsend]: 14:52
for those two scenarios

[mark]: 14:52
got you

[matt_townsend]: 14:53
so right now the deprecation schedule has us turning off the connector for use in new flows in march twenty twenty three so that means searching for the connector you will not see it in in new flows by the end of march twenty twenty three and turned off for existing flows in october twenty twenty three at the earliest and as an aditional clarification there if you have a flow that is imported through a solution or kind of store and that connector has already been added you can continue to import those that's really just in the flow designer when you're searching that's the milestone for march that hat will be turned off first as i mentioned our intent is to have support for connecting other environment support for the trigger when roas selected ahead of these milestones right to insure that folks don't get stuck without ways to reach this capabilities

[mark]: 15:49
so if people don't have a dependency on those two features that are yet to be added there's really nothing stopping them up grading now right to the new connector

[matt_townsend]: 16:00
yes so we can walk through some of the steps to share to share in more detail if just just a clare on some of the pieces but if you're using the dynamic three sixty five deprecated connector or the microsoph data verse legacy connector um in most cases you can go ahead and update to the microsophdata erseconnector and we're working to publish i think by the time this episode goes alive we'll have an updated documentation guide that goes through each of the party features that i'm describing here the migration assistant that we have in those related steps um m there are a few other limitations i want to call out so if you're using a flow most flows created before about june twenty twenty are in an older schema that we cannot yet convert using the migration assistant so a lot of the older flows that are out there actually have to be re created all right now so you can move it just has to move to the microsopdata vers legacy connector or fully recreated manually with the sorry with the microsopdatavers connector we're working adding support to the migration system so that you don't need to manually rebuild those who understand some of the challenges there and we want that to be as easy as possible

[mark]: 17:16
yeah makes sense one of things we discussed was moving data verse from pro code to low code what are you thinking about when you think about that

[matt_townsend]: 17:28
definitely so sorry let me get a sip of water here

[mark]: 17:31
yeah go for it

[matt_townsend]: 17:42
definitely and what you see is sorry definitely in what you see is um in our data verse connector today there's many aspects where you need to know internal logical names of your data verse entities or some of the c aspects and action names or things that are less less familiar you know you may have built an ap customersed a form and built out a whole experience for other users in your organization but you may not be familiar with those internal names or those pieces until you need to write a flow that connects those right so we recognize that that adds some significant challenge is just in finding the right names for things to specify you know when you're doing look ups and the data verse connector when you're riting filters today you need the logical names and odatasintext for those queries which um and in most cases we see there's ways to figure out the exact queries but we want to make those as easy as possible so kind of in a parallel work stream to the migration experiences and those parody features that i was talking about on the connector we are looking to improve the usability of the connector itself we're looking at richer look up experiences and other controls all those you you can almost say just just the more challenging low code pieces or the less low code pieces right to make sure that from power automat you can um build automations without knowing those internal names without knowing o data those other pieces each of those is a kind of a cliff right you have to go learn new syntax and find those other pieces so we're definitely looking at ways of

[mark]: 19:22
awesome

[matt_townsend]: 19:22
making um as easy as possible and i did want to call it a few resources related to that as um there's a just just in case other folks haven't seen these there's a great community browser extension called level up for dynamic to sixty five and power ups i can share the link with that after um on level um m it was a great overview of the internal field names over the form view of your data and data verso if you're trying to understand you know based on the form you can figure just what the fields are what syntax should be used um m that's a great way to get there in the built in the recently updated power apse table view for your data you can view there's a link from each row um and the schema of each column to view some of the mitmedidata from that updated page so those those are the other kind of built in way to see internal names end for look up another regarding columns and things that's the best way to find out what exactly those column names are a better output from other actions and now if you do a list rows or get a re action in the flow i think i see a lot of people doing that um m and the the other general resource is the fethxmlbuilder community project in the dynamic th five tool box fecxmlbuilder can help you create you know more advanced queries especially if you're trying to tune performance of your data verse queries or things or kind of reduce the load on your cloudflowsum by creating a fechxamlcrer you can specify the exact columns and other filter conditions right more than what you can do in data um and today the craft in those milk queries and you know that's where tools like the fehxmilbuilder can help you create those that can really lighten a ittle bit on what data is being returned kind of help you as you develop those queries and build flows that use more specific pieces of data

[mark]: 21:22
good those guys that have created those will love that you've given them a shout out and you will make sure we link them in the show notes for this episode so you can jump straight to them when we look at optimizing mark off data

[matt_townsend]: 21:32
okay

[mark]: 21:32
verse sage what are you thinking in that area around optimization and and making it a more compelling environment for people to build their solutions in

[matt_townsend]: 21:52
i'd say a lot of the usability aspects that i was talking about just around simply being able to use these pieces there's other investments that we're making around stream lining licenses discovery finding things in context of certain apse we want to make those experiences as easy as possible um but i'd say yeah i guess i said there's the mainuusability aspects that we had focus on i would you know there's just as we were speaking about cliffs and other kind of low code pieces where you know currently you know there are still cases where yo need to jump out and use a classic work full or a plug in to handle certain scenarios so did want to highlight one area where the cliff isn't phil galen but there has been lot of work to make it make it better is the customer p i experience and our business events integration from the connector so high light um m i'll shore a few the links after this as well but the custom may be i experience for um either as part of puggins or kind of on their own you can specify custom apis and data verse for specific messages or business events right and empower automate this can be exposed as actions in our winning action is performed trigger and you can also invoke those actions if there's business logic behind those through the performer band and unbound action depending on whether they're tied to a given table or not but both of those are kind of our gateways to other other customizations right kind of at arm's reach from the flue you can quickly taylor specific triggers taylor specific business logic um start you know some of the things that are not a straight forward you can figure directly within the flow you can still easily make accessible to one of your flows and start extending that way

[mark]: 23:43
nice you mentioned their classic work flows and plug ins which have of course been around for a long time back on windows work flow foundation what's the are we in closer to not needing them at all because you know i was just with a customer recently who says they're still extensively using classic work flow and not know switching over to to power automat what's where's your thinking at the moment in that space

[matt_townsend]: 24:22
definitely so yeah we definitely recognized for some of the core dynamics usage especially when you have millions of triggers or certain pieces and all of your logic is within dynamics right classic or post can be simpler to can figure in some of those scenarios definitely acknowledge that classiwrfls still have their place and we haven't fully solved that from the from the cloud pot side right but we are looking to make progress both on the scale aspect that i mentioned making sure that we can support you know the trigger volume from dynamics and there's other pieces without disruption are things that are ard to predict and we are looking into as well sincrness execution scenarios there's some longer term pieces that we do want to improve that we've finally been able to make some some progress on on the kind of i guess our planning and architecture side on those aspects so definitely acknowledge you know sincerness runs some of the weight condition aspects ah and the pre image details that are you can have kind of more finally crafted triggers and things from from certain work flows where looking to make some of those capabilities available from cold pos as well so that um you can you can leverage the whole connectorszeco system you can leverage the other power epps and betting and all the other improvements that you know come with using a called flow instead of a work flow one other piece i wanted to hire it on classic data verse work flows we do have a conversion utility that we're looking to revamp a little bit part of this is a lower party than some of the migration pieces that i mentioned on the lagastaconnector deprecation um but we do have a converter that supports i guess the significant portion of the classic work clothes that are out there and can help move you know just if you're not interested in rewriting dozens of lines of classic work flow pieces o this converter um can definitely help with it and yeah so looking to make some aspects of that more visible as well just on more than just you know the recommendation to use cloudflosare goin to make sure that you can actually make that transition

[mark]: 26:40
good

[matt_townsend]: 26:40
as as easy as possible without you know having to go reread everything from scratch and everything goes with that

[mark]: 26:41
last question i have is what questions you're seeing most coming from the community around dataverse connector what are you seeing the most and do you have any answers to any of those

[matt_townsend]: 27:03
definitely and we do see um you know tons of community feedback and engagement on our power automate ideas community and our power to mate building flows community and those other other spots so thank you for everyone who particip to help answer questions in those forms i say some of the top top requests we're hearing are looking for ways to make it easier to imbed flows in dynamic ups and model different power ups more than just that flow menu being able to say just this flow from just this context make it easier kind of an easier commanding experience for for parts of that integration we've heard a lot of feet back on bulk migration support as an man how do i move from the legacy connectors making that easier so as part of the documentation that i mentioned on the migration pieces we are looking to um publish a few scripts for edmonds as well that can help use the existing that migration migration assistant experience that i mentioned we want to there are someaditional tools for adumums that can help make that a little bit easier when you have many flows that you're looking at in your organization and you want to understand which are in scope and you know which can be moved using that tool and things offhand um m in general those other us ability pieces are the main definitely some of the most challenging feedback we hear around look up and just all those cliffs on you know now we need to go research data and now we need to you know or new syntax or how do i get this exact internal name or expansion query and i did want to give another shout out to one of our excuse me i did want to give another shout out to one of our community members dana cahill who has a incredible data vers connector cheat heat you may have seen on line um it highlights every action every trigger in a very clear way so shout out to dana for making such a great resource for the community but that has a very comprehensive view over how to use the dataversconnector and really an over view of everything where you can do improve you know you shouldn't need a giant p f to understand how to use this thing so we are trying to

[mark]: 29:11
fantastic

[matt_townsend]: 29:12
make progress all

[mark]: 29:12
fantastic

[matt_townsend]: 29:12
these aspects on the connector so um yeah piece that there's the main i want to call

[mark]: 29:13
will make sure we put links to danny's resource in there as well but yeah thanks so much for coming on the show it's been a real pleasure and really informative i appreciate it thank you

[matt_townsend]: 29:30
thank you very much more than thanks for having me