If you haven’t heard or seen the most recent update to the SDK tool Configuration Migration Tool (CMT) then it is a must check out as the latest version of the tool includes filtering capaibility as Alan Mervitz (@amervitz) noticed on Twitter and has documented on his blog.
This feature in CMT has been long sought after for years…and years. The basics of the ask were, filter the data prior to exporting, so that you could extract a subset of data. Previously CMT worked in a fashion for an entity that it was all the records or nothing. You can 100,000 contacts in your system, it would attempt to extract all of them. Now you can provide a Fetchxml filter to a data schema definition and have that applied when CMT requests the data from CDS (also known to some as Dynamics 365/CRM…still). You could previously post process exports but this required extracting and manipulating the data which added complexity.
This is huge from a portal perspective. You will often have a segment of portal configuration or have multiple portals in an instance or probably the most common issue I see people encounter with the portal and CMT is getting the annotations, or notes for a web file. All scenarios begging for a subset, or filtered data.
Now with the updated CMT, you can!
Microsoft already provides Configuration Migration data schema files for each of the portal templates but currently they are missing the filtering actually for that web site. At present they actually are only really a schema definition of entities related to a portal template not data specific to that portal. With this new filtering we can resolve 2 of the most common grips, items for that specific web site and annotations only related to web files.
As Alan has documented, you just need to go into the Import Schema Configuration while you are in Create Schema mode of CMT. Within there you are presented with a wonderful checkbox of “Use FetchXML to filter records” and an edit button.
Checkout Alan’s post for more details.
To add to Alan’s discoveries you can also include only the filter portion of the fetch definition if you are filtering with conditions on the primary entity itself.
If not setting conditions just on the primary entity, such as requiring a linked-entity type of filter which like you would need for annotation, and web file, you need to include the whole fetch definition without the attributes and order.
To help you utilize the portal schema files but with filtering capability I have updated the schema files for each template with the filters to allow the start of a more targeted export.
GitHub: koolin/PowerApps.Portal.ConfigurationDataFiles
Of course you can get a lot more narrow with filters as well. Whatever way you can filter your data in FetchXML, you can apply to the export now too. Only want Web Templates that start with a certain naming prefix, or only those modified before Christmas 🎅, add those filters and you have your schema.
The other great change contained within the new CMT is also the option to exclude pre-existing items in a system from being updated. If it already exists in the import environment and it matches on the uniqueness condition for a record being imported it will be skipped. Great way to ensure you don’t update existing data if you are aiming to only add data.
All these updates to CMT really move to solving a lot of the ALM problems portal teams previously had. CMT is now headless providing a way of scripting the import and export as part of a build process, and now it has filtering providing a way of getting a subset of data. These have been huge asks I have had on wish list for a long time and it is great to see them available now 🥳.
Colin Vermander 