Dynamics 365 Virtual Entities tips and tricks

Recently I've finally got opportunity to work with Dynamics 365 virtual entity and write custom data provider. While working on implementing custom data provider I stumbled upon pretty peculiar or not well documented behavior.

First of all there already are plenty of good articles regarding virtual entities. Two articles from Bob Guidinger: Virtual Entities in Dynamics 365 and Custom Data Providers for Virtual Entities. And three parts series from Ivan Ficko: Virtual Entities Part 1 – OData v4, Virtual Entities Part 2 – Cosmos DB, Virtual Entities Part 3 – Custom Data Provider. Four parts series from Imran Chowdhury starting from here. These articles will easily onboard you on the subject.

First thing that you will need in your custom data provider plugin is IEntityDataSourceRetrieverService service which has exactly one method RetrieveEntityDataSource.

var dataSourceRetrieverService = serviceProvider.Get<IEntityDataSourceRetrieverService>();
var dataSource = entityDataSourceRetrieverService.RetrieveEntityDataSource();

What does the code above do? IEntityDataSourceRetrieverService is injected in IServiceProvider IoC container for your data provider plugins, which run in stage 30 (Main Operation). Initially this stage was exclusively for Dynamics 365 core operations. But now we are able to use it in data source provider plugins which get registered specifically for this. RetrieveEntityDataSource returns Entity object. But what is this object? When called RetrieveEntityDataSource verifies that IPluginExecutionContext Stage property has value 30. Then it gets metadata of virtual entity (that's representation of external business data you want to be able to show in D365, e.g. new_externalcountry) it triggers on and uses DataSourceId property to find used data source entity. If it's configured properly it will retrieve your custom data source entity (that's where you store URL and credentials for your external data service) with all columns. You would actually be able to write code that would do the same. But IEntityDataSourceRetrieverService is already there and uses caching.

Another useful type you might use is ValidatingQueryExpressionVisitor from Microsoft.Xrm.Sdk.Data.Visitors namespace. Again there is no documentation on usage. When you'r developing custom data provider usually external endpoint won't support all the query capabilities we have in D365. So there is scenario when you might want quickly check query expression object and throw error if there are unsupported options. Example of usage:

var queryExpression = executionContext.InputParameterOrDefault<QueryExpression>("Query");

var validatingVisitor = new ValidatingQueryExpressionVisitor(
    AllowedQueryOptions.Filter | AllowedQueryOptions.OrderBy | AllowedQueryOptions.Project | AllowedQueryOptions.Top,
    co => co == ConditionOperator.Equal || co == ConditionOperator.NotEqual || co == ConditionOperator.Like || co == ConditionOperator.NotLike,
    pi => true);
    
queryExpression = queryExpression.Accept(validatingVisitor);

ValidatingQueryExpressionVisitor constructor accepts enum with allowed query options (filter, order by, top, etc), predicate for validating condition operator (equal, greater then, this fiscal year, etc) and predicate for validating paging info object. When calling Accept method visitor will check all validation options and when any of them is invalid it will throw InvalidQueryException with message that something is not supported. Although it seems more user-friendly to just ignore unsupported query options this class still might save you some time.

Next we need to convert query expression from Dynamics 365 schema names to external names (e.g. convert new_name to FullName) and Dynamics 365 types to external types. There is QueryMappingVisitor visitor for this in Microsoft.Xrm.Sdk.Data.Visitors namespace. But there is also handy extension method ConvertSchema for QueryExpression in Microsoft.Xrm.Sdk.Data.Extensions namespace.

var convertedQueryExpression = queryExpression.ConvertSchema(queryMap);

This methods accepts QueryMap object, which we must create first.

var queryMapFactory = new QueryMapFactory(organizationService, new DefaultTypeMapFactory());
var queryMap = queryMapFactory.Create(queryExpression);

This is will create query map with simplest type converter, which will use DefaultValueConverter for almost all types. DefaultValueConverter does nothing simply leaving object what they are.

Most probably there will be a need to convert between D365 types and external (OptionSetValue <-> int or OptionSetValue <-> string). In my case I've got all strings from external system and have to write several IAttributeValueConverter converters for GUIDs, booleans, dates and entity references. They will be registered in custom ITypeMapFactory implementation. I won't cover them here.

Converted query expression may be used then in custom visitor which will build query for external service would it be OData, SQL or custom WebApi. Response model from your service must be converted back in D365 typing. I haven't found built in tool in Sdk.Data so I wrote simple mapper which uses EntityMap object from queryMap created earlier.

var result = new EntityCollection()
{
    EntityName = queryMap.PrimaryEntityMap.NameMap.XrmName
};

foreach (var model in response.Results)
{
	result.Entities.Add(mapper.ModelToEntity(model, queryMap.PrimaryEntityMap));
}

executionContext.OutputParameters["BusinessEntityCollection"] = result;

This is it. Lets recap major steps retrieve multiple plugin for custom service provider must have:

1. Use IEntityDataSourceRetrieverService to get data source record.
2. Use ValidatingQueryExpressionVisitor for quick query validation.
3. Use QueryMapFactory to create QueryMap object.
4. Write custom attribute converters and type map factory if defaults are not enough.
5. Use ConvertSchema extension method.
6. Convert your model object to Entity using EntityMap.
7. Populate output parameter BusinessEntityCollection with EntityCollection.