How to query metadata for all existing fields

Question

We want to enable the client to post to an endpoint such as:

    [Route("Account", Name = "CreateAccount", Order = 1)]
    [HttpPost]
    public Account CreateAccount([FromBody] Account account)
    {
        var newAccount = _accountService.CreateAccountEntity(account);
        return newAccount;
    }

We know that this can be done:

POST [Organization URI]/api/data/v8.2/accounts HTTP/1.1
Content-Type: application/json; charset=utf-8
OData-MaxVersion: 4.0
OData-Version: 4.0
Accept: application/json

{
    "name": "Sample Account",
    "creditonhold": false,
    "address1_latitude": 47.639583,
    "description": "This is the description of the sample account",
    "revenue": 5000000,
    "accountcategorycode": 1
}

How do we expose to the consumer the requirements for each post/put?

To phrase it in different words, if I need to update a record on a custom or base entity using Web API as provided by CRM 2016, how do I know which fields are required to create or update the entity?

Edit: I've attempted Hank's approach, and this didn't return any metadata on the entity:

Norrie

You failed, because you used wrong Entity metadata filters. Check my answer for clarification – Ridiculous

Answer 1

You can query the Dynamics 365 metadata using the WebApi endpoint, as shown in the SDK.

For example, to retrieve all the attributes (which includes the requirement level) for account entity:

GET [Organization URI]/api/data/v8.2/EntityDefinitions(LogicalName='account')/Attributes HTTP/1.1
OData-MaxVersion: 4.0
OData-Version: 4.0
Accept: application/json
Content-Type: application/json; charset=utf-8

Distichous

thank you very much, so if i want to use web api to create or update data, can you give an example of how i would do this? would i first query the metadata, and then see what is required? an example with explanation would be fabulous – Norrie

@MeggieLuski - Please post this as a new question! And remember to mark this answer as correct if it answered the question you posted. – Distichous

Answer 2

In order to get all the metadata for an entity using SOAP endpoint, you can use RetrieveEntityRequest:

 var request = new RetrieveEntityRequest
 {
       EntityFilters = Microsoft.Xrm.Sdk.Metadata.EntityFilters.All,
       LogicalName = "account"
 }

 var response = (RetrieveEntityResponse)organizationService.Execute(request); 

EntityFiters is an enum which allows you to specify what metadata are you trying to get:

[Flags]
public enum EntityFilters
{
    //
    // Summary:
    //     Use this to retrieve only entity information. Equivalent to EntityFilters.Default.
    //     Value = 1.
    Entity = 1,
    //
    // Summary:
    //     Use this to retrieve only entity information. Equivalent to EntityFilters.Entity.
    //     Value = 1.
    Default = 1,
    //
    // Summary:
    //     Use this to retrieve entity information plus attributes for the entity. Value
    //     = 2.
    Attributes = 2,
    //
    // Summary:
    //     Use this to retrieve entity information plus privileges for the entity. Value
    //     = 4.
    Privileges = 4,
    //
    // Summary:
    //     Use this to retrieve entity information plus entity relationships for the entity.
    //     Value = 8.
    Relationships = 8,
    //
    // Summary:
    //     Use this to retrieve all data for an entity. Value = 15.
    All = 15
}

This is a flag enum so you can use it like that:

var request = new RetrieveEntityRequest
{
     EntityFilters = EntityFilters.Privileges | EntityFilters.Entity,
     LogicalName = "account"
} 

Or simply use the All value to get all necessary metadata. In your attempt you failed to retrieve metadata, because you asked only for Entity metadata and you are interested in Attributes metadata.

So, taking your code snippet as a base, I would use this in the following way:

[Route("Account", Name = "CreateAccount", Order = 1)]
[HttpPost]
public Account CreateAccount([FromBody] Account account)
{
    VerifyRequiredFields(account);
    var newAccount = _accountService.CreateAccountEntity(account);
    return newAccount;
}

private void VerifyRequiredFields(Account account)
{
     var response = GetEntityMetadata(account);
     var requiredAttributes = response.EntityMetadata.Attributes.Where(a => a.RequiredLevel?.Value == AttributeRequiredLevel.SystemRequired);
     foreach(var requiredAttribute in requiredAttributes)
     {
          if(CheckIfValueIsProvided(requiredAttribute.LogicalName, account))
          {
               throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.BadRequest, $"You are missing required value {requiredAttribute.LogicalName}"));
          }
     }
}

Method GetEntityMetadata is simply doing what was in previous example, so calling RetrieveEntityRequest and returning RetrieveEntityResponse. Of course the implementation of method CheckIfValueIsProvided depends on how your Account model class is defined, but probably you will need some kind of mapping between your model and CRM Entity model (to know how to map for example field "accountnumber" to some field in your model). This is far beyond the scope of this question, but I believe that you already know enough to get started. Just remember, that this is only an example. You should not keep this logic inside your controller class, you should move it to some utility class which you can reuse in different controllers. Metadata does not change often (and you probably have control over this changes), so you also probably would like to cache the metadata somewhere in your web application etc. I hope that you already have an idea what can be done, but the whole design if the logic is another story.

If you want to do this from JavaScript you should probably stick to the webAPI:

http://CRMADDRESS/api/data/v8.2/EntityDefinitions(LogicalName='account')/Attributes?$select=LogicalName,RequiredLevel

Will get you what you want (name of the attribute and its Required level). It will look like that:

{
  "LogicalName":"preferredcontactmethodcodename","RequiredLevel":{
    "Value":"None","CanBeChanged":false,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"MetadataId":"8663b910-af86-4dea-826e-8222706372f4"
},{
  "@odata.type":"#Microsoft.Dynamics.CRM.StringAttributeMetadata","LogicalName":"emailaddress3","RequiredLevel":{
    "Value":"None","CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"MetadataId":"97fb4aae-ea5d-427f-9b2b-9a6b9754286e"
},{
  "@odata.type":"#Microsoft.Dynamics.CRM.StringAttributeMetadata","LogicalName":"emailaddress2","RequiredLevel":{
    "Value":"None","CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"MetadataId":"98b09426-95ab-4f21-87a0-f6775f2b4210"
},{
  "@odata.type":"#Microsoft.Dynamics.CRM.StringAttributeMetadata","LogicalName":"emailaddress1","RequiredLevel":{
    "Value":"None","CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"MetadataId":"b254ab69-de5a-4edb-8059-bdeb6863c544"
},{
  "@odata.type":"#Microsoft.Dynamics.CRM.StringAttributeMetadata","LogicalName":"masteraccountidyominame","RequiredLevel":{
    "Value":"None","CanBeChanged":false,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"MetadataId":"a15dedfc-9382-43ac-8d10-7773aa3eefeb"
},{
  "@odata.type":"#Microsoft.Dynamics.CRM.StringAttributeMetadata","LogicalName":"address1_city","RequiredLevel":{
    "Value":"None","CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"MetadataId":"ca8d0a94-8569-4154-b511-718e11635449"
},{
  "@odata.type":"#Microsoft.Dynamics.CRM.LookupAttributeMetadata","LogicalName":"slaid","RequiredLevel":{
    "Value":"None","CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"MetadataId":"6bdcd7f1-5865-4fef-91b0-676824b18641"
}

You can use this to validate the request on client side, to give user a hint that he is missing important data before he sends a request to the server.

Ridiculous

thank you very much, so if i want to use web api to create or update data, can you give an example of how i would do this? would i first query the metadata, and then see what is required? an example with explanation would be fabulous – Norrie

@MeggieLuski I modified my answer to give you a hint how it can be done – Ridiculous

"Of course the implementation of method CheckIfValueIsProvided depends on how your Account model class is defined, but probably you will need some kind of mapping between your model and CRM Entity model (to know how to map for example field "accountnumber" to some field in your model). " - im sorry i dont understand the reason why we should implement an internal layer between the crm model and what the client passes in. what is the point of the added complexity? – Norrie

moreover, how would we maintain such a mapping if the schema changes over time? – Norrie

I don't know you current architecture, I was just guessing that the client model and CRM model are different. If they are different, you will have to map the values somehow, call it internal layer, a mapper, whatever, otherwise how can you compare the models? If you use the same model for both, then you don't need mapping. Let's not discuss the application architecture here, it's not related to the question, I was just giving a simple example to give you an idea how you can use this metadata to get information that you want. How you apply it, is up to you. – Ridiculous

One more thing, because I'm not sure that you are aware of that - what you get from CRM metadata is a string name of a field (for example "accountnumber"), so you have to be able to somehow map this string "accountnumber" to your model property (to check if user provided a value for it). I hope that this is clear now. – Ridiculous

i appreciate all of your clarifications and insight! i'm still unsure about how i would create a mapping against a non-static schema – Norrie

But this problem is a common problem for all the applications in the world, because model schema changes all the time. I'm not sure I understand your concerns, but if CRM schema changes (for example there is a new field), you will also have to modify your client (for example to show this field on the form), so you will have to modify the mapping. That's what unit tests are for - you can have a unit test that will fail if some fields are unmapped, It's really uncommon that only server code changes and clients stay the same, usually it's mutual, so I can't really see the problem here. – Ridiculous

thank you again for the discussion. im assuming that my question stems from ignorance, but why can't the client simply query the metadata prior to making any other type of POST/GET/PATCH request, and then having the metadata, now they are able to create a valid request"? – Norrie

You CAN build a valid request using metadata. But your problem (and I'm still referring to it, because I don't want to be off-topic here) was that you want to CHECK if the required values are filled. If you know that "accountnumber" is required from metadata, you have to somehow "MAP" this "accountnumber" to your model that you use (maybe this will be a field called "accountnumber", but maybe it will be "CrmAccountNumber" for example, I don't know your model) and check if this field has a value. I hope that is clear for you know. Discussions are forbidden on Stack Overflow, so I hope so :) – Ridiculous

Your current problem is in fact far away from the original question - you wanted to get information which fields are required and this is stored in metadata. You should now try to play with this a little and if you have problems you can create a new question on SO for example, but the initial problem has been solved, so let's not continue this discussion here. – Ridiculous

Answer 3

You can use the RetrieveEntityRequest to get the metadata for an entity.

In the following example the metadata for entity Account is retrieved:

var request = new RetrieveEntityRequest
{
    EntityFilters = EntityFilters.Entity | EntityFilters.Attributes,
    LogicalName = "account"
};

var response = (RetrieveEntityResponse)_serviceProxy.Execute(request);

The response object contains an EntityMetadata property. In it you can find the requirement setting of an attribute like this:

EntityMetadata metadata = reponse.EntityMetadata;
bool isRevenueRequired = metadata.Attributes
    .First<AttributeMetadata>(a -> a.LogicalName == "revenue")
    .RequiredLevel.Value == AttributeRequiredLevel.ApplicationRequired;

Extrapolate

This doesn't answer how to retrieve the metadata using the WebApi endpoint. – Distichous

@Distichous Looking at the edit history of the question, the C# code snippet and tags added, the question seems a bit ambiguous to me. Maybe an edit here is appropriate? – Extrapolate

@HenkvanBoeijen i've attempted your suggested approach, please view my edited question – Norrie

@MeggieLuski My mistake. The EntityFilters property is a flag enumeration. The Attributes flag must be set when the metadata for the attributes is required. In my example the Attributes collection was null. I modified my answer. – Extrapolate