Gotcha: DateTimeKind and FetchXml

I was going through the FetchXML schema and found these two interesting attributes. The first one is “utc-offset” on the “fetch” node.

utcoffset

The other one is “usertimezone” on the “attribute” node.

usertimezone

Both these attributes give an impression, that it is possible to return datetime attributes in a different timezone instead of UTC, but in reality they don’t seem to do anything.

The only way to return datetime in the user’s local timezone, is by using the deprecated ExecuteFetchRequest. I will demonstrate this with XrmToolBox. This is my simple fetch query.

fetchxml-with-datetime

Below is the raw fetch result

fetch-xml-response-raw

Below is the table grid result

fetch-xml-response-retrievemultiple

I execute the FetchXML using the FetchXML Builder tool. As you can see, the datetime values are different. This is because when you use “RetrieveMultiple” to execute the fetch query, the datetime returned is always UTC and has to be converted to the local timezone. ExecuteFetchRequest is now deprecated and RetrieveMultiple is the recommended way to execute fetchxml. These two points really puzzle me:

  1. Why user-timezone and utc-offset don’t seem to do anything. Are they internally used?
  2. Why MS decided to change the datetime behaviour

So, this is a good point to remember when you are executing fetchxml using “FetchXML Tester” or have set the Result view to Raw fetch result in “FetchXML Builder” or using a mixture of ExecuteFetchRequest and RetrieveMultiple in your code.

References:

  1. ExecuteFetchRequest class
  2. XrmToolBox Issue #326

 

Advertisements

Plugin Integration Tests using FakeXrmEasy

XrmUnitTest and FakeXrmEasy are two testing frameworks that are specifically targeted towards Dynamics CRM/Dynamics 365. It is possible to do both unit test as well as integration tests using both these frameworks. In this first post, I would like to cover FakeXrmEasy. Here is a sample plugin code that I would like to test:

using Microsoft.Xrm.Sdk;
using System;
using Microsoft.Crm.Sdk.Messages;

namespace SamplePlugin
{
    public class LastNameUpperCasePlugin : IPlugin
    {
        public void Execute(IServiceProvider serviceProvider)
        {
            ITracingService tracer = (ITracingService)serviceProvider.GetService(typeof(ITracingService));
            IPluginExecutionContext context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
            IOrganizationServiceFactory factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
            IOrganizationService service = factory.CreateOrganizationService(context.UserId);

            try
            {
                Entity entity = (Entity)context.InputParameters["Target"];
                //not required. Just added to demonstrate that we are connecting to a real crm org service
                var response = service.Execute(new WhoAmIRequest());

                var lastName = entity.GetAttributeValue<string>("lastname");
                if (!string.IsNullOrEmpty(lastName))
                {
                    entity["lastname"] = lastName.ToUpper();
                }
            }
            catch (Exception e)
            {
                throw new InvalidPluginExecutionException(e.Message);
            }
        }
    }
}

In order to test this plugin, we have to create a unit test project. Here are the steps for it:

  1. Create a unit test  project from the existing plugin solution by right clicking and choose Add->New Project->Visual C#->Test->Unit Test Project
  2. Use nuget to add the correct FakeXrmEasy reference to the test project. In this example I am using “FakeXrmEasy.365” as I am connecting to a Dynamics 365 instance online. Refer https://www.nuget.org/profiles/jmontana to find out all the different versions that are relevant to your CRM version.
  3. Create a App.config file, if it doesn’t already exist. Here is my connection string. Refer https://msdn.microsoft.com/en-us/library/mt608573.aspx to find the correct format of the connection string for you crm instance.connection-string

Below is my test for the plugin

using System;
using System.Collections.Generic;
using System.Web.Configuration;
using FakeXrmEasy;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Microsoft.Xrm.Sdk;

namespace SamplePlugin.Test
{
	[TestClass]
	public class LastNamePluginUnitTests
	{
		[TestMethod]
		public void Should_Set_LastName_In_UpperCase()
		{
			var context = new XrmRealContext
			{
				ProxyTypesAssembly = typeof(LastNameUpperCasePlugin).Assembly,
				ConnectionStringName = "CRMOnline"
			};
			var executionContext = context.GetDefaultPluginContext();
			var target = new Entity("contact")
			{
				["lastname"] = "Power",
				Id = Guid.NewGuid()
			};
			executionContext.MessageName = "Create";
			executionContext.Stage = 20;
			executionContext.PrimaryEntityId = target.Id;
			executionContext.PrimaryEntityName = target.LogicalName;
			executionContext.InputParameters = new ParameterCollection
			{
				new KeyValuePair<string, object>("Target", target)
			};
			context.ExecutePluginWith<LastNameUpperCasePlugin>(executionContext);
			Assert.AreEqual("POWER",
				((Entity) executionContext.InputParameters["Target"]).GetAttributeValue<string>("lastname"));
		}
	}
}

In this test I am creating a scenario where the plugin is running pre-create. You can change to add pre/post entity images or target a different message and stage. If we debug our integration test, we can see that we are connected to the real org service.

Debugger.png

I have used latebound entity in this example, but you can use the same code with early bound entities generated using crmsvcutil or Early Bound Generator.

You can use a similar approach for testing custom workflow assemblies as well. In that case you would be using “GetDefaultWorkflowContext” instead of “GetDefaultPluginContext” to get the execution context.

Footnote: Technically this test could have been better written as an unit test instead of a integration test. But the purpose of this post is to demonstrate the points below:

  1. How to build a fake plugin execution context
  2. How to use connection strings to create a real OrganizationService instance
  3. How to use the real OrganizationService with the fake plugin execution context

EDIT (22/11/2016): Thank you Jonas (@rappen) for pointing out the plugin stage should be pre-create for this to work. Updated context stage to 20.

Server Side Sync – Polling interval

I have been battling with getting server side sync and forward mailbox working the way I want it to over the last couple of weeks. Googling hasn’t helped me a lot on this regard, and so in this post I will explain my understanding of the server side sync process.

The biggest hurdle that I faced in understanding the server side sync process is how little detail there is in the mailbox sync errors. I am not talking about the CRM Tracing, just the standard OOB logging in the application (like workflow error). Lot of times, I have seen unknown errors and mailboxes getting disabled randomly, but no detailed log in the application to indicate what the issue could be.

I used the following query in LINQPad with the CRM driver installed.

from m in MailboxSet
where m.EmailAddress == "email@email.com"
select new {
m.EmailAddress,
PostponeMailboxProcessingUntil = m.PostponeMailboxProcessingUntil.Value.ToLocalTime(),
m.ProcessedTimes,
ProcessingLastAttemptedOn = m.ProcessingLastAttemptedOn.Value.ToLocalTime(),
ReceivingPostponedUntil = m.ReceivingPostponedUntil.Value.ToLocalTime(),
PostponeSendingUntil = m.PostponeSendingUntil.Value.ToLocalTime(),
m.NoEmailCount,
m.TransientFailureCount,
m.LastSuccessfulSyncCompletedOn,
m.LastSyncError,
m.LastSyncErrorCode,
m.LastSyncErrorCount,
m.LastSyncErrorMachineName,
m.LastSyncErrorOccurredOn,
m.LastSyncStartedOn,
m.ItemsFailedForLastSync,
m.ItemsProcessedForLastSync
}

Here is a sample result.

sss

Based my tests, here is how I think it works for incoming email. The same is true for outbound emails. I am assuming that the mailboxes are not disabled and email address is approved:

  1. Async service starts to process the email at time specified in the PostponeMailboxProcessingUntil field
  2. Async service starts processing the inbound emails and creates email records in CRM
  3. In order to set the PostponeMailboxProcessingUntil and the ReceivingPostponedUntil , for the next run async service looks into the NoEmailCount field field. The NoEmailCount field is incremented every time the async service attempts to process email. If it did create some emails in CRM, the counter is reset to zero.

Based on my tests, I believe that if no email is sent/received in 15 minutes, the polling period is fixed at around 5 minutes. Here are my test results.

sss-tests

As you can see based on my tests, the polling interval gradually starts climbing when there are no emails to process and hovers around 5 minutes, after 15 minutes of inactivity i.e. no email records were created on CRM and/or no emails were sent from CRM.

If you have a queue setup in CRM, and forward an email to the queue, 5 minutes is a very long time to wait for the async service to pickup the email from Exchange. I saw couple of posts about how this polling interval can be changed by modifying the “Minimum Polling Interval” field.

polling-interval

The recommended value for this field is “0” and even when I changed this to 1 minute, I haven’t seen it influence the “ProcessingLastAttemptedOn“. I tried to change “PostponeMailboxProcessingUntil” and “ReceivingPostponedUntil” for the mailbox though the SDK, but since they are readonly fields, they cannot be changed. All the tests were carried out in OnPrem CRM2015, and so I cannot confirm whether this is the same behaviour in CRM2016 or Dynamics 365.

EDIT (18/11/2016): This 5 minute polling interval seems to be a “by-design” behavior. Thank you Andre Margono for sending this info. Here is what the MSDN article says:

When synchronization by using server-side synchronization occurs, the process is dynamic and unique for each user’s mailbox. The synchronization algorithm ensures that mailboxes are synced according to dynamic parameters such as the number of email messages and the activity within the mailbox. Normally, email synchronization occurs every 5 minutes. When a mailbox has many email messages, the interval can be reduced dynamically to 2 minutes. If the mailbox is less active, the interval can be increased up to 12 minutes. Generally speaking, you can assume that a mailbox will be synced at least once every 12 minutes. Note that you can’t manually synchronize records through server-side synchronization and when you track email (Track button), this occurs immediately.

tl;dr; If your CRM application is not processing any email (sending/receiving) in 15 minutes, the polling interval will eventually be fixed at around 5 minutes. When an email is received, async service will start processing more frequently, but the polling period will slowly climb with no-activity until it gets to 5 minutes.

Two useful LINQ Queries

In order to run these queries you’ll have to install LINQPad and LINQPad Driver for CRM.

Query 1 – Who created the entities?

It is not possible to look at an entity and find out who created it. You can however use the createdby on the SavedQuery (System View) to find out this information, as the System View is created the same time as the entity is created and also contains the createdby user information.

from q in
(from s in SavedQuerySet.AsEnumerable()
 where s.QueryType == 0
 group s by s.ReturnedTypeCode
into g
 select new
 {
	 g.Key,
	 Views = (from s in SavedQuerySet
			  where s.ReturnedTypeCode == g.Key && s.QueryType == 0
			  orderby s.CreatedOn
			  select new { s.CreatedOn, s.CreatedBy }).First()
 })
orderby q.Views.CreatedOn descending
select new { q.Key, CreatedOn = q.Views.CreatedOn.Value.ToLocalTime(), q.Views.CreatedBy.Name }

Entity creation query.png

Query 2 – All Plugins with message, stage, filtering attributes and entity

You can use this query to get a quick snapshot of all the plugins in the system.

from m in SdkMessageProcessingStepSet
join f in SdkMessageFilterSet on m.SdkMessageFilterId.Id equals f.SdkMessageFilterId
join s in SdkMessageSet on f.SdkMessageId.Id equals s.SdkMessageId
join p in PluginTypeSet on m.PluginTypeId.Id equals p.PluginTypeId
where f.IsCustomProcessingStepAllowed.Value
&& !m.IsHidden.Value
&& m.CustomizationLevel.Value == 1
select new { Message = s.Name, Rank = m.Rank.Value, Stage = m.Stage, StageName = m.FormattedValues["stage"], m.FilteringAttributes, p.AssemblyName, PluginName= p.Name, StepName = m.Name, StepDescription = m.Description, Status = m.StatusCode, StatusName = m.FormattedValues["statuscode"]}

plugins

Bug: EntityReference, Create and latebinding

For big projects and large code sizes, it is recommended to use strongly typed classes when you are dealing with CRUD operations related to the CRM entities. You can use Early Bound Generator or “crmsvcutil” to do this. The advantages are compile type errors for mismatches types and intellisense. But if you are quickly testing out some code, latebinding is the easiest path, as it doesn’t involve the overhead of generating the classes. During this process, I discovered a bug.

Here is the code and I will explain the bug after that.

            CrmServiceClient crmSvc = new CrmServiceClient(ConfigurationManager.ConnectionStrings["CRMConnectionString"].ConnectionString);
            if (crmSvc.IsReady)
            {
                #region create contact with account entityreference

                var account = new Entity("account");
                account["name"] = "Test Account";
                account.Id = crmSvc.OrganizationServiceProxy.Create(account);
                Console.WriteLine($"Created account with id {account.Id}");

                Console.WriteLine("Creating contact with account as EntityReference");
                Console.WriteLine("----------------------------------------------------");

                var contact = new Entity("contact");
                contact["parentcustomerid"] = account.ToEntityReference();
                contact.Id = crmSvc.OrganizationServiceProxy.Create(contact);
                Console.WriteLine($"Created contact with id {contact.Id}");

                contact = crmSvc.OrganizationServiceProxy.Retrieve("contact", contact.Id, new ColumnSet("parentcustomerid"));
                Console.WriteLine($"Contact: Account Lookup {contact.GetAttributeValue<EntityReference>("parentcustomerid")?.Id}");

                #endregion

                #region update contact with account Guid

                Console.WriteLine("Updating contact with account as Guid");
                Console.WriteLine("----------------------------------------------------");
                try
                {
                    contact["parentcustomerid"] = account.Id;
                    crmSvc.OrganizationServiceProxy.Update(contact);
                    var updateContact = crmSvc.OrganizationServiceProxy.Retrieve("contact", contact.Id, new ColumnSet("parentcustomerid"));
                    Console.WriteLine($"Updated Contact: Account Lookup {updateContact.GetAttributeValue<EntityReference>("parentcustomerid")?.Id}");
                }
                catch (Exception e)
                {
                    Console.WriteLine($"Update Contact failed with Exception: {e.Message}");
                }

                crmSvc.OrganizationServiceProxy.Delete("contact", contact.Id);
                Console.WriteLine($"Deleted contact with id {contact.Id}");

                #endregion

                #region create contact with account Guid

                Console.WriteLine("\nCreating contact with account as Guid");
                Console.WriteLine("----------------------------------------------------");
                contact = new Entity("contact");
                contact["parentcustomerid"] = account.Id;
                //This should not succeed
                contact.Id = crmSvc.OrganizationServiceProxy.Create(contact);
                Console.WriteLine($"Created contact with id {contact.Id}");

                contact = crmSvc.OrganizationServiceProxy.Retrieve("contact", contact.Id, new ColumnSet("parentcustomerid"));
                Console.WriteLine($"Contact: Account Lookup {contact.GetAttributeValue<EntityReference>("parentcustomerid")?.Id}");

                crmSvc.OrganizationServiceProxy.Delete("contact", contact.Id);
                Console.WriteLine($"Deleted contact with id {contact.Id}");

                crmSvc.OrganizationServiceProxy.Delete("account", account.Id);
                Console.WriteLine($"\nDeleted account with id {account.Id}");

                #endregion

            }

Here is the result.

Guid Entityreference result

Even if you are using latebinding, CRM webservice respects type and doesn’t allow you to set an attribute to a type, that doesn’t match its definition, except in one case: You can set the EntityReference attribute to a Guid, and the code with execute without any exception. The good thing though is the fields is just set to null, and the Guid is ignored, but this a bug as you cannot do it with other types e.g. you can’t set an Optionset field to an int or a Currency field to a decimal.

I have logged this on Connect for verification. I experienced this behaviour in CRMOnline and CRM2015.

Connect issue -> https://connect.microsoft.com/site687/feedback/details/2740732/sdk-permits-setting-entityreference-attribute-to-guid-late-binding

Further reading:

MSDN: Create, retrieve, update, and delete (late bound)

 

Using Stylish to enhance CRM

What is Stylish?

Stylish is a browser extension available for both Firefox and Chrome. It helps you to alter the styles on any site that you don’t control. The change is localised i.e. it is visible only in your local machine.

How can I use this to enhance Dynamics CRM

I will give you three quick scenarios that you will find useful.

  1. Hide the new “friendly” and “informative” notification messages.
  2. Show the full width of a form label without truncation .
  3. Alter the navigation bar text and colour to easily differentiate various environments.

Hiding the notification messages

This is the CSS you’ll use to hide the notification messages. With this you can quick nuke all the messages, across all the environments you specify using the Stylish configuration. This can be done without going to “Settings” and turn each different notification classes (Get Outlook App, Get CRM App, Trial expiry, Mailbox notifications and Install Interactive Service Hub) off. The rule is for any site that ends with dynamics.com.

@namespace url(http://www.w3.org/1999/xhtml);

@-moz-document domain("dynamics.com") {
  .crmAppMessageBar_green, .crmAppMessageBar {
    display: none !important;
  }
}

Before Stylish

Message Bar Before

After Stylish

Message Bar After

Show the full label width without truncation

Starting from CRM2015, labels are not fully displayed if they are too long. The standard approach is to wrap around, but that is not how it is in CRM. Instead, they are truncated. The workaround solution for this is to increase the section width, but there is a better solution. Using Stylish, you can change the CSS that controls this behaviour. Below is the CSS for this. The rule is for any site that ends with dynamics.com.

@namespace url(http://www.w3.org/1999/xhtml);

@-moz-document domain("dynamics.com") {
  .ms-crm-InlineEditLabel, .ms-crm-InlineEditLabelText {
    white-space: normal !important;
  }
}

Before Stylish

Label Truncation Before

After Stylish

Label Truncation After

Altering navigation bar text and colour

If you are working across multiple environments, you will like this one. You can change the colour and text using CSS, so that you’ll know where you are, when are constantly switching tabs. I did an extension to do just the colour change, but Stylish does it much better, as it exposes the full power of CSS. You can obviously use themes in CRM2016 to do this. But with this approach:

  1. You can have a theme you like, that is different from the published theme
  2. CRM2015 users can get the same theming goodness

Here is the CSS, you can use for this. The style is just for the navigation bar. You can go crazy and change the form styles, command bar styles and what not.

@-moz-document url-prefix(https://[ENTER THE URL FOR THE RULE]) {
  #navBar {
    background: green !important;
  }
  .navTabLogoText:after {
    content: " DEV" !important;
  }
}

Before Stylish

Navigation Bar Before

After Stylish

Navigation Bar After

Here is some screenshots from Firefox on how to get to the Stylish menu to add custom styles. It is a little bit different in Chrome.

Stylish Menu

I have one rule to do with message bar hiding and preventing label truncation.

Stylish

Here are the links for downloading Stylish

Chrome users head to https://chrome.google.com/webstore/detail/stylish/fjnbnpbmkenffdnngjfgmeleoegfcffe

Firefox users head to https://addons.mozilla.org/en-US/firefox/addon/stylish/

IE users, head to https://www.google.com.au/intl/en/chrome/browser/desktop/index.html Smile

This is a really productive tool to have and is also free.

Level Up: A Chrome extension for CRM Power Users

EDIT (14/11/2016): The extension is now available on Edge as well. Refer https://github.com/rajyraman/Level-up-for-Dynamics-CRM-365-Edge-Browser

Today, I released a new Chrome extension that assists CRM power users. In this initial release there are 12 quick functionalities:

Form helpers

  1. Display Logical names for controls (Original script by Chris Groh http://us.hitachi-solutions.com/blog/2014/10/27/showing-entity-logical-names-on-form/)
  2. God Mode (based on http://www.magnetismsolutions.com/blog/paulnieuwelaar/2014/07/29/activate-god-mode-in-crm-2013—don-t-let-your-users-see-this)
  3. Form properties (Original script by Jared Johnson http://www.magnetismsolutions.com/blog/jaredjohnson/2014/08/03/dynamics-crm-2013-resurrecting-the-form-properties-window-with-bookmarklet)
  4. Dirty fields i.e. fields that have been modified
  5. Display record URL
  6. Display record Id

Navigation helpers

  1. Open record by Id
  2. Open security area
  3. Open System Jobs
  4. Open Solutions
  5. Open Process Definitions
  6. Open main

How do you install it

Download the extension from Chrome store -> Level up for Dynamics CRM

How does it look

Once you install this extension you will see a new rocket icon in the toolbar. If you click the icon, you will get a popup menu.

Screenshot Main

How to use it

You can have a look at this animation below to get an idea about the functionalities.

Functionality Quick Intro

Source code

The source code for the extension can be downloaded from https://github.com/rajyraman/Levelup-for-Dynamics-CRM/

The minimum supported version of Dynamics CRM for this extension is CRM2015.

In the initial version, I have 12 quick actions. Please feel free to fork the repo and add new functionalities that you are currently using with a bookmarklet and finding it useful.

Please also log any issues/feedback in the github repo. I hope this will be a useful extension for improving your productivity.

EDIT (24/06/2016): Fixed credit links for God Mode, Form Properties and Control Logical Names bookmarklets in post and repo.