CDS, Microsoft Flow and DateTime formats

Dec 6, 2018Dec 7, 2018 Natraj Yegnaraman dynamics365, flow, tricks

EDIT (7/12): You can download the Flow from https://1drv.ms/u/s!AvzjERKFC6gOwCqKnnAvMZA6mDsW if you just want to jump straight into exploring the actual Flow, instead of reading the article.

DateTime and Timezones seem to be the flavor of the month, so I thought I will take a crack at the problem as well. Coming from CRM background and spending a lot of time fiddling around the internal schema to understand the plumbings I came up with the following approach which I believe is decoupled and flexible.

In CDS, there is an entity called “UserSettings”, which stores a whole bunch of information regarding the user preferences. If you have used the awesome XrmToolBox tool called “UserSettings Utility”, you would remember this screen.

Formats

The “UserSettings” entity is the source of this information. So, why not use the same for managing the timezones and datetime formats?

For re-usability, I want to create a Flow that just returns me the timezone information based on the executing user. Here is how it looks.

Get Timezone and Formats.png

Let us understand this step by step.

Step 1:

This is the HTTP trigger, which can take in a parameter for activedirectoryguid. This is the unique identifier for the Office 365 user.

Step 1.png

Step 2:

Get the Office 365 profile of the current user. If the activedirectoryguid is not passed in step 1, the intention is to use the current user’s Office 365 information.

Step 2.png

Step 3 & 4:

Initialise the activedirectoryguid variable, and query the CDS entity “Users”, and retrieve the user that matches the activedirectoryguid.

Step 3 and 4.png

Since activedirectoryguid can be passed on the trigger as well, I use the following coalesce expression, to set the variable to either the passed value on the trigger or from the Office 365 profile in step 2. This is the expression.

coalesce(triggerBody()?['activedirectoryguid'],body('Get_my_profile_(V2)')?['id'])

Step 5 & 6:

Retrieve “User Settings” entity and the associated “Time Zone Definitions” record based on the user’s timezone.

Step 5 and 6.png

The filter for User Settings is

first(body('Get_System_Users')?['value'])['systemuserid']

The filter for Time Zone Definitions is

first(body('Get_User_Settings')?['value'])['timezonecode']

Step 7:

This is the last step, where we return all the format and timezone information.

Step 7

Below are the expressions for the returned properties:

timezone

first(body('Get_User_Timezone')?['value'])['standardname']

dateformat

replace(first(body('Get_User_Settings')?['value'])['dateformatstring'],'/',first(body('Get_User_Settings')?['value'])['dateseparator'])

timeformat

replace(first(body('Get_User_Settings')?['value'])['timeformatstring'],':',first(body('Get_User_Settings')?['value'])['timeseparator'])

dateseparator

first(body('Get_User_Settings')?['value'])['dateseparator']

timeseparator

first(body('Get_User_Settings')?['value'])['timeseparator']

Now let us look at some sample output. For a user located in US, here is how the output of the Flow looks like.

US Output

Compare this with someone who is in Australian timezone.

Australia Output

Since this Flow now contains the logic for getting the formats and timezones, it can be utilised for another Flow that needs this information. For example look at the sample Flow below

Calling Flow

Calling Flow1

Below is the expression, I use to convert the “createdon” returned by the CDS Get Record, step which returns the datetime in UTC.

convertFromUtc(body('Get_record')?['createdon'],body('Parse_JSON')?['timezone'],concat(body('Parse_JSON')?['dateformat'],' ',body('Parse_JSON')?['timeformat']))

If you compare this Flow output and the record properties from Dynamics 365, it becomes obvious that the datetime and format is correctly displayed. Note that the CDS connector returns the datetime in UTC.

Output

CRM Record Properties

Initially I wanted to call the Flow that returns the formats and timezone using the “Start Flow” action on the Flow Connector, but it doesn’t seem to be picked up the response, so I had to resort to the whole HTTP action, which is not ideal.

It seems “Start Flow”, simply enables the Flow, does not actually run the Flow. Not sure why it is named in a misleading way.

Hope this is helpful.

Using Flow to notify solution imports

Nov 18, 2018Apr 3, 2019 Natraj Yegnaraman dynamics365, flow, tricks

EDIT (03/04/2019): I made further changes to display the solution URL, so that you can click and find out the details about the solution that was imported.

EDIT (20/12/2018): I updated my Flow and made some improvements. You simply have to set the Timezone on the triggering action and you are all set. The Flow will email the solution list, to the user running the Flow. The download link has been updated to point to the updated solution.

I would not call it sneaky, but sometimes when I find the Dynamics 365 CE UI or behaviour has changed slightly, I can attribute it to some update that was applied to the environment. There are email notifications for major updates, but none for minor updates or patches that can happen frequently. So, I decided to solve this problem using Flow.

Every solution import into the system causes an Import Job record to be created. If a Flow can be scheduled to run everyday, and query the Import Job records that were started the previous day, we can easily keep track of what is happening in the environment.

The first step is to trigger the Flow on a preset schedule, and read the Import Job records.

Import Job Flow.png

Below are the expressions that I use for the boundary dates:

addDays(convertFromUtc(utcNow(),’AUS Eastern Standard Time’), -1, ‘yyyy-MM-dd’)
convertFromUtc(utcNow(),’AUS Eastern Standard Time’,’yyyy-MM-dd’)

The next step is to pick up only the information we need from the returned result, and project it to a form that is conducive for email.

Import Job Compose

With the exception of Solution Name, all the other properties are retrieved from the XML on the Data property. Below are the formula’s for those:

Publisher – xpath(xml(items(‘Parse_Import_Job_XML_and_re-map’)?[‘data’]),’string(//Publisher/Descriptions/Description[1]/@description)’)
Started On – convertFromUtc(items(‘Parse_Import_Job_XML_and_re-map’)?[‘startedon’],’AUS Eastern Standard Time’,’dd-MM-yyyy HH:mm:ss’)
Old Version – xpath(xml(items(‘Parse_Import_Job_XML_and_re-map’)?[‘data’]),’string(//upgradeSolutionPackageInformation/fileVersion)’)
New Version – xpath(xml(items(‘Parse_Import_Job_XML_and_re-map’)?[‘data’]),’string(//upgradeSolutionPackageInformation/currentVersion)’)

The last few steps are to email out the results, if any solutions were imported the previous day.

Email Solution.png

Email Step.png

The result is a barely formatted table, with the list of solutions that were imported the previous day.

Solution Import Email

With this approach, no one can sneak up a solution import on you. You have visibility over what is going on in the system.

You can download and install the Flow solution using these links:

Unmanaged: https://1drv.ms/u/s!AvzjERKFC6gOwlMIAzgRSXWhtXth

Managed: https://1drv.ms/u/s!AvzjERKFC6gOwlKvIWhW-Pb_vHiT

Hope this is helpful to stay on top of imports.

Role based Canvas Apps

Nov 14, 2018 Natraj Yegnaraman powerapps

The inspiration for this post is this post from Geetha -> https://svaghub.wordpress.com/2018/11/03/role-based-security-in-powerapps-using-spgroups/. The idea is to basically use Flow to check what kind of permission the current user has, so that some functionalities can either be lit up or hidden in the canvas app. In her post, Geetha was using SharePoint user group to get this information. Being from a CRM background, I want this to be retrieved from CDS + I am not a big fan of SharePoint.

Attempt 1: Use DataSourceInfo(‘Users’, DataSourceInfo.CreatePermission), to check if the current user can create Users, which means they are Sys Admin. But, there seems to be a limitation on DataSourceInfo, which means it does not return the correct permissions for CDS entities. It always returns true, which is not correct. So, this attempt was not successful. This would have so much easier if it had worked, as all the logic would be entirely in the canvas app.

Attempt 2: Flow magic. I didn’t want to head this route, but since DataSourceInfo did not work, I had to use Flow to solve the problem. Solution is one of the restricted CDS entities. Only certain roles have access to read this entity OOB. Below are those roles.

So, I am using this as the flag to show or hide canvas apps controls. Below is my Flow.

There are two parallel branches, one if there is an exception while retrieving Solution records, which means that the user cannot read “Solution” entity and another branch if they have read privilege on “Solution” entity. Depending on which branch the Flow takes, “canread” can be true or false. I can use the result to show or hide controls on the canvas app. The two branches have the appropriate “configure run after” set.

configure run after.png

The clunky bit about this, which I don’t like is that fact that Flow will report that the execution has failed, when Flow takes the “Cannot read Solutions” branch.

Flow result.png

In the programming world, handling an exception appropriately and continuing like nothing happened, is not considered a failure, but it looks like Flow has a different opinion on this.

Potential improvement to the design: Create a new CDS entity called “Canvas App Permission” and create attributes on this entity, to manage which areas in Canvas App should be shown or hidden. Create multiple records of this new entity and assign this to teams or individual users, depending on how you want the permissions to be applied in canvas apps. The Flow can then retrieve this entity, and PowerApps can use this result of the Flow to hide/show areas or functionality.

Hope this is useful. Credit to Geetha for coming up with the original idea.

Building Spirograph using PowerApps

Nov 2, 2018 Natraj Yegnaraman powerapps

Over the past few weeks, people have been demonstrating some cool games, built entirely using PowerApps. The reason being, that there was a contest from ThoseDynamicsGuys. If you are interested in finding out about some interesting games from this contest, watch this video from Mr.Dang himself -> https://www.youtube.com/watch?v=0-ZWqs_emQA where he reviews the games.

There were some apps that caught my eye during this period:

Power Flappy by Scott Durow
Power Roulette by Geetha Sivasailam
Coin Dog by Makato Maeda
Air Resistance by Nagao Hiroaki

I was especially fascinated by what Scott and Makato accomplished with the side scrolling nature of the games, how Geetha managed to rotate the roulette as there is no rotate angle property on images and how Nagao managed to calculate the trajectory and resistance of the ball. The first step to learning is to understand how other people do it, so I spend 4-5 days to understand how these apps have been developed.

I then wanted to develop something of my own, using the concepts that I had learnt. Spirograph was the first thing that I thought of. The easiest part was to get the equations to calculate x and y. I had to then learn the basics of starting and stopping a timer, and how to create a sense that the pattern was being “drawn”.

Since image control can render SVG, I decided to try this approach. My first challenge was how do I calculate the x and y on every tick of the timer. So, I decided to follow the approach demonstrated by Brian Dang, that involve checking and un-checking a toggle control.

Every time the toggle is checked, I can increment the iterator variable and calculate x and y for the line to be drawn and the string for the SVG path with all the lines. In SVG you can draw a line, by moving to a location specifying Mxy and then drawing the line using Lxy. Below is the formula in the OnCheck event of the slider

Set(VarI, VarI+1);
UpdateContext({x:
(((RSlider.Value-rSlider.Value) * (Cos(rSlider.Value*VarI/RSlider.Value))) + (aSlider.Value * Cos(VarI*(1-(rSlider.Value/RSlider.Value))))),
y:
(((RSlider.Value-rSlider.Value) * (Sin(rSlider.Value*VarI/RSlider.Value))) - ((aSlider.Value * Sin(VarI*(1-(rSlider.Value/RSlider.Value))))))
});
Collect(Lines, {Row: VarI,
X: x,
Y: y});
Set(PathVar, PathVar & If(PathVar="", "M", "L") & x & "," & y);
true

I can then simply set the SVG path’s d property from the PathVar variable, that has all the line co-ordinates. Below is the value for the Image property of the Image control that renders the Spirograph.

SVG Path.png

Below a video of the app in action. You can play around with the sliders and be fascinated by the patterns that it generates.

Here are some interesting patterns the app generated.

You can download the app from the PowerApps Community Gallery -> https://powerusers.microsoft.com/t5/Community-Apps-Gallery/Spirograph/m-p/175447

References:

SVG Paths – https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths#Line_commands

Math behind Spirograph – http://www.mathematische-basteleien.de/spirographs.htm

Building a MVP Notifier using Flow and Azure Functions

Sep 3, 2018 Natraj Yegnaraman flow, functions

Since MVPs are now announced every month, I have found it hard to track down the new awardees in Business Applications area before 3rd of every month. So, I thought I will build a notifier using Flow and Functions. This is the logic:

Seed the inital MVP data from mvp.microsoft.com, to figure out the new MVPs every month into Azure Tables
Schedule the Flow at 1st of every month at 5 p.m. PDT. Hopefully by this time, everyone has filled out atleast their name in the MVP profile.
Retrieve the MVP data again from mvp.microsoft.com
Figure out the new MVP and post a message to Slack. Add the new MVP details to Azure Tables

The logic that retrieves the details from mvp.microsoft.com uses Azure Functions. Below is the project.json for that Function App.

{
  "frameworks": {
    "net46":{
      "dependencies": {
        "AngleSharp": "0.9.10",
        "Newtonsoft.Json": "11.0.0.2"
      }
    }
   }
}

Next, is the actual logic that returns MVP list for the category passed in the URL.

using System.Net;
using AngleSharp;
using AngleSharp.Dom;
using AngleSharp.Dom.Html;
using System;
using System.Runtime.InteropServices;
using System.Text;
using Newtonsoft.Json;

public static async Task Run(HttpRequestMessage req, TraceWriter log)
{
    log.Info("C# HTTP trigger function processed a request.");

    // parse query parameter
    string mvpCategory = req.GetQueryNameValuePairs()
        .FirstOrDefault(q => string.Compare(q.Key, "category", true) == 0)
        .Value;

    if (mvpCategory == null)
    {
        // Get request body
        dynamic data = await req.Content.ReadAsAsync();
        mvpCategory = data?.category;
    }

	var pageNumber = 1;
	var webClient = new WebClient();
	IHtmlDocument doc;
	IHtmlCollection mvpProfiles = null;
	List mvps = new List();
	var htmlParser = new AngleSharp.Parser.Html.HtmlParser(new Configuration().WithDefaultLoader());
	do
	{
		var data = webClient.DownloadData(new Uri($"https://mvp.microsoft.com/en-us/MvpSearch?ex={WebUtility.UrlEncode(mvpCategory)}&sc=s&pn={pageNumber++}&ps=48"));
		doc = htmlParser.Parse(Encoding.UTF8.GetString(data));

		mvpProfiles = doc.QuerySelectorAll(".profileListItem");
		var currentPage = mvpProfiles
		.Select(d =>
		{
			var nameAndLocation = d.QuerySelector(".profileListItemFullName a");
            var mvpUrl = nameAndLocation.GetAttribute("href");
			return new MVPInfo
			{
				Name = nameAndLocation.TextContent,
				Url = $"https://mvp.microsoft.com{mvpUrl}",
				Country = d.QuerySelector(".profileListItemLocation .subItemContent").TextContent,
                Id = mvpUrl.Substring(mvpUrl.LastIndexOf("-")+1)
			};
		})
		.ToList();
		mvps.AddRange(currentPage);
	} while (mvpProfiles.Any());

    var json = JsonConvert.SerializeObject(mvps, Formatting.Indented);
    return mvpCategory == null
        ? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a MVP category on the query string")
        : new HttpResponseMessage(HttpStatusCode.OK)
          {
                Content = new StringContent(json, Encoding.UTF8, "application/json")
          };
}

public class MVPInfo
{
	public string Name { get; set; }
	public string Url { get; set; }
	public string Country { get; set; }
    public string Id { get; set; }
}

It uses the AngleSharp library to parse the response returned by mvp.microsoft.com search.

Below is the Flow that uses this Function, to populate the Azure Table.

Flow Overview.png

As you can see below the Flow runs at 5 p.m on 1st of every month, and gets the current MVP list from Azure Table, MVPAwards.

Azure Tables.png

The partion key for the table is month and year and the row key is the actual MVP Id.

Azure Tables Definition.png

Since this Flow and the associated Function were built for my personal use, I call the Function directly without using a custom connector. I pass the award category on the URL of the Functions itself. If you want to use a custom connector, refer one of my earlier posts.

Azure Function Call.png

After parsing the JSON returned by the Azure Function, I can use the “select” function to map the returned data, so that I can compare the data based on the MVP Id and insert the new MVPs into the Azure Table, if needed. The partion key will the current year and month (yyyyMM).

Flow Select.png

Next is the crucial bit, where I do the actual comparison.

Compare MVPs.png

The “filter” function can be used to see whether there are any matches in MVP list that was retrieved from Azure Table i.e. the previous month MVPs. If no results were returned, that means they were not a MVP last month. So, they are a new MVP and this can be stored in Azure Tables and can also be posted to Slack.

IF Then Logic.png

Post to Slack is using the Webhooks registered on the channel.

Slack Web Hooks.png

After the Flow has finished running, the output gets posted to Slack.

MVP awards.png

References:

Improving entity forms using embedded PowerApps

Jun 22, 2018Jun 25, 2018 Natraj Yegnaraman dynamics365, microsoftflow, powerapps

I have been looking into scenarios with PowerApps and Flow that can benefit Dynamics 365 Customer Engagement user experience. One of the scenarios that can add value right away is on the entity forms. PowerApps can be embedded as an IFrame on the normal entity forms, and can be used similar to Dialogs to offload some processing to PowerApps and Flow.

Here is the finished product.

Embed PowerApps in UCI

This works without any JavaScript at all in UCI, with the normal IFrame control on the form. Make sure to tick the option that passes the record id and objecttype code and untick cross site scripting restriction.

IFrame Properties.png

No scripts are needed on the form to embed the PowerApp.

Form Scripts

Once PowerApps is in place, the current form context can be inferred using the “id” parameter that is passed on to the form.

PowerApps Initial

I use a known Guid during the design phase to assist me with the app design process, as the PowerApps calls the Flow during the OnStart event and sets the ProblemDetails variable.

A Flow can be associated to an event, from the Action->Flows area.

Associate Flow.png

When the PowerApps loads, it calls the Flow with the Guid, to retrieve the case details. The Flow that responds to PowerApps with these details on the case: Title, Customer Name, Type of Customer (Account or Contact).

Case Details Flow.png

In this Flow I just use “Respond to PowerApps”action and return the three outputs.

Return to PowerApps.png

I used variables to store the Client, which could be the Account’s name or Contact’s FullName, depending on what is on the case. The client type could be either Account or Contact. Account Details and Contact details are retrieved based on the result of the Client Type branch.

For the second Flow, the user presses the “Check” button which performs some additional checks based on business criteria. For this Flow, I used the “Response” action, which allows me to return JSON results. I stored the cases I am interested in on an array variable.

For each case.png

From the variable, I used Select action to grab only the properties I am interested in.

I can then use the “Response” action to return these to PowerApps.

One weird thing that I encountered in PowerApps/Flow integration, is that I would simply see the result as “True” from Flow, when I tried to return the return the response straight from the variable.

True Response.png

When I used Parse JSON and then Select to reduce the properties it started working. This can happen when there is something wrong with the schema validation, but I am not sure how this can happen when I copy-paste the JSON response from the previous steps to auto-generate the schema.

One more thing: When the Flow that is associated with the PowerApps changes, just make sure to disassociate and reassociate the Flow. I had issues when I did not do this, due to PowerApps caching the Flow definition.

References;

https://preview.flow.microsoft.com/en-us/blog/howto-upload-return-file/

https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/get-sessionid

Using Virtual Entities to query metadata

Jun 14, 2018Sep 13, 2018 Natraj Yegnaraman dynamics365, opensource, tricks, virtualentity

After my previous post, I continued to explore virtual entities to see what real life problems I can solve using them. One problem I could think of was metadata. How awesome would it be, if I can use Advanced Find to query entity and attribute metadata, or visualise them as a normal entity! It is not a dream anymore. I have developed an open-source solution to do this.

Here is some of the sample queries:

Query all attributes of type customer

Query by Attribute Type

Query by Attribute Type Results

Query all mandatory attributes

Query All Required.png

Query All Required Results.png

Query by Attribute Type and Entity

Query by Attribute Type and Entity.png

Query by Attribute Type and Entity Results.png

Query all Many to Many intersect entities

Query MM Entity.png

Query MM Entity Results

Query entities that have quick create enabled

Query Quick Create

Quick Create Results

You can open the entity and see more details.

Entity Form

You can open the attribute and view more details as well.

Attribute Form

All this is awesomeness is possible using the power of virtual entities. There are two virtual entities that you can query. They are called Entity and Attribute.

VE Solution

You can download the source code and managed solution from https://github.com/rajyraman/Metadata-Virtual-Entity/releases.

This step is important

After importing the managed solution, change the data source for the attribute entity from “Entity Datasource” to “Attribute Datasource”. You have to do this from the Customization area and not from the managed solution.

Change Datasource.png

This is because by default, the system does not allow relationships between two virtual entities that have different datasources. This exception is shown when you try to do this.

Solution Exception

Stack Trace.png

In order to workaround this exception, you keep the data source same for the “Entity” virtual entity (parent) and “Attribute” (child) virtual entity, create the relationship and then change to the right datasource. Hence, the managed solution has the datasource set to “Entity Datasource” for the “Attribute” virtual entity, which has to be changed manually after importing the solution.

I hope this solution would be really useful for administrators. Please let me know any feedback on the post or on GitHub issues.

Sourcecode: https://github.com/rajyraman/Metadata-Virtual-Entity

Managed Solution: https://github.com/rajyraman/Metadata-Virtual-Entity/releases/latest

Dreaming in CRM, PowerApps & Flow

My thoughts on PowerApps, Flow, Dynamics 365, JavaScript, C# and Software Development. Sometimes, I post when I am half-asleep