Using Chrome DevTools to debug Bot Framework

When debugging the Bot Framework it is trivial to debug if you are developing in C#. It was bit of pain for me to do the same is node, as JavaScript is not my everyday language. But, after researching googling about this, I can say that it is really easy in node as well.

Below are the version details that I tried this on:

  • node: 8.1.2
  • npm: 5.0.3
  • OS: Windows 10
  • Chrome: Canary 61.0.3137.0
  • Bot Framework Emulator: 3.5.29

Below is my package.json for a simple “Hello World” bot.

{
  "name": "echo-bot",
  "version": "0.1.0",
  "license": "MIT",
  "description": "Echo bot example",
  "scripts": {
    "start": "node --inspect app.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "botbuilder": "^3.8.4",
    "dotenv": "^4.0.0",
    "restify": "^4.3.0"
  }
}

Since I am developing this in Visual Studio Code, this is my tasks.json

{
    // Use IntelliSense to learn about possible Node.js debug attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "node",
            "request": "launch",
            "cwd": "${workspaceRoot}",
            "runtimeExecutable": "npm",
            "runtimeArgs": [
                "start"
            ],
            "name": "Launch Program"
        }
    ]
}

Now, to run the bot application in VSCode using Ctrl+F5 or typing “npm start” in the command prompt.

To debug the bot in Chrome DevTools, type “chrome://inspect/#devices“. Then click the “Open dedicated DevTools for Node” link. The page should be like this once you do this.

Chrome DevTools to debug Bot Framework.png

Chrome will now display the node application that it can detect in the Remote Target area. Click in the inspect link to open the DevTools. Now, set your breakpoint and use the emulator to send a message.

Debug botframework source

In the screenshot above you can see that I am already debugging using Chrome DevTools and debugger is waiting in the breakpoint that I set.

Reference:

https://nodejs.org/en/docs/inspector/

 

 

 

 

 

 

 

Advertisements

User not found error

Word of warning: This change is unsupported as it involves editing the CRM database directly. Use it at your own risk.

I encountered a weird error in CRM, that basically said that the user cannot be found. When I checked the event log, this appeared to be the underlying exception:

Exception type: CrmException
Exception message: No Microsoft Dynamics CRM user exists with the specified domain name and user ID
at Microsoft.Crm.Authentication.Claims.AuthenticationProvider.GetOrganizationId(ClaimsPrincipal principal)
at Microsoft.Crm.Authentication.Claims.AuthenticationProvider.Authenticate(HttpApplication application)
at Microsoft.Crm.Authentication.AuthenticationStep.Authenticate(HttpApplication application)
at Microsoft.Crm.Authentication.AuthenticationPipeline.Authenticate(HttpApplication application)
at Microsoft.Crm.Authentication.AuthenticationEngine.Execute(Object sender, EventArgs e)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

But, the user was indeed there from before and all I did was reactivate a long dormant user. My initial suspicion was that this had something to do with AD. I ran the following query to check the SID of the user against the MSCRM_CONFIG database:

select c.FriendlyName,b.FullName,d.AuthInfo,b.DomainName
from SystemUserOrganizations a
inner join [ORG_MSCRM].dbo.SystemUser b on a.CrmUserId=b.SystemUserId
inner join Organization c on c.Id=a.OrganizationId
inner join SystemUserAuthentication d on d.UserId=a.UserId
where b.DomainName='CONTOSO\powerm'

This gave me the SID of the user when the user was first created in CRM. I next ran this command in to get the current SID of the user:

wmic useraccount where name='powerm' get sid

The SID did not match to the one already in CRM, so I had to update it in the MSCRM_CONFIG database:

update d
set d.AuthInfo='W:[SID OUTPUT FROM THE DOS COMMAND PROMPT]'
from SystemUserOrganizations a
inner join [ORG_MSCRM].dbo.SystemUser b on a.CrmUserId=b.SystemUserId
inner join Organization c on c.Id=a.OrganizationId
inner join SystemUserAuthentication d on d.UserId=a.UserId
where b.DomainName='CONTOSO\powerm'

Once I updated this, the user was able to login with the same AD account.

Server Side Sync: View for Synced and Not-Synced Appointments

If you decide not to use the CRM App for Outlook, syncing emails, contacts and appointments using just Server Side Sync and Forward Mailbox can be really painful sometimes, especially when you are troubleshooting why something is not being synced. The “Server-Side Synchronization Monitoring” Dashboard provides some assistance to the Administrator, but when you want to dig into the details, I often end up using “FetchXML Builder” or “LinqPad” to query the TraceLog entity. Below is the fetchxml I use

<fetch top="50" >
  <entity name="tracelog" >
    <attribute name="tracestatus" />
    <attribute name="tracedetailxml" />
    <attribute name="tracelogid" />
    <attribute name="machinename" />
    <attribute name="tracecode" />
    <attribute name="traceactionxml" />
    <attribute name="traceparameterxml" />
    <attribute name="traceparameterhash" />
    <attribute name="errortypedisplay" />
    <attribute name="modifiedon" />
    <attribute name="text" />
    <attribute name="level" />
    <attribute name="collationlevel" />
    <filter>
      <condition attribute="tracestatus" operator="eq" value="0" />
    </filter>
    <order attribute="createdon" descending="true" />
  </entity>
</fetch>

Another common scenario that I troubleshoot is, why an appointment is not synced to Outlook after it was created in CRM. The first thing to check in this case is the user’s sync filter. I use the XrmToolBox tool “Sync Filter Manager” to check what the user’s sync filter for appointment. After confirming that the fetchxml picks up the appointment that has to be synced, we can now proceed to stage 2 of troubleshooting.

The field that is key in this scenario is called “GlobalObjectId“. This field in the appointment entity will be set, after the appointment is synced to the user’s Outlook. Unfortunately, this field is marked as not searcheable, which means it cannot be used in Advanced Find.

Global ObjectId.png

But, we can still create view that shows the synced appointments and appointments that have not been synced, using FetchXML Builder. Below are the steps

  1. Install FetchXML Builder from XrmToolBox store, if you don’t have it already
  2. Save an existing view into a new view on appointment entity. Don’t worry about the filters yet as we will update this using FetchXML BuilderSave a new view.png
  3. Next step is to open this view in FetchXML BuilderOpen View.png
  4. Update the fetchxml and save the view. Confirm the user’s sync filter for appointment using the Sync Filter Manager and make sure the new fetchxml criteria match with the user’s sync filter criteriaSave View.png

Synced Appointments – FetchXML

<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="true" >
  <entity name="appointment" >
    <attribute name="subject" />
    <attribute name="scheduledstart" />
    <attribute name="scheduledend" />
    <attribute name="regardingobjectid" />
    <attribute name="prioritycode" />
    <attribute name="activityid" />
    <attribute name="instancetypecode" />
    <attribute name="location" />
    <order attribute="createdon" descending="true" />
    <filter type="and" >
      <condition attribute="scheduledstart" operator="not-null" />
      <condition attribute="instancetypecode" operator="neq" value="2" />
      <condition attribute="scheduledend" operator="not-null" />
      <condition attribute="globalobjectid" operator="not-null" />
    </filter>
    	<link-entity name="activityparty" from="activityid" to="activityid" alias="ad" >
      <filter type="and" >
        <condition attribute="participationtypemask" operator="ne" value="9" />
      </filter>
    </link-entity>
  </entity>
</fetch>

Not Synced Appointments – FetchXML

<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="true" >
  <entity name="appointment" >
    <attribute name="subject" />
    <attribute name="scheduledstart" />
    <attribute name="scheduledend" />
    <attribute name="regardingobjectid" />
    <attribute name="prioritycode" />
    <attribute name="activityid" />
    <attribute name="instancetypecode" />
    <attribute name="location" />
    <order attribute="createdon" descending="true" />
    <filter type="and" >
      <condition attribute="scheduledstart" operator="not-null" />
      <condition attribute="instancetypecode" operator="neq" value="2" />
      <condition attribute="scheduledend" operator="not-null" />
      <condition attribute="globalobjectid" operator="null" />
    </filter>
    	<link-entity name="activityparty" from="activityid" to="activityid" alias="ac" >
      <filter type="and" >
        <condition attribute="participationtypemask" operator="ne" value="9" />
      </filter>
    </link-entity>
  </entity>
</fetch>

You should now be able to use these new view from the grid area.

Synced Appointments.png

If you try to use the view from Advanced Find, you will get this error.Advanced Find.png

This means you have to use FetchXML Builder to update you fetchxml, if any changes are required in the future. If you want to add more columns to the view, you’ll have to use “View Designer” to do that, as you would not be able to use Advanced Find to do this, because of this error.

I hope this post will help you troubleshoot future appointment sync issues.

Tools used:

  1. LinqPad
  2. Dynamics CRM LinqPad Driver
  3. XrmToolBox
  4. FetchXML Builder
  5. View Designer
  6. Sync Filter Manager

Executing large FetchXML with WebAPI

You can easily execute fetchxml in WebAPI using the “fetchXml” query parameter. But this “GET” method won’t work, if the fetchxml is too big. In this case, you have to use the “POST” method to execute the fetchxml.

Sample Request Header:

Accept: application/json
OData-MaxVersion: 4.0
OData-Version: 4.0
Content-Type: multipart/mixed;boundary=batch_contactfetch

Sample Request Body:

--batch_contactfetch
Content-Type: application/http
Content-Transfer-Encoding: binary

GET https://[CRM URL]/api/data/v8.2/contacts?fetchXml=<fetch count="10" ><entity name="contact" ><attribute name="fullname" /></entity></fetch> HTTP/1.1
Content-Type: application/json
OData-Version: 4.0
OData-MaxVersion: 4.0

--batch_contactfetch--

Sample Code:

var req = new XMLHttpRequest();
req.open("POST", Xrm.Page.context.getClientUrl() + "/api/data/v8.2/$batch", true);
req.setRequestHeader("OData-MaxVersion", "4.0");
req.setRequestHeader("OData-Version", "4.0");
req.setRequestHeader("Accept", "application/json");
req.setRequestHeader("Content-Type", "multipart/mixed;boundary=batch_contactfetch");
req.onreadystatechange = function() {
    if (this.readyState === 4) {
        req.onreadystatechange = null;
        if (this.status === 200) {
            var response = JSON.parse(this.response.substring(this.response.indexOf('{'),this.response.lastIndexOf('}')+1));
			console.log(response.value);
        } else {
            Xrm.Utility.alertDialog(this.statusText);
        }
    }
};

var body = '--batch_contactfetch\n'
body += 'Content-Type: application/http\n'
body += 'Content-Transfer-Encoding: binary\n'
body += '\n'
body += 'GET ' + Xrm.Page.context.getClientUrl()+'/api/data/v8.2/contacts?fetchXml=<fetch count="10" ><entity name="contact" ><attribute name="fullname" /></entity></fetch> HTTP/1.1\n'
body += 'Content-Type: application/json\n'
body += 'OData-Version: 4.0\n'
body += 'OData-MaxVersion: 4.0\n'
body += '\n'
body += '--batch_contactfetch--'

req.send(body);

Request Screenshot

Fetch Post Request.png

Response Screenshot

fetch-post-response

Bookmarklet: Copy and Paste Lookup

Lookups (including partylist and customer) cannot be copy pasted. I have encountered couple of scenarios where I needed this capability. I have developed these bookmarklets to solve this issue.

Copy Lookup

Select and the drag the code below to the bookmark bar.

javascript:(function(){let contentPanels=Array.from(document.querySelectorAll('iframe')).filter(function(d){return d.style.visibility!=='hidden'});if(contentPanels&&contentPanels.length>0){let Xrm=contentPanels[0].contentWindow.Xrm;let currentControl=Xrm.Page.ui.getCurrentControl();if(currentControl&¤tControl.getControlType()==='lookup'){let currentLookup=currentControl.getAttribute().getValue();if(currentLookup){let serialisedLookupValue=JSON.stringify(currentLookup.map(x=>{let c={};({id:c.id,name:c.name,type:c.type,typename:c.typename,entityType:c.entityType}=x);return c;}));sessionStorage.setItem('ryr_serialisedLookup',serialisedLookupValue);alert('Lookup copied. Ready to paste');}}else{alert('No field has been selected or the currently selected field is not a lookup');}}else{alert('Entity form not detected');}})();void 0;

Source:

(function () {
	let contentPanels = Array.from(document.querySelectorAll('iframe')).filter(function (d) {
			return d.style.visibility !== 'hidden'
		});
	if (contentPanels && contentPanels.length > 0) {
		let Xrm = contentPanels[0].contentWindow.Xrm;
		let currentControl = Xrm.Page.ui.getCurrentControl();
		if (currentControl && currentControl.getControlType() === 'lookup') {
			let currentLookup = currentControl.getAttribute().getValue();
			if (currentLookup) {
				let serialisedLookupValue = JSON.stringify(
						currentLookup.map(x =  > {
								let c = {};
								({
									id : c.id,
									name : c.name,
									type : c.type,
									typename : c.typename,
									entityType : c.entityType
								}
										 = x);
								return c;
							}));
				sessionStorage.setItem('ryr_serialisedLookup', serialisedLookupValue);
				alert('Lookup copied. Ready to paste');
			}
		} else {
			alert('No field has been selected or the currently selected field is not a lookup');
		}
	} else {
		alert('Entity form not detected');
	}
})();

Paste Lookup

Select and the drag the code below to the bookmark bar.

javascript:(function(){let contentPanels=Array.from(document.querySelectorAll('iframe')).filter(function(d){return d.style.visibility!=='hidden'});if(contentPanels&&contentPanels.length>0){let Xrm=contentPanels[0].contentWindow.Xrm;let currentControl=Xrm.Page.ui.getCurrentControl();if(currentControl&¤tControl.getControlType()==='lookup'){let currentLookup=currentControl.getAttribute();let copiedLookupValue=sessionStorage.getItem('ryr_serialisedLookup');if(copiedLookupValue){currentLookup.setValue(JSON.parse(copiedLookupValue));} else{alert('Please select a lookup to copy first before pasting');}}else{alert('No field has been selected or the currently selected field is not a lookup');}}else{alert('Entity form not detected');}})();void 0;

Source:

(function () {
	let contentPanels = Array.from(document.querySelectorAll('iframe')).filter(function (d) {
			return d.style.visibility !== 'hidden'
		});
	if (contentPanels && contentPanels.length > 0) {
		let Xrm = contentPanels[0].contentWindow.Xrm;
		let currentControl = Xrm.Page.ui.getCurrentControl();
		if (currentControl && currentControl.getControlType() === 'lookup') {
			let currentLookup = currentControl.getAttribute();
			let copiedLookupValue = sessionStorage.getItem('ryr_serialisedLookup');
			if(copiedLookupValue){
				currentLookup.setValue(JSON.parse(copiedLookupValue));
			}
			else{
				alert('Please select a lookup to copy first before pasting');
			}
		} else {
			alert('No field has been selected or the currently selected field is not a lookup');
		}
	} else {
		alert('Entity form not detected');
	}
})();

Instructions

  1. Select the lookup field to be copied. The lookup will usually be highlighted with a blue background when it is selectedlookup-selected
  2. Run the Copy Lookup bookmarklet. You will get a alert message confirming that the lookup has been copied
  3. Select the lookup field that is the target of the paste. The lookup has to be of the same type i.e. if the copied lookup was a contact lookup, the target lookup also has to be a contact lookup
  4. Run the Paste Lookup bookmarklet
  5. Save the record

I have tested these bookmarklets only in Chrome 53 and Firefox 50.

Export to Excel using Dynamics 365 SDK

With the new Dynamics 365 release, a new message has been added that making exporting FetchXML results really simple. This message is not documented, hence is technically unsupported. With that word of warning, I will show you how to utilise this new message to export data from Dynamics 365 to an Excel file.

ExportToExcel message definition

Parameter Type
View EntityReference
FetchXml string
LayoutXml string
QueryApi string
QueryParameters InputArgumentCollection

Code

using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Linq;
using System.ServiceModel;
using Microsoft.Crm.Sdk.Messages;
using Microsoft.Xrm.Client;
using Microsoft.Xrm.Client.Services;
using Microsoft.Xrm.Sdk;

namespace Experiments
{
    class Program
    {
        private static OrganizationService _orgService;
        static void Main(string[] args)
        {
            try
            {
                CrmConnection connection = CrmConnection.Parse(
                    ConfigurationManager.ConnectionStrings["CRMConnectionString"].ConnectionString);

                using (_orgService = new OrganizationService(connection))
                {
                    var exportToExcelRequest = new OrganizationRequest("ExportToExcel");
                    exportToExcelRequest.Parameters = new ParameterCollection();
                    //Has to be a savedquery aka "System View" or userquery aka "Saved View"
                    //The view has to exist, otherwise will error out
                    //Guid of the view has to be passed
                    exportToExcelRequest.Parameters.Add(new KeyValuePair<string, object> ("View",
                        new EntityReference("userquery", new Guid("{0B915102-24A7-E611-8101-1458D05B1178}"))));
                    exportToExcelRequest.Parameters.Add(new KeyValuePair<string, object>("FetchXml", @"
                    <fetch distinct='false' no-lock='false' mapping='logical' returntotalrecordcount='true'>
	                    <entity name='contact'>
		                    <attribute name='fullname' />
	                    </entity>
                    </fetch>"));
                    exportToExcelRequest.Parameters.Add(new KeyValuePair<string, object>("LayoutXml", @"
                    <grid name='resultset' object='2' jump='fullname' select='1' icon='1' preview='1'>
	                    <row name='result' id='contactid'>
		                    <cell name='fullname' width='300' />
	                    </row>
                    </grid>"));
                    //need these params to keep org service happy
                    exportToExcelRequest.Parameters.Add(new KeyValuePair<string, object>("QueryApi", ""));
                    exportToExcelRequest.Parameters.Add(new KeyValuePair<string, object>("QueryParameters",
                        new InputArgumentCollection()));
                    var exportToExcelResponse = _orgService.Execute(exportToExcelRequest);
                    if (exportToExcelResponse.Results.Any())
                    {
                        File.WriteAllBytes("Active Contacts.xlsx", exportToExcelResponse.Results["ExcelFile"] as byte[]);
                    }
                }
            }
            catch (FaultException<OrganizationServiceFault> ex)
            {
                string message = ex.Message;
                throw;
            }
        }
    }
}

Closing Notes:

  1. “View” parameter can accept “userquery” or “savedquery”, but they have to exist i.e. you can’t pass empty Guid.
  2. The fetchxml and layoutxml can be different from what is in the “savedquery” or “userquery”. Hence, you can create a “Personal View” just so that you can use it in this message, but modify the fetchxml and layoutxml to whatever you want.
  3. The name of the tab in the Excel output file will be the name of the view specified in the “View” parameter
  4. This message can be executed from Javascript as well, but you will get a base64 string instead of a byte array in the response.

Please vote up my request on Connect (logged Feb 2015) -> https://connect.microsoft.com/site687/feedback/details/1127874/export-to-excel-sdk-message so that this message can be made available as an unbound WebAPI action.

Two useful LINQ Queries

In order to run these queries you’ll have to install LINQPad and LINQPad Driver for CRM.

Query 1 – Who created the entities?

It is not possible to look at an entity and find out who created it. You can however use the createdby on the SavedQuery (System View) to find out this information, as the System View is created the same time as the entity is created and also contains the createdby user information.

from q in
(from s in SavedQuerySet.AsEnumerable()
 where s.QueryType == 0
 group s by s.ReturnedTypeCode
into g
 select new
 {
	 g.Key,
	 Views = (from s in SavedQuerySet
			  where s.ReturnedTypeCode == g.Key && s.QueryType == 0
			  orderby s.CreatedOn
			  select new { s.CreatedOn, s.CreatedBy }).First()
 })
orderby q.Views.CreatedOn descending
select new { q.Key, CreatedOn = q.Views.CreatedOn.Value.ToLocalTime(), q.Views.CreatedBy.Name }

Entity creation query.png

Query 2 – All Plugins with message, stage, filtering attributes and entity

You can use this query to get a quick snapshot of all the plugins in the system.

from m in SdkMessageProcessingStepSet
join f in SdkMessageFilterSet on m.SdkMessageFilterId.Id equals f.SdkMessageFilterId
join s in SdkMessageSet on f.SdkMessageId.Id equals s.SdkMessageId
join p in PluginTypeSet on m.PluginTypeId.Id equals p.PluginTypeId
where f.IsCustomProcessingStepAllowed.Value
&& !m.IsHidden.Value
&& m.CustomizationLevel.Value == 1
select new { Message = s.Name, Rank = m.Rank.Value, Stage = m.Stage, StageName = m.FormattedValues["stage"], m.FilteringAttributes, p.AssemblyName, PluginName= p.Name, StepName = m.Name, StepDescription = m.Description, Status = m.StatusCode, StatusName = m.FormattedValues["statuscode"]}

plugins