Welcome to your DQ for Dynamics™ Knowledge Base Overview

This Knowledge Base is your comprehensive guide to navigate you through the steps and procedures to successfully implement DQ for Dynamics™ . 

It sets out the best practices and pre-requisites to ensure your application will correctly operate and interface with D365.

Pre-requisites:

1. It is advised that all testing take place in a D365 UAT environment. The pre-requisites are as follows:

1. That it is a direct replica of Production. This includes:
   1. Data that is representative of Production data, even if this is a sample dataset.
   2. Workflows and plug-ins replicate the Production environment.
2. The same DQ for Dynamics™ solution and version is installed in both test/UAT and Production environments.
3. Use of the same connection method between D365 CE and the DQ for Dynamics™ service.
4. Apply the same security roles to all relevant users across each environment.

Capture & Cleanse:

Installation

1. Create an account in the DQ Shop and buy credits.
2. Download either the Online Solution or On Premise Solution
3. Provision DQ for Dynamics™ to connect directly with CRM
4. Download your DQ for Dynamics™ Solutions:
   1. Base; and one or all of the below:
   2. Phone;
   3. Email;
   4. Address
5. Import to your chosen Environment in Power Apps Maker Portal
6. Repeat the import process for all Solutions required
7. Now from within DQ for Dynamics Cleanse Configuration, you can edit all of the environment variables that you saw during the solution import
8. In Power Apps Maker Portal, navigate to the form where you want the capture tool to feature
9. Find the field required
10. Add a Component to the field
11. Complete the Control Properties to the Component 

The tool will carry out the following respectively:

• Phone: allows users to carry out data quality checks including checking a phone number's validity, structure, and authenticity in real time when capturing data. 
• Email: again, allows users to check an email address's validity, structure, and authenticity in real time when capturing data.  
• Address: allows users to look up addresses in real-time during data capture to avoid data entry mistakes and ensure accurate addressing information. 

Match Testing:

1. Define your dataset by creating Dynamics CRM views.
   1. These can be loaded into DQ for Dynamics™ during the match session creation.
2. Define your default match configuration, match transforms and display and auto-fill.
3. Create your first match session to group duplicates.
4. Ask data stewards to verify the match results and feedback on incorrect grouping of duplicate records.
   1. This step may also highlight that records are missing from duplicate groups
5. Alter matching/transform rules and iterate session and repeat step 4, until the data stewards are happy with the matching results.

Manual Merge Testing:

1. Having correctly grouped duplicate records, data stewards must manually select a ‘Best Record’ from a set of duplicates and promote this to be the master record.
2. Select the ‘best data’ from a duplicate or multiple duplicates to update your master record. 
3. Once the merge has completed, verify that:
   1. The record selected as your master is still active in D365
   2. Duplicate record(s) are de-activated and linked to the surviving master record
   3. Data filled from a duplicate(s) record has been updated onto the master record
   4. Related Entities data has been re-assigned to the master record
   5. If you have defined Post Merge De-Duplication rules, please check that the Related Entities included in your configuration have been correctly deduplicated. 
   6. To gain further insights, analyse your D365 Audit History which will show the updates performed to your master and duplicate records. Check both master AND duplicate record Audit History. 
      1. This process will tell you if there are any issues when updating fields on the master record, or plug-ins that fire when certain fields get updated. 
   7. Please feedback any issues to your CRM administrator, who will be able to provide clarity on any issues found during testing.

Automated Merge Testing:

Once you have correctly grouped and manually tested your merge process, it is important to test the batch processing of duplicate groups.

Defining & Testing Best Record Rules

1. Go to the ‘Master Record Rules’ section in your default settings and define how you would automatically select your best record.
2. Now, go back to a set of duplicate records and select ‘Auto Promote’ on the bottom left of your screen. This will call the rules you have defined in step 1, and automatically select your best record from within your duplicate group.
3. Once finished, analyse the results, and ensure that the correct record has been selected. 
   1. If the automated rules have not selected the best record, think through the logic that you have applied and tweak your rules. 
   2. If you believe the rules are not executing correctly, please reach out to support@dqglobal.com

Note: It is important that you run this process over a sample of data so that you can be confident that you are testing over a representative dataset. 

Defining & Testing Auto Fill Rules

1. Go to the ‘Display & Auto Fill’ section in your default settings and define how you would automatically select best data from a set of duplicate records.
2. Now, go back to a set of duplicate records and select ‘Auto Fill on the bottom left of your screen. This will call the rules you have defined in step 1, and automatically select your best data from within your duplicate group.
3. Once finished, analyse the results, and ensure that the correct data has been selected. 
   1. If the automated rules have not selected the best data, think through the logic that you have applied and tweak your rules. 
   2. If you believe the rules are not executing correctly, please reach out to support@dqglobal.com

Note: It is important that you run this process over a sample of data so that you can be confident that you are testing over a representative dataset. 

Batch Testing

You have now completed the execution of automated processing on INDIVIDUAL duplicate groups. The final process is to test your configuration in Batch. This test will ensure that hundreds/thousands of duplicates can be processed sequentially. The reason for carrying out this test is to ensure that the system behaves in the same way as individual merge tests, and performance can be monitored.

Test Sample:

Find a sample of data between 100-1000 duplicate groups. This can be achieved by limiting the number of records fed to the match engine through an CRM advanced find query.

Auto Promote Testing:

1. On the duplicate listings screen select ‘Auto Promote All’
2. Monitor in ‘Auto Process Management’ in ‘Settings’ 
   1. Process should take no longer than 1 minute per 1000 records
3. Eyeball a sample of duplicate groups and ensure the correct record has been selected as your master.

Auto-Fill Testing:

1. On the duplicate listings screen, select ‘Auto Fill All’
2. Monitor within ‘Auto Process Management’ in ‘Settings’ 
   1. Process should take no longer than 30 – 90 minutes per 1000 records
3. Eyeball a sample of duplicate groups and ensure the correct data has been populated onto your master record.

Note: The bulk Auto Fill process will update record in CRM. Once the Auto Fill process is complete, go to CRM and check records are updated as per your Auto Fill rules.

Auto Accept & Merge All:

1. On the duplicate listings screen select ‘Auto Accept & Merge All’
2. Monitor within ‘Auto Process Management’ in ‘Settings’ 
   1. Process should take no longer than 30–90 minutes per 1000 records
3. Eyeball a sample of duplicate groups and ensure the merge process has executed correctly. Use the same steps performed in the manual merge test to verify that the merge has taken place as expected. 
4. Testing procedure:

1. The record selected as your master is still active in D365
2. Duplicate record(s) are de-activated and linked to the surviving master record
3. Data filled from a duplicate(s) record has been updated onto the master record
4. Related entities data has been re-assigned to the master record
5. If you have defined Post Merge De-Duplication rules, please check that the Related Entities included in your configuration have been correctly deduplicated. 
6. To gain further insights, analyse your D365 Audit History which will show the updated performed to your master and duplicate records. Check both master AND duplicate record Audit History. 
   1. This process will tell you if there are any issues when updating fields on the master record, or plug-ins that fire when certain fields get updated. 
7. Please feedback and issues to your CRM administrator, who will be able to provide clarity on any issues found during testing.

Final Acceptance Criteria:

Before moving to Production, consider the following factors:

• Speed and performance impacts
• Effect of Plug-ins/workflows
  1. Including sub entities of Related Entities
• Update/Merge Errors and how to mitigate/rectify these issues

Note: Every client is different, and the guidelines set out above are DQ Global’s best practices and advice. Each client may have customised solutions which require customised testing. Please consider these scenario’s during this process to ensure data integrity is preserved when moving to your production systems.

Now we've run through, let's get started breaking your journey into these manageable steps: 

Capture		Cleanse		Consolidate		Admin Centre
Capture ensures every entry into your CRM is consistent, complete and unique, preventing you from the chaos of data errors.		Cleanse offers a user-friendly bulk data cleansing tool, saving you from the hard task of constant CRM data cleaning.		Consolidate steps in when standard tools fall short, helping you handle duplicate records without the headache.		Your Admin Centre holds all the adhoc guides to assist in addition to the walkthroughs in your Knowledge Base.
Take me here				Take me here		Take me here