Mixed Reality Solutions with Power Apps / Dynamics 365 CRM / CDS

In recent times Pokemon Go was a mixed reality game, it showed pokemon characters in your real surrounding using the camera of your mobile phone.

And that kind of brought about ideas to present customers with the ability to view the products in their own surroundings that would help them try placing the objects in their environment and visualize it before making the decision.

Designing Mixed Reality Solutions in the past has required specialized technical knowledge, but with the aim to democratize technology, Microsoft Power Apps now offers Mixed Reality Components that are ready to use for everyone without any development experience.

Here I take an example of product images. The product details are stored in the Product entity available with Dynamics 365 CRM Sales and the images for the products have been stored in the notes section of the record.

Let us start with placing a gallery component to list out all the products.

Place the gallery control from the command bar.

In the datasource of the gallery – choose Products (you may have to make a connection with Common Data Service (Current) connector, if you do not already have one).

And choose the appropriate view from the Product entity that you want to use to list the products from.

In the field selection list, we choose ID and Name to be displayed.

Now once a user selects one of the products, we would like to display the images associated with the product. The images have been stored in the notes area of the record.

To present this as a scrolling carousel, choose the horizontal gallery layout.

Set the Data source of this gallery control to point to the notes of the selected product from the product gallery.

To display the image, we set the source of the image control to the document field of the note.

The files uploaded as notes are stored in base64 format in the document field.

The image control support Data URI format. Data URI format is a combination of the mimetype of the file along with the base64 value.

We get the base64 value from the document field, to this we concatenate the mimetype to generate the data uri.

Concatenate(“data:image/jpg;base64,”, ThisItem.Document)

To get the navigation buttons to show up change the following settings of the gallery:

Now we can see the images stored in notes in CDS within a gallery in a canvas app.

To be able to visualize this image in your surrounding we now need to go ahead and add the Mixed Reality component.

To see these components in your list in the canvas studio, make sure to enable the feature:

File -> Settings -> Experimental Features -> Mixed Reality features.

Once enabled, you should now see the following components available to add.

Select the View in MR component and add it outside the Preview Gallery control as shown below:

You can visualize the following types of data using the View in MR control:

• Publicly accessible, CORS-compliant URLs
• Base64-encoded URIs
• Attachments or media content accessed through data connectors

Ref: https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/mixed-reality-component-view-3d

We are using the Base64-encoded URIs.

Set the Source property of this control to the same as that we set for the image control above for preview.

Unlike most of the Power Apps controls that we can preview from our desktop environment, to see the View in MR in action, you need a device that is MR compatible, for me that is a mobile device that I would try this from.

Save and publish this app so that you can now access this through the Power Apps mobile app.

Download the Power apps mobile app from Appstore for iOS or Android.

Login to this app using the credentials for the Power Apps environment where you designed and published your app.

You see the images associated with the product.

Click on ‘View in MR’ and this is how you get to select the surface where you would like to place image.

Click to choose the position and place the image and here is the final result:

You could also set the dimensions of the object so that it is placed and appears accordingly in the real surroundings.

If instead of the image of the object, you were only interested to check if the object of given dimensions would fit in a specified space, you could use the View Shape in MR component instead.

Set the dimensions of the object on the control as shown below:

And this is how this one shows up in MR:

Note: You could place the image files on a public location that can be accessed. SharePoint does not provide a direct URL access to the image due to CORS issue.