Working with the Sitecore Universal Tracker Version 1

Sitecore Universal Tracker Initial Release Version 1.0 was released along with Sitecore 9.1 on November 28th 2018.

Sitecore Universal Tracker is an API first, a micro-service-based tracking service to track actions from mobile apps, IoT, AR, VR and other emerging channels.

Please read through my previous blog posts Offline tracking with the Sitecore Universal Tracker & Installing Sitecore Universal Tracker 1.x on Sitecore 9.1

Once you read through the above two links, it will be easy to understand the following. I spent over 2 months trying to figure things out with the help of the product team.

Using the Universal Tracker, we can store Interactions and Events within those interactions. The interactions and events in the interaction are sent to the Web API endpoint, this is called the Collection Stage.

During the Collection stage all the interactions and events are stored in temporary storage (SQL Server). An individual interaction can be closed by a completed request or by the visit being timed out. Once the interaction is closed the next stage is Processing.

During the Processing stage you can enhance the data before sending it to the xDB by adding filters which can do lookups like the GeoIP lookup on an IP. Once processing is complete the data is sent to the xConnect service to be stored in the XDB.

The Universal Tracker Client SDK is a Portable Class Library that supports all interaction and event types registered in xConnect, including the standard interactions; event, goal, outcome, campaign. The SDK also supports additional events, like Device Information, Geolocation, Page Opened, Page Closed and Error. The SDK will be open source and is pre-baked into the Sitecore Mobile SDK, powered by Xamarin. I would say that the current focus is on getting UT stable in the Mobile SDK and cover all use cases.

The expectation is that you start an Interaction, send a few goals or events, and then close out the session. Unfortunately in my use case, I need to be able to start multiple interactions for different contacts and send events. Currently, there are only three public web API available, put events, put interaction, complete interaction.

I also need to be able to search for an active Interaction by the contact identifier, which is not available. There is a hidden method available to search for interaction by interaction id but that won’t help me.

Setup

Let’s get started by setting up some basic items so we can track our anonymous contacts. I needed to setup a channel called SUGCON.

UT does not work with the ref data service so we need to sync the channels we need from /Sitecore/system/Marketing Control Panel/Taxonomies/Channel to the \Sitecore.Tracking.Processing.Service\sitecore\Sitecore.Tracking.Processing.ChannelManagement\PipelinesConfig\channelTypes.json file manually.

Next, setup an event and goals. Be sure to deploy what you have created.

Once they are setup, we can use the goal and event id’s to trigger events for UT.

In my scenario, I need the anonymous contacts to be indexed. Modify the IndexAnonymousContactData setting to true in the xconnectpath\root\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\SearchIndexer\sc.Xdb.Collection.IndexerSettings.xml file. There is an identical setting in xconnectpath\root\App_data\Config\Sitecore\SearchIndexer\sc.Xdb.Collection.IndexerSettings.xml file which is not used.

<Settings> <Sitecore> <XConnect> <SearchIndexer> <Services> <IndexerSettings> <Type>Sitecore.Xdb.Collection.Indexing.IndexerSettings, Sitecore.Xdb.Collection</Type> <LifeTime>Singleton</LifeTime> <Options> <SplitRecordsThreshold>25000</SplitRecordsThreshold> <IndexPIISensitiveData>false</IndexPIISensitiveData> <IndexAnonymousContactData>true</IndexAnonymousContactData> </Options> </IndexerSettings> <IIndexer> <Type>Sitecore.Xdb.Collection.Indexing.DecoratedIndexer, Sitecore.Xdb.Collection</Type> <As>Sitecore.Xdb.Collection.Indexing.IIndexer, Sitecore.Xdb.Collection</As> <LifeTime>Singleton</LifeTime> </IIndexer> <IIndexerInitializer> <Type>Sitecore.Xdb.Collection.Indexing.DefaultIndexInitializer, Sitecore.Xdb.Collection</Type> <As>Sitecore.Xdb.Collection.Indexing.IIndexInitializer, Sitecore.Xdb.Collection</As> <LifeTime>Singleton</LifeTime> </IIndexerInitializer> <IIndexRebuildFlow> <Type>Sitecore.Xdb.Collection.Indexing.IndexRebuildFlow, Sitecore.Xdb.Collection</Type> <As>Sitecore.Xdb.Collection.Indexing.IIndexRebuildFlow, Sitecore.Xdb.Collection</As> <LifeTime>Singleton</LifeTime> <Options> <IncomingDataLagOnCompletion>0.00:00:05</IncomingDataLagOnCompletion> <BatchSize>1000</BatchSize> </Options> </IIndexRebuildFlow> <IIndexRebuilderCountersDecorator> <Type>Sitecore.XConnect.Configuration.ServiceDecorator`2[[Sitecore.Xdb.Collection.Indexing.IIndexRebuilder, Sitecore.Xdb.Collection], [Sitecore.Xdb.Collection.Indexing.IndexRebuilderCountersDecorator, Sitecore.Xdb.Collection]], Sitecore.XConnect.Configuration</Type> <LifeTime>Singleton</LifeTime> </IIndexRebuilderCountersDecorator> </Services> </SearchIndexer> </XConnect> </Sitecore></Settings>

The Universal Tracker is responsible for collecting, processing and sending interactions to xConnect. The Sitecore processing and reporting roles are responsible for the data aggregation and reporting. UT interactions are ignored by default by the WebVisitFilter because the interactions are not PageViews or Web visits. In order for us to process the Universal Tracker interactions we need to disable WebVisitFilter for all dimensions in the sitecorewebpath\App_Config\Sitecore\ExperienceAnalytics\Sitecore.ExperienceAnalytics.Aggregation.config file.

Here are a couple of useful links to help you create custom filters, segements and reports: https://doc.sitecore.com/users/91/sitecore-experience-platform/en/create-a-custom-report-filter-and-segment.html https://doc.sitecore.com/users/82/sitecore-experience-platform/en/creating-a-custom-experience-analytics-report.html

The Sitecore.Tracking.Collection.Service\sitecore\Sitecore.Tracking\Config\config.xml configuration file controls how submitted events are mapped to the corresponding xConnect events defined in the Sitecore Marketing Control panel. If you submit an event that is unknown and is not specified in this config, there is a risk of data loss or improper parsing.

You can specify additional types by adding the published type to the config.xml file, with a fully qualified name with the syntax AssemblyName.Namespace.ClassName, AssemblyName.

<?xml version="1.0" encoding="utf-8" ?><Settings> <Sitecore.Tracking> <TypeMappings> <event>Sitecore.XConnect.Event, Sitecore.XConnect</event> <goal>Sitecore.XConnect.Goal, Sitecore.XConnect</goal> <outcome>Sitecore.XConnect.Outcome, Sitecore.XConnect</outcome> <pageview>Sitecore.XConnect.Collection.Model.PageViewEvent, Sitecore.XConnect.Collection.Model</pageview> <campaign>Sitecore.XConnect.Collection.Model.CampaignEvent, Sitecore.XConnect.Collection.Model</campaign> <download>Sitecore.XConnect.Collection.Model.DownloadEvent, Sitecore.XConnect.Collection.Model</download> <search>Sitecore.XConnect.Collection.Model.SearchEvent, Sitecore.XConnect.Collection.Model</search> </TypeMappings> </Sitecore.Tracking></Settings>

The Sitecore.Tracking.Collection.Service\sitecore\Sitecore.Tracking.Plugin.Status\Config\config.xml file controls the UT Status plugin, which displays the status page. By default, the setting LocalAccessOnly is set to true, which means the status controller only accepts requests from the local machine. External requests will return an HTTP 404 (not found) error. Set LocalAccessOnly to false to allow external requests.

Starting an Interaction & pushing events

You can work with UT without writing a line of code by using Postman. You can start interactions for known contacts by specifing the Source and Identifier for the contact. Go through “Get contact by identifier” section on https://doc.sitecore.com/developers/91/sitecore-experience-platform/en/get-contacts.html. In my case, the contacts are all anonymous. Let’s go through some sample requests.

The example below shows how we are starting an interaction and also triggering a goal that the user has attended the conference.

The call to start interaction returns a 201 (Created) along with the Interaction Id. We can use this Id to push more events to the UT as shown in the following example. All subsequent calls would need the id to be passed.

Closing Interactions

You can close the interactions by sending a PUT request as shown below.

You can also set a timeout attribute per channel to closing out Interactions in the Sitecore.Tracking.Processing.Service\sitecore\Sitecore.Tracking.Processing.ChannelManagement\PipelinesConfig\channelTypes.json file.

All the active interaction and interactions are stored temporarily in the UT database before they are processed. You can run queries to see the values. Be sure to convert the binary data field to text to investigate.

The following is some sample C# code to push an interaction with an event and a goal.

using System; using Sitecore.UniversalTrackerClient.Entities; using Sitecore.UniversalTrackerClient.Request.RequestBuilder; using Sitecore.UniversalTrackerClient.Session.SessionBuilder; namespace MyFriendsConsoleApp { class Program { static void Main(string[] args) { Console.WriteLine("Hello Friend!"); Program.TestUT(); Console.ReadLine(); } private static async void TestUT() { string instanceUrl = "https://sitecore.tracking.collection.service"; //Your collection service string channelId = "f3f8307a-78d3-4899-97aa-543410a93ecd"; // SUGCON Channel - /sitecore/system/Marketing Control Panel/Taxonomies/Channel/Offline/Event/SUGCON string eventDefinitionId = "581f7198-8be9-4162-8d98-254808cffee2"; // Attended Conference - /sitecore/system/Marketing Control Panel/Events/Attended Conference string goalDefinitionId = "26d7693d-f8cb-4c69-8e60-46273111158f"; // Attended Offline Tracking with Universal Tracker Session - /sitecore/system/Marketing Control Panel/Goals/Conference/Offline Tracking with Universal Tracker var defaultInteraction = UTEntitiesBuilder.Interaction() .ChannelId(channelId) .Initiator(InteractionInitiator.Contact) .Contact("SUGCONConference", "Attendee 22") //Anonymous Contact .Build(); using (var session = SitecoreUTSessionBuilder.SessionWithHost(instanceUrl) .DefaultInteraction(defaultInteraction) .BuildSession() ) { //Send event of type event var eventRequest = UTRequestBuilder.EventWithDefenitionId(eventDefinitionId) .AddCustomValues("key1", "value1") //pass any custom values along with the event .Timestamp(DateTime.Now) .Build(); var eventResponse = await session.TrackEventAsync(eventRequest); Console.WriteLine("Track EVENT RESULT: " + eventResponse.StatusCode.ToString()); //Send event of type goal var goalRequest = UTRequestBuilder.EventWithDefenitionId(goalDefinitionId) .Timestamp(DateTime.Now) .Build(); var goalResponse = await session.TrackEventAsync(goalRequest); Console.WriteLine("Track Goal RESULT: " + goalResponse.StatusCode.ToString()); } } } }

Processing

The interactions are not processed in real time by we can modify some setting to make it more efficient. By default, the processing engine checks for closed interactions every 5 seconds. We can modify the BatchSize and ProcessingThresholdPercent in Sitecore.Tracking.Processing.Service\sitecore\Sitecore.Tracking.Processing.Engine\Config\config.xml file to optimize the processing.

Feature requests: - Get/search for an active interactions by the contact identifier. - Manually trigger UT to process the completed interactions. - Search for an active Interaction by the contact identifier. - Manage multiple interactions at one time. (switching between active interactions)

Result

Credits: Alistair Deneys Alexey Vashchenko

Links: Installing Sitecore Universal Tracker 1.x on Sitecore 9.1 Offline tracking with the Sitecore Universal Tracker Universal Tracker SDK Sitecore.UniversalTracker.MobileSDK Sitecore Universal Tracker Documentation

If you have any questions or concerns, please get in touch with me. (@akshaysura13 on Twitter or on Slack)