Confidence score is one of the key criteria for deciding what assets needs to be shown to end users based on your search query in AEM. Higher confidence score tagged assets are displayed first followed by low confidence score tagged asset. 

In this tutorial we will learn :-

• Why AEM Smart tagging is required
• Understand how AEM search behaves with Smart Tags.
• Update confidence score for Smart Tags.

Why Smart Tagging is Required:-

Enterprise organisations deals with large amount of digital assets which grows with time. In such scenerios it become very important that your digital assets has proper metadata and tags. If proper asset metadata is not present then those assets become dark assets means they wont be visible to end users when they search for specific term and these assets becomes un usable. To resolve this problem with dark assets AEM provides automated smart tagging of assets. Adding Smart tags to your assets helps in improving overall end user search experience.

You can always curate your automatic applied smart tags to avoid inaccurate tagging, but applying smart tags is a one way process we cannot send feedback to Adobe Sensei service to avoid specific tags on similar assets in future. Adobe AEM smart tagging sensei AI model gets trained regularly based on Assets uploaded to Adobe Stock Photography. For training Adobe sensei AI model with customer specific tags use Enhanced Smart Tagging.

Understand AEM search with Smart tags:-

By default, AEM omni search combines the search terms with an AND clause. Using smart tags does not change this default behavior. However smart tags adds an OR clause to find any of the search terms in the applied smart tags.

For example, consider searching for man jumping. Assets with just man or just jumping keyword in the metadata do not appear in the search results by default. However, an asset tagged with either man or jumping using smart tags appears in such a search query. So the search results are a combination of:-

• Assets with man and jumping keywords in the metadata.
• Assets smart tagged with either of the keywords.

Order of Precedence how search results are evaluated:-

The search results that match all search terms in metadata fields are displayed first, followed by the search results that match any of the search terms in the smart tags. In the above example, the approximate order of display of search results is:-

• Matches man and jumping term in various metadata fields.
• Matches man and jumping term in smart tags.
• Matches man or jumping term in smart tags.

Understand confidence Score and update it for Smart tags:-

AEM applies a minimum confidence threshold for smart tags to avoid having too many tags for each video asset, which slows down indexing. Your asset search results are ranked based on the confidence scores, which generally improve search results beyond what an inspection of the assigned tags of any video asset suggests. Inaccurate tags often have low confidence scores so they seldom appear at the top of the Smart Tags list for assets.

The default value for Smart tag for image asset is 0.5 and for video asset is 0.7 (It should be between 0 and 1). The default value might might not hold good for all customers as an Architect you should discuss it with business users to understand what threshold holds good for your customer.

Note:- Manual Tags are always assigned with 100% (maximum confidence score). Hence if any manual tag matches the search term it will be listed on top of search results.

Smart Tag OSGI config for updating confidence Score in AEM As Cloud:

To update confidence score OSGI Config to the project deployed to Adobe Experience Manager as a Cloud Service through Cloud Manager. Follow below steps:-

In the Adobe Experience Manager project (ui.config since Archetype 24, or previously ui.apps) the config.author OSGi configuration, include a config file named com.adobe.cq.assetcompute.impl.senseisdk.SenseiSdkImpl.cfg.json with the following contents:-

{
  "minVideoActionConfidenceScore":0.7,
  "minVideoObjectConfidenceScore":0.7,
  "maxTextWords:Integer":3,
  "minTextConfidenceScore":0.7,
  "minColorCoverage":0.05,
  "maxColors:Integer":0,
  "colorMask:Integer":0,
  "minImageConfidenceScore":0.7
}

Note :- Please add/update only those JSON properties that you wish to update in AEMasCloud. For example if you want to update confidence score value only for images then below JSON is enough.

{
  "minImageConfidenceScore":0.7
}

To validate your changes on local environment after code deployment. Navigate to Felix console and search for OSGI configuration “Adobe CQ Sensei SDK Service” as shown below.

Update confidence score OSGI Config to the project deployed to Adobe Experience Manager 6.5 or below :

Include a config file named com.day.cq.dam.similaritysearch.internal.impl.SimilaritySearchServiceImpl with the following contents:-

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
    jcr:primaryType="sling:OsgiConfig"
    max.tags="10"
    min.tag.confidence="0.7"/>

Options available to configure in AEM 6.5 and below instances are:-

• max.tags: Maximum number of tags. Default value is 25.
• min.tag.confidence: Minimum confidence score below which tags will not be added. Default value is 0.5
• training.namespace: Separate training namespace for storing tags under predictedTags node. For Ex:- “custom_namespace.tag_name1”

Testing Smart tag Confidence score on Development or Higher environment:-

Because Smart tag service integration is not available in local development environment. We need to deploy our code to higher environment to complete our final testing.

Screenshot after customising confidence score for smart tags

As you can see now number of smart tags applied to an asset is reduce and asset is automatically tagged with only higher confidence score tag.

Note:- Do not keep confidence score too high because Smart Tag service is still in learning phase and continuous evolving, if it is not able to identify an asset then it will tag it with low confidence score and due to high threshold that smart tag won’t be applied and we will eventually face same dark asset issue again.