Page Comparison

...

XperienCentral allows you to make fields of custom content item facets searchable via the Advanced Search UI. This topic explains how the mechanism works and what its restrictions and possibilities are. Note that the API is not specifically about creating custom facets, but rather about how to add custom fields to the index. The ability to turn a field into a facet is just one of the possible options.

...

...

Tutorial

The API is explained on the basis of a simple example. The following shows the custom media item definition:

...

Let's assume the implementation of this recipe interface annotates getName() with @Property but nothing else. If we were to index this content item now, the result of getName() will be added to the content item's body, but nothing else will happen. This is the same behavior as in XperienCentral versions 10.0.0 through 10.11.1. Because this is very limited, we are going to go further. We start by indexing the simple properties and then index the more complex properties, and finally we'll turn some of the properties into facets.

@Indexable

This is a type of annotation which can be used on interfaces, classes, and so forth. It indicates that the annotated type should be indexed in a Solr document. This means that if, for example, RecipeVersion has this annotation, a document will be created for each recipe version and if a property of this document matches a user search, then this document will be returned. Suppose we add the @Indexable annotation to our recipe. With this annotation only, nothing at all will happen and only the default properties for an article version will be indexed, therefore merely adding @Indexable to your class has little effect on our interface. To be able to do more with it, we need to use at least one of the other annotations.

@Field

This annotation is used only if @Indexable is present. It indicates that the result of the annotated method should be indexed. The result may be a single value (for a single-valued field), a collection or an array of values (for a multi-valued field), a map of key/values (for multiple unique fields), or null (ignore the value). Primitive types are supported as well. In the map case, each value is indexed with their unique key, but the parameters of the method's annotation are used. This annotation is inherited, so if it is present on an overridden method for example, the overriding method does not need to specify the annotation as well - it can override it, but it cannot remove an annotation.

Now we expand our interface.

@Indexable
public interface RecipeVersion extends MediaItemArticleVersion {

    @Field
    public String getName();

    @Field
    public String getDescription();

    public Locale getLanguage();

    public Country getCountry();

    public List<Ingredient> getIngredients();
}

When indexing a recipe version with above definition, the method getName() will be invoked and its result will be indexed. The name of the recipe will be stored and indexed, but we cannot search on it yet. Furthermore, since it is some sort of title, we want its value to weigh more during a search. getDescription() will also be stored and indexed, but it is unlikely the description should be stored; only being able to search on its value should be enough. We can achieve this using the following parameters (the value after '=' is the default value):

• stored=true - Specifies whether this field should be stored. A field that is not stored is not retrievable and usable on the client, but it can be used when searching. The possible values are true and false.
• indexed=false -Specifies whether this field should be indexed. A field is indexed by default if it is either boosted, or a facet. The possible values are true and false. There is typically no reason to explicitly set this parameter to true.
• body=false - By default, a value is not indexed in an aggregate field. Setting this property causes the values to be indexed to "body which makes it possible for the document to be found when searching on the value. The possible values are true and false.
• heading=false - This is the same as body, except that setting this property to true promotes "body" to "heading" which gives the value a slightly bigger boost. The possible values are true and false.
• boost=0.0 - Setting this property allows this field to influence the containing document's score based on the relevance of the field compared to other fields in the document. A positive boost makes the document score higher, while a negative boost lowers the score. Note that a boost between 0 and 1 sounds like a negative boost, but it is not. Reasonable values for boost are between -1 and 1. A boost of 0 means no boost is used.

What we actually want is: 

@Indexable
public interface RecipeVersion extends MediaItemArticleVersion {

    @Field(heading = true, boost = 1)
    public String getName();
    
    @Field(stored = false, body = true)
    public String getDescription();
 
    public Locale getLanguage();
 
    public Country getCountry();
 
    public List<Ingredient> getIngredients();
}

In this case, the name will be stored and copied to the heading and a match on the name will cause the document to appear higher up in the Advanced Search. The description will not get its own field but will instead be added to the body.

Now we are going to include the language of the recipe. We want to store and index this, and don't need it to be in the body or heading, so simply adding @Field should be enough. The only problem here is that Locale is not a SOLR type because SOLR can only handle the following types at this time (and arrays/collections of them):

• int, float, long, double, boolean (note that shorts are not supported)
• String
• Date

So we need a way to take our Locale and turn it into a String. This can be achieved using the adapter parameter.

• adapter=null - Adapters allow developers to change the value returned by the annotated method before it is indexed/stored. Adapters should also be used if a value should be indexed in a language-specific way. An adapter may be
  • a concrete class, in which case it should have a zero-arguments constructor;
  • an interface of which an implementation is exposed via OSGi (whiteboard pattern).
• An adapter may decide to adapt a value to nothing (that is, return null), causing no value to be indexed/stored. This is also what happens if an adapt method terminates exceptionally, although in this case a warning is logged as well.
• If an annotated method returns null, this value is given to the adapter anyway in order to give it the chance to make something out of it.

The adapter interface looks like this:

/**
 * Class that takes specific values and transforms them into (possibly language specific) other values.
 *
 * @see Field
 */
public interface FieldAdapter<T> {

    /**
     * Whether the adapted value can depend on the language.
     *
     * @return true or false
     */
    boolean isLanguageSpecific();

    /**
     * Adapts a T to return the value in the specified language. The specified language may be null,
     * in which case no specific language may be used. An implementation may return null, which is
     * equivalent to "do not index this field for this language". The returned value is the one that
     * is stored and is thus what is returned and displayed in Advanced Search. If this adapter is
     * language specific, its adapt methods will be called for each language and for a null language
     * to index the value in a language agnostic way (if possible). If it is not language specific,
     * only the latter is the case. This method is only invoked if this adapter's field is stored.
     * The returned value should match one of the registered {@link FieldTypeDefinition SOLR types}.
     *
     * @param value the value to adapt
     * @param forLocale the target locale to represent the value in
     * @return the adapted value, or null if no value should be stored
     * @see FieldAdapter#adapt(Object, Locale)
     * @see Field#stored()
     */
    Object adapt(T value, Locale forLocale);
}

so, the implementation for a locale looks like this:

public class LocaleFieldAdapter implements FieldAdapter<Locale> {

    @Override
    public boolean isLanguageSpecific() {
        return true;
    }

    @Override
    public Object adapt(Locale value, Locale forLocale) {
        if (value == null) { // If the language is missing, we can't do anything
            return null;
        }

        if (forLocale == null) { // If there is no target language, just return the language tag
            return value.getLanguage();
        }

        return value.getDisplayLanguage(forLocale); // Else, return the display language
    }
}

Now we update our recipe to index the language as well:

@Indexable
public interface RecipeVersion extends MediaItemArticleVersion {

    @Field(heading = true, boost = 1)
    public String getName();
    
    @Field(stored = false, body = true)
    public String getDescription();
    
    @Field(adapter = LocaleFieldAdapter.class)
    public Locale getLanguage();
    
    public Country getCountry();

    public List<Ingredient> getIngredients();
}

At this point, the following custom data would be indexed:

• name: "French Fries" (indexed, stored, boost = 1)

• heading: ["French Fries"]

• body: ["Long and yellow deliciousness."]

• language: "en" (indexed, stored)

• language (NL): "Engels" (indexed, stored)

• language (EN): "English" (indexed, stored)

@Document

Now that we have indexed a few trivial properties, we want to index some more complex properties: Country and List<Ingredient>. We could index this using an adapter, just like we did for Locale, but since we have access to the Country and Ingredient classes, it is much nicer to make use of @Document and annotate the referred classes. This annotation tells the indexer to not index the value, but instead to treat the value as an object that we should parse and scan for annotations as well. This looks like this.

@Indexable
public interface RecipeVersion extends MediaItemArticleVersion {

    @Field(heading = true, boost = 1)
    public String getName();
    
    @Field(stored = false, body = true)
    public String getDescription();
    
    @Field(adapter = LocaleFieldAdapter.class)
    public Locale getLanguage();
    
    @Document
    public Country getCountry();

    @Document
    public List<Ingredient> getIngredients();
}

@Indexable // Don't forget to use this annotation here!
public interface Country {

    @ReferField
    public String getName();
    
    public String getCapital();
    
    public String getMotto();
}

@Indexable // Don't forget to use this annotation here!
public interface Ingredient {

    @ReferField(body = true)
    public String getName();
}

This is straight-forward; two things should be noted:

1. getIngredients() returns a list, but you do not need to take care of this explicitly because XperienCentral iterates it into lists for you and considers all Ingredients in the list instead of the list itself.

2. Instead of @Field we use @ReferField in the referred classes. We do this to indicate that this is a field that should be included if the document is included in an other document, rather than being a document in itself. This makes it possible to store a reference to a type rather than the complete type, which is redundant. Say we want to index countries as well, we would basically annotate country as follows:
   
   

   Code Block
   theme Eclipse
   @Indexable public interface Country { @Field( ... ) @ReferField public String getName(); @Field( ... ) public String getCapital(); @Field( ... ) public String getMotto(); }

   
   Thanks to ReferField, only the name of the country will be included in other documents, and the name, capital, and motto will be included if the document is the root document (that is, if countries themselves are indexed). For the same reason we have a ReferDocument as well.

The document annotations have an adapter parameter as well so that you can change the returned value before it is scanned for annotations. A document adapter behaves in exactly the same way as a field adapter. You are allowed to return any object or primitive, but remember that only annotated fields in the resulting class will be indexed, so returning a string does not index the string value. Instead, it scans String.class for annotations (which it won't find) and in the end does not index anything. Field, ReferField, Document, and ReferDocument may all be used at the same time.

At this point, the following custom data would be indexed:

• name: "French Fries" (indexed, stored, boost = 1)

• heading: ["French Fries"]

• body: ["Long and yellow deliciousness.", "Potatoes", "Sunflower oil", "Salt"]

• language: "en" (indexed, stored)

• language (NL): "Engels" (indexed, stored)

• language (EN): "English" (indexed, stored)

• country name: "France"

• ingredient names: ["Potatoes", "Sunflower oil", "Salt"] (indexed)

There's one problem with this: all fields will be indexed in a language-independent way. This means that stopwords will not be removed, synonyms are not used, no words are stemmed, and so forth. This limits the searchability of our recipe, therefore we should fix this with an @Indexable parameter:

• languageGetter=null - By default, a document is language agnostic. It is possible to make document indexing website language specific by specifying a LanguageGetter here:
  
  

  Code Block
  theme Eclipse
  /** * Class that can read specific documents and determine their language. */ public interface LanguageGetter<T> { /** * Extracts the language from the given document. If the document cannot be associated * with a language, null should be returned. * * @param document the document * @return the document's language */ Locale getLanguage(T document); }

  
  A language getter may be

  • a concrete class, in which case it should have a zero-arguments constructor;
  • an interface of which an implementation is exposed via OSGi (whiteboard pattern).

So what we would do is create a language getter and add the parameter.

public class RecipeLanguageGetter implements LanguageGetter<RecipeVersion> {

    @Override
    public Locale getLanguage(RecipeVersion document) {
        return document.getLanguage();
    }
}

@Indexable(languageGetter = RecipeLanguageGetter.class)
public interface RecipeVersion extends MediaItemArticleVersion {
    ...
}

Now we will index the following:

• name: "French Fries" (indexed, stored, boost = 1)

• name (EN): "French Fries" (indexed, stored, boost = 1)

• heading: ["French Fries"]

• heading (EN): ["French Fries"]

• body: ["Long and yellow deliciousness.", "Potatoes", "Sunflower oil", "Salt"]

• body (EN): ["Long and yellow deliciousness.", "Potatoes", "Sunflower oil", "Salt"]

• language: "en" (indexed, stored)

• language (NL): "Engels" (indexed, stored)

• language (EN): "English" (indexed, stored)

• country name: "France"

• ingredient names: ["Potatoes", "Sunflower oil", "Salt"] (indexed)

Note that the country and ingredients are not indexed in a language-specific way yet. These items will be indexed in the language their language getter returns. It is possible however, to override this behaviour with a @(Refer)Document parameter

• inheritLanguage=false - If this is set to true, the language of the target document will be set/overridden to the language of the current document. This may either be the language explicitly defined for the object declaring the annotation or the language inherited from a parent document. This setting is useful if referred documents should be indexed with the same language as the root document, for example, elements on pages.

Facets

You can turn any field into a facet. All you have to do for this is use the following parameter.

...

Our implementation of the SearchFacetComponentDefinition looks like this:

public class RecipeFacetComponentDefinition extends ComponentDefinitionImpl implements SearchFacetComponentDefinition {

    private final List<String> myPath;
    private final Map<Locale, String> myTitles;
    private final int myPosition;

    public RecipeFacetComponentDefinition(List<String> path, Map<Locale, String> titles, int position) {
        super(false);

        myPath = path;
        myTitles = titles;
        myPosition = position;
    }

    @Override
    public String getWidgetId() {
        return "stringFacetWidget";
    }

    @Override
    public String getTitle(Locale forLocale) {
        return myTitles.get(forLocale);
    }

    @Override
    public int getPosition() {
        return myPosition;
    }

    @Override
    public Map<String, String> getFacetProperties() {
        return null;
    }

    @Override
    public int accept(SearchFacetDescriptor facet) {
        if (facet.getOwnerType().getName().equals(RecipeVersion.class.getName()) 
		  && Collections.indexOfSubList(facet.getPath(), myPath) != -1) {
            return 10;
        }

        return -1;
    }
}

...

Now we're finished. This is what we see after opening the Advanced Search after our recipe is indexed (we titled it "My recipe"). You will find that if you open the search dialog in Dutch, the facet titles will be the ones we added via the facet definition. Furthermore, "Recepttaal" will have the option "Engels" instead of "English".

Image Removed

The  The complete source and deployable jar can be found in the attachments.

...

• One parameter not described is the extension parameter:
  • extension=null - By default, an annotated class is, except for the referred classes, the only class that is scanned for annotations. In most cases this is fine, but in some cases it is not sufficient. For example, if one needs to invoke an external service that is not accessible from within the object or should not be a public/API method. To address, it is possible to define an extension of a class; a class with the annotated class as constructor parameter. This extension is scanned in addition to the class and comes therefore with more flexibility. It is possible to define an extension per type in the hierarchy level.
    • Extensions are treated like any other class, so if an extension is not annotated with @Indexable, it is not scanned.
    • The owner of a field originating from an extension is not the extension class, but rather the class requiring the extension.
    • An extension must be a concrete class with a single-argument constructor of the type the extension is for. For example: public RecipeVersionExtension(RecipeVersion recipe).

• It is possible to index result sets from a query as well. To do so, you should create a custom media item that invokes the SQL command that returns the ResultSet you want to index. Then, you should create a Map based on this result set and return this - the returned values will then be indexed with their key as part of the field name. You can use an extension for this. For example:
  
  

  Code Block
  theme Eclipse
  @Field(body = true) public Map<String, String> getFields() { Map<String, String> fields = new HashMap<>(); DataSource dataSource = ...; String selectQuery = ...; try(Connection conn = dataSource.getConnection(); PreparedStatement stmt = conn.prepareStatement(selectQuery)) { ResultSet rs = stmt.executeQuery(); for (int i = 0; rs.next(); ++i) { fields.put(rs.getMetaData().getColumnName(i), rs.getString(i)); } } catch (SQLException e) { LOG.log(Level.WARNING, "Could not (fully) index resultset", e); } return fields; }

  
  
  

• Both interfaces and classes (including implementations) can be indexed, so implementation-specific data is allowed as well.
• P(ackage-p)rivate/protected methods can be indexed but will be made accessible.
• Annotated methods may not have parameters and may not return void.
• If a class that is not exported through the Export-Package directive in the pom.xml adds a facet, this facet's SearchFacetDescriptor will have owner void.class.

Back to top 

...

Overview

Packages

• nl.gx.webmanager.services.contentindex.annotation

• nl.gx.webmanager.services.contentindex.adapter

...

• If your facet does not appear in the Advanced Search, ensure that:
  • you have created one of your annotated content items;
  • the value you expect to see is actually used and does not return null;
  • the content item is indexed;
  • your path is fully lowercased.
• If a class that is not exported through the Export-Package directive in the pom.xml adds a facet, this facet's SearchFacetDescriptor will have owner void.class.
• Many things are logged at log level FINE. This should give you detailed information about what values are indexed for a specific method, and why.

Back to top