Ascend’15 – Find advanced developer scenarios

Ascend ‘15 was a great event! It is really great that EPiServer organizes a big event for partners, developers, marketers and for everyone who’s interested. I was asked by EPiServer to give a lab session about EPiServer Find together with Mari Jørgensen. Mari works as a team lead/developer for Geta based in Norway. Together we covered some advanced topics about Find, of course it was quite hard to choose only a couple topics. We had 70 minutes for our session and it was a real challenge to fit all topics in the timeframe because we had so many things to talk about. In this blog I’ll cover the topics I discussed in the session, Mari’s blog will follow soon.

Unified Search

When Find is installed in EPiServer, pages and blocks are automatically indexed. In some scenario’s you would like to index custom .NET types. With the Find APi it is real simple to index those types. With unified search you can search on different types that have been indexed. A huge benefit of unified search is that you don’t have to worry about projection because Find will take care of this. Find will return a collection of UnifiedSearchHit objects when doing a unified search, I will discuss this later on. Pages and blocks are automatically searchable with unified search. For custom .NET types there are two options to make them searchable with unified search:

Implement ISearchContent interface
Define a property or extension method with the same as how it is defined in the ISearchContent interface.

public class Company
        {
            public string Name { get; set; }
            public string CatchPhrase { get; set; }
            public string Bs { get; set; }
            public string Phone { get; set; }
            public string Street { get; set; }
            public string Zipcode { get; set; }
            public string City { get; set; }
     
            public virtual string SearchTitle { get { return Name; } }
            public virtual string SearchText
            {
                get
                {
                    return string.Format("{0} {1} {2} {3} {4} {5}", CatchPhrase, Bs, Phone, Street, Zipcode, City);
                }
            }
            public virtual string SearchSection { get { return "Company"; } }
        }

EPiServer self uses the second option for pages and blocks, namely extension methods. That can be very useful especially when you don’t want to implement all the properties of the ISearchContent. For the second option, it is mandatory to also register your type on the UnifiedSearchRegistry class. It is only necessary to execute this line of code while the application is starting up, so put this line of code in the Global.asax or in an initializable module.

SearchClient.Instance.Conventions.UnifiedSearchRegistry.Add<Company>();

What I mentioned earlier is that Find will take care of projection. That means it is not necessary to map the different types to your own result type.

Say for example I have indexed ArticlePage objects (which is a normal EPiServer page) and two custom .NET types Company and User objects. I can use unified search to search on the different types. What Find automatically does is map the different result types to the type UnifiedSearchHit. The reason Find can automatically do this is because types that are searched on are implementing the ISearchContent interface or defined a property of extension method with the same name. So Find knows that for example the SearchTitle (ISearchContent) property needs to be mapped to the Title property of the UnifiedSearchHit. It is also possible to customize projection.

SearchClient.Instance.Conventions.UnifiedSearchRegistry
                    .ForInstanceOf<Company>()
                    .ProjectTitleFrom(x => x.Name);

There are a number of methods available for customize projection:

ProjectExcerptUsing
ProjectHighlightedExcerptUsing
ProjectImageUriFrom
ProjectMetaDataFrom
ProjectTitleFrom
ProjectTypeNameFrom
ProjectUrlFrom

The same as for registering types it is only necessary to execute the customize projection code when the application is starting up.

The code for performing a unified search is quite simple.

_client.UnifiedSearchFor(query)
                    .GetResult();

Highlighting

Example: Search for ‘to’:

impossible to keep up with it all. But, with times constantly changing, we have to do our best to keep up … that everyone should know more about is FTAs, free to air satellite receivers and files. An FTA receiver

With highlighting, it is possible to highlight the matching searching query in the result. The word will be surrounded by HTML tags, default the <em> is used. Find returns fragments of the resulting text that contain the matching search query. There are a number of configuration options that can be set:

Number of fragments: Configure the number of fragments that should be returned. Default two fragments are returned.
Fragment size: The length of the fragment can be set with this property. Default 100 characters are returned.
Concatenation: Configure how fragments should be separated. Default three dots are used.
Post tag: Highlighted words are surrounded with HTML tags. Default value is </em> tag.
Pre tag: Highlighted words are surrounded with HTML tags. Default value is <em> tag.

var hitSpecification = new HitSpecification();
    hitSpecification.HighlightExcerpt = true;
    hitSpecification.HighlightTitle = true;
    hitSpecification.PreTagForAllHighlights = "<i>";
    hitSpecification.PostTagForAllHighlights = "</i>";
    hitSpecification.ExcerptHighlightSpecAction = x =>
    {
                   x.FragmentSize = 4;
                   x.NumberOfFragments = 200;
                   x.Concatenation = fragment => string.Join("___", fragment);
    };
     
    _client.UnifiedSearchFor(query)
        GetResult(hitSpecification);

In the above code example, you can see that the HitSpecification type contains the configuration properties. Not only unified search supports highlighting but also typed search.

_client.Search<ArticlePage>()
                    .For(query)
                    .Select(x => new
                    {
                        Text = x.BodyText.AsHighlighted(new HighlightSpec
                        {
                            FragmentSize = 200,
                            NumberOfFragments = 4,
                            PostTag  = "</i>",
                            PreTag = "<i>",
                            Concatenation = frag => frag.Concatenate(" ___ ")
                        })
                    })
                    .GetResult();

Statistics

EPiServer Find can track statistics, this can be enabled in code. Find can track the search queries and the click behavior from the visitor on the result items. In the Find interface the following statistics can be viewed:

Most frequent search: The most popular search queries that were done by visitors. This statistic can be viewed by last 24 hours, week, month or even by year, same goes for the other statistics available in the Find interface.
Search without hits: Search queries that didn’t return results.
Search without relevant hits: Search queries that did return results, but the visitor didn’t click on any of the result items.

When using unified search it is quite easy to track statistics.

_client.UnifiedSearchFor(query)
                    .StatisticsTrack()
                    .GetResult();

The StatisticsTrack method makes sure that the search query is tracked, but also that the result item URL is enriched with a number of querystring parameters. The following querystring parameter are added to the URL:

_t_id: Track id.
_t_q: Search query.
_t_tags: Used to categorize statistics for example in a multi-language or multi-site environment.
_t_ip: IP address from the visitor.
_t_hit.id: Result hit id.
_t_hit.pos: The position of the result hit in the returned list.

Find use the querystring parameters to track the click behavior of the visitors. In the layout (or masterpage) file the following line of code needs to be added. This makes sure that EPiServer can inject Javascript. Find use Javascript to track the click behavior of the visitor.

@Html.RequiredClientResources(RenderingTags.Footer)

Beside unified search it is also possible to track the statistics when a typed search is done. More information about this can be found in a blog of Henrik Fransas.

When tracking statistics it is possible to build functionality based on those statistics. I will cover that in the next section.

Statistics - Autocomplete

With the autocomplete functionality, it is possible to provide suggestions to the visitor. There are two types of suggestions: editorial and statistical. Suggestions of the type editorial are added manually in the Find interface. These suggestions are always ordered on top of the list. Statistical suggestions are based on search queries done by previous visitors that actually returned some results. This functionality can be implemented with the .NET client APi.

client.Statistics().GetAutocomplete(query);

Or with Javascript by calling the URL:

/find_v2/_autocomplete?prefix=' + query + '&size=5

Statistics - Did you mean

The did you mean functionality also provides suggestion to the visitor. The same as autocomplete functionality suggestions are based on statistics or items that have been added manually in the Find interface. In the Find interface items can be added by specifying the search query and a ‘related query’ which will be provided as a suggestion. What happened with the statistical suggestion is that a previous visitor also searched on the same word and immediately searched for something else, that makes it a related search query.

Use .NET client APi.

client.Statistics().GetDidYouMean(query);

Or with Javascript by using this URL:

/find_v2/_didyoumean?query=' + query + '&size=5

Statistics - Spellcheck

The spellcheck functionality provides suggestions when a visitor makes a typo in the search query. Suggestions are provided based on statistical data. The following JSON is returned by Find when using the spellcheck functionality.

{
        "hits": [
            {
                "suggestion": "group",
                "type": "statistical",
                "distance": 0.8
            }
        ],
        "status": "ok"
    }

As you can see the distance field indicates how close the suggestion is from the actual search query.

Use the .NET client APi:

client.Statistics().GetSpellcheck(query);

Or with Javascript by using this URL:

/find_v2/_spellcheck?query=' + query + '&size=5

Lab

Where I focussed on the topics that I just discussed, Mari covered indexing, filters and facets in her presentation. For the lab part we created some exercises about filtering. The exercises were based on the hotel Find index from Allan Thraen.

Filter on price –> Get all hotels that have price between 100 and 200.
Filter on rating/review –> Find all hotels that have a star rating of either 4 or 5, OR review rate of 9 or 10 with more than 50 reviews.
Filter on location –> Get all hotels within 5 km of the cosmopolitan hotel, order by distance from cosmopolitan hotel (closest first).
Advanced filtering –> Show hotels with more than 2 stars within 10 km of the cosmopolitan hotel that offer room service, have air condition and have chain specified by user input* (*using Console.ReadLine()).
Range facets –> Create range facets for price ranges 20-50, 51-100 and 101-150 USD.
Facet for Country –> List name of all countries that have hotels.
Basic text search query entered by user, in fields Name and Description.

If you are interested in the implementation of the exercises you can find them on GitHub. The demo website I used for my part of the presentation is also available on GitHub.

You can view the presentation slides on my SlideShare account. Mari’s blog will follow soon!