Solutions Playbook

Appearance

Meilisearch ​

Intro ​

Technologies:

Implementation:

  • Launched a dedicated Meilisearch instance on AWS using the community build by Meilisearch. Being hosted at https://meilisearch.campus.salthq.co/ on an elastic IP
  • Launched a dev server which will allow for index testing https://dev.meilisearch.salthq.co/
  • (For now at least) The above server is going to be the central search driver for ALL Campuses (and possibly other Salt platforms). They are differentiated by a ENV defined SCOUT_PREFIX

What this enables:

  • Any Laravel models can be defined to use Searchable; trait, this means we can use the ::search() method on eloquent queries and power a really powerful full text search 🚀
  • One of the main differences is that we can add searching to models based on custom meta: eg: (see pic) We can search users based on their role name, or group names etc. This was NOT possible before. This feature will be hugely powerful with analytics and progress insights etc.
  • It allows fuzzy search (searching with typos)

Platform configuration ​

The Meilisearch driver is configured in the .env file with the following variables

SCOUT_DRIVER=meilisearch
SCOUT_PREFIX={unique prefix if the meilisearch server is shared}
MEILISEARCH_HOST={meilisearch server}
MEILISEARCH_KEY={key}

Rules for defining the Scout_Prefix ​

We will follow a set format for all Scout Prefixes. Scout Prefixes need to be critically unique across all platforms Production, Staging, Local Dev etc.

Formatting ​

  • Partner Code - 3 Char lowercase Abbreviations e.g. Advantage => alt, Raizcorp => rzp
  • Product Code - 3 Char lowercase Abbreviations e.g. Engauge Hub => enh
  • Environment codes:
    • Production - prd
    • Staging - stg
    • Local Development - dev
  • IF env is you local dev set additionally your name abbreviation - 3 Char lowercase Abbreviations e.g. Mike -> mik

Hosted Environments

{Partner Code}_{Product Code}_{Environment Code}

Local Development Environments

{Partner Code}_{Product Code}_{Environment Code}_{Dev Name Abbrv}

Importing indexes to Meilisearch ​

To import indexes to Meilisearch you can use the artisan command php artisan scout:import 'App\Model'. When run, this will upsert the index to Meilisearch.

In scout.php config if the queue is set to true 'queue' => env('SCOUT_QUEUE', true), the import will only happen if the queue driver is running.

This command can be run in a migration:

    public function up()
    {
        Artisan::call("scout:import 'App\\\User'");
    }

    public function down()
    {
        Artisan::call("scout:flush 'App\\\User'");
    }

Setting up a model to be Searchable ​

To make a model searchable you need to add the Scout use Searchable; trait.

Once added a model can use the ::search() method on eloquent queries. You can use eloquent paginate() & get() after the search() method.

Eg: User::search($search_term)->paginate();

Customizing the searcable array ​

By default all the Model attributes are searchable. One of the powerful features of Meilisearch is the ability to customize the searcable array. You can do this by using the toSearchableArray() method in the model class.

Eg, adding user groups to the user searchable array to allow you to search users by their user group names:

    public function toSearchableArray()
    {
        $array = $this->toArray();
 
        $groups = $this->groups()->pluck('name');
        $array['user_groups'] = $groups;

        return $array;
    }

| NB: Everytime you change the searchable array you need to update the Meilisearch index using a migration or calling the artisan command.

Meilisearch Service class ​

A custom MeilisearchService.php class has been added to allow for advanced functionality. The functionality includes:

  • Searchable attributes
  • Sortable attributes
  • Filterable attributes
  • Typo tolerance settings

Searchable attributes ​

By default all the index attributes are searchable. If you want to customize the searchable attributes you can specify them in a getSearchableAttributes() method in the model.

public function getSearchableAttributes() {
    return [
        'name',
        'email'
    ];
}

Sortable attributes ​

By default none of the index attributes are sortable, and the results will be sorted by 'id'. To add sortable attributes you can specify them in a getSortableAttributes() method in the model.

public function getSortableAttributes() {
    return [
        'name',
        'email'
    ];
}

Once added you can sort a search by adding the orderBy() method onto a Meilisearch query. Eg: User::search($search_term)->orderBy('name','asc')->paginate();

Filterable attributes ​

By default none of the index attributes are filterable. To add filterable attributes you can specify them in a getFilterableAttributes() method in the model.

public function getFilterableAttributes() {
    return [
        'last_login_at',
    ];
}

Once added you can filter a search by adding the where() method onto a Meilisearch query. Eg: User::search($search_term)->where('last_login_at', '>=', Carbon::now()->subDays('7'))->paginate();

Type tolerance ​

Meilisearch handles small typo errors in the search query. By default, Meilisearch accepts one typo for query terms containing five or more characters, and up to two typos if the term is at least nine characters long. Typo tolerance

You can define this per index using the getTypoToleranceSettings() method:

public function getTypoToleranceSettings() {
        return [
            'minWordSizeForTypos' => [
                'oneTypo' => 3,
                'twoTypos' => 5
            ]
        ];
}

Updating attributed in the MeilisearchService class ​

Once a setting changes in an index the MeilisearchService class needs to be called to update for a specific index. This is handled in the ProductionSeeder class that is run on every production deployment.

 // Update any changes on the meilisearch functions
$meilisearch_service = app()->make(MeilisearchService::class);
$meilisearch_service->updateIndexSettings(new User());

Sort builder macro ​

To enable an easier Implementation of sort for a Meilisearch query, a sort macro has been added to a ScoutMacroServiceProvider class. This macro checks for sort request attributes and adds the orderBy() method to the Meilisearch query builder.

 Builder::macro('sort', function () {
          
    $sort_column = isset($_GET['column']) && !empty($_GET['column']) ? $_GET['column'] : null;
    $sort_direction = isset($_GET['direction']) ? $_GET['direction'] : 'asc';

    if ($sort_column) {
        return $this->orderBy($sort_column, $sort_direction);
    }
    return $this;
});

Now dynamic sorting can be easily added to any Meilisearchquery User::search($term)->sort()->paginate();

Using eloquent with Meilisearch ​

Often you may find you want to use eloquent alongside Meilisearch, eg: adding a relationship eager load. The search() builder method cannot be used on an eloquent collection and neither visa versa. However, you can still use Eloquent alongside Meilisearch using a query() method:

User::search($term)->query( function( $query ) use ($role,$group) {
    // Eloquent queries here
    $query->with('roles')
    ->ofRole($role)
    ->ofGroup($group)
    ->perPagePaginate();
} )->sort()->paginate();

| NB: Some eloquent methods do not work, eg: orderBy(), so always test before taking to production.