Elasticsearch and Laravel scout-elasticsearch-driver return an empty response

Question

First i use scout-elasticsearch-driver for Laravel Scout: https://github.com/babenkoivan/scout-elasticsearch-driver

I followed step by step the readme, create the index, migrate the index, configure a mapping and User::search()->get() returns an empty array.

Obviously my database is migrated and populated.

I would like to search a user by:

• His first_name
• His last_name
• His nick_name

So i have created an IndexConfigurator:

class UserIndexConfigurator extends IndexConfigurator
{
   use Migratable;

   /**
   * @var array
   */
   protected $settings = [
     //
   ];
}

Created a SearchRule:

class UserSearchRule extends SearchRule
{
   public function buildQueryPayload()
   {
       $query = $this->builder->query;

       return [
          'should' => [
              [
                'match' => [
                   'first_name' => [
                              'query' => $query
                          ]
                      ]
                  ],
                  [
                      'match' => [
                          'last_name' => [
                              'query' => $query
                          ]
                      ]
                  ],
                  [
                      'match' => [
                          'nick_name' => [
                              'query' => $query
                          ]
                      ]
                  ]
              ]
          ];
   }
}

Configured my User Model accordingly:

<?php
class User extends Authenticatable

  {
    useSearchable;
    /**
     * @var string
     */
    protected $indexConfigurator = UserIndexConfigurator::class;

    /**
     * @var array
     */
    protected $searchRules = [UserSearchRule::class ];

    /**
     * @var array
     */
    protected $mapping = ['properties' => ['first_name' => ['type' => 'text'], 'last_name' => ['type' => 'text'], 'nick_name' => ['type' => 'text'], ]];
    /**
     * The attributes that are mass assignable.
     *
     * @var array
     */
    protected $fillable = ['id', 'first_name', 'last_name', 'nick_name', ];
  }

Do i missed something?

EDIT 1: Result of curl request to elasticsearch cluster

curl -XGET "http://localhost:9200/user/_search?pretty=true&q=*:*"

EDIT 2:

curl -XGET "http://localhost:9200/user?pretty=true"

Print of the query that this produced by:

dd(User::search('lo')->explain());

EDIT 3 Laravel scout and elasticsearch driver configuration:

scout_elastic.php

<?php declare(strict_types = 1);

return [
    'client' => [
        'hosts' => [
            env('SCOUT_ELASTIC_HOST', 'localhost:9200'),
        ],
    ],
    'update_mapping' => env('SCOUT_ELASTIC_UPDATE_MAPPING', true),
    'indexer' => env('SCOUT_ELASTIC_INDEXER', 'single'),
    'document_refresh' => env('SCOUT_ELASTIC_DOCUMENT_REFRESH', 'wait_for'),
];

scout.php

<?php declare(strict_types = 1);

return [

    /*
    |--------------------------------------------------------------------------
    | Default Search Engine
    |--------------------------------------------------------------------------
    |
    | This option controls the default search connection that gets used while
    | using Laravel Scout. This connection is used when syncing all models
    | to the search service. You should adjust this based on your needs.
    |
    | Supported: "algolia", "null", "elastic"
    |
    */

    'driver' => env('SCOUT_DRIVER', 'elastic'),

    /*
    |--------------------------------------------------------------------------
    | Index Prefix
    |--------------------------------------------------------------------------
    |
    | Here you may specify a prefix that will be applied to all search index
    | names used by Scout. This prefix may be useful if you have multiple
    | "tenants" or applications sharing the same search infrastructure.
    |
    */

    'prefix' => env('SCOUT_PREFIX', ''),

    /*
    |--------------------------------------------------------------------------
    | Queue Data Syncing
    |--------------------------------------------------------------------------
    |
    | This option allows you to control if the operations that sync your data
    | with your search engines are queued. When this is set to "true" then
    | all automatic data syncing will get queued for better performance.
    |
    */

    'queue' => env('SCOUT_QUEUE', true),

    /*
    |--------------------------------------------------------------------------
    | Chunk Sizes
    |--------------------------------------------------------------------------
    |
    | These options allow you to control the maximum chunk size when you are
    | mass importing data into the search engine. This allows you to fine
    | tune each of these chunk sizes based on the power of the servers.
    |
    */

    'chunk' => [
        'searchable' => 500,
        'unsearchable' => 500,
    ],

    /*
    |--------------------------------------------------------------------------
    | Soft Deletes
    |--------------------------------------------------------------------------
    |
    | This option allows to control whether to keep soft deleted records in
    | the search indexes. Maintaining soft deleted records can be useful
    | if your application still needs to search for the records later.
    |
    */

    'soft_delete' => false,

    /*
    |--------------------------------------------------------------------------
    | Algolia Configuration
    |--------------------------------------------------------------------------
    |
    | Here you may configure your Algolia settings. Algolia is a cloud hosted
    | search engine which works great with Scout out of the box. Just plug
    | in your application ID and admin API key to get started searching.
    |
    */

    'algolia' => [
        'id' => env('ALGOLIA_APP_ID', ''),
        'secret' => env('ALGOLIA_SECRET', ''),
    ],

];

EDIT 4: Updated elasticsearch mapping

php artisan elastic:update-mapping App\\Models\\User

Gives:

{
  "user" : {
    "aliases" : {
      "user_write" : { }
    },
    "mappings" : {
      "users" : {
        "properties" : {
          "first_name" : {
            "type" : "text",
            "analyzer" : "standard"
          },
          "id" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "last_name" : {
            "type" : "text",
            "analyzer" : "standard"
          },
          "nick_name" : {
            "type" : "text",
            "analyzer" : "standard"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1552675390883",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "hdtsA8ncQNC8rrI5Tr853A",
        "version" : {
          "created" : "6050499"
        },
        "provided_name" : "user"
      }
    }
  }
}

Gina

You need to provide the mapping for your index. – Garrygarson

@ArchitSaxena thanks for reply, but protected $mapping in my User model it is not the mapping of my index? – Gina

Oh i'm sorry i must have gleamed over it. It looks good to me. Can you print the elastic DSL query that gets generated through your code. Your mapping in the code looks good, but just to make sure that the actual index you are querying has the same mapping, can you post the response of curl -XGET host:9200/user_index? – Garrygarson

@ArchitSaxena thanks for reply, it's ok, see my edit :) – Gina

Hey, this doesn't tell me much. Try curl -XGET host:9200/user This should give you the actual mapping of the index. What you added is the result of search. Also, can you print the query that this produces User::search()->get()? – Garrygarson

@ArchitSaxena thanks again for helping me, seed my second edit – Gina

I think there's some problem with how you've setup scout. On going through the documentation it seems that scout ships with Algolia driver by default. I am assuming you added some driver for elasticsearch? The query that you printed doesn't look like an elasticsearch DSL query (only the hits._payload.body part is relevant). Also, i don't understand why curl -XGET "http://localhost:9200/user?pretty=true" returned mappings: {}. All of it points to the fact that there's some problem with how scout is configured because it can't seem to communicate with elasticsearch. – Garrygarson

You'll need to post what engine and it's configuration (if any) that you added, specific to elasticsearch. I haven't used scout or Laravel before, but i can take a look. – Garrygarson

i have updated my post with the config files. – Gina

@ArchitSaxena by running php artisan elastic:update-mapping App\\Models\\User the mapping was updated, see my fourth update. – Gina

Answer 1

Solution was to update the UserSearchRule:

<?php declare(strict_types = 1);

namespace App\Models\SearchRules;

use ScoutElastic\SearchRule;

/**
 * Class UserSearchRule
 *
 * @package App\Models\SearchRules
 */
class UserSearchRule extends SearchRule
{

    /**
     * @inheritdoc
     */
    public function buildQueryPayload()
    {
        $query = $this->builder->query;

        return [
            'must'  => [
                'multi_match'   => [
                    'query'     => $query,
                    'fields'    => ['first_name', 'last_name', 'nick_name'],
                    'type'      => 'phrase_prefix',
                ],
            ],
        ];
    }
}

Gina

it's works. Great. Thank you so much. – Effy

Answer 2

I tried the answer but doesn't work on my end.

If the answer doesnt work, we can do this manually and get the hit results.

   $query = request()->has('q') ? request('q') : '*';
   $results = Product::search($query)->raw()['hits'];
   $products = collect($results['hits'])->pluck('_source');

Blastoff