Google App Engine: Is it possible to do a Gql LIKE query?

Question

Simple one really. In SQL, if I want to search a text field for a couple of characters, I can do:

SELECT blah FROM blah WHERE blah LIKE '%text%'

The documentation for App Engine makes no mention of how to achieve this, but surely it's a common enough problem?

Unguiculate

The continued problem revolves around people trying to use GAE Datastore as if it were a Relational/~SQL database. By Google introducing GQL, it further leads people into thinking in terms of SQL systems. However I understand that Google is trying to make the transition for everyone much easier even though I'm not sure this is the right approach. – Ker

Answer 1

BigTable, which is the database back end for App Engine, will scale to millions of records. Due to this, App Engine will not allow you to do any query that will result in a table scan, as performance would be dreadful for a well populated table.

In other words, every query must use an index. This is why you can only do =, > and < queries. (In fact you can also do != but the API does this using a a combination of > and < queries.) This is also why the development environment monitors all the queries you do and automatically adds any missing indexes to your index.yaml file.

There is no way to index for a LIKE query so it's simply not available.

Have a watch of this Google IO session for a much better and more detailed explanation of this.

Eleanoreleanora

Answer 2

i'm facing the same problem, but i found something on google app engine pages:

Tip: Query filters do not have an explicit way to match just part of a string value, but you can fake a prefix match using inequality filters:

db.GqlQuery("SELECT * FROM MyModel WHERE prop >= :1 AND prop < :2",
            "abc",
            u"abc" + u"\ufffd")

This matches every MyModel entity with a string property prop that begins with the characters abc. The unicode string u"\ufffd" represents the largest possible Unicode character. When the property values are sorted in an index, the values that fall in this range are all of the values that begin with the given prefix.

http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html

maybe this could do the trick ;)

Chord

+1 Tho it's worth pointing out this is case-sensitive. Luckily the data in the field I'm querying is converted to lowercase prior to storing. – Admissive

Answer 3

Altough App Engine does not support LIKE queries, have a look at the properties ListProperty and StringListProperty. When an equality test is done on these properties, the test will actually be applied on all list members, e.g., list_property = value tests if the value appears anywhere in the list.

Sometimes this feature might be used as a workaround to the lack of LIKE queries. For instance, it makes it possible to do simple text search, as described on this post.

Bacteroid

the post doesn't exist anymore – Lombardi

Answer 4

You need to use search service to perform full text search queries similar to SQL LIKE.

Gaelyk provides domain specific language to perform more user friendly search queries. For example following snippet will find first ten books sorted from the latest ones with title containing fern and the genre exactly matching thriller:

def documents = search.search {
    select all from books
    sort desc by published, SearchApiLimits.MINIMUM_DATE_VALUE
    where title =~ 'fern'
    and genre =  'thriller'
    limit 10
}

Like is written as Groovy's match operator =~. It supports functions such as distance(geopoint(lat, lon), location) as well.

Cowherd

Answer 5

App engine launched a general-purpose full text search service in version 1.7.0 that supports the datastore.

Details in the announcement.

More information on how to use this: https://cloud.google.com/appengine/training/fts_intro/lesson2

Ironhanded

Answer 6

Have a look at Objectify here , it is like a Datastore access API. There is a FAQ with this question specifically, here is the answer

How do I do a like query (LIKE "foo%")
You can do something like a startWith, or endWith if you reverse the order when stored and searched. You do a range query with the starting value you want, and a value just above the one you want.

String start = "foo";
    ... = ofy.query(MyEntity.class).filter("field >=", start).filter("field <", start + "\uFFFD");

Glomma

it will search "starts with" not "Contains". – Charbonnier

Answer 7

Just follow here: init.py#354">http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/ext/search/init.py#354

It works!

class Article(search.SearchableModel):
    text = db.TextProperty()
    ...

  article = Article(text=...)
  article.save()

To search the full text index, use the SearchableModel.all() method to get an
instance of SearchableModel.Query, which subclasses db.Query. Use its search()
method to provide a search query, in addition to any other filters or sort
orders, e.g.:

  query = article.all().search('a search query').filter(...).order(...)

Goltz

Answer 8

I tested this with GAE Datastore low-level Java API. Me and works perfectly

    Query q = new Query(Directorio.class.getSimpleName());

    Filter filterNombreGreater = new FilterPredicate("nombre", FilterOperator.GREATER_THAN_OR_EQUAL, query);
    Filter filterNombreLess = new FilterPredicate("nombre", FilterOperator.LESS_THAN, query+"\uFFFD");
    Filter filterNombre =  CompositeFilterOperator.and(filterNombreGreater, filterNombreLess);

    q.setFilter(filter);

Unorthodox

this works for prefix, but what if I want to match from the end of the string? For e.g - i want to search abc in sdfdsabc, then it should return sdfdsabc – Jaqitsch

Answer 9

In general, even though this is an old post, a way to produce a 'LIKE' or 'ILIKE' is to gather all results from a '>=' query, then loop results in python (or Java) for elements containing what you're looking for.

Let's say you want to filter users given a q='luigi'

users = []
qry = self.user_model.query(ndb.OR(self.user_model.name >= q.lower(),self.user_model.email >= q.lower(),self.user_model.username >= q.lower()))

for _qry in qry:
 if q.lower() in _qry.name.lower() or q.lower() in _qry.email.lower() or q.lower() in _qry.username.lower():
      users.append(_qry)

Fraunhofer

Answer 10

It is not possible to do a LIKE search on datastore app engine, how ever creating an Arraylist would do the trick if you need to search a word in a string.

@Index
    public ArrayList<String> searchName;

and then to search in the index using objectify.

List<Profiles> list1 = ofy().load().type(Profiles.class).filter("searchName =",search).list();

and this will give you a list with all the items that contain the world you did on the search

Inhibitory

Answer 11

If the LIKE '%text%' always compares to a word or a few (think permutations) and your data changes slowly (slowly means that it's not prohibitively expensive - both price-wise and performance-wise - to create and updates indexes) then Relation Index Entity (RIE) may be the answer.

Yes, you will have to build additional datastore entity and populate it appropriately. Yes, there are some constraints that you will have to play around (one is 5000 limit on the length of list property in GAE datastore). But the resulting searches are lightning fast.

For details see my RIE with Java and Ojbectify and RIE with Python posts.

Immortality

Answer 12

"Like" is often uses as a poor-man's substitute for text search. For text search, it is possible to use Whoosh-AppEngine.

Lindseylindsley