Difference between com.google.datastore.v1 and com.google.cloud.datastore / Missing option to disable index

Question

I'm currently building a Google Cloud Dataflow job which parses XML files and saves the entries using Google Datastore, bu the different Java libraries seem to be very confusing.

First I found com.google.datastore.v1, which works great with Dataflow, but later on I realized that the option to excludes fields from being indexed is missing. (Most of my fields do not need an index and will never be used in a query.)

Then I found com.google.cloud.datastore, which has a method named "setExcludeFromIndexes" to achieve exactly what I was looking for, but Dataflow is not able to save entities generated using this library.

Is one of the libraries the newer one or what is the difference at all? And is there a way to disable indexes for single fields using the v1-library?

Malapropism

Answer 1

v1-library is a thin layer that provides generated proto sources and some helper functions.
google-cloud-datastore is a wrapper that uses v1-library aimed to be more user friendly.
To allow different Datastore client libraries to interface with Dataflow, we use the v1 library as it is the least common denominator.

Ideally you would use the google-cloud-datastore library, but depending on the versions, v1-library and google-cloud-datastore might have conflicting dependencies (protobuf 3.0.0 v 3.0.0-beta-1 in particular) in which case you either have to pick the compatible versions or use v1-library directly.

The v1-library helper functions might not be enough to exclude fields from indexing but you could always create your own helpers by dealing with proto messages directly. In this particular case, you would be creating your own Value and setting exclude_from_indexes explicitly.

Caryncaryo

Answer 2

These two libraries are not currently compatible with each other (you cannot convert between the two data representations), but you can exclude values from indexes in com.google.datastore.v1:

Value value = Value.newBuilder()
    .setStringValue("foo")
    .setExcludeFromIndexes(true)
    .build();

Ricardo

Answer 3

I have faced the same Issue. I have updated the Datastore version in pom.xml to the latest version and Issue has been resolved.

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-datastore</artifactId>
  <version>1.106.0</version>
</dependency>

Bradstreet