Relation passed to #or must be structurally compatible. Incompatible values: [:references]

Question

I have two queries, I need an or between them, i.e. I want results that are returned by either the first or the second query.

First query is a simple where() which gets all available items.

@items = @items.where(available: true)

Second includes a join() and gives the current user's items.

@items =
  @items
  .joins(:orders)
  .where(orders: { user_id: current_user.id})

I tried to combine these with Rails' or() method in various forms, including:

@items =
  @items
  .joins(:orders)
  .where(orders: { user_id: current_user.id})
  .or(
    @items
    .joins(:orders)
    .where(available: true)
  )

But I keep running into this error and I'm not sure how to fix it.

Relation passed to #or must be structurally compatible. Incompatible values: [:references]

Mudcat

Answer 1

There is a known issue about it on Github.

According to this comment you might want to override the structurally_incompatible_values_for_or to overcome the issue:

def structurally_incompatible_values_for_or(other)
  Relation::SINGLE_VALUE_METHODS.reject { |m| send("#{m}_value") == other.send("#{m}_value") } +
    (Relation::MULTI_VALUE_METHODS - [:eager_load, :references, :extending]).reject { |m| send("#{m}_values") == other.send("#{m}_values") } +
    (Relation::CLAUSE_METHODS - [:having, :where]).reject { |m| send("#{m}_clause") == other.send("#{m}_clause") }
end

Also there is always an option to use SQL:

@items
  .joins(:orders)
  .where("orders.user_id = ? OR items.available = true", current_user.id)

Racing

Went with SQL queries instead. Thanks for the info. – Mudcat

@Andrey Should have referred to my answer instead of editing yours and adding the same. Anyway, +1 for the first solution. – Holston

@RSB i was going to start with sql part, than googled error message and found a github issue. After all I still decided to add my initial thought, since it's the less code to make it work. – Racing

@AndreyDeineko Where to add structurally_incompatible_values_for_or method in rails application? – Alpenglow

Answer 2

Hacky workaround: do all your .joins after the .or. This hides the offending .joins from the checker. That is, convert the code in the original question to...

@items =
  @items
  .where(orders: { user_id: current_user.id})
  .or(
    @items
    .where(available: true)
  )
  .joins(:orders) # sneaky, but works! 😈

More generally, the following two lines will both fail

A.joins(:b).where(bs: b_query).or(A.where(query))  # error! 😞 

A.where(query).or(A.joins(:b).where(bs: b_query))  # error! 😞 

but rearrange as follows, and you can evade the checker:

A.where(query).or(A.where(bs: b_query)).joins(:b)  # works 😈 

This works because all the checking happens inside the .or() method. It's blissfully unaware of shennanigans on its downstream results.

One downside of course is it doesn't read as nicely.

Maryjanemaryjo

This fixed the issue for me. This really should be the answer. – Frigorific

Fixed for me to. Thanks ! joins and includes must be added after or. Can someone explain why, if this works, rails does not accept us to add it before 'or' ? – Fullblown

@Fullblown for joins, I don't think any good reason. In fact, after rails 6.1.3 I think you you can put the .joins before the .or, and I know for fact you can put .includes before the .or. As for why this exists in the first place... Rails has to restrict certain operations like .limit or .distinct, as allowing these before .or would be untranslatable to sql. (What would it mean to .or together two clauses with different .limits?) Longer explanation of all cases here: github.com/rails/rails/issues/24055#issuecomment-793274937 – Maryjanemaryjo

@Maryjanemaryjo thanks for these explainations. I will upgrade my Rails 6.0 application to 6.1.3 soon ! – Fullblown

Would have been interesting to know what the generated SQL looked like for your answer, because more than working it's important it does the right thing. – Spinal

@NunoCosta Sure thing! here's a (partially redacted) test I just ran at the console: puts A.where(id: 1).or(A.where(bs: { id: 1 })).joins(:b).to_sql yields SELECT "as".* FROM "as" INNER JOIN "bs" ON "bs"."id" = "as"."b_id" WHERE ("as"."id" = 1 OR "bs"."id" = 1). This looks like good SQL to me. This did reveal an error in pluralization in my original examples. I'll update that now. – Maryjanemaryjo

Answer 3

You can write the query in this good old way to avoid error

@items = @items.joins(:orders).where("items.available = ? OR orders.user_id = ?", true, current_user.id)

Hope that helps!

Holston

Answer 4

I ran into the same issue, however the code was defined in a different place and was very difficult to change directly.

# I can't change "p"
p = Post.where('1 = 1').distinct # this could also be a join

And I needed to add an or statement to it

p.or(Post.where('2 = 2'))

The following code won't raise an error, because it has distinct like the initial relationship.

p.or(Post.where('2 = 2').distinct)

The problem with it it that it only works as long as you know the relationship. It may or not have a join, or distinct.

This works regardless of what the relationship is:

p.or(p.unscope(:where).where('2 = 2'))
=> SELECT DISTINCT `posts`.* FROM `posts` WHERE ((1 = 1) OR (2 = 2))

Carbolize

Answer 5

It occurs when you try to combine two multi-active records of the same type, but one of them has a joins value or an includes value, or in your case a reference value, that the other does not. Therefore we need to match the values between them, and I found a general way to do this without knowing the actual values in advance.

items_1 = @items.joins(:orders)
                .where(orders: { user_id: current_user.id})

items_2 = @items.where(available: true)
                .joins(items_1.joins_values)
                .includes(items_1.includes_values)
                .references(items_1.references_values)

@items = items_1.or(items_2)

Breadfruit

Answer 6

just solve it!

  def exec_or_statement(q1, q2)
    klass = q1.klass
    key = klass.primary_key

    query_wrapper_1 = {}
    query_wrapper_1[key] = q1

    query_wrapper_2 = {}
    query_wrapper_2[key] = q2

    klass.where(query_wrapper_1).or(klass.where(query_wrapper_2))
  end

  query_1 = @items.where(available: true)

  query_2 =
    @items
    .joins(:orders)
    .where(orders: { user_id: current_user.id})
  
  exec_or_statement(query_1, query_2)

Eccrine