Polymorphic embedded structs

Question

I'm wanting to store a tree structure in Postgres, and I'm hoping to embed an arbitrary Elixir struct on each node of the tree, something like this:

defmodule Node do
  use Ecto.Schema

  schema "nodes" do
    belongs_to :parent_node, Node
    embeds_one :struct, ArbitraryDataType
  end
end

However, I believe embeds_one requires a specific struct data type to be specified, which won't work for my case. Is there a way around this?

---

My backup plan is to use two fields, one for the struct type and one for the struct fields, like this:

defmodule Node do
  use Ecto.Schema

  schema "nodes" do
    belongs_to :parent_node, Node
    field :struct_type, :string
    field :fields, :map
  end
end

In order to save the record in the first place, I would need to use the __struct__ field to determine the struct type. Then, I would use logic something like the following to rebuild the original struct after retrieving the node from the database:

Enum.reduce(
  retrieved_node.fields,
  String.to_atom("Elixir.#{retrieved_node.struct_type}") |> struct,
  fn {k,v}, s -> Map.put(s, String.to_atom(k), v) end
)

Downes

Why would you want to store struct_type? Structs are basically nothing but a map with __struct__ field. I would just store it as is (as a plain old good map) and you’ll get ArbitraryDataType out of the box. Am I missing something? – Indecipherable

I agree with @mudasobwa. Maybe you want to rethink the design decision of storing a struct_type in the first place. – Resolve

That's exactly what I'm hoping to do, and there may well be a way to do it. However, when I save a struct to a json column in Postgres, Ecto does not persist the __struct__ key/value info. Therefore, when I retrieve the row from the database, I get a "regular" old map, not the struct I was hoping for — the type information is lost. – Downes

Did you ever figure out a solution? I'm also interested in using arbitrary structs and being able to persist and query and get back the same structs. I posted a comment on this thread with similar question to yours: elixirforum.com/t/… – Mutable

Not yet, though I've been too busy to work on that project again. It'll likely be sitting on the shelf awhile longer… :/ – Downes

Answer 1

I've recently solved a similar issue, and as I see it you have two options. Either you ...

Use a custom Ecto.Type

This allows you to exactly control what kind of data you want to encode into the field. By doing this you can retain the module of the struct and the fields with relative ease.

A possible implementation might look like this:

defmodule EctoStruct do
  use Ecto.Type

  def type, do: :map

  def cast(%_{} = struct), do: {:ok, struct}
  def cast(_), do: :error

  def dump(%module{} = struct) do
    data = %{
      "module" => Atom.to_string(module),
      "fields" => Map.from_struct(struct)
    }

    {:ok, data}
  end

  def load(%{"module" => module, "fields" => fields}) do
    module = String.to_existing_atom(module)
    fields = Enum.map(fields, fn {k, v} -> {String.to_existing_atom(k), v} end)

    {:ok, struct!(module, fields)}
  rescue
    _ -> :error
  end
end

With this in place you can "simply" use field :my_struct, EctoStruct in your schema.

Alternatively you ...

Reconsider your choice of database

A tree is an inherently connected data structure. Depending on your exact requirements and how deep your tree becomes, traversing said tree with Postgres can become very slow very fast.

While I solved the issue mentioned earlier I hit said performance issues quite early, having to use recursive joins and materialized views to stay close to usable response times.

Since then I switched over to a graph database (Neo4j) and my performance issues completely disappeared. This would also easily allow you to encode various differing struct types into your tree by making use of Labels.

Depending on your particular requirements this might be worth a thought.

Gramarye

Thank you for this idea, I've probably written ~5 different variants of this so far and none of them felt great since they always needed that extra step where you convert the string keys to atom keys after loading something from the database. I'm actually taking this idea a bit further by also using the new Ecto.Type.embedded_dump/embedded_load functions to dump/load the values while transforming the keys into atoms. I'll also be writing a generic cast_polymorphic_embed/4 function which I want to use instead of cast_embed. – Cling

Answer 2

The following library brings support for polymorphic embeds:

https://github.com/mathieuprog/polymorphic_embed

Minster