Insert multiple rows at once with Ecto. "protocol Enumerable not implemented for #Ecto.Changeset"

Question

I have two tables. A table of topics which has_many tweets. My table of tweets belongs_to a topic.

Topic Schema:

defmodule Sentiment.Topic do
  use Sentiment.Web, :model

  schema "topics" do
    field :title, :string

    has_many :tweets, Sentiment.Tweet
  end

  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:title])
    |> validate_required([:title])
  end
end

Tweet Schema:

defmodule Sentiment.Tweet do
  use Sentiment.Web, :model

  schema "tweets" do
    field :message, :string
    belongs_to :topic, Sentiment.Topic

  end

  @doc """
  Builds a changeset based on the `struct` and `params`.
  """
  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:message])
    |> validate_required([:message])
  end
end

I am attempting to insert a topic into my table, followed by 500 tweets after I run a twitter search for that topic.

In my controller, I use Ecto.Multi to group my repo operations, however, each time I run my operation I get an error protocol Enumerable not implemented for #Ecto.Changeset<action: nil, changes: %{message: "\"aloh....

This is how I am attempting to insert my topic first, obtain it's id, and then insert a tweet message with the associated id with one transaction.

 def create(conn, %{"topic" => topic}) do
    # create a topic changeset
    topic_changeset = Topic.changeset(%Topic{}, topic)

    # obtain a list of tweet messages: ["hello", "a tweet", "sup!"]
    %{"title" => title} = topic
    all_tweets = title
    |> Twitter.search

# create an Ecto.Multi struct.
multi =
  Ecto.Multi.new
  |> Ecto.Multi.insert(:topics, topic_changeset) #insert topic
  |> Ecto.Multi.run(:tweets, fn %{topics: topic} ->
    changeset_tweets = all_tweets
    |> Enum.map(fn(tweet) ->
      %{topic_id: topic.id, message: tweet}
    end)

    Repo.insert_all(Tweet, changeset_tweets)

  end)

      # Run the transaction
      case Repo.transaction(multi) do # ERROR HERE!
        {:ok, result} ->
          conn
          |> put_flash(:info, "Success!")
          |> redirect(to: topic_path(conn, :index))
        {:error, :topics, topic_changeset, %{}} ->
          conn
          |> put_flash(:error, "Uh oh...")
          |> render("new.html", changeset: topic_changeset)
        {:error, :tweets, topic_changeset, %{}} ->
          conn
          |> put_flash(:error, "Something really bad happened...")
          |>render("new.html", changeset: topic_changeset)
      end
  end

How can I insert_all about 500 rows in one transaction using Ecto.Multi?

Update I have converted the list of changesets into a list of maps and my error has changed to something even more confusing.

Water

Can you post the complete error message including the stacktrace? The error should be in the line containing Repo.insert_all since it only supports list of maps and lists of keyword lists, not list of changesets. – Medallist

@Medallist I posted an image of the error for you. – Water

As @Medallist said, you are trying to pass in a list of changesets to Repo.insert_all. however, that function does not accept a list of changesets. It only takes a list of maps or a list of keyword lists. – Condescending

@JustinWood I have created a list of maps, each map containing two properties. topic_id and message. When I attempt to run the transaction, my error is no case clause matching: {5, nil} – Water

Have you considered to use cast_assoc(:tweets) in the topic changeset, instead of performing the same thing manually? – Isosteric

Answer 1

For Ecto.Multi to properly progress with steps, every one of them has to return either {:ok, value} or {:error, reason} tuple.

When inserting, updateing or deleteing a changeset, it will return such a tuple automatically, but for run, you need to return it explicitly.

Please, consider the following:

Ecto.Multi.new
|> Ecto.Multi.insert(:topics, topic_changeset) #insert topic
|> Ecto.Multi.run(:tweets, fn %{topics: topic} ->
   maps = 
     Enum.map(all_tweets, fn(tweet) ->
       %{topic_id: topic.id, message: tweet}
     end)

    {count, _} = Repo.insert_all(Tweet, maps)
    {:ok, count} # <----
end)

Period

Thank you for the clear explanation! This helped me a lot. I'm new to phoenix :) – Water

Ecto.Multi can be used in any Elixir project. This doesn't have to be Phoenix. Just saying! Glad you found it useful! – Gretta

I'll be sure to read the docs more carefully next time. Thank you! – Water

Answer 2

Alex, this is not a direct answer to your question with Ecto.Multi, but a suggestion, that it might be easier to use cast_assoc(:tweets) inside your topic changeset.

This would look like this:

# Topic.ex
...
def changeset(struct, params \\ %{}) do
  struct
  |> cast(params, [:message])
  |> cast_assoc(:tweets) 
  |> validate_required([:message])
end

# create_topic..
...
tweets = [%{message: "Tweet 1"}, %{message: "Tweet 2"}]

{:ok, topic} = 
  %Topic{}
  |> Topic.changeset(Map.put(topic, :tweets, tweets))
  |> Repo.insert()

Isosteric

I tried this but get a changeset error saying I'm missing the foreign_key ie. topic_id. – Propensity

Answer 3

Note that as of now (2021) Ecto 3 provides the function Multi.insert_all already, so that one wouldn't need to use Multi.run manually in this scenario.

Demagogic