Adaptive Card response from a WaterfallStep Dialog MS Bot framework v4

Question

I am trying to send an adaptive card which has 2 options for user to select. When user submit the response from adaptive card I am receiving :

Newtonsoft.Json.JsonReaderException: Error reading JArray from JsonReader. Current JsonReader item is not an array: StartObject. Path ‘[‘BotAccessors.DialogState’].DialogStack.$values[0].State.options.Prompt.attachments.$values[0].content.body’.

Full code example Link : Manage a complex conversation flow with dialogs

Modification made in HotelDialogs.cs:-

public static async Task<DialogTurnResult> PresentMenuAsync(
                WaterfallStepContext stepContext,
                CancellationToken cancellationToken)
            {
                // Greet the guest and ask them to choose an option.
                await stepContext.Context.SendActivityAsync(
                    "Welcome to Contoso Hotel and Resort.",
                    cancellationToken: cancellationToken);
                //return await stepContext.PromptAsync(
                //    Inputs.Choice,
                //    new PromptOptions
                //    {
                //        Prompt = MessageFactory.Text("How may we serve you today?"),
                //        RetryPrompt = Lists.WelcomeReprompt,
                //        Choices = Lists.WelcomeChoices,
                //    },
                //    cancellationToken);

                var reply = stepContext.Context.Activity.CreateReply();
                reply.Attachments = new List<Attachment>
                {
                    new Attachment
                    {
                        Content = GetAnswerWithFeedbackSelectorCard("Choose: "),
                        ContentType = AdaptiveCard.ContentType,
                    },
                };
                return await stepContext.PromptAsync(
                    "testPrompt",
                    new PromptOptions
                    {
                        Prompt = reply,
                        RetryPrompt = Lists.WelcomeReprompt,
                    },
                    cancellationToken).ConfigureAwait(true);
            }

Note: ["testPrompt"] I tried with Text Prompt and slightly customizing the TextPrompt to read Activity Value. If Text prompt is not the appropriate prompt for adaptive card response,please let me know is there any other prompt that can be used or some custom prompt will be helpful for this kind of scenario.

Custom Prompt:-

using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Schema;

namespace HotelBot
{
    public class CustomPrompt : Prompt<string>
    {
        public CustomPrompt(string dialogId, PromptValidator<string> validator = null)
            : base(dialogId, validator)
        {
        }

        protected async override Task OnPromptAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, bool isRetry, CancellationToken cancellationToken = default(CancellationToken))
        {
            if (turnContext == null)
            {
                throw new ArgumentNullException(nameof(turnContext));
            }

            if (options == null)
            {
                throw new ArgumentNullException(nameof(options));
            }

            if (isRetry && options.RetryPrompt != null)
            {
                await turnContext.SendActivityAsync(options.RetryPrompt, cancellationToken).ConfigureAwait(false);
            }
            else if (options.Prompt != null)
            {
                await turnContext.SendActivityAsync(options.Prompt, cancellationToken).ConfigureAwait(false);
            }
        }

        protected override Task<PromptRecognizerResult<string>> OnRecognizeAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, CancellationToken cancellationToken = default(CancellationToken))
        {
            if (turnContext == null)
            {
                throw new ArgumentNullException(nameof(turnContext));
            }

            var result = new PromptRecognizerResult<string>();
            if (turnContext.Activity.Type == ActivityTypes.Message)
            {
                var message = turnContext.Activity.AsMessageActivity();
                if (!string.IsNullOrEmpty(message.Text))
                {
                    result.Succeeded = true;
                    result.Value = message.Text;
                }
                else if (message.Value != null)
                {
                    result.Succeeded = true;
                    result.Value = message.Value.ToString();
                }
            }

            return Task.FromResult(result);
        }
    }
}

Card Creation Method:-

private static AdaptiveCard GetAnswerWithFeedbackSelectorCard(string answer)
        {
            if (answer == null)
            {
                return null;
            }

            AdaptiveCard card = new AdaptiveCard();
            card.Body = new List<AdaptiveElement>();
            var choices = new List<AdaptiveChoice>()
            {
                new AdaptiveChoice()
                {
                    Title = "Reserve Table",
                    Value = "1",
                },
                new AdaptiveChoice()
                {
                    Title = "Order food",
                    Value = "0",
                },
            };
            var choiceSet = new AdaptiveChoiceSetInput()
            {
                IsMultiSelect = false,
                Choices = choices,
                Style = AdaptiveChoiceInputStyle.Expanded,
                Value = "1",
                Id = "Feedback",
            };
            var text = new AdaptiveTextBlock()
            {
                Text = answer,
                Wrap = true,
            };
            card.Body.Add(text);
            card.Body.Add(choiceSet);
            card.Actions.Add(new AdaptiveSubmitAction() { Title = "Submit" });
            return card;
        }

Thanks!

Quodlibet

It looks like you're generating the adaptive card in a method called GetAnswerWithFeedbackSelectorCard. Can we see that method? – Cockloft

@KyleDelaney Updated with the card creation method. – Quodlibet

Answer 1

After digging for some way forward I came across:

Issue#614

Thus to make adaptive card response work from Dialog, I made a compatible adaptive card prompt by one modification each in Prompt.cs and TextPrompt.cs from Microsoft bot framework.

Prompt.cs => Prompt2.cs ; TextPrompt.cs => CustomPrompt.cs

Prompt2.cs :

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Bot.Builder.Dialogs.Choices;
using Microsoft.Bot.Schema;
using Newtonsoft.Json;

namespace Microsoft.Bot.Builder.Dialogs
{
    //Reference: Prompt.cs
    /// <summary>
    /// Basic configuration options supported by all prompts.
    /// </summary>
    /// <typeparam name="T">The type of the <see cref="Prompt{T}"/>.</typeparam>
    public abstract class Prompt2<T> : Dialog
    {
        private const string PersistedOptions = "options";
        private const string PersistedState = "state";

        private readonly PromptValidator<T> _validator;

        public Prompt2(string dialogId, PromptValidator<T> validator = null)
            : base(dialogId)
        {
            _validator = validator;
        }

        public override async Task<DialogTurnResult> BeginDialogAsync(DialogContext dc, object options, CancellationToken cancellationToken = default(CancellationToken))
        {
            if (dc == null)
            {
                throw new ArgumentNullException(nameof(dc));
            }

            if (!(options is PromptOptions))
            {
                throw new ArgumentOutOfRangeException(nameof(options), "Prompt options are required for Prompt dialogs");
            }

            // Ensure prompts have input hint set
            var opt = (PromptOptions)options;
            if (opt.Prompt != null && string.IsNullOrEmpty(opt.Prompt.InputHint))
            {
                opt.Prompt.InputHint = InputHints.ExpectingInput;
            }

            if (opt.RetryPrompt != null && string.IsNullOrEmpty(opt.RetryPrompt.InputHint))
            {
                opt.RetryPrompt.InputHint = InputHints.ExpectingInput;
            }

            // Initialize prompt state
            var state = dc.ActiveDialog.State;
            state[PersistedOptions] = opt;
            state[PersistedState] = new Dictionary<string, object>();

            // Send initial prompt
            await OnPromptAsync(dc.Context, (IDictionary<string, object>)state[PersistedState], (PromptOptions)state[PersistedOptions], false, cancellationToken).ConfigureAwait(false);

            // Customization starts here for AdaptiveCard Response:
            /* Reason for removing the adaptive card attachments after prompting it to user,
             * from the stat as there is no implicit support for adaptive card attachments.
             * keeping the attachment will cause an exception : Newtonsoft.Json.JsonReaderException: Error reading JArray from JsonReader. Current JsonReader item is not an array: StartObject. Path ‘[‘BotAccessors.DialogState’].DialogStack.$values[0].State.options.Prompt.attachments.$values[0].content.body’.
             */
            var option = state[PersistedOptions] as PromptOptions;
            option.Prompt.Attachments = null;
            /* Customization ends here */

            return Dialog.EndOfTurn;
        }

        public override async Task<DialogTurnResult> ContinueDialogAsync(DialogContext dc, CancellationToken cancellationToken = default(CancellationToken))
        {
            if (dc == null)
            {
                throw new ArgumentNullException(nameof(dc));
            }

            // Don't do anything for non-message activities
            if (dc.Context.Activity.Type != ActivityTypes.Message)
            {
                return Dialog.EndOfTurn;
            }

            // Perform base recognition
            var instance = dc.ActiveDialog;
            var state = (IDictionary<string, object>)instance.State[PersistedState];
            var options = (PromptOptions)instance.State[PersistedOptions];

            var recognized = await OnRecognizeAsync(dc.Context, state, options, cancellationToken).ConfigureAwait(false);

            // Validate the return value
            var isValid = false;
            if (_validator != null)
            {
            }
            else if (recognized.Succeeded)
            {
                isValid = true;
            }

            // Return recognized value or re-prompt
            if (isValid)
            {
                return await dc.EndDialogAsync(recognized.Value).ConfigureAwait(false);
            }
            else
            {
                if (!dc.Context.Responded)
                {
                    await OnPromptAsync(dc.Context, state, options, true).ConfigureAwait(false);
                }

                return Dialog.EndOfTurn;
            }
        }

        public override async Task<DialogTurnResult> ResumeDialogAsync(DialogContext dc, DialogReason reason, object result = null, CancellationToken cancellationToken = default(CancellationToken))
        {
            // Prompts are typically leaf nodes on the stack but the dev is free to push other dialogs
            // on top of the stack which will result in the prompt receiving an unexpected call to
            // dialogResume() when the pushed on dialog ends.
            // To avoid the prompt prematurely ending we need to implement this method and
            // simply re-prompt the user.
            await RepromptDialogAsync(dc.Context, dc.ActiveDialog).ConfigureAwait(false);
            return Dialog.EndOfTurn;
        }

        public override async Task RepromptDialogAsync(ITurnContext turnContext, DialogInstance instance, CancellationToken cancellationToken = default(CancellationToken))
        {
            var state = (IDictionary<string, object>)instance.State[PersistedState];
            var options = (PromptOptions)instance.State[PersistedOptions];
            await OnPromptAsync(turnContext, state, options, false).ConfigureAwait(false);
        }

        protected abstract Task OnPromptAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, bool isRetry, CancellationToken cancellationToken = default(CancellationToken));

        protected abstract Task<PromptRecognizerResult<T>> OnRecognizeAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, CancellationToken cancellationToken = default(CancellationToken));

        protected IMessageActivity AppendChoices(IMessageActivity prompt, string channelId, IList<Choice> choices, ListStyle style, ChoiceFactoryOptions options = null, CancellationToken cancellationToken = default(CancellationToken))
        {
            // Get base prompt text (if any)
            var text = prompt != null && !string.IsNullOrEmpty(prompt.Text) ? prompt.Text : string.Empty;

            // Create temporary msg
            IMessageActivity msg;
            switch (style)
            {
                case ListStyle.Inline:
                    msg = ChoiceFactory.Inline(choices, text, null, options);
                    break;

                case ListStyle.List:
                    msg = ChoiceFactory.List(choices, text, null, options);
                    break;

                case ListStyle.SuggestedAction:
                    msg = ChoiceFactory.SuggestedAction(choices, text);
                    break;

                case ListStyle.None:
                    msg = Activity.CreateMessageActivity();
                    msg.Text = text;
                    break;

                default:
                    msg = ChoiceFactory.ForChannel(channelId, choices, text, null, options);
                    break;
            }

            // Update prompt with text and actions
            if (prompt != null)
            {
                // clone the prompt the set in the options (note ActivityEx has Properties so this is the safest mechanism)
                prompt = JsonConvert.DeserializeObject<Activity>(JsonConvert.SerializeObject(prompt));

                prompt.Text = msg.Text;
                if (msg.SuggestedActions != null && msg.SuggestedActions.Actions != null && msg.SuggestedActions.Actions.Count > 0)
                {
                    prompt.SuggestedActions = msg.SuggestedActions;
                }

                return prompt;
            }
            else
            {
                msg.InputHint = InputHints.ExpectingInput;
                return msg;
            }
        }
    }
}

CustomPrompt.cs :

using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Schema;

namespace HotelBot
{
    //Reference: TextPrompt.cs
    public class CustomPrompt : Prompt2<string>
    {
        public CustomPrompt(string dialogId, PromptValidator<string> validator = null)
            : base(dialogId, validator)
        {
        }

        protected async override Task OnPromptAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, bool isRetry, CancellationToken cancellationToken = default(CancellationToken))
        {
            if (turnContext == null)
            {
                throw new ArgumentNullException(nameof(turnContext));
            }

            if (options == null)
            {
                throw new ArgumentNullException(nameof(options));
            }

            if (isRetry && options.RetryPrompt != null)
            {
                await turnContext.SendActivityAsync(options.RetryPrompt, cancellationToken).ConfigureAwait(false);
            }
            else if (options.Prompt != null)
            {
                await turnContext.SendActivityAsync(options.Prompt, cancellationToken).ConfigureAwait(false);
            }
        }

        protected override Task<PromptRecognizerResult<string>> OnRecognizeAsync(ITurnContext turnContext, IDictionary<string, object> state, PromptOptions options, CancellationToken cancellationToken = default(CancellationToken))
        {
            if (turnContext == null)
            {
                throw new ArgumentNullException(nameof(turnContext));
            }

            var result = new PromptRecognizerResult<string>();
            if (turnContext.Activity.Type == ActivityTypes.Message)
            {
                var message = turnContext.Activity.AsMessageActivity();
                if (!string.IsNullOrEmpty(message.Text))
                {
                    result.Succeeded = true;
                    result.Value = message.Text;
                }
                /*Add handling for Value from adaptive card*/
                else if (message.Value != null)
                {
                    result.Succeeded = true;
                    result.Value = message.Value.ToString();
                }
            }

            return Task.FromResult(result);
        }
    }
}

Thus workaround until official release of Adaptive Card Prompt for dialog in V4 botframework, is to use this custom prompt.

Usage: (Only for sending adaptive cards which have submit actions)

Referring to the example in the question section:

Add(new CustomPrompt("testPrompt"));

The response for the adaptive card submit action will be received in the next waterfall step : ProcessInputAsync()

var choice = (string)stepContext.Result;

choice will be JSON string of the body posted by the adaptive card.

Quodlibet

Answer 2

this is current problem, do we know when we will be able create multi-turn conversation flow using adaptive card in V4 bot framework and wait for Adaptive card's response in stepcontext.result variable instead always sending user to original OnTurn method.

Conch

Answer 3

Hit this issue today. This looks like a known issue and there is a workaround available on GitHub

Attachment attachment = new Attachment()
{
ContentType = AdaptiveCard.ContentType,
Content = JsonConvert.DeserializeObject(JsonConvert.SerializeObject(AdpCard)),
};

https://github.com/Microsoft/AdaptiveCards/issues/2148#issuecomment-462708622

Brookes