Toolbar shadow disappears when background color is dark

Question

I am developing an app that uses a Toolbar wrapped in AppBarLayout and it allows the user to change themes of the app by applying a theme style. For some reason, when the theme has a dark Toolbar, the drop shadow disappears (or becomes less visible?).

Here is my Toolbar in XML:

<android.support.design.widget.AppBarLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content">

    <android.support.v7.widget.Toolbar
        xmlns:android="http://schemas.android.com/apk/res/android"
        android:id="@+id/toolbar"
        android:background="?attr/colorPrimary"
        app:itemIconTint="@android:color/black"
        android:layout_width="match_parent"
        android:layout_height="?attr/actionBarSize"
        app:titleTextAppearance="?attr/actionBarTheme"
        android:theme="@style/ThemeOverlay.AppCompat.ActionBar" >

    <TextView
        android:id="@+id/toolbar_title"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        style="?attr/actionBarTheme"
        android:layout_gravity="center" />

    </android.support.v7.widget.Toolbar>

    </android.support.design.widget.AppBarLayout>

I am using ?attr/colorPrimary to set the background color of the Toolbar.

Theme with white Toolbar (shadow is present and very visible):

<style name="WhiteBlack" parent="Theme.AppCompat.Light.NoActionBar">
    <!-- Customize your theme here. -->
    <item name="colorPrimary">@android:color/white</item>
    <item name="colorPrimaryDark">@android:color/white</item>
    <item name="colorAccent">@android:color/black</item>
    <item name="preferenceTheme">@style/PreferenceThemeOverlay</item>
    <item name="actionBarTheme">@style/ToolbarTitleDark</item>
    <item name="actionBarStyle">@style/ToolbarTitleDark</item>
</style>

<style name="ToolbarTitleDark" parent="@style/TextAppearance.Widget.AppCompat.Toolbar.Title">
    <item name="android:textSize">20sp</item>
    <item name="android:textColor">@android:color/black</item>
    <item name="colorControlNormal">@android:color/black</item>
</style>

Theme with darker Toolbar (shadow is missing or barely visible):

<style name="AppTheme" parent="Theme.AppCompat.Light.NoActionBar">
    <!-- Customize your theme here. -->
    <item name="colorPrimary">@color/colorPrimary</item>
    <item name="colorPrimaryDark">@color/colorPrimaryDark</item>
    <item name="colorAccent">@color/colorAccent</item>
    <item name="preferenceTheme">@style/PreferenceThemeOverlay</item>
    <item name="actionBarTheme">@style/ToolbarTitleLight</item>
    <item name="actionBarStyle">@style/ToolbarTitleLight</item>
</style>

<style name="ToolbarTitleLight" parent="@style/TextAppearance.Widget.AppCompat.Toolbar.Title">
    <item name="android:textSize">20sp</item>
    <item name="android:textColor">@android:color/white</item>
    <item name="colorControlNormal">@android:color/white</item>
</style>

How can I make the shadow look the same regardless of the color of the Toolbar background? I want the shadow to always look like it does in the white Toolbar example.

Thank you!!

Vanwinkle