this page might be helpful - How to close a Dialog IFrame in Liferay 6.2
If you define your modal window like this (let's say in view.jsp):
<aui:button name="openDialog" type="button" value="open-dialog" />
<liferay-portlet:renderURL var="dialogURL" windowState="<%=LiferayWindowState.POP_UP.toString() %>">
<liferay-portlet:param name="mvcPath" value="/dialog.jsp" />
</liferay-portlet:renderURL>
<aui:script use="liferay-util-window">
A.one('#<portlet:namespace/>openDialog').on('click', function(event) {
Liferay.Util.openWindow({
dialog: {
centered: true,
height: 300,
modal: true,
width: 400
},
id: '<portlet:namespace/>dialog',
title: '<liferay-ui:message key="i-am-the-dialog" />',
uri: '<%=dialogURL %>'
});
});
</aui:script>
and create button trigger (or onsubmit event listener in your case) inside the dialog page (dialog.jsp):
<aui:button name="closeDialog" type="button" value="close" />
<aui:script use="aui-base">
A.one('#<portlet:namespace/>closeDialog').on('click', function(event) {
// Let's suppose that "data" contains the processing results
var data = ...
// Invoke a function with processgin results and dialog id
Liferay.Util.getOpener().<portlet:namespace/>closePopup(data, '<portlet:namespace/>dialog');
});
</aui:script>
you will get the window that opened the dialog by getOpener() function. In page that creates the dialog (view.jsp), you have to provide the closePopup function like this:
<aui:script>
Liferay.provide(
window,
'<portlet:namespace/>closePopup',
function(data, dialogId) {
var A = AUI();
// Here you can use "data" parameter
// Closing the dialog
var dialog = Liferay.Util.Window.getById(dialogId);
dialog.destroy();
},
['liferay-util-window']
);
</aui:script>