Liferay performance issue

Question

We are facing performance issue with liferay 6.2 after migrating from 6.1.

Almost all our custom as well as liferay's plugin portlet are responding very slow for non admin users.

We observe that permission checks seems to be taking lot of time. Also following query is being hit so many times...

 (
    SELECT DISTINCT Group_.groupId AS groupId
    ,REPLACE(Group_.NAME, ' LFR_ORGANIZATION', '') AS groupName
    ,Group_.type_ AS groupType
    ,Group_.friendlyURL AS groupFriendlyURL
    FROM Group_
    INNER JOIN Users_Groups ON (Users_Groups.groupId = Group_.groupId)
    WHERE (Group_.liveGroupId = 0)
    AND (Group_.active_ = 1)
    AND (Group_.liveGroupId = 0)
    AND (Users_Groups.userId = 3391488)
    AND (Group_.companyId = 10132)
    AND (
    Group_.classNameId = 10012
    OR Group_.classNameId = 10024
    )
    AND (Group_.parentGroupId != - 1)
    AND (Group_.liveGroupId = 0)
    AND (Group_.NAME != 'Control Panel')
    AND (
    Group_.NAME LIKE NULL
    OR NULL IS NULL
    )
    AND (
    Group_.description LIKE NULL
    OR NULL IS NULL
    )
    AND (Group_.type_ != 4)
    )

    UNION

    (
    SELECT DISTINCT Group_.groupId AS groupId
    ,REPLACE(Group_.NAME, ' LFR_ORGANIZATION', '') AS groupName
    ,Group_.type_ AS groupType
    ,Group_.friendlyURL AS groupFriendlyURL
    FROM Group_
    INNER JOIN Users_Orgs ON (Users_Orgs.organizationId = Group_.classPK)
    WHERE (Group_.liveGroupId = 0)
    AND (Group_.active_ = 1)
    AND (Users_Orgs.userId = 3391488)
    AND (Group_.companyId = 10132)
    AND (
    Group_.classNameId = 10012
    OR Group_.classNameId = 10024
    )
    AND (Group_.parentGroupId != - 1)
    AND (Group_.liveGroupId = 0)
    AND (Group_.NAME != 'Control Panel')
    AND (
    Group_.NAME LIKE NULL
    OR NULL IS NULL
    )
    AND (
    Group_.description LIKE NULL
    OR NULL IS NULL
    )
    AND (Group_.type_ != 4)
    )

    UNION

    (
    SELECT DISTINCT Group_.groupId AS groupId
    ,REPLACE(Group_.NAME, ' LFR_ORGANIZATION', '') AS groupName
    ,Group_.type_ AS groupType
    ,Group_.friendlyURL AS groupFriendlyURL
    FROM Group_
    INNER JOIN Groups_Orgs ON (Groups_Orgs.groupId = Group_.groupId)
    INNER JOIN Users_Orgs ON (Users_Orgs.organizationId = Groups_Orgs.organizationId)
    WHERE (Group_.liveGroupId = 0)
    AND (Group_.active_ = 1)
    AND (Group_.liveGroupId = 0)
    AND (Users_Orgs.userId = 3391488)
    AND (Group_.companyId = 10132)
    AND (
    Group_.classNameId = 10012
    OR Group_.classNameId = 10024
    )
    AND (Group_.parentGroupId != - 1)
    AND (Group_.liveGroupId = 0)
    AND (Group_.NAME != 'Control Panel')
    AND (
    Group_.NAME LIKE NULL
    OR NULL IS NULL
    )
    AND (
    Group_.description LIKE NULL
    OR NULL IS NULL
    )
    AND (Group_.type_ != 4)
    )

    UNION

    (
    SELECT DISTINCT Group_.groupId AS groupId
    ,REPLACE(Group_.NAME, ' LFR_ORGANIZATION', '') AS groupName
    ,Group_.type_ AS groupType
    ,Group_.friendlyURL AS groupFriendlyURL
    FROM Group_
    INNER JOIN Groups_UserGroups ON (Groups_UserGroups.groupId = Group_.groupId)
    INNER JOIN Users_UserGroups ON (Users_UserGroups.userGroupId = Groups_UserGroups.userGroupId)
    WHERE (Group_.liveGroupId = 0)
    AND (Group_.active_ = 1)
    AND (Group_.liveGroupId = 0)
    AND (Users_UserGroups.userId = 3391488)
    AND (Group_.companyId = 10132)
    AND (
    Group_.classNameId = 10012
    OR Group_.classNameId = 10024
    )
    AND (Group_.parentGroupId != - 1)
    AND (Group_.liveGroupId = 0)
    AND (Group_.NAME != 'Control Panel')
    AND (
    Group_.NAME LIKE NULL
    OR NULL IS NULL
    )
    AND (
    Group_.description LIKE NULL
    OR NULL IS NULL
    )
    AND (Group_.type_ != 4)
    )
    ORDER BY groupName ASC

Wondering why liferay is not caching it....

Freda

Are you using CE or EE version? Liferay appears to currently have issues scaling a User's permissions checking. As the number of roles/groups that the User is a member of grows, the impact is not linear. Performance issues are especially evident in Search portlet searches. – Arquebus

We are using 6.2 EE SP7 and yes we are observing performance issues mainly in permission check as for Admin users permission check is ignored it's working fine for Admin users but not for non admin users. – Freda

We too have EE support and are currently awaiting a response from Liferay on this matter. As evidence of the problem, a co-worker hooked the permission checking code and was able to dramatically reduce our permission checking execution times (from minutes down to milliseconds). I would file a support ticket and await the hotfix to see if it resolves the issues within your environment. – Arquebus

Thanks @Arquebus , would you mind sharing those permission hook ? It would act as a temp fix for us as well till the time liferay come up with the solution. – Freda

@Arquebus Can you give some hints as to how did you hook into the permission checking? through EXT or hook and which classes would be required to be overridden? – Verism

For our particular issue, we only care about the performance of search and Asset Publisher activities. The method doGetPermissionQuery_6 within SearchPermissionCheckerImpl was our bottleneck, called multiple times (sometimes taking 2 minutes) for an overall execution over 20 minutes. The issue was a result of the number of UserGroups added in the if/else (lines 304-328). In an EXT, we tailored those lines to fit our needs. – Arquebus