Indexing Sitecore Item security and restricting returned search results

Question

I have several roles defined, each with different restrictions to content and media items and I would like to restrict the search results that are returned based on the access rights of the currently logged in user, rather than displaying the result and the user then presented with an "Access Denied" page. Some content will obviously be accessible to extranet\anonymous so they should be returned for all users regardless.

The security follows the standard Sitecore practices so Role inheritance (roles within roles) will be used, so it will need to take this into account also.

I couldn't see anything in the Advanced Database Crawler module that would help and I've looked through the Sitecore Search and Indexing Guide (version 6.6 and version 7) but couldn't find any information about indexing the security applied to items. The following articles have some suggestions:

• How can I set up a Lucene index in Sitecore that handles security correctly?

This feels "dirty" and has the potential for performance issues, particularly when there are a large number of items returned. Also, (see in the comments) the issue with paging results.

• Security (aka Permissions) and Lucene - How ? Should it be done?

The above looks more realistic, and would filter out the results based on indexed security roles, there would obviously be a need to expand the roles to handle roles within roles. My concern here would be that we would need to handle denied permissions, when we specifically need to deny/restrict access for certain roles to content items (I know this is not recommended practice, but there is a very specific need to always deny).

I'm at the planning stage at the moment so with the release of Sitecore 7 today there is also the possibility to use the updated Lucene libraries and/or SOLR if that makes life easier - assuming of course that some of the modules like WebForms for Marketers and Email Campaign Manager are updated before too long.

What are the solutions that people are using for returning search results taking into account security? Any alternatives than the linked questions above? Maybe something in Sitecore 7 that I can leverage, the updated Lucene libraries or SOLR?

I'd prefer to keep this all "out of the box" Sitecore and not use other third party search products if at all possible.

Phila

I've only done it with a custom crawler that adds a metadata value like MembersOnly: 1. I think if you had a field like AuthorizedRoles that contained all the roles allowed to see the item you could tailor your query that way. But maybe permissions is something you do as a post process? I'd be interested in seeing where the answers go - great question. – Rune

Answer 1

A slight alternative to the suggestion from Klaus:

In Sitecore.ContentSeach.config you'll find a pipeline called contentSearch.getGlobalSearchFilters

Processors added to this pipeline will be applied to any query, so if we drop in one that applies a filter based on roles we're good.

ComputedField

To start, we want a computed field added to our index configuration:

<fields hint="raw:AddComputedIndexField">
   <field fieldName="read_roles"           returnType="stringCollection">Sitecore.ContentSearch.ComputedFields.ReadItemRoles,Sitecore.ContentSearch</field>
</fields>

NOTE the stored type is a collection of strings. We'll use it to index all the names of roles that can read an item.

Implementation

1. We have a base abstract class to handle the extraction of item security details

   public abstract class ItemPermissions: IComputedIndexField
   {
       public string FieldName { get; set; }
       public string ReturnType { get; set; }
   
       public object ComputeFieldValue(IIndexable indexable)
       {
           var indexableItem = indexable as SitecoreIndexableItem;
           if (indexableItem == null) return null;
   
           var security = indexableItem.Item.Security;
   
           return GetPermissibles(security);
       }
   
       protected abstract object GetPermissibles(ItemSecurity security);
   }
   

2. We implement the above with the abstracted method

   public class ReadItemRoles : ItemPermissions
   {
       protected override object GetPermissibles(ItemSecurity security)
       {
           var roles = RolesInRolesManager.GetAllRoles();
           return roles.Where(security.CanRead).Select(r => r.Name);
       }
   }
   

NOTE There's obviously a performance impact here, this will reduce your indexing speed. To reduce the impact, only add the the computed field to the index configuration for the index that contains secured content. E.g. If your web content is only accessed by the anonymous user it will add no benefit.

Pipeline

Add the entry in to the config

<contentSearch.getGlobalSearchFilters>
    <processor type="Sitecore.ContentSearch.Pipelines.GetGlobalFilters.ApplyGlobalReadRolesFilter, Sitecore.ContentSearch" />
  </contentSearch.getGlobalSearchFilters>

Implementation

Implement the pipeline filter to check the roles of the context user

public class ApplyGlobalReadRolesFilter : GetGlobalFiltersProcessor
{
    public override void Process(GetGlobalFiltersArgs args)
    {
        var query = (IQueryable<SitecoreUISearchResultItem>)args.Query;

        var userRoles = Context.User.Roles.Select(r => r.Name.Replace(@"\", @"\\"));

        var predicate = PredicateBuilder.True<SitecoreUISearchResultItem>();
        predicate = userRoles.Aggregate(predicate, (current, role) => current.Or(i => i["read_roles"].Contains(role)));

        if(predicate.Body.NodeType != ExpressionType.Constant)
            args.Query = query.Filter(predicate);
    }
}

Summary

1. Create a ComputedField that returns a list of all valid roles for a given access right
2. Apply a pipeline processor to contentSearch.getGlobalSearchFilters to add a query filter to each search request.
3. Use the PredicateBuilder class to ensure the role names are OR'ed together

The big benefit here is that you take the hit at index time and the handling of item restriction is handled through a search query as normal. No need to worry about the facet numbers or search counts being incorrect.

You can restrict the roles you are checking to compute the field and you can vary the application of the pipeline filter. You can even take out the pipeline filter and just update your queries to filter when you require it.

NOTE The biggest problem with this set up is the requirement to re-index your content when security restrictions change. Should you be applying security restrictions to users themselves, you'll have to include additional computed fields.

Edit 02/06/2013

I was just tinkering with this in a project and realised that it was AND'ing the roles in the query. If a user had multiple roles assigned then both roles would have to have declared rights to the item. I've updated the pipeline processor to use the PredicateBuilder class to OR the roles. A check is also added to ensure the predicate is not a constant, this ensures the query is updated only if we have a filter to apply.

Lambart

Great answer, I would rather take the hit at index time rather than query time and re-indexing due to security changes is acceptable. To be fair, in this project there are not that many items so checking permissions at runtime will not have a huge impact but I would like to know the options for future requirements. – Phila

Article on Computed Index Fields in Sitecore 7 – Phila

Should probably add a slight disclaimer... I'm the newest member of the Sitecore 7 team ;-) Computed fields are a big thing, they let you manipulate properties of your indexable any way you can imagine and save straight to the index. If you can do the processing while indexing your queries become much simpler and more powerful. – Lambart

Answer 2

After some more searching around, the Linq to Sitecore article pointed me to the following lines of code:

var index = SearchManager.GetIndex("sitecore_master_index");
var context = index.CreateSearchContext(SearchSecurityOptions.EnableSecurityCheck))

Digging through Sitecore.ContentSearch.dll and Sitecore.ContentSearch.LuceneProvider.dll in dotPeek decompiler and the mention of the indexing.filterIndex.outbound pipeline in the Sitecore 7 Search document I found the following code:

Sitecore.ContentSearch.LuceneProvider.LuceneSearchReults

public IEnumerable<SearchHit<TElement>> GetSearchHits()
{
  for (int idx = this.startIndex; idx <= this.endIndex; ++idx)
  {
    Document doc = this.context.Searcher.IndexReader.Document(this.searchHits.ScoreDocs[idx].Doc, (FieldSelector) this.fieldSelector);
    if (!this.context.SecurityOptions.HasFlag((Enum) SearchSecurityOptions.DisableSecurityCheck))
    {
      string secToken = doc.GetField("_uniqueid").StringValue;
      string dataSource = doc.GetField("_datasource").StringValue;
      if (!string.IsNullOrEmpty(secToken))
      {
        bool isExcluded = OutboundIndexFilterPipeline.CheckItemSecurity(new OutboundIndexFilterArgs(secToken, dataSource));
        if (!isExcluded)
          yield return new SearchHit<TElement>(this.searchHits.ScoreDocs[idx].Score, this.configuration.IndexDocumentPropertyMapper.MapToType<TElement>(doc, this.selectMethod, this.virtualFieldProcessors, this.context.SecurityOptions));
      }
    }
    else
      yield return new SearchHit<TElement>(this.searchHits.ScoreDocs[idx].Score, this.configuration.IndexDocumentPropertyMapper.MapToType<TElement>(doc, this.selectMethod, this.virtualFieldProcessors, this.context.SecurityOptions));
  }
}

Sitecore.ContentSearch.Pipelines.IndexingFilters

public class ApplyOutboundSecurityFilter : OutboundIndexFilterProcessor
{
    public override void Process(OutboundIndexFilterArgs args)
    {
      if (args.IndexableUniqueId == null || !(args.IndexableDataSource == "Sitecore"))
        return;
      ItemUri uri = new ItemUri(args.IndexableUniqueId);
      if (args.AccessRight != AccessRight.ItemRead || Database.GetItem(uri) != null)
        return;
      args.IsExcluded = true;
    }
}

So it looks like Sitecore 7 gives us the ability to filter the the search results using the security rights of the context user straight out of the box, albeit using a very similar method of checking the item read permissions that Mark Cassidy suggested. Which is good news, since if this is a requirement for Sitecore 6 implementations then we can easily update the Advanced Database Crawler to do the same thing.

I'm still not convinced on the performance of this though, given that Sitecore 7 comes with Item Buckets and the possibility of storing millions of items. It should be possible to create several indexes though, and only EnableSecurityCheck on indexes with security enabled content, but then we need to think about combining the results from several indexes for "global search" results and also take into acccount Boosting which will mean re-ordering the combined results.

Phila

The InBound and OutBound pipelines allow to you apply logic to include/exclude an item as it is added into the index and as it is retrieved from it. As you mention the more logic you do here the more you have to consider performance as it will be applied to every item. – Lottielotto

This is built in to Sitecore 7+ but as Sergejs commented, the args.IndexableDataSource will fail since the actual value is lowercase "sitecore". From what I can see, they fixed this in 8.0 but it fails in 7.5 all revs as well. – Babylon

Answer 3

well - your considerations seems quite on target. The easy implementation is to check the item through the database lookup - but paging, facetting and other statistics will fail in that.

The approach we do is to index security tokens for roles - having an inclusion field for allow and an exclusion field for deny rights. You then need to build a query for this - and expand all roles in roles for the query.

There are two issues you might be facing. One is very complex OR queries for all roles in roles memberships. Might be less then well-performing. The other is indexing congestion - as you will need to index major parts of the contentitems when security and permissions change as then are inherited.

The alternative is to use join queries - but normally that would be bad performing.

Umbra

One of the options I am looking into is using Filtering Options as suggested in second SO answer I linked. It might perform better and easier to code using your suggestions. – Phila

I cannot comment on Kieranties post due to too little reputation :-) But I still see an indexing issue. When you use inherited security, you will not change an subitem - but security rules have changed due to the inheritance. Therefore you will not get the subitem indexed and the security filter will not reflect actual security rules as the item in the index will hold old security info. You need some kind of subitem indexing to inflict the indexing of all subitems on change of security. – Umbra

Technically I think the solution is exactly the same as yours, which is you index all the roles of the item and the actual implementation of how you get those roles is up to you but the code provided should do that in GetPermissibles method. I would have accepted this as the answer as well if i could have. – Phila

Answer 4

Sitecore developers made a silly mistake, it will never work, because of that statement: if ((args.IndexableUniqueId != null) && (args.IndexableDataSource == "Sitecore"))

as args.IndexableDataSource will always be equal to "sitecore" not "Sitecore". I'm currently upgrading big project to latest 7.2 Update and found out that silly mistake, oh Sitecore Devs usual mistakes :)

Pension