Implications of using AzureSearch SDK with static Dictionary of 30-40 ISearchIndexClients

Question

I have an ASP.NET Web Application that uses 30-40 different search indexes across 5-6 search services (various clients are in different pricing tiers).

Currently I am marshaling a new instance of the ISearchServiceClient followed by the appropriate ISearchIndexClient for the specific index needed based on the client making the call.

In an effort to increase performance I was thinking about marshaling up ALL of the ISearchIndexClients at application startup and placing them into a Dictionary object:

public static Dictionary<String, SearchIndexClient> SearchIndexes;

so that any specific index can be called up directly from the static Dictionary and used like so:

SearchIndexes["IndexName"].Documents.Search(searchText, searchParameters);

My hope is that this will speed up query and index update times especially on a "hot" index. My concern is that this may introduce memory leaks, performance issues and other unknowns.

I have not seen any examples using a statically available SearchServiceClient or SearchIndexClient so I am a bit uneasy going forward with this approach. My questions to the community are:

1. Is my plan sound?
2. Will it actually increase performance.
3. What are the drawbacks or implications (if any?)
4. If the amount of Indexes increases over time (to 60-70 for example) will I start to see drawbacks then?

5. Would it make more sense to marshal up the SearchServiceClients into a dictionary and connect to the appropriate SearchIndexClient from there as needed like so:

   public static Dictionary<String, SearchServiceClient> SearchServices;
   
   var searchIndexClient = SearchServices["ServiceName"].Indexes.GetClient("IndexName");
   searchIndexClient.Documents.Search(searchText, searchParameters);
   

Delorisdelorme

Answer 1

This strategy will probably not scale to the number of indexes you want. The most likely outcome is that you will exhaust the pool of available TCP connections. A better approach would be to implement a cache of SearchIndexClient instances keyed by index name. On a cache miss, you could get exclusive access to the least-recently-used client and set the IndexName property on it. That settable property was added to SearchIndexClient for exactly this scenario (note that it replaces the deprecated TargetDifferentIndex method).

You can find more discussions and background information about the implications of sharing SearchIndexClients on GitHub, the MSDN forums, and this related StackOverflow question.

Darwinism

Thanks Bruce. What are your thoughts on caching a single SearchIndexClient for EACH SearchServiceClient and calling TargetDifferentIndex on it whenever it is used. This way I would ONLY have to cache ONE IndexClient per Service, and target the index I need per request. If this is a good way to go, should I check which index it is set to before retargeting it (in case it is already the one I need)? – Delorisdelorme

I'm also wondering if I just stay with my current method of marshaling a new SerchServiceClient + SearchIndexClient if I will be running into any issues at scale. Right now this is working fine for me and I might be OK with the performance until I have enough clients to warrant revisiting. If I would be only making minimal performance gains for the added complexity I may just hold off until my app grows. Regardless: does upping my (System.Net.ServicePointManager.DefaultConnectionLimit to 12 * Environment.ProcessorCount) still help me in either case? Thoughts appreciated!!!! – Delorisdelorme

TargetDifferentIndex is very low-impact, so calling it when you don't need to is probably not a big deal (as long as you ensure you do not call it concurrently from multiple threads). Increasing the size of the connection pool can also help, but sockets aren't free. Ultimately you know your app and its workloads best, so I encourage you to experiment. – Darwinism

Thanks Bruce. I just tried using an ObjectCache of type MemoryCache.Default to store each SearchIndexClient when created (with an AbsoluteExpiration of about 15 min) and did the same with their parent SearchServiceIndex (60 minutes for those) and saw an immediate speed increase as well as 2/3 less SearchIndex errors when I hammered my application with requests... Going to test it at scale now and see how it goes. – Delorisdelorme