How to Properly Batch Index in Core Spotlight
Asked Answered
U

1

6

I am starting to use Core Spotlight to index items in my app. As I figured it would, the app crashes on my device, but not the simulator, because I have 2000+ items to index. I noticed I start getting memory warning by the time I add about 427 items to my NSMutableArray holding my items, and crashes completely by the time it gets to 540 items.

I am aware of the ability to batch these indexes, and I can see the code presented by Apple on their site Apple - Index App Content - Using Advanced Features. However, I'm not sure exactly how to implement this.

Could someone add some code or point me in the right direction to accomplishing this?

Update #1 (2016/06/21)

The items I'm trying to index are about 2000+ names, with phone numbers, emails, title, and department information for my company. I have a file built in but I also download a file periodically to get up-to-date info. I save the data as a plist, not into CoreData. Once the file downloads I initiate the index, by first deleting everything currently indexed and then adding all the items. From the server I cannot determine if data has changed.

My goal is to index about 200-400 items at a time until all items have been indexed. Below is how I'm currently indexing without batching.

- (void)setupCoreSpotlightSearchFor:(NSString *)fileName {
        if ([fileName isEqual:kPlistFileNameEmployee]) {
            NSFileManager *fileManager = [NSFileManager defaultManager];
            NSURL *downloadFileUrl = kDirectoryEmployeeListDownload;

            if ([fileManager fileExistsAtPath:[downloadFileUrl path]]) {
                NSArray *dataArray = nil;
                NSError *error = nil;
                dataArray = [NSArray arrayPropertyListWithURL:downloadFileUrl error:&error];
                if (!error && dataArray) {
                    NSMutableArray *items = nil;
                    int counter = 0;
                    for (NSDictionary *person in dataArray) {
                        NSNumberFormatter *f = [[NSNumberFormatter alloc] init];
                        f.numberStyle = NSNumberFormatterDecimalStyle;
                        NSNumber *idNum = [f numberFromString:[[person objectForKey:@"Id"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]]];
                        NSString *firstName = [[person objectForKey:@"FirstName"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]];
                        NSString *lastName = [[person objectForKey:@"LastName"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]];
                        NSString *email = [[person objectForKey:@"Email"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]];
                        NSString *phone = [[person objectForKey:@"Extension"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]];
                        NSString *title = [[person objectForKey:@"Title"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]];
                        NSString *organization = [[person objectForKey:@"Organization"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]];
                        NSString *department = [[person objectForKey:@"Department"] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]];

                        CSSearchableItemAttributeSet *attributeSet = [[CSSearchableItemAttributeSet alloc] initWithItemContentType:(NSString *)kCIAttributeTypeImage];

                        Person *person = [Person createPersonWithId:idNum
                                                          FirstName:[NSString stringOrNil:firstName]
                                                           LastName:[NSString stringOrNil:lastName]
                                                              Email:[NSString stringOrNil:email]
                                                        PhoneNumber:[NSString stringOrNil:phone]
                                                              Title:[NSString stringOrNil:title]
                                                       Organization:[NSString stringOrNil:organization]
                                                         Department:[NSString stringOrNil:department]
                                          ];

                        if ([NSString stringOrNil:[person fullName]] != nil) {
                            [attributeSet setTitle:[person fullName]];
                            [attributeSet setContentDescription:[person title]];

                            NSMutableArray *keywords = [[NSMutableArray alloc] init];

                            if ([NSString stringOrNil:[person firstName]] != nil) {
                                [keywords addObject:[person firstName]];
                            }
                            if ([NSString stringOrNil:[person lastName]] != nil) {
                                [keywords addObject:[person lastName]];
                            }
                            if ([NSString stringOrNil:[person title]] != nil) {
                                [keywords addObject:[person title]];
                            }
                            if ([NSString stringOrNil:[person department]] != nil) {
                                [keywords addObject:[person department]];
                            }
                            if ([NSString stringOrNil:[person phoneNumber]] != nil) {
                                [keywords addObject:[person phoneNumber]];
                            }
                            if ([NSString stringOrNil:[person email]] != nil) {
                                [keywords addObject:[person email]];
                            }

                            [attributeSet setKeywords:keywords];

                            UIImageView *imageView = [[UIImageView alloc] initWithFrame:CGRectMake(0, 0, 180, 180)];
                            [imageView setImageWithString:[person initialsWithCapitalization:InitialsCapitalizationTypeBoth andLastNameFirst:NO] color:[UIColor colorWithRed:0.77 green:0.07 blue:0.19 alpha:1.0] circular:YES];
                            [attributeSet setThumbnailData:UIImagePNGRepresentation([imageView image])];

                            if ([NSString stringOrNil:[person phoneNumber]] != nil) {
                                [attributeSet setSupportsPhoneCall:[NSNumber numberWithInt:1]];
                                [attributeSet setPhoneNumbers:@[[person phoneNumber]]];
                            }
                            if ([NSString stringOrNil:[person email]] != nil) {
                                [attributeSet setEmailAddresses:@[[person email]]];
                            }

                            CSSearchableItem *item = [[CSSearchableItem alloc] initWithUniqueIdentifier:[NSString stringWithFormat:@"%@", [person id]] domainIdentifier:@"com.myapp.index.employee" attributeSet:attributeSet];

                            if (item) {
                                if (!items) {
                                    items = [[NSMutableArray alloc] init];
                                }
                                [items addObject:item];
                                counter++;
                            }
                        }
                    }

                    if (items) {
                        // delete current items
                        [[CSSearchableIndex defaultSearchableIndex] deleteAllSearchableItemsWithCompletionHandler:^(NSError * _Nullable error) {
                            if (error) {
                                DLog(@"%@",error);
                            } else {
                                DLog(@"Deleted Index!");

                                // add new items
                                [[CSSearchableIndex defaultSearchableIndex] indexSearchableItems:items completionHandler:^(NSError * _Nullable error) {
                                    if (error) {
                                        DLog(@"%@",error);
                                    } else {
                                        DLog(@"Added Index!");
                                    }
                                }];
                            }
                        }];
                    }
                }
            }
        }
    }

Thanks!

Ultra answered 13/6, 2016 at 23:34 Comment(8)
I'm not sure what you're asking for... You want to bath your indexes and you can see the code implemented by Apple, but you're still confused? Here's a link to start with I guess toptal.com/ios/ios-9-spotlight-search-for-developersCorenda
@KFDoom, no I got that working. I'm trying to Batch them as stated in the link I supplied.Ultra
So you want to batch updates while the app isn't running? That's what the Advanced section from the Apple documentation you link to mostly discusses.Corenda
I disagree with you. The advanced sections discusses two different functions: 1) Batch updates and 2) Index while not running. These two are separate features. Listing 4-6 can be done while the app is running and I'm trying to get examples of how people are getting it done.Ultra
Ah I see! We don't disagree at all, actually. I'm just trying to figure out precisely what you're asking. No we can get cooking..Corenda
Ah ok cool! =) Thanks!Ultra
I'm about to add more info which might help or even might change the implementation.Ultra
Roger, roger. I'll take a look at it post work.Corenda
E
0

So your memory problem isn't actually anything to do with core spotlight, because your code is crashing before you even get there. You should profile to check exactly what's taking all the memory, I guess the images you're generating are a large part of it, particularly because you're using PNG data instead of JPEG (with significant compression given the simple nature of the images). It also seems very wasteful to create an image view when what you really want is the image, and it's doubly wasteful to create a new image view on each iteration of the loop...

So, you shouldn't really be batching the core spotlight processing, you should be batching your file processing. There are a number of ways you could do that, but the main point is that you process say 100 items from the file into an array, then pass that to core spotlight and wait for it to finish. Then you start the next 100 items in the file.

You could do that with an asynchronous NSOperation. You could do it with a recursive block and GCD.

Etch answered 22/6, 2016 at 21:21 Comment(3)
got it. interesting. I haven't used NSOperation. I'll have to look it up. would you mind giving me some code to work with?Ultra
lorenzoboaro.io/2016/01/05/…Etch
Sorry was on vacation. I'll look into it now. However, I did ask for working code/sample to receive the 50 bounty. If you can do so, thanks! BTW: it's in OBJ-CUltra

© 2022 - 2024 — McMap. All rights reserved.