Programmatically sending tap event to coordinates in background in iOS using KIF

Question

The idea is to make an app that can send a 'tap the screen at coordinates (x,y)' event continuously in the background of another app in iOS. Think of a 'Cookie Clicker' cheat. I am currently assisting my friends in his app, and my job is to attempt to bug it out as much as possible. I have a slight feeling that tapping one of his buttons as much as possible will bug out his app, thus why I'm here.

The method is using KIF along with the background execution in iOS to achieve this.

1. In one of the classes, the file KIFUITestActor.m has the code:

   - (void)tapScreenAtPoint:(CGPoint)screenPoint
   {
   
   [self runBlock:^KIFTestStepResult(NSError **error) {
   
   // Try all the windows until we get one back that actually has something in it at the given point
   UIView *view = nil;
   for (UIWindow *window in [[[UIApplication sharedApplication] windowsWithKeyWindow] reverseObjectEnumerator]) {
       CGPoint windowPoint = [window convertPoint:screenPoint fromView:nil];
       view = [window hitTest:windowPoint withEvent:nil];
   
       // If we hit the window itself, then skip it.
       if (view != window && view != nil) {
           break;
       }
   }
   
   KIFTestWaitCondition(view, error, @"No view was found at the point %@", NSStringFromCGPoint(screenPoint));
   
   // This is mostly redundant of the test in _accessibilityElementWithLabel:
   CGPoint viewPoint = [view convertPoint:screenPoint fromView:nil];
   [view tapAtPoint:viewPoint];
   
   return KIFTestStepResultSuccess;
       }];
   }
   

   Would this be the code I'm looking for, and how would I implement it in Swift? (Or Objective-C if it makes things easier?)

2. Would running the class as a background service achieve the background execution?

Many thanks if you can answer this question.

Accretion

If I am understanding correctly, you are wanting to test user interactions that involve tap gestures, in order to verify that the app will not withstand repeat UITouch/Tap gestures... Can you clarify? – Teador

Yes indeed, it does sound like a strange request/procedure. I have been asked to intentionally bug out an app, on which I expect will not withstand multiple taps on a button that calls a server. As a programmer, the accepted logic is that I am too...call it 'lazy' to tap the button myself, and if my friend (the app developer) would like to test out the bug for himself I can thus send him this tapping 'hack'. The goal is to send touch gestures to certain coordinates on the screen, which would be the button. i.e. Tap tap tap without fingers on the screen on an app. Is this clearer? – Accretion

What is the significance of having the "automated tap tester" in A) being in a second app, and B) running in background mode? If a framework is available to automate the testing as desired, after having been added to a duplicate codebase of your friends app, would it be something you're up for trying? – Teador

I'm operating this more on a hunch than actual fact. Truth be told, I am not the only tester, and the job at hand is for the purpose of obtaining a prize that he will give. So a) yours is a more than possible method, but I should not want to waste his time; and b) he's not going to help me give up his prize. All in all, this is a personal procedure. Besides, I might adapt the solution to this problem into future projects. Thanks. – Accretion

I now realize what you're asking and do believe that is something that can be done. I say this because I have run into spyware that reads data from iOS device keyboards, called keylogger, which is in the same realm of what you're looking for. – Teador

Answer 1

A couple options, depending on what you have access to. If you have the actual IPA of the other app that you want to 'bug out', and you can attach a test target to it, then KIF is a great option.

This is a little guide on UI testing with KIF (easily googlable, link here). A quick summary though is:

  [tester tapViewWithAccessibilityLabel:@"Clear"];

That kind of code should have an equivalent named tapViewAtPoint or something along those lines (I last used KIF over a year ago, but it should be well documented as it was then).

However, if you DO NOT have access to the app you are trying to attack, which seems more likely for this type of question, then you can use a different approach altogether.

Automator can help you script this up if you can at least download the app in the iPhone simulator. Since Xcode 9 I believe allows you to do this, this part should not be hard, as long as the target app IS on the app store.

If it's not, and you don't have the code, I'd be curious to hear how you are being distributed the app, as there might still be a way to use one or the other option I suggested. Good luck!

Edit: update based on re-reading the question + bounty request one more time

Would running the class as a background service achieve the background execution?

This again depends on if you CAN run it as a background service. If you have access to the code itself, then it could be done, although I'm not sure what the goal of background execution is in your described question.

KIF won't be able to work on anything that isn't in the foreground. In general, iOS devices never funnel taps to an app in the background, so if the attack vector being tested is to repeatedly tap an app, background execution is likely not the answer.

From your question this looks more like a use case for parallelizing a tap loop, and your problem seems like it could benefit from Xcode 9's parallel simulator feature addition.

Although I have not personally tried this yet, you could do parallel KIF runs in multiple simulators on a machine running Xcode 9. This is likely the best way to get as many data points as possible about what caused issues, as with UI tests you can take a screenshot of the app at any point. In Xcode 9, I believe there is also a new feature to record a video straight from the Simulator (and no longer need Quicktime).

Uitlander

As far as I understand, this is a very helpful answer. With regards to the app itself, it is an IPA given to me, but since I am helping my friend test it, I'd like the method to be such that it works even if it has been downloaded from the App Store, debug purposes you see. With regards to the method of parallelizing a tap loop, how would that be done and under what circumstances? Thanks much for the response. – Accretion

well, parallelizing on simulator is only possible if you're using Xcode 9 (to be out of beta quite soon). Not sure "under what circumstances", I think that's kind of for you to determine, but the mechanism would be to use Automator most likely, and have Automator go simulator by simulator in rapid succession tapping where you want a tap to occur. – Uitlander

I will certainly give that a try. Have your 200, sir! – Accretion

If I could trouble you once more, the method I am searching for is such that even if the app was downloaded from the App Store, the tap event would still function properly. The method you proposed is useful only if I was given the .ipa, but that would not be the case for a live app. Any ideas? – Accretion

Ah, I'm actually not sure in that case. I do know Workflow is an app similar to Automator, but on iOS itself. I've used it, it works pretty well, might be able to support the app you need to test (it can definitely loop an action for you). fwiw the company that makes Workflow was recently acquired by Apple :). – Uitlander