How to ignore certain UITouch Points in multitouch sequence

Question

I am working on a drawing app for ipad, I am providing full screen for drawing. So as we all now, user might write with his wrist support or by resting his hand on the screen. So my aim is to allow the user to write freely with his wrist/hand support.

But the app should only detect finger drawing, or ignore/reject wrist and hand touches and delete them

I started working on it , I created a sample project with multitouch enabled.

Below is my code

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {

    mouseSwiped = NO;

    for (UITouch *touch in touches)
    {
        NSString *key = [NSString stringWithFormat:@"%d", (int) touch];
        lastPoint = [touch locationInView:self.view];

        [touchPaths setObject:[NSValue valueWithCGPoint:lastPoint] forKey:key];
    }

}


- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{    
    mouseSwiped = YES;
    CGPoint lastPoint3;

    for (UITouch *touch in touches)
    {
        NSString *key = [NSString stringWithFormat:@"%d", (int) touch];

        lastPoint = [[touchPaths objectForKey:key] CGPointValue];


        currentPoint1 = [touch locationInView:self.view];

        NSLog(@"Y:%f",currentPoint1.y);


        UIGraphicsBeginImageContext(self.view.frame.size);
        [self.tempDrawImage.image drawInRect:CGRectMake(0, 0, self.view.frame.size.width, self.view.frame.size.height)];
        CGContextSetLineCap(UIGraphicsGetCurrentContext(), kCGLineCapRound);
        CGContextSetLineWidth(UIGraphicsGetCurrentContext(), brush );
        CGContextSetRGBStrokeColor(UIGraphicsGetCurrentContext(), red, green, blue, 1.0);
        CGContextSetBlendMode(UIGraphicsGetCurrentContext(),kCGBlendModeNormal);
        CGContextBeginPath(UIGraphicsGetCurrentContext());
        CGContextMoveToPoint(UIGraphicsGetCurrentContext(), lastPoint.x, lastPoint.y);
        CGContextAddLineToPoint(UIGraphicsGetCurrentContext(), currentPoint1.x, currentPoint1.y);

        CGContextStrokePath(UIGraphicsGetCurrentContext());
        self.tempDrawImage.image = UIGraphicsGetImageFromCurrentImageContext();
        [self.tempDrawImage setAlpha:opacity];
        UIGraphicsEndImageContext();

       [touchPaths setObject:[NSValue valueWithCGPoint:currentPoint1] forKey:key];
    }
}

So this works fine with any number of touches, But I am not understanding how can I reject those palm/hand touches while drawing and only draw, what user draws with his finger/ stylus.

Presently if I draw, I get this thing, below is the image

Here I have drawn with my hand support, you can see below "Hello" their is few weird drawing has happened. How I can reject those touches and delete them and only draw hello

Thanks

Ranjit

Forward

@user2277872 Most drawing apps actually do filter out false touches this way. – Palmate

Answer 1

One solution is to store the topmost tap in touchesBegan and only draw this one.

As you have pointed out, you are not supposed to retain the UITouch instance, so I recommend using a weak reference instead.

This will only draw a single touch. If you wish to draw the touches of multiple fingers, you need another way of filtering out the hand (many drawing apps have user settings for telling the app the pose of the hand, for example, but this is of course more complicated).

Here is an idea on how to do it:

#import <QuartzCore/QuartzCore.h>

@interface TViewController () {
    // We store a weak reference to the current touch that is tracked
    // for drawing.
    __weak UITouch* drawingTouch;
    // This is the previous point we drawed to, or the first point the user tapped.
    CGPoint touchStartPoint;
}
@end
@interface _TDrawView : UIView {
@public
    CGLayerRef persistentLayer, tempLayer;
}
-(void)commitDrawing;
-(void)discardDrawing;
@end

@implementation TViewController

- (void) loadView
{
    self.view = [[_TDrawView alloc] initWithFrame:[UIScreen mainScreen].bounds];
    self.view.opaque = YES;
    self.view.multipleTouchEnabled = YES;
    self.view.backgroundColor = [UIColor whiteColor];
}

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
    // Start with what we currently have
    UITouch* topmostTouch = self->drawingTouch;
    // Find the top-most touch
    for (UITouch *touch in touches) {
        CGPoint lastPoint = [touch locationInView:self.view];
        if(!topmostTouch || [topmostTouch locationInView:self.view].y > lastPoint.y) {
            topmostTouch = touch;
            touchStartPoint = lastPoint;
        }
    }
    // A new finger became the drawing finger, discard any previous 
    // strokes since last touchesEnded
    if(self->drawingTouch != nil && self->drawingTouch != topmostTouch) {
        [(_TDrawView*)self.view discardDrawing];
    }
    self->drawingTouch = topmostTouch;
}

- (void) touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
    // Always commit the current stroke to the persistent layer if the user
    // releases a finger. This could need some tweaking for optimal user experience.
    self->drawingTouch = nil;
    [(_TDrawView*)self.view commitDrawing];
    [self.view setNeedsDisplay];
}

- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{
    const CGFloat red=0, green=0, blue=0, brush=1;
    for (UITouch *touch in touches) {
        // Find the touch that we track for drawing
        if(touch == self->drawingTouch) {
            CGPoint currentPoint = [touch locationInView:self.view];

            // Draw stroke first in temporary layer
            CGContextRef ctx = CGLayerGetContext(((_TDrawView*)self.view)->tempLayer);
            CGContextSetLineCap(ctx, kCGLineCapRound);
            CGContextSetLineWidth(ctx, brush );
            CGContextSetRGBStrokeColor(ctx, red, green, blue, 1.0);
            CGContextSetBlendMode(ctx,kCGBlendModeNormal);
            CGContextBeginPath(ctx);
            CGContextMoveToPoint(ctx, touchStartPoint.x, touchStartPoint.y);
            CGContextAddLineToPoint(ctx, currentPoint.x, currentPoint.y);
            CGContextStrokePath(ctx);
            // Update the points so that the next line segment is drawn from where
            // we left off
            touchStartPoint = currentPoint;
            // repaint the layer
            [self.view setNeedsDisplay];
        }
    }
}

@end

@implementation _TDrawView

- (void) finalize {
    if(persistentLayer) CGLayerRelease(persistentLayer);
    if(tempLayer) CGLayerRelease(tempLayer);
}

- (void) drawRect:(CGRect)rect {
    CGContextRef ctx = UIGraphicsGetCurrentContext();
    if(!persistentLayer) persistentLayer = CGLayerCreateWithContext(ctx, self.bounds.size, nil);
    if(!tempLayer) tempLayer = CGLayerCreateWithContext(ctx, self.bounds.size, nil);

    // Draw the persistant drawing
    CGContextDrawLayerAtPoint(ctx, CGPointMake(0, 0), persistentLayer);
    // Overlay with the temporary drawing
    CGContextDrawLayerAtPoint(ctx, CGPointMake(0, 0), tempLayer);
}

- (void)commitDrawing {
    // Persist the temporary drawing
    CGContextRef ctx = CGLayerGetContext(persistentLayer);
    CGContextDrawLayerAtPoint(ctx, CGPointMake(0, 0), tempLayer);
    [self discardDrawing];
}
- (void)discardDrawing {
    // Clears the temporary layer
    CGContextRef ctx = CGLayerGetContext(tempLayer);
    CGContextClearRect(ctx, self.bounds);
    CGContextFlush(ctx);
}
@end

EDIT: I added the logic that if a new touch is detected, if there is currently any stroke being drawn with a higher y-value, it is removed, as we discussed in the comments.

The overlaying is done by painting two CGLayers. This code could be optimized a lot for performance, it should be looked at more as an illustration than production-ready code.

Palmate

How we can delete the drawing which happened below the highest position of y – Forward

For example , before I draw with my finger, I will rest my hand , so touch will be detected and will draw some lines, and when I touch my finger, it detects that , the finger has highest "y" postion and writes with it. So I want to know how we can delete that lower position touch drawing happened when it detects the higher postion touch. – Forward

I tried your code, some times it draws some times it doesnt , their is lag in drawing, did you tried it? – Forward

Hello @Krumelur, please look at my comments – Forward

I ran the code, I did not notice any lag, though it is not the most efficient way of drawing since the bitmap is recreated every time you draw a new line segment. – Palmate

Regarding detecting hand vs. finger, there is no silver bullet that will solve all your problems. You have to experiment with a mix of heuristics and user settings. Maybe require the point to move some minimum distance? – Palmate

ok.can you tell me why we need to use NSDictionary, is it required. Secondly, I was just checking bamboo app on ipad, what they do is they are erasing the line which is at lower position when it gets the line at higher position. So any suggestions how that can be achieved.Just give me some directions. – Forward

No, in my code example, you can remove the NSDictionary. I was just keeping your code with minimal change. If you only need one finger, you don't need to store more than one point. – Palmate

let us continue this discussion in chat – Forward

Hello @Krumelur, I increase and decrease my canvas size on demand, so suppose user draws in canvas with size 200*200, then I incrase the canvas size to 200*300 then how we should maintain the previous drawing in 200*200 and next drawing in 300*300 – Forward

Then you need to create a new layer and paint the old layer onto the new layer. – Palmate

you suggest that, everytime, I increase the canvas size, I should create a new Layer and paint the persist layer into newLayer and destroy the persist layer? – Forward

I tried your code, In my case I am using three points, bcoz I want to get a smooth curve, but it is not working, Here is the link for the project dropbox.com/s/jpy7o0xg6oe0c01/MultiTouch.zip, Whenever I touch with two fingers and line is drawn between them, – Forward

In the above code, when I was testing I saw that, for example, if I draw a line, and without taking out my finger, If I try to draw one more line with lower y value, at first it doesnt draw, but if I try second time, it draws. Can you tell me why this is happening and how can we resolve this issue – Forward

Hello @Krumelur, please look at this #21935564 – Forward

Hello @Palmate please look at this #21952774 – Forward

Answer 2

Use a UIPanGestureRecognizer to handle the touches instead, and set its maximumNumberOfTouches property to 1, so that it will only recognize one touch at a time.

The gesture recognizer will handle ignoring things that are just taps, because it's specifically for recognizing pans. Also by setting the max number of touches to 1, once they start writing, no other touches will have any effect on it, it'll automatically continue to track just the first touch.

EDIT:

Here's a simple example, starting with a basic single view application template and replacing everything below the #import statement with the following:

@interface ViewController ()

@property (nonatomic, strong) UIPanGestureRecognizer *panGestureRecognizer;

@property (nonatomic, strong) UIImageView *imageView;

@end

@implementation ViewController

- (void)viewDidLoad
{
    [super viewDidLoad];

    self.imageView = [[UIImageView alloc] initWithFrame:self.view.bounds];
    [self.view addSubview:self.imageView];

    self.panGestureRecognizer = [[UIPanGestureRecognizer alloc] initWithTarget:self action:@selector(performPanGesture:)];
    self.panGestureRecognizer.maximumNumberOfTouches = 1;
    [self.view addGestureRecognizer:self.panGestureRecognizer];
}

- (void)viewDidLayoutSubviews
{
    [super viewDidLayoutSubviews];

    self.imageView.frame = self.view.bounds;
}

- (void)performPanGesture:(UIPanGestureRecognizer *)panGesture
{
    if (panGesture == self.panGestureRecognizer) {
        CGPoint touchLocation = [panGesture locationInView:self.view];
//        NSLog(@"%f, %f", touchLocation.x, touchLocation.y);

        UIGraphicsBeginImageContextWithOptions(self.view.frame.size, YES, 0.0);

        CGContextRef context = UIGraphicsGetCurrentContext();

        [self.view.layer renderInContext:context];

        CGContextAddEllipseInRect(context, CGRectMake(touchLocation.x - 1, touchLocation.y - 1, 3, 3));

        CGContextDrawPath(context, kCGPathFill);

        UIImage *outputImage = UIGraphicsGetImageFromCurrentImageContext();

        UIGraphicsEndImageContext();

        self.imageView.image = outputImage;
    }
}

@end

This will draw little circles on the screen wherever a touch is received. You could also modify it to keep track of the last touch point and draw lines between the points so you have something that's continuous. You can also see that touching with a second finger won't start making points in a new spot, because it only handles one touch, and a second touch is simply ignored.

Tanga

@Forward This answer should work. How is your testing? – Rolling

@sunkehappy, I am not following this answer.I have not tried it. – Forward

@Forward this example does a good job of handling just a single finger, which should take care of your problem. – Speedwell

@Tanga please look at this #21952774 – Forward

Answer 3

When the touches start, take a touch from the set and keep a reference to it. How you decide which one to take is up to you - hopefully there would be only one, or you could check the location of each touch and choose the 'highest' on the screen.

As the touches moved, check that your stored touch is still valid (contained in touches) and, if it is, use it only (you can ignore all the other touches).

---

At its most trivial:

self.trackingTouch = [touches anyObject];

but there are other (better) ways of choosing which touch to store.

---

Not really sure why the docs say You should never retain an UITouch object when handling an event when they also say A UITouch object is persistent throughout a multi-touch sequence.

I haven't seen issues with storing the touch previously, but that doesn't mean it couldn't cause issues in the future (somehow). The alternative, based on A UITouch object is persistent throughout a multi-touch sequence is to store only the pointer to the touch and use it for pointer comparison only (so calling no methods on the UITouch object behind the pointer).

Murage

Hey @Wain, when touches start, there can be many touches, I dont know the count so how can I keep reference to it. And as you can see I am storing all my touch points in NsDictionary – Forward

Just hold an @property of your chosen touch (they aren't recreated each time the handler methods are called. The point is that currently you're looping over all touches and you should just choose one and use that. – Murage

I am getting confused. Can you show me a code snippet. How can I chose a touch? – Forward

documentation says we should never retain a touch object – Forward

Which documentation are you looking at? – Murage

UITouch documentation from apple.developer.apple.com/library/ios/documentation/uikit/reference/… – Forward

so what should I do, should I create a property or no, if no then how should I do it. The line of code which you have written says [touches anyObject].But this will get me only one touch right.So if user first rest his palm then it will get its touch and will reject finger . Please correct me , if I am wrong – Forward

I would (do) store the touch in a property. anyObject is the easy way to prove it works, but they you will want to iterate the touches and check the locations in the view and choose better which one to use... Resting their palm first is an issue, you need to decide how to handle it from a UX point of view - should you cancel that touch (and delete the line) if a new touch starts higher up the screen? How do you want to handle the number of touches changing while the user is drawing. (aside, I have often said 1 touch draws, multiple touches cancel the whole operation...) – Murage

let us continue this discussion in chat – Forward

Hello @Wain, I tried your approach, first simple and then iterating over it. So now I found the highest y function of set of touches and I will draw that and ignore the rest(with lower values), But How to delete these lower position touches. For example , before I draw with my finger, I will rest my hand, so hand touch will be detected and will draw some lines, and when I touch my finger, it detects that , the finger has highest "y" postion and writes with it. So I want to know how we can delete that lower position touch drawing happened when it detects the higher postion touch. – Forward

I would hold the count of touches. If it changes (grows), check if you want to replace the tracking touch. If you do, empty the array of touch points and add the new one for the new tracking touch. If your touch points array holds points from a previous 'session' then I would maintain a new array for each session and only add those points to the main array when each session is fully completed. – Murage

I am confused, can you please elaborate – Forward

let's go back to the chat: chat.stackoverflow.com/rooms/44237/… – Murage

Hello @Murage please look at this #21952774 – Forward

Answer 4

The Documentation for UITouch has changed over the past years, and now it says:

A touch object persists throughout a multi-touch sequence. You may store a reference to a touch while handling a multi-touch sequence, as long as you release that reference when the sequence ends.

So simply keep a reference to the touch while you're tracking the draw gesture, and discard all moved/canceled/ended events for other touches.

Unassuming