Is Perl's flip-flop operator bugged? It has global state, how can I reset it?

Question

I'm dismayed. OK, so this was probably the most fun Perl bug I've ever found. Even today I'm learning new stuff about Perl. Essentially, the flip-flop operator .. which returns false until the left-hand-side returns true, and then true until the right-hand-side returns false keep global state (or that is what I assume.)

Can I reset it (perhaps this would be a good addition to Perl 4-esque hardly ever used reset())? Or, is there no way to use this operator safely?

I also don't see this (the global context bit) documented anywhere in perldoc perlop is this a mistake?

Code

use feature ':5.10';
use strict;
use warnings;

sub search {
    my $arr = shift;
    grep { !( /start/ .. /never_exist/ ) } @$arr;
}

my @foo = qw/foo bar start baz end quz quz/;
my @bar = qw/foo bar start baz end quz quz/;

say 'first shot - foo';
say for search \@foo;

say 'second shot - bar';
say for search \@bar;

---

Spoiler

$ perl test.pl
first shot
foo
bar
second shot

Sulfonal

Good question! Looks like its acting like a closure. – Pheon

Oh i encountered this - I wrote a function using the flipflop, called it twice and the second time it had preserved state from the first. broke my app! – Masinissa

Wow, I never even knew reset existed. ++ just for that. :) – Contrecoup

@friedo, for shits and giggles, google VarStructor ah, the good ole' days. – Sulfonal

ah, good old VarStructor, a blast from the past – Abiosis

Answer 1

Can someone clarify what the issue with the documentation is? It clearly indicates:

Each ".." operator maintains its own boolean state.

There is some vagueness there about what "Each" means, but I don't think the documentation would be well served by a complex explanation.

Note that Perl's other iterators (each or scalar context glob) can lead to the same problems. Because the state for each is bound to a particular hash, not a particular bit of code,each can be reset by calling (even in void context) keys on the hash. But for glob or .., there is no reset mechanism available except by calling the iterator until it is reset. A sample glob bug:

sub globme {
    print "globbing $_[0]:\n";
    print "got: ".glob("{$_[0]}")."\n" for 1..2;
}
globme("a,b,c");
globme("d,e,f");
__END__
globbing a,b,c:
got: a
got: b
globbing d,e,f:
got: c
Use of uninitialized value in concatenation (.) or string at - line 3.
got: 

For the overly curious, here are some examples where the same .. in the source is a different .. operator:

Separate closures:

sub make_closure {
    my $x;
    return sub {
        $x if 0;  # Look, ma, I'm a closure
        scalar( $^O..!$^O ); # handy values of true..false that don't trigger ..'s implicit comparison to $.
    }
}
print make_closure()->(), make_closure()->();
__END__
11

Comment out the $x if 0 line to see that non-closures have a single .. operation shared by all "copies", with the output being 12.

Threads:

use threads;
sub coderef { sub { scalar( $^O..!$^O ) } }
coderef()->();
print threads->create( coderef() )->join(), threads->create( coderef() )->join();
__END__
22

Threaded code starts with whatever the state of the .. had been before thread creation, but changes to its state in the thread are isolated from affecting anything else.

Recursion:

sub flopme {
    my $recurse = $_[0];
    flopme($recurse-1) if $recurse;
    print " "x$recurse, scalar( $^O..!$^O ), "\n";
    flopme($recurse-1) if $recurse;
}
flopme(2)
__END__
1
 1
2
  1
3
 2
4

Each depth of recursion is a separate .. operator.

Abiosis

Answer 2

The trick is not use the same flip-flop so you have no state to worry about. Just make a generator function to give you a new subroutine with a new flip-flop that you only use once:

sub make_search {
    my( $left, $right ) = @_;
    sub {
        grep { !( /\Q$left\E/ .. /\Q$right\E/ ) } @{$_[0]};
        }
}

my $search_sub1 = make_search( 'start', 'never_existed' );
my $search_sub2 = make_search( 'start', 'never_existed' );


my @foo = qw/foo bar start baz end quz quz/;

my $count1 = $search_sub1->( \@foo );
my $count2 = $search_sub2->( \@foo );

print "count1 $count1 and count2 $count2\n";

I also write about this in Make exclusive flip-flop operators.

Vanhomrigh

Answer 3

The "range operator" .. is documented in perlop under "Range Operators". Looking through the doucmentation, it appears that there isn't any way to reset the state of the .. operator. Each instance of the .. operator keeps its own state, which means there isn't any way to refer to the state of any particular .. operator.

It looks like it's designed for very small scripts such as:

if (101 .. 200) { print; }

The documentation states that this is short for

if ($. == 101 .. $. == 200) { print; }

Somehow the use of $. is implicit there (toolic points out in a comment that that's documented too). The idea seems to be that this loop runs once (until $. == 200) in a given instance of the Perl interpreter, and therefore you don't need to worry about resetting the state of the .. flip-flop.

This operator doesn't seem too useful in a more general reusable context, for the reasons you've identified.

Cotonou

right, I didn't see in the docs ''where the global context of the operator was mentioned'' -- while I mentioned perldoc perlop, I didn't qualify the doc-question enough sorry for the confusion. – Sulfonal

Answer 4

A workaround/hack/cheat for your particular case is to append the end value to your array:

sub search { 
  my $arr = shift;
  grep { !( /start/ .. /never_exist/ ) } @$arr, 'never_exist';
} 

This will guarantee that the RHS of range operator will eventually be true.

Of course, this is in no way a general solution.

In my opinion, this behavior is not clearly documented. If you can construct a clear explanation, you could apply a patch to perlop.pod via perlbug.

Saltwort

This is of course a simple valid workaround (a viable solution), and I up-voted all of the answers on this question. But, it currently I'm looking to learn more about the operator. – Sulfonal

Or don't push it onto the array in the first place: grep { !( /start/ .. /never_exist/ ) } (@$arr, 'never_exist'); – Unbend

Answer 5

I found this problem, and as far as I know there's no way to fix it. The upshot is - don't use the .. operator in functions, unless you are sure you are leaving it in the false state when you leave the function, otherwise the function may return different output for the same input (or exhibit different behaviour for the same input).

Masinissa

This should probably be made clear in the documentation. pokes brian d foy – Unarm

I can submit a doc patch tomorrow, but it would seem as if it was better fixed by patching perl to make the .. not so global, and possibly reset with reset() – Sulfonal

I added a very short note to perlop, but I thought it was clear before. The problem always comes in when you read in things that aren't there, like assuming its scoped when nothing says that it is. – Vanhomrigh

Answer 6

Each use of the .. operator maintains its own state. Like Alex Brown said, you need to leave it in the false state when you leave the function. Maybe you could do something like:

sub search {
  my $arr = shift;
  grep { !( /start/ || $_ eq "my magic reset string" ..
            /never_exist/ || $_ eq "my magic reset string" ) } 
      (@$arr, "my magic reset string");
}

Gariepy

I wouldn't use the word use. I think it is more accurate to say appearance in the source – Sulfonal

@Evan Carroll: see my answer for where that's inaccurate. – Abiosis

Yep yep, you're obviously the better perl'er great post. (I would have guessed the threads) – Sulfonal