Is there a pythonic way of knowing when the first and last loop in a for is being passed through?

Question

I have a template in which I placed, let's say 5 forms, but all disabled to be posted except for the first one. The next form can only be filled if I click a button that enables it first.

I'm looking for a way to implement a Django-like forloop.last templatetag variable in a for loop inside an acceptance test to decide whether to execute a method that enables the next form or not.

Basically what I need to do is something like:

for form_data in step.hashes:
    # get and fill the current form with data in form_data
    if not forloop.last:
        # click the button that enables the next form
# submit all filled forms

Papistry

This question is similar; take a look at it: https://mcmap.net/q/209154/-quot-for-quot-loop-first-iteration – Instructions

Keep in mind that if you're using break or an exception is raised the loop can end before the "last" iteration of the loop. – Disorganize

Answer 1

If I am understanding your question correctly, you want a simple test for whether you are at the beginning or end of the list?

If that's the case, this would do it:

for item in list:
    if item != list[-1]:
        #Do stuff

For the first item in the list, you would replace "-1" with 0.

Sidestep

But what if there are duplicates in the list? Your list could be [5, 8, 3, 3], and your item != list[-1] would match the next-to-last as well as the last item. – Instructions

@Vanessa correct -- this should be item is not list[-1] and that still won't work correctly for interned values like some short strings, True, False, None, and 0 to 255. – Disorganize

Answer 2

I don't know of anything built-in, but you can easily write a generator to give you the required information:

def firstlast(seq):
    seq = iter(seq)
    el = prev = next(seq)
    is_first = True
    for el in seq:
        yield prev, is_first, False
        is_first = False
        prev = el
    yield el, is_first, True


>>> list(firstlast(range(4)))
[(0, True, False), (1, False, False), (2, False, False), (3, False, True)]
>>> list(firstlast(range(0)))
[]
>>> list(firstlast(range(1)))
[(0, True, True)]
>>> list(firstlast(range(2)))
[(0, True, False), (1, False, True)]
>>> for count, is_first, is_last in firstlast(range(3)):
    print(count, "first!" if is_first else "", "last!" if is_last else "")


0 first! 
1  
2  last!

Karsten

@S.Lott I don't think you can yield before the loop because you need to know whether or not there's another element in the sequence for the is_last value. Feel free to prove me wrong. – Karsten

The 1-element degenerate case is a sticky issue in this question. – Acquirement

Answer 3

You could use enumerate and compare the counter with the length of the list:

for i, form_data in enumerate(step.hashes):
    if i < len(step.hashes):
        whatever()

Feeble

Answer 4

for form_data in step.hashes[:-1]:
    # get and fill the current form with data in form_data
for form_data in step.hashes[-1:]:
    # get and fill the current form with data in form_data
    # click the button that enables the next form
# submit all filled forms

Don't like the repetition of get and fill the current form with data in form_data? Define a function.

Acquirement

Answer 5

Generator with buffer.

def first_last( iterable ):
    i= iter(iterable)
    f= next(i)
    yield f, "first"
    n= next(i)
    for another in i:
        yield n, None
        n= another
    yield n, "last"

for item, state in first_list( iterable ):
    # state is "first", None or "last". 

Zipping two sequences

flags = ["first"] + (len(iterable)-2)*[None] + ["last"]
for item, state in zip( iterable, flags ):
    # state is "first", None or "last".

Acquirement

Your generator doesn't tell you "last" for a sequence with exactly one element. – Karsten

@Duncan. It can't. It has to say "first" for that element. Do you want it to say both "First" and "Last"? Or perhaps "Only"? – Acquirement

I suspect the question is what Gerard wants rather than what I want. – Karsten

Answer 6

I think he wants to have a wrapper around the iterator that provides first / last queries, also the parameter could be an iterator so all sort of len() would faild

Here it is what I came up so far, the trick is to use a double iterator, one that looks ahead one step of the first:

class FirstLastIter(object):

    def __init__(self, seq):
        self._seq_iter = iter(seq)
        self._seq_iter_next = iter(seq)
        self._idx = -1
        self._last = None
        self.next_next()

    @property
    def first(self):
        return self._idx == 0

    @property
    def last(self):
        return self._last == True

    def __iter__(self):
        return self

    def next_next(self):
        try:
            self._seq_iter_next.next()
        except StopIteration:
            self._last = True

    def next(self):
        val = self._seq_iter.next()
        self._idx += 1
        self.next_next()
        return val

for x in FirstLastIter([]):
    print x

iterator = FirstLastIter([1])
for x in iterator:
    print x,iterator.first,iterator.last

iterator = FirstLastIter([1,2,3])
for x in iterator:
    print x,iterator.first,iterator.last

returns:

1 True True
1 True False
2 False False
3 False True

Piercing

Answer 7

Before anyone goes sharpening their torches or igniting the pitch-forks I'm no expert in what is Pythonic, that stated, seems to me that if first and/or last is wanted from a list, in the since of if first or if last within a loop, it seems expected to super the class, and add-in the functionality that is desired... maybe, so what follows is totally pre-alpha version e-1^11% sorta code that may cause havoc if looked at on a Tuesday just the right way...

import sys
## Prevent `.pyc` (Python byte code) files from being generated
sys.dont_write_bytecode = True

from collections import OrderedDict


class MetaList(list):
    """
    Generates list of metadata dictionaries for list types

    ## Useful resources

    - [C Source for list](https://github.com/python/cpython/blob/master/Objects/listobject.c)
    - [Supering `list` and `collections.MutableSequence`](https://mcmap.net/q/161865/-overriding-append-method-after-inheriting-from-a-python-list)
    """

    # List supering methods; maybe buggy but seem to work so far...
    def __init__(self, iterable = [], **kwargs):
        """
        > Could not find what built in `list()` calls the initialized lists during init... might just be `self`...
        > If feeling cleverer check the C source. For now this class will keep a copy

        ## License [GNU_GPL-2](https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html)

            Generates list of metadata dictionaries for lists types
            Copyright (C) 2019  S0AndS0

            This program is free software; you can redistribute it and/or
            modify it under the terms of the GNU General Public License
            as published by the Free Software Foundation; version 2
            of the License.

            This program is distributed in the hope that it will be useful,
            but WITHOUT ANY WARRANTY; without even the implied warranty of
            MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
            GNU General Public License for more details.

            You should have received a copy of the GNU General Public License
            along with this program; if not, write to the Free Software
            Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
        """
        self.metadata = []
        for index, value in enumerate(iterable):
            if isinstance(value, list):
                sub_kwargs = {}
                sub_kwargs.update(kwargs)
                sub_kwargs['address'] = kwargs.get('address', [index])
                sub_list = MetaList(iterable = value, **sub_kwargs)
                self.append(sub_list, **kwargs)
            else:
                self.append(value, **kwargs)

        # Note; supering order matters when using built in methods during init
        super(MetaList, self).__init__(iterable)

    def __add__(self, other):
        """
        Called when adding one list to another, eg `MetaList([1,2,3]) + [9,8,7]`

        - Returns copy of list plus `other`, sorta like `self.extend` but without mutation

        ## Example input

            test_list = MetaList([1 ,2, 3])
            longer_list = test_list + [4, 5, 6]

        ## Example output

            print("#\ttest_list -> {0}".format(test_list))
            #   test_list -> [1, 2, 3]
            print("#\tlonger_list -> {0}".format(longer_list))
            #   longer_list -> [1, 2, 3, 4, 5, 6]
        """
        super(MetaList, self).__add__(other)
        output = MetaList(self)
        output.extend(other)
        return output

    def __setitem__(self, index, item, **kwargs):
        """
        Called when setting values by index, eg `listing[0] = 'value'`, this updates `self` and `self.metadata`
        """
        super(MetaList, self).__setitem__(index, item)

        address = kwargs.get('address', []) + [index]
        value = item

        dictionary = self.__return_dictionary(
            address = address,
            index = index,
            value = value)

        self.metadata[index] = dictionary
        self.__refresh_first()
        self.__refresh_last()
        self.__refresh_indexes(start = index)

    def append(self, item, **kwargs):
        """
        Appends to `self.metadata` an `OrderedDict` with the following keys

        - `address`: `[0]` or `[0, 1, 5]` list of indexes mapping to `value`
        - `index`: `0` or `42` integer of index within current listing
        - `value`: `string`, `['list']`, `{'dict': 'val'}`, etc; not enabled by default
        - `first`: `True`/`False` boolean; item is first in current listing
        - `last`: `True`/`False` boolean; item is last in current listing
        """
        super(MetaList, self).append(item)

        # Update last status of previously last item within `self.metadata`
        if self.metadata:
            self.metadata[-1]['last'] = False

        index = len(self.metadata)
        address = kwargs.get('address', []) + [index]

        value = item

        dictionary = self.__return_dictionary(
            address = address,
            index = index,
            value = value)

        dictionary['first'] = False
        dictionary['last'] = True

        if len(self.metadata) == 0:
            dictionary['first'] = True

        self.metadata += [dictionary]

    def extend(self, listing, **kwargs):
        """
        Extends `self.metadata` with data built from passed `listing`

        - Returns: `None`

        > `kwargs` is passed to `MetaList` when transmuting list types
        """
        super(MetaList, self).extend(listing)
        for index, value in enumerate(listing):
            if isinstance(value, list):
                last_address = []
                if self.metadata:
                    # Grab `address` list minus last item
                    last_address = self.metadata[-1]['address'][0:-1]
                # Add this `index` to `address` list for recursing
                sub_list = MetaList(value, address = last_address + [index], **kwargs)
                self.append(sub_list, **kwargs)
            else:
                self.append(value, **kwargs)

    def insert(self, index, item, **kwargs):
        """
        Inserts `item` at `index` for `self` and dictionary into `self.metadata`

        - Returns: `None`

        Note: `self.metadata[index + 1]` have the following data mutated

        - `data['index']`
        - `data['address']`

        Additionally: `self.metadata[0]` and `self.metadata[-1]` data mutations will occur

        - `data['first']`
        - `data['last']`
        """
        super(MetaList, self).insert(index, item)

        address = kwargs.get('address', []) + [index]
        dictionary = self.__return_dictionary(
            address = address,
            index = index,
            value = item,
            **kwargs)
        self.metadata.insert(index, dictionary)

        self.__refresh_first()
        self.__refresh_last()
        self.__refresh_indexes(start = index)
        # Off-set to avoid n +- 1 errors ;-)
        self.__refresh_addresses(
            start = index + 1,
            index = len(address) - 1,
            modifier = 1)

    def pop(self, index = -1, target = None):
        """
        Pop value from `self` and `self.metadata`, at `index`

        - Returns: `self.pop(i)` or `self.metadata.pop(i)` depending on `target`
        """
        popped_self = super(MetaList, self).pop(index)
        popped_meta = self.__pop_metadata(index)

        if 'metadata' in target.lower():
            return popped_meta

        return popped_self

    def remove(self, value):
        """
        Removes `value` from `self` and `self.metadata` lists

        - Returns: `None`
        - Raises: `ValueError` if value does not exsist within `self` or `self.metadata` lists
        """
        super(MetaList, self).remove(value)
        productive = False
        for data in self.metadata:
            if data['value'] == value:
                productive = True
                self.__pop_metadata(data['index'])
                break

        if not productive:
            raise ValueError("value not found in MetaList.metadata values")

    # Special herbs and spices for keeping the metadata fresh
    def __pop_metadata(self, index = -1):
        """
        Pops `index` from `self.metadata` listing, last item if no `index` was passed

        - Returns: `<dictionary>`
        - Raises: `IndexError` if `index` is outside of listed range
        """
        popped_metadata = self.metadata.pop(index)
        addr_index = len(popped_metadata['address']) - 1

        ## Update values within `self.metadata` dictionaries
        self.__refresh_first()
        self.__refresh_last()
        self.__refresh_indexes(start = index)
        self.__refresh_addresses(start = index, index = addr_index, modifier = -1)

        return popped_metadata

    def __return_dictionary(self, address, index, value, **kwargs):
        """
        Returns dictionaries for use in `self.metadata` that contains;

        - `address`: list of indexes leading to nested value, eg `[0, 4, 2]`
        - `index`: integer of where value is stored in current listing
        - `value`: Duck!... Note list types will be converted to `MetaList`
        - `first`: boolean `False` by default
        - `last`: boolean `False` by default

        > `kwargs`: passes through to `MetaList` if transmuting a list `value`
        """
        if isinstance(value, list):
            kwargs['address'] = address
            value = MetaList(value, **kwargs)

        dictionary = OrderedDict()
        dictionary['address'] = address
        dictionary['index'] = index
        dictionary['value'] = value
        dictionary['first'] = False
        dictionary['last'] = False
        return dictionary

    def __refresh_indexes(self, start = 0):
        """
        Update indexes from `start` till the last

        - Returns: `None`
        """
        for i in range(start, len(self.metadata)):
            self.metadata[i]['index'] = i

    def __refresh_addresses(self, start = 0, end = None, index = 0, modifier = -1):
        """
        Updates `address`es within `self.metadata` recursively

        - Returns: `None`
        - Raises: `TODO`

        > `index` is the *depth* within `address` that `modifier` will be applied to
        """
        if not start or start < 0:
            start = 0
        if not end or end > len(self.metadata):
            end = len(self.metadata)

        for i in range(start, end):
            metadata = self.metadata[i]
            if isinstance(metadata['value'], list):
                metadata['value'].__refresh_addresses(index = index, modifier = modifier)
            else:
                if len(metadata['address']) - 1 >= index:
                    metadata['address'][index] += modifier
                else:
                    raise Exception("# TODO: __refresh_addresses append or extend address list")

    def __refresh_last(self, quick = True):
        """
        Sets/re-sets `self.metadata` `last` value

        - Returns `True`/`False` based on if `self.metadata` was touched

        If `quick` is `False` all items in current listing will be touched
        If `quick` is `True` only the last item and second to last items are touched
        """
        if not self.metadata:
            return False

        if len(self.metadata) > 1:
            self.metadata[-2]['last'] = False

        if not quick and len(self.metadata) > 1:
            for i in range(0, len(self.metadata) - 1):
                self.metadata[i]['last'] = False

        self.metadata[-1]['last'] = True
        return True

    def __refresh_first(self, quick = True):
        """
        Sets first dictionary within `self.metadata` `first` key to `True`

        - Returns `True`/`False` based on if `self.metadata` was touched

        If `quick` is `False` all items will be touched in current listing
        If `quick` is `True` the first and second items are updated
        """
        if not self.metadata:
            return False

        if len(self.metadata) > 1:
            self.metadata[1]['first'] = False

        if not quick and len(self.metadata) > 1:
            for i in range(1, len(self.metadata)):
                self.metadata[i]['first'] = False

        self.metadata[0]['first'] = True
        return True

    # Stuff to play with
    def deep_get(self, indexes, iterable = None):
        """
        Loops over `indexes` returning inner list or value from `self.metadata`

        - `indexes` list of indexes, eg `[1, 3, 2]`
        - `iterable` maybe list, if not provided `self.metadata` is searched
        """
        referance = self.metadata
        if iterable:
            reference = iterable

        for index in indexes:
            reference = reference[index]

        return reference

    def copy_metadata(self):
        """
        Returns copy of `self.metadata`
        """
        return list(self.metadata)

    def yield_metadata(self, iterable = None, skip = {'first': False, 'last': False, 'between': False}, **kwargs):
        """
        Yields a *flat* representation of `self.metadata`,

        Prefilter via `skip = {}` dictionary with the following data

        - `first`: boolean, if `True` skips items that are first
        - `last`: boolean, if `True` skips items that are last
        - `between`: boolean, if `True` skips items that are not last or first
        """
        metadata = self.metadata
        if iterable:
            metadata = MetaList(iterable).metadata

        for item in metadata:
            if isinstance(item.get('value'), list):
                # Recurse thy self
                for data in item['value'].yield_metadata(skip = skip, **kwargs):
                    yield data
            else:
                if skip:
                    if skip.get('first', False) and item['first']:
                        continue
                    if skip.get('last', False) and item['last']:
                        continue
                    if skip.get('between', False) and not item['first'] and not item['last']:
                        continue
                # If not skipped get to yielding
                yield item

... and it maybe buggier than the light fixtures of that one friend that talked to publicly of close encounters, they know who they are... but this does do some nifty tricks

Example input one

meta_list = MetaList([1, 2, 3, 4, 5])

for data in meta_list.metadata:
    if data['first']:
        continue
    if data['last']:
        continue

    print("self[{0}] -> {1}".format(data['index'], data['value']))

Example output one

self[1] -> 2
self[2] -> 3
self[3] -> 4

Example input two

meta_list = MetaList(['item one', ['sub item one', ('sub', 'tuple'), [1, 2, 3], {'key': 'val'}], 'item two'])

for data in meta_list.yield_metadata():
    address = "".join(["[{0}]".format(x) for x in data.get('address')])
    value = data.get('value')
    print("meta_list{0} -> {1} <- first: {2} | last: {3}".format(address, value, data['first'], data['last']))

Example output two

meta_list[0] -> item one <- first: True | last: False
meta_list[1][0] -> sub item one <- first: True | last: False
meta_list[1][1] -> ('sub', 'tuple') <- first: False | last: False
meta_list[1][2][0] -> 1 <- first: True | last: False
meta_list[1][2][1] -> 2 <- first: False | last: False
meta_list[1][2][2] -> 3 <- first: False | last: True
meta_list[1][3] -> {'key': 'val'} <- first: False | last: True
meta_list[2] -> item two <- first: False | last: True

If you feel your brains go all minty fresh, but it kinda isn't totally okay and that is in it's own way all the better... to me that be the most Pythonic

Enjoy, and perhaps if there be interest I'll push this to GitHub for all to Pull and Fork.

Side note @fabrizioM +1 for superb use of @property magics

Tightfisted