How to pass "nothing" as an argument to `[` for subsetting?

Question

I was hoping to be able to construct a do.call formula for subsetting without having to identify the actual range of every dimension in the input array. The problem I'm running into is that I can't figure out how to mimic the direct function x[,,1:n,] , where no entry in the other dimensions means "grab all elements."

Here's some sample code, which fails. So far as I can tell, either [ or do.call replaces my NULL list values with 1 for the index.

x<-array(1:6,c(2,3))
dimlist<-vector('list', length(dim(x)))
shortdim<-2
dimlist[[shortdim]] <- 1: (dim(x)[shortdim] -1)
flipped <- do.call(`[`,c(list(x),dimlist)) 

I suppose I could kludge a solution by assigning the value -2*max(dim(x)) to each element of dimlist, but yuck.
(FWIW, I have alternate functions which do the desired job either via melt/recast or the dreaded "build a string and then eval(parse(mystring)) , but I wanted to do it "better.")

Edit: as an aside, I ran a version of this code (with the equivalent of DWin's TRUE setup) against a function which used melt & acast ; the latter was several times slower to no real surprise.

Immemorial

Need definition of flipdim. And dimList was an empty list after its creation because there didn't happen to be an x object in my workspace. There is now, and it's not clear whether you really wanted dimlist to be 6 items-long. – Ivied

Lazy evaluation alert: (dimlist<-vector('list', length(dim(x))) throws an error. – Ivied

I think I know how. I'm waiting for an example to work on that makes sense. – Ivied

@DWin it works for me. Except I apologize for not putting my statements in the proper order. Fixed now. – Immemorial

I have asked a related Question motivated by @HongOoi's Answer. It would be nice to be able to do what @Dwin shows but via an alist instead of a list. – Leavitt

@GavinSimpson good idea. I'll wait a bit and check DWin's answer here as the best solution. – Immemorial

Related (and contains the answer) Select along one of n dimensions in array – Tablecloth

@JoshuaUlrich thanks for finding that link. So I guess the answer is that one way or another you can't put "nothing" into the call. – Immemorial

Answer 1

After some poking around, alist seems to do the trick:

x <- matrix(1:6, nrow=3)
x
     [,1] [,2]
[1,]    1    4
[2,]    2    5
[3,]    3    6

# 1st row
do.call(`[`, alist(x, 1, ))
[1] 1 4

# 2nd column
do.call(`[`, alist(x, , 2))
[1] 4 5 6

From ?alist:

‘alist’ handles its arguments as if they described function arguments. So the values are not evaluated, and tagged arguments with no value are allowed whereas ‘list’ simply ignores them. ‘alist’ is most often used in conjunction with ‘formals’.

---

A way of dynamically selecting which dimension is extracted. To create the initial alist of the desired length, see here (Hadley, using bquote) or here (using alist).

m <- array(1:24, c(2,3,4))
ndims <- 3
a <- rep(alist(,)[1], ndims)
for(i in seq_len(ndims))
{
    slice <- a
    slice[[i]] <- 1
    print(do.call(`[`, c(list(m), slice)))
}

     [,1] [,2] [,3] [,4]
[1,]    1    7   13   19
[2,]    3    9   15   21
[3,]    5   11   17   23

     [,1] [,2] [,3] [,4]
[1,]    1    7   13   19
[2,]    2    8   14   20

     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

Mahmoud

Not to kill the buzz, but I don't see how this answers the OP's question as far as providing something where the subsetted dimension is dynamic (the input variable short dim)... – Wert

@Wert I don't really understand the bit about dynamic subsetted dimensions (especially since the code in the question doesn't work), but you can use alist in place of list in the call to do.call. So it should also work for that, assuming OP's comment about mimicing [,,n] is relevant. – Mahmoud

@Wert I think this does answer the question. The point is to generate a call to '[' and the OP wants to have empty arguments. – Leavitt

I see you were much faster on the draw than I. Deleting my Answer, but I might edit in the extract from ?alist so it is clear why this works. – Leavitt

@GavinSimpson Will do. – Mahmoud

Trying to clarify: the OP does not know in advance if he wants x[1 ,] or x[, 1]. That's what his (dynamic) variable shortdim will tell. alist has the same synopsis (commas and possibility for missing arguments) as [ so I'm pretty confident it won't solve the OP's problem. – Wert

@Wert It could if an alist of dynamic length could be formed. Once the list is formed (of empty arguments plus x), you can index that like any other list and set alist[[shortdim]] to be what values you want. Creating a dynamic alist seems difficult at the moment though I have a Question asking exactly that. – Leavitt

@Wert You can do this with alist too. The problem really is that you have to create the initial list with eval(parse). – Mahmoud

@Wert and @Gavin : The whole point of this exercise is to avoid eval(parse()) so I don't think alist does anything I couldn't have done by creating the whole string x[, 2:5,,] and evalparsing it (which is what my first go-round does, quite successfully). And yes, I need my arguments to be dynamically generated. So unless I could create something like alist(x,dimlist) this approach won't work. I'll have to go off and try that out. – Immemorial

@CarlWitthoft I realise that, hence my new Question - see comments on your question for link. – Leavitt

after much futzing around, we've finally managed to kick eval(parse) out. – Mahmoud

Answer 2

I've always used TRUE as a placeholder in this instance:

> x
     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6
> do.call("[", list(x, TRUE,1))
[1] 1 2

Let's use a somewhat more complex x example: x <- array(1:36, c(2,9,2), then if the desire is for a vector to be substituted in a list of subscripts that will recover all of the first and second dimensions and only the second "slice" of the third dimension:

shortdim <- 3
short.idx <- 2
dlist <- rep(TRUE, length(dim(x)) )
dlist <- as.list(rep(TRUE, length(dim(x)) ))

> dlist
[[1]]
[1] TRUE

[[2]]
[1] TRUE

[[3]]
[1] TRUE

> dlist[shortdim] <- 2
> do.call("[", c(list(x), dlist) )
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
[1,]   19   21   23   25   27   29   31   33   35
[2,]   20   22   24   26   28   30   32   34   36

Another point sometimes useful is that the logical indices get recycled so you can use c(TRUE,FALSE) to pick out every other item:

(x<-array(1:36, c(2,9,2)))
, , 1

     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
[1,]    1    3    5    7    9   11   13   15   17
[2,]    2    4    6    8   10   12   14   16   18

, , 2

     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
[1,]   19   21   23   25   27   29   31   33   35
[2,]   20   22   24   26   28   30   32   34   36

> x[TRUE,c(TRUE,FALSE), TRUE]
, , 1

     [,1] [,2] [,3] [,4] [,5]
[1,]    1    5    9   13   17
[2,]    2    6   10   14   18

, , 2

     [,1] [,2] [,3] [,4] [,5]
[1,]   19   23   27   31   35
[2,]   20   24   28   32   36

And further variations on every-other-item are possible. Try c(FALSE, FALSE, TRUE) to get every third item starting with item-3.

Ivied

This in my opinion will answer the OP's problem, but you should make that more clear via the use of shortdim. – Wert

If I could, I would. I could not figure out what was meant by shortdim and asked for clarification for the example that I thought was erroneous. – Ivied

As I understand, shortdim is the dimension the OP wants to subset. So your list should have TRUE everywhere except there. – Wert

+1 As there doesn't seem to be an easy way to create and alist() of variable length, it would seem that list() with elements TRUE` would be the easiest way to respond to a dynamic shortdim – Leavitt

@flodel. Thanks for the clarification. Hope the code is clear. – Ivied

@Wert is correct. I hope I caught all the mistakes in my original post. Using TRUE is at least more pleasing than using -2*max(dim(x)) as the default placeholder. – Immemorial

Answer 3

Not a straight answer, but I'll demo asub as an alternative as I am pretty sure this is what the OP is eventually after.

library(abind)

Extract 1st row:

asub(x, idx = list(1), dims = 1)

Extract second and third column:

asub(x, idx = list(2:3), dims = 2)

Remove the last item from dimension shortdim as the OP wanted:

asub(x, idx = list(1:(dim(x)[shortdim]-1)), dims = shortdim)

You can also use negative indexing so this will work too:

asub(x, idx = list(-dim(x)[shortdim]), dims = shortdim)

Last, I will mention that the function has a drop option just like [ does.

Wert

Interesting. I suppose I should confess that my original goal was a "flip" function which reversed an array along the designated dimension. Looks like asub can do this just fine. – Immemorial

Then maybe you meant idx = list(rev(seq_len(dim(x)[shortdim]))) – Wert

Yes, that's what I will do. But I appreciate your solution since it's applicable to all sorts of array manipulations. If I end up selecting DWin's, it's because it solves my question about [ , even though your solution is a great solution to my root problem. Maybe I'll pick a winner after a time trial :-) – Immemorial

If asub is slower, which wouldn't surprise me that much, it will probably be because it does all sorts of checks and returns meaningful error messages (see getAnywhere("asub.default")). Think of asub as a robust, tested, and widely approved implementation of the [ call you were trying to build programmatically. – Wert

... However, looking at the code, it seems that asub is building a [ call with missing args e.g. x[, 2,] which should be noticeably faster than x[TRUE, 2, TRUE] for a large input array. – Wert

Answer 4

Ok, here's the code for four versions, followed by microbenchmark . The speed appears to be pretty much the same for all of these. I'd like to check all answers as accepted, but since I can't, here are the chintzy criteria used: DWin loses because you have to enter "TRUE" for placeholders.
flodel loses because it requires a non-base library My original loses, of course, because of eval(parse()). So Hong Ooi wins. He advances to the next round of Who Wants to be a Chopped Idol :-)

flip1<-function(x,flipdim=1) {
    if (flipdim > length(dim(x))) stop("Dimension selected exceeds dim of input")
    a <-"x["
    b<-paste("dim(x)[",flipdim,"]:1",collapse="")
    d <-"]"
    #now the trick: get the right number of commas
    lead<-paste(rep(',',(flipdim-1)),collapse="")
    follow <-paste(rep(',',(length(dim(x))-flipdim)),collapse="")
    thestr<-paste(a,lead,b,follow,d,collapse="")
    flipped<-eval(parse(text=thestr))
    return(invisible(flipped))
    }       

flip2<-function(x,flipdim=1) {
    if (flipdim > length(dim(x))) stop("Dimension selected exceeds dim of input")
    dimlist<-vector('list', length(dim(x))  )  
    dimlist[]<-TRUE  #placeholder to make do.call happy 
    dimlist[[flipdim]] <- dim(x)[flipdim]:1 
    flipped <- do.call(`[`,c(list(x),dimlist) )
    return(invisible(flipped))
    }       

# and another...
flip3 <- function(x,flipdim=1) {
    if (flipdim > length(dim(x))) stop("Dimension selected exceeds dim of input")
    flipped <- asub(x, idx = list(dim(x)[flipdim]:1), dims = flipdim)
    return(invisible(flipped))
}

#and finally, 
flip4 <- function(x,flipdim=1) {
    if (flipdim > length(dim(x))) stop("Dimension selected exceeds dim of input")
    dimlist <- rep(list(bquote()), length(dim(x)))
    dimlist[[flipdim]] <- dim(x)[flipdim]:1
    flipped<- do.call(`[`, c(list(x), dimlist))
    return(invisible(flipped))
}

Rgames> foo<-array(1:1e6,c(100,100,100))
Rgames> microbenchmark(flip1(foo),flip2(foo),flip3(foo),flip4(foo)


   Unit: milliseconds
       expr      min       lq   median       uq      max neval
 flip1(foo) 18.40221 18.47759 18.55974 18.67384 35.65597   100
 flip2(foo) 21.32266 21.53074 21.76426 31.56631 76.87494   100
 flip3(foo) 18.13689 18.18972 18.22697 18.28618 30.21792   100
 flip4(foo) 21.17689 21.57282 21.73175 28.41672 81.60040   100

Immemorial

Answer 5

You can use substitute() to obtain an empty argument. This can be included in a normal list.

Then, to programmatically generate a variable number of empty arguments, just rep() it:

n <- 4
rep(list(substitute()), n)

Broderickbrodeur