Find immediate neighbors by group using data table or igraph

Question

I have a data.table:

groups <- data.table(group = c("A", "B", "C", "D", "E", "F", "G"), 
                     code_1 = c(2,2,2,7,8,NA,5),
                     code_2 = c(NA,3,NA,3,NA,NA,2),
                     code_3 = c(4,1,1,4,4,1,8))

group code_1 code_2 code_3
  A      2     NA      4
  B      2      3      1
  C      2     NA      1
  D      7      3      4
  E      8     NA      4
  F     NA     NA      1
  G      5      2      8

What I would like to achieve, is for each group to find the immediate neighbors based on the available codes. For example: Group A has immediate neighbors groups B, C due to code_1 (code_1 is equal to 2 in all groups) and has immediate neighbor groups D,E due to code_3 (code_3 is equal to 4 in all those groups).

What I tried is for each code, subsetting the first column (group) based on the matches as follows:

groups$code_1_match = list()
for (row in 1:nrow(groups)){

  set(groups, i=row, j="code_1_match", list(groups$group[groups$code_1[row] == groups$code_1]))
}

  group code_1 code_2 code_3          code_1_match
    A      2     NA      4              A,B,C,NA
    B      2      3      1              A,B,C,NA
    C      2     NA      1              A,B,C,NA
    D      7      3      4                  D,NA
    E      8     NA      4                  E,NA
    F     NA     NA      1 NA,NA,NA,NA,NA,NA,...
    G      5      2      8                  NA,G

This "kinda" works but I would assume there is a more data table kind of way of doing this. I tried

groups[, code_1_match_2 := list(group[code_1 == groups$code_1])]

But this doesn't work.

Am I missing some obvious data table trick to deal with it?

My ideal case result would look like this (which currently would require using my method for all 3 columns and then concatenating the results):

group code_1 code_2 code_3    Immediate neighbors
  A      2     NA      4         B,C,D,E
  B      2      3      1         A,C,D,F
  C      2     NA      1         A,B,F
  D      7      3      4           B,A
  E      8     NA      4           A,D
  F     NA     NA      1           B,C
  G      5      2      8           

Encroach

Could be done using igraph. – Nanette

My aim is to feed the result to igraph to create an adjacency matrix. If I am missing some functionality which would do that please point me to it, that would be really helpful! – Encroach

@Nanette please consider posting a solution involving igraph, it could be really interesting. – Lamar

@Lamar posted, though thinking there might be a better igraph way of doing it. – Nanette

Answer 1

Using igraph, get 2nd degree neighbours, drop numeric nodes, paste remaining nodes.

library(data.table)
library(igraph)

# reshape wide-to-long
x <- melt(groups, id.vars = "group")[!is.na(value)]

# convert to graph
g <- graph_from_data_frame(x[, .(from = group, to = paste0(variable, "_", value))])

# get 2nd degree neighbours
x1 <- ego(g, 2, nodes = groups$group)

# prettify the result
groups$res <- sapply(seq_along(x1), function(i) toString(intersect(names(x1[[ i ]]),
                                                                   groups$group[ -i ])))

#    group code_1 code_2 code_3        res
# 1:     A      2     NA      4 B, C, D, E
# 2:     B      2      3      1 A, C, D, F
# 3:     C      2     NA      1    A, B, F
# 4:     D      7      3      4    B, A, E
# 5:     E      8     NA      4       A, D
# 6:     F     NA     NA      1       B, C
# 7:     G      5      2      8           

---

More info

This is how our data looks like before converting to igraph object. We want to ensure code1 with value 2 is different from code2 with value 2, etc.

x[, .(from = group, to = paste0(variable, "_", value))]
#     from       to
#  1:    A code_1_2
#  2:    B code_1_2
#  3:    C code_1_2
#  4:    D code_1_7
#  5:    E code_1_8
#  6:    G code_1_5
#  7:    B code_2_3
#  8:    D code_2_3
#  9:    G code_2_2
# 10:    A code_3_4
# 11:    B code_3_1
# 12:    C code_3_1
# 13:    D code_3_4
# 14:    E code_3_4
# 15:    F code_3_1
# 16:    G code_3_8

Here is how our network looks like:

Note that A..G nodes are always connected through code_x_y. So we need to get the 2nd degree, ego(..., order = 2) gives us neighbours up to including 2nd degree neighbours, and returns a list object.

To get the names:

lapply(x1, names)
# [[1]]
# [1] "A"        "code_1_2" "code_3_4" "B"        "C"        "D"        "E"       
# 
# [[2]]
# [1] "B"        "code_1_2" "code_2_3" "code_3_1" "A"        "C"        "D"        "F"       
# 
# [[3]]
# [1] "C"        "code_1_2" "code_3_1" "A"        "B"        "F"       
# 
# [[4]]
# [1] "D"        "code_1_7" "code_2_3" "code_3_4" "B"        "A"        "E"       
# 
# [[5]]
# [1] "E"        "code_1_8" "code_3_4" "A"        "D"       
# 
# [[6]]
# [1] "F"        "code_3_1" "B"        "C"       
# 
# [[7]]
# [1] "G"        "code_1_5" "code_2_2" "code_3_8"

To prettify the result, we need to remove code_x_y nodes and the origin node (1st node)

sapply(seq_along(x1), function(i) toString(intersect(names(x1[[ i ]]), groups$group[ -i ])))
#[1] "B, C, D, E" "A, C, D, F" "A, B, F"    "B, A, E"    "A, D"       "B, C"       ""   

Nanette

Without being an expert in igraph, this looks really strange. It seems to work :) If I understand it correctly it first creates a graph where the codes are the immediate neighbors and then it finds the actual immediate neighbors as the second neighbors from that graph? – Encroach

@Encroach btw not expert at all, just like to solve igraph problems sometimes. Still waiting on some expert to suggest a better way. – Nanette

Yeah I am considering offering a bounty just in case. But let's see in 2 days :) – Encroach

Answer 2

There is probably some more practical way of achieving this but you could do something like this, using melts and joins:

mgrp <- melt(groups, id.vars = "group")[!is.na(value)]
setkey(mgrp, variable, value)
for (i in seq_along(groups$group)) {
  let = groups$group[i]
  set(
    groups, 
    i = i, 
    j = "inei", 
    value = list(mgrp[mgrp[group == let], setdiff(unique(group), let)])
  )
}

groups
#    group code_1 code_2 code_3    inei
# 1:     A      2     NA      4 B,C,D,E
# 2:     B      2      3      1 A,C,D,F
# 3:     C      2     NA      1   A,B,F
# 4:     D      7      3      4   B,A,E
# 5:     E      8     NA      4     A,D
# 6:     F     NA     NA      1     B,C
# 7:     G      5      2      8       

Porpoise

Answer 3

As mentioned by zx8754, using data.table::melt with combn and then igraph::as_adjacency_matrix

library(data.table)
df <- melt(groups, id.vars="group", na.rm=TRUE)[,
    if (.N > 1L) transpose(combn(group, 2L, simplify=FALSE)), value][, (1) := NULL]

library(igraph)
as_adjacency_matrix(graph_from_data_frame(df, FALSE))

output:

7 x 7 sparse Matrix of class "dgCMatrix"
  A B C E D G F
A . 1 1 1 1 1 .
B 1 . 2 . 1 1 1
C 1 2 . . . 1 1
E 1 . . . 1 1 .
D 1 1 . 1 . . .
G 1 1 1 1 . . .
F . 1 1 . . . .

---

or without using igraph

x <- df[, unique(c(V1, V2))]
df <- rbindlist(list(df, data.table(x, x)))
tab <- table(df)   #or xtabs(~ V1 + V2, data=df)
ans <- t(tab) + tab
diag(ans) <- 0L
ans

output:

   V1
V2  A B C D E F G
  A 0 1 1 1 1 0 1
  B 1 0 2 1 0 1 1
  C 1 2 0 0 0 1 1
  D 1 1 0 0 1 0 0
  E 1 0 0 1 0 0 1
  F 0 1 1 0 0 0 0
  G 1 1 1 0 1 0 0

Ballyrag

Could xtabs create a similar output as the igraph step? – Smaltite

This is a really helpful and (to my eyes) elegant answer, thank you! – Encroach

@Cole, yeah can use table or xtabs – Ballyrag

Answer 4

This is inspired by @sindri_baldur's melt. This solution:

1. Melts the groups
2. Performs a cartesian self-join.
3. Pastes together all the groups that matches.
4. Joins back to the original DT

library(data.table)
#> Warning: package 'data.table' was built under R version 3.6.2
groups <- data.table(group = c("A", "B", "C", "D", "E", "F", "G"), code_1 = c(2,2,2,7,8,NA,5), code_2 = c(NA,3,NA,3,NA,NA,2), code_3=c(4,1,1,4,4,1,8))

molten_grps = melt(groups, measure.vars = patterns("code"), na.rm = TRUE)

inei_dt = molten_grps[molten_grps,
            on = .(variable, value),
            allow.cartesian = TRUE
            ][,
              .(inei = paste0(setdiff(i.group, .BY[[1L]]), collapse = ", ")),
              by = group]

groups[inei_dt, on = .(group), inei := inei]

groups
#>     group code_1 code_2 code_3       inei
#>    <char>  <num>  <num>  <num>     <char>
#> 1:      A      2     NA      4 B, C, D, E
#> 2:      B      2      3      1 A, C, D, F
#> 3:      C      2     NA      1    A, B, F
#> 4:      D      7      3      4    B, A, E
#> 5:      E      8     NA      4       A, D
#> 6:      F     NA     NA      1       B, C
#> 7:      G      5      2      8

Smaltite