Golang Reusing Memory Address Copying from slice?

Question

I was hitting an issue in a project I'm working on. I found a way around it, but I wasn't sure why my solution worked. I'm hoping that someone more experience with how Go pointers work could help me.

I have a Model interface and a Region struct that implements the interface. The Model interface is implemented on the pointer of the Region struct. I also have a Regions collection which is a slice of Region objects. I have a method that can turn a Regions object into a []Model:

// Regions is the collection of the Region model
type Regions []Region

// Returns the model collection as a list of models
func (coll *Regions) ToModelList() []Model {
    output := make([]Model, len(*coll))
    for idx, item := range *coll {
        output[idx] = &item
    }
    return output
}

When I run this code, I end up with the first pointer to the Region outputted multiple times. So, if the Regions collection has two distinct items, I will get the same address duplicated twice. When I print the variables before I set them in the slice, they have the proper data.

I messed with it a little bit, thinking Go might be reusing the memory address between loops. This solution is currently working for me in my tests:

// Returns the model collection as a list of models
func (coll *Regions) ToModelList() []Model {
    output := make([]Model, len(*coll))
    for idx, _ := range *coll {
        i := (*coll)[idx]
        output[idx] = &i
    }
    return output
}

This gives the expected output of two distinct addresses in the output slice.

This honestly seems like a bug with the range function reusing the same memory address between runs, but I always assume I'm missing something in cases like this.

I hope I explained this well enough for you. I'm surprised that the original solution did not work.

Thanks!

Yabber

Side note, you almost never want to use a pointer to a slice (i.e. your coll *Regions which is effectively *[]Region). Slices are small and already contain a pointer to the data. The only time you'd want to use a pointer receiver for Regions is if you needed to change the length and/or capacity within the method. – Indonesia

Answer 1

In your first (non working) example item is the loop variable. Its address is not changing, only its value. That's why you get the same address in output idx times.

Run this code to see the mechanics in action;

func main() {

    coll := []int{5, 10, 15}

    for i, v := range coll {
       fmt.Printf("This one is always the same; %v\n", &v)
       fmt.Println("This one is 4 bytes larger each iteration; %v\n", &coll[i])
    }
}

Natalee

I was under the impression that since I was storing a reference to the pointer in the list, that Go shouldn't reuse that memory address. Am I mistaken in thinking that? – Yabber

@Yabber give me a few to code up an example. The problem is you're taking the address of item, it's just the loop variable. You actually want to assign it's value since item itself is already a pointer. – Natalee

There is no concept of "reference" in Go. There is not and thinking in terms of references results in bugs. – Stardom

@Natalee - That makes sense. The reason that I was taking the pointer was that the pointer reciever implements the model interface, so I have to actually use the pointer or the compiler yells at me. – Yabber

@Stardom - I think I have to do some more research on how the GC is working, and how Go uses these memory addresses. I'm assuming that you mean it simply stores pointers as memory addresses, but doesn't consider them as referenced memory. Do you have any good links explaining it? – Yabber

@Natalee - Thank you for the example. This makes more sense to me now. I was referencing the temporary variable, just as you had said. I actually had to reference where the data was stored! Thanks for all your help. – Yabber

@Yabber np, glad I can help. – Natalee

Answer 2

There is just one item variable for the entire loop, which is assigned the corresponding value during each iteration of the loop. You do not get a new item variable in each iteration. So you are just repeatedly taking the address of the same variable, which will of course be the same.

On the other hand, if you declared a local variable inside the loop, it will be a new variable in each iteration, and the addresses will be different:

for idx, item := range *coll {
    temp := item
    output[idx] = &temp
}

Scrimshaw

This is a very concise easy to understand description, thanks. Its worth noting that creating a new variable and assigning it to the pointer to the item doesn't work as well. I tried that too :-) – Yabber

Answer 3

Actually, Go is currently a mess. Sometimes the loop variables are reused between iterations, sometimes they are not, and new memory is allocated at each iteration, depending on how the wind blows, more or less.

Example code:

package main

func main() {
    numbers := []int{1, 2, 3}

    //  var result *int

    for iteration, number := range numbers {
        println(iteration, &number)
        //      result = &number
    }

    //  println(result)
}

If you just run the code, without the commented lines, you indeed get the same memory address printed at each iteration, as many have mentioned in the answers:

0 0xc00008fee8
1 0xc00008fee8
2 0xc00008fee8

However, if you uncomment the commented lines, the Go compiler detects that you are storing a reference to the loop variable inside a variable accessible outside the loop, and it will allocate a different memory area for the loop variable at each iteration. So this will be the output of the code with the commented lines included:

0 0xc00000a038
1 0xc00000a050
2 0xc00000a058
0xc00000a058

So, in MOST simple cases it is now, theoretically, safe to store references to memory allocated for the for loop variables.

However, in more complex code, for example iterating on slice of structs and storing the address of some item from a struct in a variable accessible outside the loop, sometimes the compiler fails to detect that and still uses a single piece of memory for the loop variables for all iterations, and then all kind of "interesting" bugs will be introduced if you didn't expect that.

Of course, if you then try to create a concise example that reproduces the Go compiler bug, you will find that it is almost impossible to create one (still working on that, I wasn't able to reproduce this compiler bug on simple code far).

Bottom line: Always assume the memory is reused for the for variables, even if that's not actually always true, unless you don't have a problem with losing hours debugging esoteric Go compiler bugs.

Sukiyaki

Later comment: The behavior seems to have changed between go 1.19 and go 1.22. in 1.19 it cannot detect outside references, and it will not allocate separate memory at each iteration, no matter what. 1.22 can detect outside references and the code behaves like I mentioned in my original answer. – Sukiyaki