54

I have concurrent goroutines which want to append a (pointer to a) struct to the same slice. How do you write that in Go to make it concurrency-safe?

This would be my concurrency-unsafe code, using a wait group:

var wg sync.WaitGroup
MySlice = make([]*MyStruct)
for _, param := range params {
    wg.Add(1)
    go func(param string) {
        defer wg.Done()
        OneOfMyStructs := getMyStruct(param)
        MySlice = append(MySlice, &OneOfMyStructs)
    }(param)
}
wg.Wait()

I guess you would need to use go channels for concurrency-safety. Can anyone contribute with an example?

1

3 Answers 3

58

There is nothing wrong with guarding the MySlice = append(MySlice, &OneOfMyStructs) with a sync.Mutex. But of course you can have a result channel with buffer size len(params) all goroutines send their answers and once your work is finished you collect from this result channel.

If your params has a fixed size:

MySlice = make([]*MyStruct, len(params))
for i, param := range params {
    wg.Add(1)
    go func(i int, param string) {
         defer wg.Done()
         OneOfMyStructs := getMyStruct(param)
         MySlice[i] = &OneOfMyStructs
     }(i, param)
}

As all goroutines write to different memory this isn't racy.

Sign up to request clarification or add additional context in comments.

10 Comments

It's very interesting your last consideration: in case the size of the slice is known and you are just dealing with pointers to the objects, you don't need to use a concurrency mechanism at all
This does not depend on "slice of pointers": It would work also for "slice of MyStruct". Again the code never writes to the same memory.
I was assuming that the memory allocation for a pointer is fixed, while the memory allocation for a struct is not fixed. I suppose I am wrong then.
Hu? What is "fixed"? Any type in Go has a certain memory layout which is determined completely a compile time. No difference between a pointer and something else.
I've read that you should usually not have buffering in channels, unless there's a specific need. For your suggestion about the goroutines writing to a shared channel, isn't that a case where you would just deliver synchronously (unbuffered)?
|
28

The answer posted by @jimt is not quite right, in that it misses the last value sent in the channel and the last defer wg.Done() is never called. The snippet below has the corrections.

https://play.golang.org/p/7N4sxD-Bai

package main

import "fmt"
import "sync"

type T int

func main() {
    var slice []T
    var wg sync.WaitGroup

    queue := make(chan T, 1)

    // Create our data and send it into the queue.
    wg.Add(100)
    for i := 0; i < 100; i++ {
        go func(i int) {
            // defer wg.Done()  <- will result in the last int to be missed in the receiving channel
            queue <- T(i)
        }(i)
    }

    go func() {
        // defer wg.Done() <- Never gets called since the 100 `Done()` calls are made above, resulting in the `Wait()` to continue on before this is executed
        for t := range queue {
            slice = append(slice, t)
            wg.Done()   // ** move the `Done()` call here
        }
    }()

    wg.Wait()

    // now prints off all 100 int values
    fmt.Println(slice)
}

3 Comments

Coming from the future: why does the @jimt solution not work? The wg.Done() is deferred, so it is only called after the value is sent through the channel. What am I missing?
@chris Can I use an unbuffered channel? queue := make(chan T)
using two different sync primitives is overkill, keeping only channel will do the job
1

I wanted to add that since you know how many values you are expecting from the channel, you may not need to make use of any synchronization primitives. Just read from the channel as much data as you are expecting and leave it alone:

borrowing @chris' answer

package main

import "fmt"

type T int

func main() {
    var slice []T

    queue := make(chan T)

    // Create our data and send it into the queue.
    for i := 0; i < 100; i++ {
        go func(i int) {
            queue <- T(i)
        }(i)
    }

    for i := 0; i < 100; i++ {
        select {
        case t := <-queue:
            slice = append(slice, t)
        }
    }

    // now prints off all 100 int values
    fmt.Println(slice)
}

The select will block until the channels receives some data, so we can rely on this behaviour to just read from the channel 100 times before exiting.

In your case, you can just do:

package main

func main() {
    MySlice = []*MyStruct{}
    queue := make(chan *MyStruct)

    for _, param := range params {
        go func(param string) {
            OneOfMyStructs := getMyStruct(param)
            queue <- &OneOfMyStructs
        }(param)
    }

    for _ := range params {
        select {
        case OneOfMyStructs := <-queue:
            MySlice = append(MySlice, OneOfMyStructs)
        }
    }
}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.