In Go I cannot figure out how to get the following setup.
- 1st level go routines should stop their 2nd level goroutines before they terminate.
- If the function
runGoroutineis not in a loop (I don't want it to loop) then how would the goroutine exit when a message gets sent to the channel? If there was no loop thencase <-stopChan:would only get called once at the very start. - What if the function
runGoroutinecalls other async functions? Would those get exited too when the return is hit?
Maybe I am approaching this wrong. In my case I have x task Groups that the user can start, when the user starts a task group a goroutine is made (1st level) to return CL user access. This task group (1st level) is a collection of y API requests (y is 200-300 in practice). For each API request I make a new go-routine (2nd level). I am targeting speed; If I and performing hundreds of API requests a second, is it faster/efficient to put each request into its own goroutine? Or is it the same to do it on 1 thread?
Parent/Main process
| |
Child goroutine Child goroutine
| |
X bottom goroutines X bottom goroutines (API level)
package main
import (
"fmt"
"time"
)
func runGoroutine(stopChan <-chan struct{}) {
for {
select {
case <-stopChan:
return // Exit the goroutine
default:
fmt.Println("Goroutine is working...")
time.Sleep(1 * time.Second)
}
}
}
func main() {
stopChan := make(chan struct{})
// Start the goroutine
go runGoroutine(stopChan)
// User should be able to close the routines they want
close(stopChan)
}
New Code
package main
import (
"context"
"log"
"net/http"
"sync"
"time"
)
func main() {
// Initialize context with cancellation
ctx, cancel := context.WithCancel(context.Background())
// Call the worker pool in a separate goroutine to allow main to return immediately
// We can start any task group like this
go startWorkerPool(ctx, 20, 100)
// User has CL access on main thread
// Main function returns, but `cancel` is accessible to allow later cancellation if needed
log.Println("Worker pool started; you can cancel it by calling cancel()")
waitForNine()
}
func waitForNine() {
var input int
for input != 9 {
fmt.Print("Enter a number: ")
fmt.Scan(&input)
}
}
// startWorkerPool starts a fixed-size worker pool to handle `numRequests` requests.
func startWorkerPool(ctx context.Context, numWorkers, numRequests int) {
var wg sync.WaitGroup
work := make(chan string)
// Launch the specified number of workers
for i := 0; i < numWorkers; i++ {
wg.Add(1)
go func(ctx context.Context) {
defer wg.Done()
client := http.Client{}
for url := range work {
if req, err := http.NewRequestWithContext(ctx, "GET", url, nil); err != nil {
log.Println("failed to create new request:", err)
} else if resp, err := client.Do(req); err != nil {
log.Println("failed to make request:", err)
} else {
resp.Body.Close()
}
}
}(ctx)
}
// Enqueue URLs for workers and close the channel when done
go func() {
for i := 0; i < numRequests; i++ {
work <- "https://httpbin.org/get"
}
close(work)
}()
// Wait for all workers to finish
wg.Wait()
log.Println("All requests processed")
}
