Preserving Order in Go Concurrency

Solving the problem of lost ordering in Async operation

Imagine this scenario: You’re building an API that fetches a list of products from your database. Each product needs to be decorated with pricing and availability data from other services before being returned to your clients. When I first encountered this problem, my initial instinct was to leverage Go’s powerful concurrency — spinning up goroutines to fetch this additional data asynchronously. However, I quickly ran into a common challenge: the results were being returned out of order, causing confusion for users expecting items in a specific sequence.

Why does this happen, and how can you ensure your asynchronous Go code maintains deterministic ordering? Let’s explore this subtle but significant challenge together.

The Problem: Lost Ordering

When using goroutines in Go, there’s no guarantee on the order of completion. Each goroutine might finish at different times due to network latency, processing time, or other factors.

For example, consider a scenario where you’re fetching details for a list of user profiles:

items := fetchUserProfiles() // returns []Profile
resultsChan := make(chan DecoratedProfile)
for _, item := range items {
    go func(it Profile) {
        decorated := fetchAdditionalDetails(it)
        resultsChan <- decorated
    }(item)
}
var decoratedProfiles []DecoratedProfile
for range items {
    decoratedProfiles = append(decoratedProfiles, <-resultsChan)
}

Here, decoratedProfiles could easily be out-of-order, disrupting the user's expected experience.

The Goal: Deterministic Order

To return items in their original order, explicitly manage their positions.

Solution #1: Use an Indexed Result

Passing along an index allows you to reconstruct the original order later:

items := fetchUserProfiles()
type indexedResult struct {
    idx int
    profile DecoratedProfile
}
resultsChan := make(chan indexedResult)
for idx, item := range items {
    go func(i int, it Profile) {
        decorated := fetchAdditionalDetails(it)
        resultsChan <- indexedResult{idx: i, profile: decorated}
    }(idx, item)
}
orderedResults := make([]DecoratedProfile, len(items))
for range items {
    res := <-resultsChan
    orderedResults[res.idx] = res.profile
}

Using this method, your API response reliably preserves the intended order.

Solution #2: Using sync.WaitGroup

Another straightforward solution involves using sync.WaitGroup:

var wg sync.WaitGroup
items := fetchUserProfiles()
orderedResults := make([]DecoratedProfile, len(items))
for idx, item := range items {
    wg.Add(1)
    go func(i int, it Profile) {
        defer wg.Done()
        orderedResults[i] = fetchAdditionalDetails(it)
    }(idx, item)
}
wg.Wait()

This approach ensures all goroutines finish execution and preserves the original sequence.

Which to Use?

• Indexed Channel Approach: Ideal for streaming results or needing flexibility in handling individual items.
• WaitGroup Approach: Simpler, clean, and suitable for batch operations where completion of all tasks is required.

Final Thoughts

Concurrency can dramatically improve performance, but it demands careful handling. Keeping the original order intact is crucial, particularly when user experience depends heavily on predictable sequencing.