How does zero allocation work when dealing with strings and byte slices in Go? When converting string ↔ []byte is new memory allocated, and when can allocation be avoided?

Answer.

In Go, conversion between string and []byte usually triggers new memory allocation because strings are immutable while byte slices are mutable. There are exceptions due to some internal compiler optimizations (escape analysis), but they are not guaranteed and may change between Go versions.

Direct conversion:

• []byte(s string): always allocates a new byte array and copies data.
• string(b []byte): also always copies data from the byte array into a new string.

Zero allocation is an approach where we avoid unnecessary allocations. The Go standard library has an unsafe conversion through the unsafe package, allowing you to refer to the same data without copying. This should only be used with full understanding of the risks involved.

import (
    "reflect"
    "unsafe"
)

func BytesToString(b []byte) string {
    return *(*string)(unsafe.Pointer(&b))
}

func StringToBytes(s string) []byte {
    sh := (*reflect.StringHeader)(unsafe.Pointer(&s))
    bh := reflect.SliceHeader{
        Data: sh.Data,
        Len:  sh.Len,
        Cap:  sh.Len,
    }
    return *(*[]byte)(unsafe.Pointer(&bh))
}

Everywhere except for special cases, it’s better to avoid these practices as they break data safety (you can change the contents of a string via a shared slice, which can lead to errors).

---

Tricky Question.

It's often asked: "Is it true that there won't be allocation when converting string to []byte if the string is small or constant?"

Correct answer: No. Regardless of the string size, the compiler will always create a new byte slice and copy the contents. The only exceptions are in unsafe optimizations, which do not guarantee safety and are not supported by the official Go documentation.

---

Examples of real errors due to ignorance of the intricacies of the topic.

---

Story

The team was writing a high-load parsing service for logs and constantly converted incoming strings to byte slices and back for processing. During peak situations, the garbage collector spent up to 30% of CPU time collecting short-lived copies. Profiling revealed that each conversion between string↔[]byte allocated a separate memory area. After implementing pools and redesigning the API, a significant portion of conversions could be eliminated, halving the load on the GC.

---

Story

One of the developers optimized JSON handling using unsafe conversion bytes→string to avoid allocations. Initially, the performance boost was noticeable, but after a month crashes occurred: a byte buffer was reused, resulting in the string pointing to old modified data. The issue could only be fixed by reverting to standard copies and redesigning the interprocess API.

---

Story

When transmitting large binary data over the network, they decided to "optimize" serialization using BytesToString (without copying). One day, the string being sent became publicly visible, and the content of the byte slice was immediately overwritten, leading to junk being sent and leaking private parts of data into the error packet log. Ultimately, memory deduplication resulted in leaking private data!