A generic, goroutine-safe buffer for batching any type of entry — log events, metrics, webhooks, DB rows. Flush on size, on a timer, or on demand.
Four methods. Seven options. Does one thing and does it well.
Set WithMaxEntries(n) and the buffer auto-flushes the moment it fills up — no polling, no guesswork.
WithFlushInterval(d) starts a background goroutine that drains the buffer on a ticker. Set to 0 to disable.
Call FlushSync() before your process exits and every pending entry makes it out — clean shutdown guaranteed.
Return a Temporary() error from your handler and the buffer retries up to MaxRetries before calling your error handler.
WithFlushTimeout(d) wraps each flush in a bounded context. Slow downstreams can't stall your pipeline.
Mutex-protected internals. Push from as many goroutines as you like — no external locking needed.
Push entries and watch the buffer behave exactly as it would in production.
Real, production-grade patterns.
package main
import (
"context"
"fmt"
"time"
buffer "github.com/dogxhood/go-buffer"
)
func main() {
// Flush every 5 s or when 10 entries accumulate.
b := buffer.New(
buffer.WithFlushHandler(func(ctx context.Context, entries []interface{}) error {
fmt.Printf("flushing %d entries\n", len(entries))
return nil
}),
buffer.WithMaxEntries(10),
buffer.WithFlushInterval(5 * time.Second),
)
defer b.Stop()
for i := 0; i < 42; i++ {
b.Push(fmt.Sprintf("event_%d", i))
}
b.FlushSync() // drain before exit
}package logger
import (
"context"
"encoding/json"
"net/http"
"time"
buffer "github.com/dogxhood/go-buffer"
)
type LogEvent struct {
Level string
Message string
At time.Time
}
// NewLogPipeline batches log events and ships them in bulk.
func NewLogPipeline(endpoint string) *buffer.Buffer {
client := &http.Client{Timeout: 5 * time.Second}
return buffer.New(
buffer.WithMaxEntries(100),
buffer.WithFlushInterval(10 * time.Second),
buffer.WithFlushTimeout(8 * time.Second),
buffer.WithFlushHandler(func(ctx context.Context, entries []interface{}) error {
batch := make([]LogEvent, 0, len(entries))
for _, e := range entries {
batch = append(batch, e.(LogEvent))
}
body, _ := json.Marshal(batch)
req, _ := http.NewRequestWithContext(ctx, "POST", endpoint, body)
_, err := client.Do(req)
return err
}),
)
}// Implement the Temporary() interface to signal a retryable error.
type tempErr struct{ msg string }
func (e *tempErr) Error() string { return e.msg }
func (e *tempErr) Temporary() bool { return true }
b := buffer.New(
buffer.WithMaxRetries(3),
buffer.WithFlushHandler(func(ctx context.Context, entries []interface{}) error {
if upstream.Unavailable() {
return &tempErr{"upstream down"} // retried up to 3×
}
return upstream.Ship(entries)
}),
buffer.WithErrorHandler(func(err error) {
// only called after MaxRetries exhausted
log.Printf("permanent failure: %v", err)
}),
)quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
go func() {
for i := 0; ; i++ {
b.Push(fmt.Sprintf("metric_%d", i))
}
}()
<-quit
log.Println("shutting down — draining buffer…")
b.Stop() // stop the interval goroutine
b.FlushSync() // block until the last batch ships
log.Println("done")| Option | Type | Default | Description |
|---|---|---|---|
| WithFlushHandler(fn) | Option | noop | Called on every flush with a context and the batch. Required to do anything useful. |
| WithErrorHandler(fn) | Option | noop | Called after MaxRetries temporary failures — permanent error path. |
| WithMaxEntries(n) | Option | 250 | Size-based flush threshold. Push triggers an async flush at this count. |
| WithMaxRetries(n) | Option | 3 | Retries before the error handler is invoked on temporary errors. |
| WithFlushInterval(d) | Option | 30s | Timer-based flush cadence. Set to 0 to disable periodic flushing. |
| WithFlushTimeout(d) | Option | 15s | Context deadline passed to each flush invocation. |
| Method | Description |
|---|---|
| Push(v interface{}) | Add a value. Thread-safe. Auto-flushes asynchronously when MaxEntries is hit. |
| Flush() | Flush pending entries asynchronously. Returns immediately. |
| FlushSync() | Flush pending entries and block until the handler returns. Use on shutdown. |
| Stop() | Stop the interval goroutine. Call FlushSync() afterwards to drain any remainder. |
DefaultFlushInterval to 30sFlushSync() for synchronous drainTemporary() errorsPublic roadmap — August through December 2026.
Landing page, interactive demo, API reference, OG image & full SEO.
Type-safe Buffer[T] using Go 1.18+ generics. No more interface{} casts.
First-class metrics via OpenTelemetry — flush latency, queue depth, error rate.
Configurable overflow strategies — block, drop, or sample when the buffer is full.
Full generics API, observability, backpressure — stable release with migration guide.
go-buffer exists because batching should be a one-liner, not a design problem.
Every production system eventually needs to batch writes — log events to an aggregator, bulk DB inserts, webhook calls hitting a rate limit. The naive approach is channels and goroutines, but the bookkeeping compounds fast: you need a ticker, a mutex, a shutdown signal, and retry logic before you've written a single line of business logic. go-buffer handles all of that. You write the flush handler. The library handles everything else.
New(WithFlushHandler(fn)) is a complete, production-ready setup.Push() from any number of goroutines without external locking.FlushSync() blocks until the last batch clears, giving you a clean drain on process exit — no lost events.Go developers building log pipelines, metrics collectors, event streaming systems, or anything where calling a downstream sink once per event is wasteful or unsafe. If you're writing a loop that forwards individual records to a database, queue, or HTTP endpoint — go-buffer is the missing layer between your code and your sink.
go-buffer is released under the MIT License — free to use in any project, commercial or otherwise, with no strings attached. The source code and full documentation are available on the package registry.