Skip to content

Introduction ​

Firehose is a type-safe event processing framework for Go that enables you to build composable event pipelines with conditional execution and middleware support.

The Problem ​

Applications process events from various sources (HTTP requests, message queues, timers, system events, user input) and react with side effects. Without a structured approach, event handling becomes:

  • Scattered across the codebase
  • Difficult to test in isolation
  • Hard to modify without breaking things
  • Impossible to compose or reuse

The Solution ​

Firehose provides a declarative framework for event processing pipelines:

Select → Into → From → Where → Having

This ordering intentionally resembles SQL clauses: SELECT ... INTO ... FROM ... WHERE ... HAVING ...

Execution order (driven by runtime flow): From → Where → Select → Having → Into

Each stage is:

  • Select: Transformation logic converting input events to output events
  • Into: Destination handling the output event (side effects, storage, forwarding)
  • From: Event source producing events of a specific type
  • Where: Optional input condition evaluated against event attributes
  • Having: Optional output condition evaluated against transformed output

Key Features ​

Type Safety ​

Rules enforce type safety between pipeline stages. The compiler ensures transformations match:

go
// HTTP request events → Order events
SQLRule[HTTPRequest, OrderPlaced]

// Order events → Email notifications
SQLRule[OrderPlaced, EmailSent]

Event Source Fanout ​

Register multiple rules with the same source instance. The framework detects this and starts the source only once, fanning events out to all rules:

go
kafkaSource := &KafkaConsumer{Topic: "orders"}

// Both rules share kafkaSource - it starts once, events fan out
head, _ = Add(ctx, head, &SQLRule[OrderEvent, Email]{From: kafkaSource, ...})
head, _ = Add(ctx, head, &SQLRule[OrderEvent, Metrics]{From: kafkaSource, ...})

Middleware System ​

Apply cross-cutting concerns like logging, metrics, retry logic, or rate limiting:

go
type LoggingMiddleware[I, O any] struct{}

func (m LoggingMiddleware[I, O]) WrapAction(
    ctx context.Context,
    rule Rule,
    action Action[I, O],
) (Action[I, O], error) {
    return loggingAction[I, O]{ruleID: rule.GetID(), next: action}, nil
}

Next Steps ​

Released under the MIT License.