Introduction
Firehose is a type-safe event processing framework for Go that enables you to build composable event pipelines with conditional execution and middleware support.
The Problem
Applications process events from various sources (HTTP requests, message queues, timers, system events, user input) and react with side effects. Without a structured approach, event handling becomes:
- Scattered across the codebase
- Difficult to test in isolation
- Hard to modify without breaking things
- Impossible to compose or reuse
The Solution
Firehose provides a declarative framework for event processing pipelines:
Select → Into → From → Where → HavingThis ordering intentionally resembles SQL clauses: SELECT ... INTO ... FROM ... WHERE ... HAVING ...
Execution order (driven by runtime flow): From → Where → Select → Having → Into
Each stage is:
- Select: Transformation logic converting input events to output events
- Into: Destination handling the output event (side effects, storage, forwarding)
- From: Event source producing events of a specific type
- Where: Optional input condition evaluated against event attributes
- Having: Optional output condition evaluated against transformed output
Key Features
Type Safety
Rules enforce type safety between pipeline stages. The compiler ensures transformations match:
// HTTP request events → Order events
SQLRule[HTTPRequest, OrderPlaced]
// Order events → Email notifications
SQLRule[OrderPlaced, EmailSent]Event Source Fanout
Register multiple rules with the same source instance. The framework detects this and starts the source only once, fanning events out to all rules:
kafkaSource := &KafkaConsumer{Topic: "orders"}
// Both rules share kafkaSource - it starts once, events fan out
head, _ = Add(ctx, head, &SQLRule[OrderEvent, Email]{From: kafkaSource, ...})
head, _ = Add(ctx, head, &SQLRule[OrderEvent, Metrics]{From: kafkaSource, ...})Middleware System
Apply cross-cutting concerns like logging, metrics, retry logic, or rate limiting:
type LoggingMiddleware[I, O any] struct{}
func (m LoggingMiddleware[I, O]) WrapAction(
ctx context.Context,
rule Rule,
action Action[I, O],
) (Action[I, O], error) {
return loggingAction[I, O]{ruleID: rule.GetID(), next: action}, nil
}Next Steps
- Quick Start - Build your first event pipeline
- Core Concepts - Deep dive into events, rules, and components
- API Reference - Complete API documentation