Core Types
Core type definitions for the Firehose framework.
SQLRule
Defines a complete event processing pipeline.
type SQLRule[I, O any] struct {
ID string
Environments []string
Select Action[I, O]
Into Destination[O]
From Source[I]
Where Condition[I]
Having Condition[O]
Middlewares []Middleware[I, O]
}Fields
ID (string)
Unique identifier for the rule. Used in logs, reports, and monitoring.
ID: "order_processing"Environments ([]string)
Optional list of environments where this rule is active. If empty, rule is active in all environments.
Environments: []string{"production", "staging"}Rule is included only when the ENV environment variable matches one of the listed values.
Select (Action[I, O])
Transformation that converts input events (type I) to output events (type O).
Select: ProcessOrder{}Into (Destination[O])
Destination that consumes output events of type O.
Into: OrderDatabase{}From (Source[I])
Event source that produces events of type I.
From: HTTPServer{Addr: ":8080"}Where (Condition[I])
Optional condition that filters input events. If nil, all input events pass through.
Where: condition.Cond[OrderEvent]("amount > 1000")Having (Condition[O])
Optional condition that filters transformed output events before sending to destination.
Having: condition.Cond[ProcessedOrder]("status = \"ready\"")Middlewares ([]Middleware[I, O])
Pipeline interceptors applied in reverse registration order (first middleware wraps last).
Middlewares: []Middleware[I, O]{
&middlewares.Panic[I, O]{},
&middlewares.Slog[I, O]{},
}Pipeline Field Mapping
SQLRule, ScenarioRule, and StreamRule express the same pipeline: source → input condition → action → output condition → destination. MapReduceRule adds an intermediary Reduce stage for stateful accumulation.
| Stage | SQLRule | ScenarioRule (BDD) | StreamRule (Kafka Streams) | MapReduceRule |
|---|---|---|---|---|
| Source | From | When | Source | Source |
| Input condition | Where | Given | Filter | Filter |
| Action | Select | Then | Map | Map |
| Reduce | — | — | — | Reduce |
| Output condition | Having | GivenOutput | FilterOutput | FilterOutput |
| Destination | Into | To | Sink | Sink |
ScenarioRule
BDD-inspired rule with Given-When-Then semantics. Note the field is named When (not Give) to align with the "when" step in BDD scenarios.
type ScenarioRule[I, O any] struct {
ID string
Environments []string
When Source[I]
Given Condition[I]
Then Action[I, O]
GivenOutput Condition[O]
To Destination[O]
Middlewares []Middleware[I, O]
}Fields
When (Source[I])
Event source that produces events of type I. Named after the "when" step in BDD Given-When-Then scenarios.
Given (Condition[I])
Optional condition that filters input events. Equivalent to Where on SQLRule.
Then (Action[I, O])
Transformation that converts input events to output events. Equivalent to Select.
GivenOutput (Condition[O])
Optional condition that filters transformed output events before sending to destination. Equivalent to Having.
To (Destination[O])
Destination that consumes output events. Equivalent to Into.
StreamRule
Kafka Streams-inspired rule with Source/Filter/Map/FilterOutput/Sink semantics.
type StreamRule[I, O any] struct {
ID string
Environments []string
Source Source[I]
Filter Condition[I]
Map Action[I, O]
FilterOutput Condition[O]
Sink Destination[O]
Middlewares []Middleware[I, O]
}Fields
Source (Source[I])
Event source that produces events of type I. Equivalent to From on SQLRule.
Filter (Condition[I])
Optional condition that filters input events. Equivalent to Where.
Map (Action[I, O])
Transformation that converts input events to output events. Equivalent to Select.
FilterOutput (Condition[O])
Optional condition that filters transformed output events before sending to destination. Equivalent to Having.
Sink (Destination[O])
Destination that consumes output events. Equivalent to Into.
MapReduceRule
MapReduce-inspired rule with Source/Filter/Map/Reduce/FilterOutput/Sink semantics. Uses three type parameters: I (input), M (intermediary), Out (accumulated output). The Reduce stage maintains a thread-safe accumulator via sync.Mutex.
type MapReduceRule[I, M, Out any] struct {
ID string
Environments []string
Source Source[I]
Filter Condition[I]
Map Action[I, M]
Reduce Reducer[M, Out]
FilterOutput Condition[Out]
Sink Destination[Out]
Middlewares []Middleware[I, M]
}Fields
Source (Source[I])
Event source that produces events of type I. Equivalent to From on SQLRule.
Filter (Condition[I])
Optional condition that filters input events. Equivalent to Where.
Map (Action[I, M])
Transformation that converts input events of type I to intermediary values of type M. Equivalent to Select.
Reduce (Reducer[M, Out])
Combines an intermediary value with the current accumulator to produce a new output value. The accumulator is thread-safe and persists across events from the same source.
type Reducer[M, Out any] interface {
Reduce(ctx context.Context, value M, accumulator Out) (Out, error)
}The accumulator starts at the zero value of Out and is updated atomically after each successful Reduce call.
FilterOutput (Condition[Out])
Optional condition that filters reduced output before sending to sink. Equivalent to Having.
Sink (Destination[Out])
Destination that consumes accumulated output events. Equivalent to Into.
Accumulator Behavior
The Reduce accumulator is updated atomically before the FilterOutput check — even if FilterOutput rejects the result, the accumulator retains the new value.
// Pipeline execution order for MapReduceRule:
// 1. Source emits event I
// 2. Filter (optional) — skip if false, return ErrInputNoMatch
// 3. Map — transforms I → M
// 4. Reduce — combines M with accumulator Out → new Out
// 5. FilterOutput (optional) — skip if false, return ErrOutputNoMatch
// 6. Sink — sends Out to destinationSource Interface
type Source[T any] interface {
Start(ctx context.Context, cb Callback[T]) (done <-chan struct{}, err error)
}Action Interface
type Action[I, O any] interface {
Process(ctx context.Context, event I, syms boolexpr.Symbols) (O, error)
}Destination Interface
type Destination[T any] interface {
Send(ctx context.Context, event T) error
}Condition Interface
type Condition[I any] interface {
Evaluate(ctx context.Context, event I, syms boolexpr.Symbols) (bool, error)
}Middleware Interface
type Middleware[I, O any] interface {
WrapCallback(ctx context.Context, rule Rule, cb Callback[I]) (Callback[I], error)
WrapAction(ctx context.Context, rule Rule, action Action[I, O]) (Action[I, O], error)
WrapDestination(ctx context.Context, rule Rule, dest Destination[O]) (Destination[O], error)
}Callback and ErrorHandler
type Callback[I any] func(context.Context, I, ErrorHandler)
type ErrorHandler func(err error)Callback is called by sources to deliver events to the pipeline.
ErrorHandler receives errors from rule execution back to the source.
Rule
Opaque interface representing registered rules.
type Rule interface {
GetID() string
GetSource() any
GetEnvironments() []string
GetNext() Rule
SetNext(n Rule)
GetPrev() Rule
SetPrev(p Rule)
GetNextSameSource() Rule
SetNextSameSource(n Rule)
SetPrevSameSource(p Rule)
Start(ctx context.Context) (<-chan struct{}, error)
Init(ctx context.Context) error
}RuleError
Wraps errors with the originating rule ID.
type RuleError struct {
Rule string
Err error
}
func NewRuleError(rule string, err error) errorHelper Functions
// EventID computes a hash-based identifier for an event.
func EventID(event any) (uint64, error)
// EventSymbols extracts symbols from an event if it implements boolexpr.Symbols.
func EventSymbols(event any) boolexpr.SymbolsError Types
ErrInputNoMatch
Indicates an input condition (Where / Given / Filter) evaluated to false (normal control flow, not an error).
var ErrInputNoMatch = errors.New("no match")ErrOutputNoMatch
Indicates an output condition (Having / GivenOutput / FilterOutput) evaluated to false (normal control flow, not an error).
var ErrOutputNoMatch = errors.New("output no match")ConditionError
Wraps errors from condition evaluation.
type ConditionError struct {
Err error
}
func (e ConditionError) Error() string
func (e ConditionError) Unwrap() errorActionError
Wraps errors from action processing.
type ActionError struct {
Err error
}
func (e ActionError) Error() string
func (e ActionError) Unwrap() errorDestinationError
Wraps errors from destination operations.
type DestinationError struct {
Err error
}
func (e DestinationError) Error() string
func (e DestinationError) Unwrap() errorReduceError
Wraps errors from the Reduce operation in a MapReduceRule.
type ReduceError struct {
Err error
}
func (e ReduceError) Error() string
func (e ReduceError) Unwrap() errorReducer Interface
Stateful accumulator for MapReduceRule.
type Reducer[M, Out any] interface {
Reduce(ctx context.Context, value M, accumulator Out) (Out, error)
}