[<<][erlang][>>][..]
Thu Feb 25 14:31:37 EST 2016

Some different stream abstractions

In a project I'm using two different stream abstractions.

- sinks
- lists as (abortable) folds

The latter is "pure" in that it doesn't rely on processes: it has the
sematics of a left fold, with a pure folding function that combines an
element with the old accumulator to yield a new accumulator

The former is parameterized by a function that consumes the data in
the sequence using a side effect.  I.e. it behaves as a message send.



The appearance of these two forms is very typical in stream processing
applications that are based on a receive/send or read/write core: part
of it is pure (functional), written as a map or fold over a stream of
events, and part of it has a side-effect, sending data "out", e.g. a
sink.

There is some design freedom about where to introduce the "sink".

In most cases it seems best to do that as late as possible, keeping
most of the sequence representation in pure form.



Basic ideas:

- Write the primitive event stream as a fold (abortable).

  This works as long as it is not necessary to see individual elements
  in the sequence. e.g. treat the sequence as a sequence only.

- Define 'map' for folds, essentially implementing "map fusion".  This
  is equivalent to stateless stream processing. This allows basic
  chained event processing without having to introduce processes.

- Define new folds in terms of folds.  This is equivalent to stateful
  stream processing.

- Apply a composed fold to code that sends each element.  This is
  where processes and outputs are introduced.



The main guideline is that a function is easier to handle than a
collection of processes.  This increases testability.




[Reply][About]
[<<][erlang][>>][..]