Wed Sep 12 11:56:51 EDT 2018
Strict stream processing
I have a file in, file out stream processor operating on lines, but I
can't figure out how to add strictness to keep sane memory usage.
If I understand correctly: this needs to be done at the pulling end.
EDIT: Adding random bang patterns and nothing helps. Getting
frustrated. I don't know where to start looking.
I'm thinking that the problem might be in "show" and "++".
Likely this needs lazy byte streams. Why else would there be a need
for such an abstraction? Strings and (++) are not ok because all the
(++) thunks need to be unpacked before the first line can be read.
The lesson seems to be: monoids are not good for infinite streams.
I've run into this before, e.g. the writer monad.
Googling "monoid space leak":