These are just another feedback relation. No big deal. Maybe make it part of fold? There are mainly 2 way to organize delay memory: per shared or separate. The only thing that's important is memory locality, which is not going to be very predictable. Maybe best to organize delay lines in cache line chunks, i.e. multi-channel.