Yes, I think you can because you can stream files over swarm.
While the routing protocol developed for swarm is likely to be abstracted out so that it can be used for any targeted data/message transfer from one participating ethereum node to another and this may be adaptable to streaming directly from you to your listeners