serialize messages to kafka - use middleware
notes:
serde - serialization library
sync
-- producer.produce(ctx, pm) -> sends to kafka
-- sent to chat.message
-- consumers: messageHydration, appIntWorker, hcopenchats-workers, hc-filter-messages-worker
- hc-filter-messages-worker consumes chat.message
- checks if its a direct or a group
- categorizes direct / group
- has a producer and a consumer
if direct -> send notificationWorker -> notificiationWorker
if group -> send to another worker for further processing.
- order log of events - partitions ( do not confuse this with replica partition)
- configure # of consumers for a topic
- 1 2to
- consumer groups scale up and down on their own on the workers
- commiting the offset means you read the message
- each consumer knows where it reads from the name of topic, offset