InferenceController implementations should use connection pooling

Description

Changes to be made to Inference controller implementations:

  • Currently stateless should be singleton and use connection pooling to reuse connections

  • Close response

Changes to be made to KafkaInferenceLogger:

  • Close KafkaProducer

Assignee

Robin

Reporter

Robin

Labels

None

Fix versions

Priority

Highest
Configure