I would monitor the MQTT messages, for example, with MQTT Explorer. A better approach would be to use Node-RED. In a simple flow, an MQTT subscriber node (mqtt in) can receive and count the messages. Perhaps the messages are being sent too frequently.
They will never arive the broker, there is even no try to send them, tcpdump is quiet at that time when the queue overflow occurs.
It must be a problem with the firmware(s)
It would help a little bit if the firmware won‘t throw away the messages it cannot deliver. It even ignores QoS 1 or QoS 2, it’s just gone to nirvana