The second Task/UI server won't start when using activeMQ, and this error message appears on the log. WARN main activemq.transport.failover.FailoverTransport:283 - Transport (tcp://localhost:61616) failed, attempting to automatically reconnect java.io.IOException: Wire format negotiation timeout: peer did not send his wire format.
This occurs when multiple IIQ servers are being used. The host/port pair are entered into the database during initial IIQ setup using the default /WEB-INF/config/dataextract/MessageBus.xml file. Subsequent servers are confused by the localhost host entry.
This can be changed and imported after modifying the /WEB-INF/config/dataextract/MessageBus.xml file.
<entry key="clientConnectionString" value="failover:(tcp://localhost:61616)?initialReconnectDelay=10&maxReconnectAttempts=10&randomize=false&startupMaxReconnectAttempts=10&jms.watchTopicAdvisories=false&jms.redeliveryPolicy.initialRedeliveryDelay=900000&jms.redeliveryPolicy.redeliveryDelay=900000"/>
It can also be updated within the IIQ UI at Global Settings>Messaging Configuration>Connection Settings>Client Connection String
Replace tcp://localhost:61616 with a resolvable host or ip address. (tcp://IIQHost1.example.com:61616 or tcp://10.10.10.10:61616)
If the hostname can change, an IP address is the most stable.
If failover is desired a second comma separated entry can be added (tcp://IIQHost1.example.com:61616,tcp://IIQHost2:61616)
When adding a secondary Message Server to the configuration, be sure to update the iiq.properties file with the correct port if you do not use the same port. The 0.0.0.0 means it is listening to any incoming IP on that port.
activeMQMessageServiceManager.brokerUri=tcp://0.0.0.0:61616?transport.trace=true&transport.soTimeout=10000
When configuring the message configuration client connection string for the first time, use the default values and change the port(s) in the UI first, then change it in the iiq.properties file to ensure the connections can happen on restart.
When adding new servers afterwards, update the client connection string in the UI before starting a new failover host, change to the configured port in the iiq.properties file on the new host, then start the host.
Another gotcha is you need to have the ports open in the firewall for each host, since it will timeout without a good indication of why it timed out.