
Action
1.
Resolve the DNS server reachability issue.
2.
If the DNS server cannot be reached, add each team member to the
/etc/hosts
file on
each controller.
Controller and application data differs among controllers in a team
Symptom
Controllers and applications in a team do not display the same data, or data appears to be out
of synchronization between controllers in a team.
Cause
The system clocks for the controllers in the team are not synchronized.
For controllers that are configured to use NTP (Network Time Protocol) but connect to different
NTP servers, the server clocks can drift over time, resulting in a significant relative difference
between clock times. Cassandra serializes write operations by timestamp, so differences between
system clock times can result in inconsistent data across those systems.
Action
1.
Ensure that all the servers on which the controllers in the team are installed are configured
to use NTP.
2.
Ensure that the servers are configured to use a centralized NTP daemon.
Application data is not synchronized after a controller rejoins the team
Symptom
Data displayed in the user interfaces and the output of programmatic interfaces of applications
that use Cassandra differs among the controllers in a team after a controller rejoins the team
after a long outage (such as more than 1 hour).
Controller data and data for applications that do not use the Cassandra database are not affected.
Cause
After a Cassandra instance is offline for longer than is supported for automatic synchronization
by the Cassandra database, the Cassandra databases in a controller team are not synchronized
automatically.
Action
You must stop the controller that was offline, remove that Cassandra instance from the cluster,
delete all the Cassandra data on the controller, and then restart the controller:
1.
Log in to Linux on the server on which the controller with the incorrect data is installed.
Typically this is the controller that came back online after a long outage.
2.
Start a root user command shell by entering the following command at the prompt:
~$
sudo su
3.
Stop the controller and administrative services:
~#
service sdnc stop
~#
service sdna stop
4.
Log in to Linux on a server on which a controller with the correct data is installed.
5.
Start a root user command shell by entering the following command at the prompt:
~$
sudo su
Troubleshooting teamed environments
165