Ansible Tower Licenese Limit Exceeded

Hi all, we noticed that our ansible tower reached its license limit of 1500 hosts. Apparent cause is misconfigured inventories and inventories with "manual" source that list same instances twice under different names or contain stale instances that have been removed from the AWS, but not from the inventory.

As an emergency action, Site-Infra will force update the misconfigured inventories to ensure we can continue using Tower to manage production environment. As a result, stale (i.e. stopped or decomissioned) hosts will be removed from dynamically generated inventories.

Time permitting we will also force-update settings of dynamically generated inventory groups to "Overwrite", i.e. remove stale or manually added hosts in the group.

To prevent Tower from miscounting instances, inventory admins should make sure to:

enable "Overwrite" option in "Update Options" section of your inventory groups, which use "Amazon EC2" as their source
provide recommended "Source Variables" values (especially if your instances reside in EC2 Classic environment)
avoid creation of inventory groups with "manual" source or manually adding hosts to your inventories

More information and guidelines on creating inventories is provided in the docs https://29022131.atlassian.net/wiki/x/kLHTCw.

Risk:

When number of hosts exceeds licence, operations that add new hosts to inventory will fail. This can result in jobs not running against all required hosts.

Time:

Wednesday 2017-12-27, 13:30:00 UTC+7.

Notes:

If you have any problem to access tower.noc.tvlk.cloud please contact us in #site-infra-channel.