This is what I did after upgrading from version 2 to version 17 and all the configuration was lost. Paths in this tutorial might be different for you, depending on your configuration. This might also be a bit sketchy, but hopefully it can give you some ideas on how to continue to fix your installation.

Based on what I have read: You need to have the same secrets configured in your old and new inventory when running the installation playbook, otherwise the encrypted credentials cannot be opened.

Basic procedure

  • old database A (with your data) in directory a
  • new database B (empty) in directory b
  1. dump A to some path
  2. rename B to free its name for the next step
  3. restore A with the old name of B in location b
  4. awx-manage migrate in the awx_task docker container

Figuring out where the data is and verifying that it is not migrated

sudo docker container ls

lists your containers. There were two postgres containers: postgres and awx_postgres. The displayed date made it clear that postgres is from our original installation.

sudo docker exec -it awx_postgres bash  # entering the docker container
psql -U awx
\list

There is the "awx" database. You can for example find users via

SELECT * FROM auth_user;

which was empty for me except for the admin.

From an older version there is also a container called "postgres". Doing the same thing in there yields all of my old data. But that database is outdated and some columns are therefore missing, like managed_by_tower in main_credentials. If you only have an awx_postgres docker container your data is possibly still in some folder, it's just that no container is configured to be able to access that (outdated) data.

After looking around, I found a Migration to add the column of the example: https://github.com/ansible/awx/blob/devel/awx/main/migrations/0120_galaxy_credentials.py, so I wanted to now figure out a way to run this and all other migrations.

Going back to the host machine (out of the docker bash) via Ctrl + D, you should check the path in your inventory file (cloned from the repo) or vars.yml (if you use one) that leads to the postgres data:

cat awx/installer/inventory | grep postgres_data
postgres_data_dir="~/.awx/pgdocker

cat awx/installer/vars.yml | grep postgres_data
postgres_data_dir: /var/lib/awx/pgdocker

Is it correct? If not you might have also screwed up your upgrade procedure. If you use vars.yml for the installation the setting there takes priority over the inventory file.

Turned out in /var/lib/awx/pgdocker there are two postgres data locations:

/var/lib/awx/pgdocker$ sudo find . -type f -name postgresql.conf
./pgdata/postgresql.conf
./12/data/postgresql.conf

You might just compare the timestamps of those files to figure out which directory contains your database with your data.

Migrating the old data

I created a dump of the old databse and moved it to the directory of the new container (so that it is accessible from a bash in there). Depending on your container and path setup paths might be different! 

sudo docker exec -it postgres bash  # old container, old db
pg_dump -U awx -d awx > /var/lib/postgresql/data/12/data/dump
# ctrl + d exit container

the dump should now be in the postgres_data_dir on the host. but you can put it anywhere if that doesn't work for you and use docker mv to get it into the awx_postgres container.

In the new container restore that dump (Also see https://www.postgresqltutorial.com/postgresql-rename-database/)

sudo docker exec -it awx_postgres bash  # new container, new db
# preparation and backup
psql -U awx
\connect postgres
SELECT pg_terminate_backend (pid) FROM pg_stat_activity WHERE datname = 'awx'; # closes all db connections
ALTER DATABASE awx RENAME TO awx_empty;
CREATE DATABASE awx;

# resteoring the dump
# ctrl + d exit psql
psql -U awx -d awx < /var/lib/postgresql/data/dump
# takes a few minutes

The new container now contains the old data, but in an outdated database format. The migrations need to be executed.

# leave awx_postgres and enter awx_task:
sudo docker exec -it awx_task bash
awx-manage migrate

# output:
Operations to perform:
 Apply all migrations: auth, conf, contenttypes, main, oauth3_provider, sessions, sites, social_django, sso, taggit
Running migrations:
 Applying auth.0009_alter_user_last_name_max_length... OK
 Applying auth.0010_alter_group_name_max_length... OK
 Applying auth.0011_update_proxy_permissions... OK
 Applying conf.0006_v331_ldap_group_type... OK
...

Since the connections to the db have been closed manually previously, I restart the docker service which restarts AWX.

sudo systemctl restart docker

I then was able to log in into the new version with my old account.