Cleanup
Jobs cut off by a restart, and how long finished jobs are kept
Abandoned jobs
A running job is tracked by the AnsibleForms process that started it. When that tracking stops, the job is marked abandoned:
- At startup, the jobs that this process followed before it restarted, and the jobs from before 7 that no process follows, are marked abandoned if they are still
runningor aborting; the log showsAbandoned <n> jobs. A job that runs on an RTE is not touched: it carries on without the process that started it. - Every minute, the jobs followed by another AnsibleForms process (or RTE) that stopped answering for 2 minutes are marked abandoned; the log shows
Abandoned <n> jobs followed by a node that stopped answering. - Every hour, jobs that started more than a day ago, are still
runningand wrote no output for a day are marked abandoned. A job that runs on an RTE is left alone here too: the RTE cleans up its own jobs when it restarts.
From 7.3.0 a template job on AWX, AAP or Ascender that a restart or a stopped node abandoned is picked up again, if it started within the last 24 hours: one AnsibleForms process takes it over, follows it on the runner again, and the job ends with what really happened there. Jobs on the other runners, playbooks, workflows and multistep jobs stay abandoned.
An abandoned playbook may have run partly. Check the managed hosts before you relaunch it; see A job shows the status abandoned.
Job retention
Jobs are kept until they are deleted, unless the administrator sets a retention period.
With JOB_RETENTION_DAYS set, a nightly task (at 2:30) deletes the finished jobs, with their output and steps, that ended more than that many days ago. It only removes jobs with status success, failed, warning, aborted, rejected or abandoned, so running jobs and jobs waiting for approval are kept however old they are. The default, 0, keeps every job.