Maintenance Jobs

Database Cleanup Jobs

Distr includes built-in CLI tasks for database and storage pruning to prevent unbounded growth of resources such as metrics, logs, and status entries.

The cleanup routine can be executed manually.

distr cleanup $CLEANUP_TARGET

Available cleanup targets:

Target	Description
DeploymentLogRecord	Deployment log entries
DeploymentTargetLogRecord	Deployment target log entries
DeploymentRevisionStatus	Deployment revision status entries
DeploymentTargetMetrics	Deployment target metrics entries
OIDCState	Expired OIDC state entries
ArtifactBlob	Unreferenced registry blobs from S3 (requires registry to be enabled)

For production deployments we recommend scheduling these jobs automatically, either using the built-in job scheduler for single instance deployments or using Kubernetes CronJobs for high-availability deployments.

Automated job scheduling for Single Instance deployments

If you only have one instance of Distr running (e.g., using Docker Compose), you can use the integrated job scheduling.

The internal scheduling can be configured via environment variables.

An example configuration file can be found on github.com/distr-sh/distr/deploy/docker:

# Cron interval for cleaning deployment revision statuses older than STATUS_ENTRIES_MAX_AGE
CLEANUP_DEPLOYMENT_REVISION_STATUS_CRON="0 * * * *"
CLEANUP_DEPLOYMENT_REVISION_STATUS_TIMEOUT="10m"
# Cron interval for cleaning metrics older than METRICS_ENTRIES_MAX_AGE
CLEANUP_DEPLOYMENT_TARGET_METRICS_CRON="0 * * * *"
CLEANUP_DEPLOYMENT_TARGET_METRICS_TIMEOUT="10m"
# Cron interval for cleaning deployment log entries exceeding LOG_RECORD_ENTRIES_MAX_COUNT
CLEANUP_DEPLOYMENT_LOG_RECORD_CRON="0 * * * *"
CLEANUP_DEPLOYMENT_LOG_RECORD_TIMEOUT="10m"
# Cron interval for cleaning deployment target log entries
CLEANUP_DEPLOYMENT_TARGET_LOG_RECORD_CRON="0 * * * *"
CLEANUP_DEPLOYMENT_TARGET_LOG_RECORD_TIMEOUT="10m"
# Cron interval for cleaning expired OIDC state entries
CLEANUP_OIDC_STATE_CRON="0 * * * *"
CLEANUP_OIDC_STATE_CRON_TIMEOUT="10m"
# Cron interval for cleaning unreferenced artifact blobs from S3 (requires registry)
CLEANUP_ARTIFACT_BLOB_CRON="0 * * * *"
CLEANUP_ARTIFACT_BLOB_TIMEOUT="10m"
CLEANUP_ARTIFACT_BLOB_MIN_AGE="24h"

If these variables are not set, no cleanup jobs are scheduled.

Automated job scheduling for High-Availability deployments

For high-availability deployments with multiple instances of Distr (e.g., using Kubernetes), the built-in job scheduling is not suitable, as it would lead to multiple instances trying to perform the same cleanup tasks concurrently.

Therefore, we recommend using CronJobs in Kubernetes to handle the cleanup tasks. But the concept can apply to any HA setup where the cleanup jobs are triggered externally.

These jobs can also be configured via our Helm Chart.

An example configuration can be found in github.com/distr-sh/distr/deploy/charts/distr:

cronJobs:
  - name: deployment-log-record-cleanup
    labels:
      distr.sh/job: deployment-log-record-cleanup
    args: [cleanup, DeploymentLogRecord, --timeout, 10m]
  - name: deployment-target-log-record-cleanup
    labels:
      distr.sh/job: deployment-target-log-record-cleanup
    args: [cleanup, DeploymentTargetLogRecord, --timeout, 10m]
  - name: deployment-revision-status-cleanup
    labels:
      distr.sh/job: deployment-revision-status-cleanup
    args: [cleanup, DeploymentRevisionStatus, --timeout, 10m]
  - name: deployment-target-metrics-cleanup
    labels:
      distr.sh/job: deployment-target-metrics-cleanup
    args: [cleanup, DeploymentTargetMetrics, --timeout, 10m]
  - name: oidcstate-cleanup
    labels:
      distr.sh/job: oidcstate-cleanup
    args: [cleanup, OIDCState, --timeout, 10m]
  - name: artifact-blob-cleanup
    labels:
      distr.sh/job: artifact-blob-cleanup
    args: [cleanup, ArtifactBlob, --timeout, 10m]

Additional configuration

Some cleanup targets allow for additional customization using the following environment variables:

STATUS_ENTRIES_MAX_AGE="24h"
METRICS_ENTRIES_MAX_AGE="24h"
LOG_RECORD_ENTRIES_MAX_COUNT="500"
CLEANUP_ARTIFACT_BLOB_MIN_AGE="24h"

Caution

STATUS_ENTRIES_MAX_AGE, METRICS_ENTRIES_MAX_AGE and LOG_RECORD_ENTRIES_MAX_COUNT do not have a default value. If unset, the cleanup routine for these targets will have no effect.

Removed features

Deployment Target Status (removed in v2.20.0)

Prior to v2.20.0, Distr tracked a separate DeploymentTargetStatus per deployment target. This concept has been removed and is no longer available as a cleanup target.