Lately I was required to clean redundant Elasticsearch indices. Every midnight. Easy thing for a
cron job, right? The main thing was, that the whole infrastructure is built around Docker containers and there is no server on which it is possible to run
After googling for a while I haven't found nice and short image for this and therefore start to creating my own.
I wanted to have: the smallest image possible; robust solution - the simplest
bash scripts; configurability.
The Alpine image is the smallest available. The small issue I've instantly found: the
date util from it doesn't support proper
date: invalid date '2 days ago'
This can be fixed with installing the
coreutils. Thus my
Dockerfile look like this:
FROM alpine:latest MAINTAINER "Andrew Prokhorenkov" USER root RUN apk update && apk add --no-cache curl coreutils COPY crontab /etc/crontabs/root COPY clean_indices.sh /tmp/clean_indices.sh COPY run_crond.sh /run_crond.sh RUN chmod -v +x /run_crond.sh RUN mkdir -p /var/log/cron && touch /var/log/cron/cron.log CMD ["/run_crond.sh"]
The command is a separate script in which I put the environment variables into
clean_indices.sh script and run the crond with logging.
#!/bin/sh env | egrep '^ES_' | cat - /tmp/clean_indices.sh > /clean_indices.sh crond -L /var/log/cron/cron.log "$@" && tail -f /var/log/cron/cron.log
Easy, huh? Easy. :)
The actual script in cleaning the indices is also simple:
#!/bin/sh for day in $(seq $KEEP $(($KEEP + 7))) do IDX_DATE=$(date -d "$day days ago" +%Y-%m-%d) echo 'DELETE '$ES_INDEX-$IDX_DATE curl -s -XDELETE -u $ES_USER:$ES_PASS https://$ES_URL/$ES_INDEX-$IDX_DATE || true echo done
Where all the variables (
$ES_INDEX) is being passed to container via environment file in Docker Compose:
KEEP=30 ES_USER=bob ES_PASS=bob_strongest_password ES_URL=bob.space ES_INDEX=log
With the final touch of
crontab it is finished.
0 0 * * * sh /clean_indices.sh >> /var/log/cron/cron.log 2>&1 # An empty line is required at the end of this file for a valid cron file.
Now you only need to build it and use in
docker-compose.yml or standalone run.
I hope this will help someone to run the cron in Docker container.