When running apt-get install foo, the result is cached and requires an apt-get clean to remove this cache.
However, adding
RUN echo "Acquire::http {No-Cache=True;};" > /etc/apt/apt.conf.d/no-cache
As an initial step will mean that packages are not cached with apt, and will not require an apt-get clean.
Thoughts?