I personally always use and recommend FROM scratch images. It not only saves storage, traffic and reduces deployment times, but it also removes most, if not all, lateral movement avenues that attackers could use to escalate after an initial exploitation.

The only situations where you would want to reach for alpine or debian base images is if you need external tools and programs that can't easily be compiled and embedded in your FROM scratch image (e.g. chromium, ffmpeg, pandoc...). In this situation, I will always prefer alpine images because packages are more up-to-date and generally work without any problems.