validatebackups

Validates some personal google cloud backups

This utility is intended to download a sample of random files from backup locations. These files can then be manually verified to make sure the backups are working.

This is a port of my Python script of the same name to Go, fixing some bugs and design issues along the way.

High level procedure

• Connect to google cloud storage
• Run validation procedure for known buckets
• These validation procedures are similar but not identical
• Download files that need to be manually validated
• Report success/failure for each bucket

Things to be addressed

• Python script needs to be restarted from the beginning if a download fails partway through.
  • Catch a failing download partway through and just restart for that file.
    • This utility retries each failed download a configurable number of times
  • Serialize progress somehow so restarting the utility resumes from where it left off.
    • This utility skips already downloaded files and restart from the next file in sequence instead of the beginning all over again.
• Python script's unit test suite runs integration tests that depend on accessing google cloud.
  • Go's mocking couldn't quite handle abstracting this away, so we connect to google cloud too ¯\(ツ)/¯