RIZZASTER

Weekly Leaderboard Reset Incident Review

/archive/2022/old/2021/ · last edited 08 Nov 2019 · Community

Last weekend, we encountered a minor issue with the weekly leaderboard reset for Rizzaster. The reset didn’t occur at the scheduled time, which was supposed to be 10:00 PM on Saturdays. Instead, it happened at 2:00 AM, leading to confusion among players and affecting the integrity of the leaderboard for a brief period.

Upon investigation, we found that a recent software update inadvertently altered the system’s cron job for the reset. The update had introduced a syntax error in the cron configuration file, which delayed the execution of the script. This error was unnoticed during the initial testing phase due to the complexity of the new features added in the update.

To prevent such incidents in the future, we have updated our testing procedures. We now include a dedicated phase where all cron job configurations are thoroughly reviewed and tested before the update goes live. Additionally, we have established a post-update monitoring system to promptly detect and address any unexpected changes or issues that might arise.

Related pages

Up to /archive/2022/old/2021/ · Archive home