(11/18) chainBB downtime report

chainBB was down for approximately 3 hours today due to a hard reboot (not of my doing) of the server it's API and blockchain processors exist on. Thanks to @lukestokes for the heads up that chainBB wasn't working.

The services are still catching up (indicated by the message at the top of the screen) using public RPC nodes while the local full node it uses comes back online. steemd has a nasty habit of corrupting the block_log when not shut down gracefully, so currently the server is pulling a fresh copy from a seed node and will then replay/reindex the blockchain afterwards. I imagine this will take another 3-4 hours to complete, and during that time, new content may be slower than usual to appear on the site.

Currently the server infrastructure behind chainBB is robust, but not incredibly fault tolerant. I have plans written up to migrate the entire system to a high availability cluster at some point early next year, it's just a matter of time and money, since building out a true non-beta environment will cost around 10x-20x more than what I'm paying now.

I'll do my best to report whenever these outages occur and the reasons behind them - hopefully in a more structured way in the future.

$ 23.501 SBD