Safe Online Decision-Making For Non-Stationary Systems