So I am going to start this by saying I do not like to nor do I generally agree with changes during the business day, for this one I pushed as much as I felt comfortable with against mgmt to let me do it off hours. It was politely rejected and I was told to do it during lunch time after sending out a notice. I agreed after making my point about potential extended downtime and made sure they understood.
We have Exch2013 on a 2k8 server (previous IT) which is being replaced by a Exch2016/Server2016 VM that I stood up recently, moved over all of the objects to 2016 so just mailflow/transport is left to move. I was told to make those changes at lunch (I know...), since we have Reflexion handling spam anything not delivered should be queued up on their end. All my change entails is a few firewall objects and some testing. Well I make my changes and double check it all, then I try mailflow tests from gmail and a few others both inbound and out. NOTHING EXTERNAL WORKS! After a little troubleshooting I find the messages in the Reflexion portal in an error state listed SMTP errors. I assume it's my change and after confirming all is what I planned I decided to fall back as my window was shrinking. I roll it back and nothing, same issue. Internal mail is fine but external in/out is not. I then tried to call Reflexion support to no avail, open a web case, and then tried chat. I got lucky with chat and was given this lovely link... My change started @ 12:30p EST, about 2 hours into their issue that we had no notice on.
At the end of the day I feel it's my fault for not knowing they had an outage, I should have checked before my change but since we never had an outage with them it never occurred to be until part way into troubleshooting. Just had to vent to someone who could understand...
[link] [comments]