Friday night, almost there
It started on Friday evening, yes Friday for all but not for me.
Waiting for the EMC engineer to do a webex and configure our Recoverpoint appliance. Normally, if you are planning for DR site (big project indeed) for the last 6 months; how much time will you give for the lifeline to work yes, data replication. 1 month would be a decent guess, we have to do in 2 days. Don't laugh, very serious.
We had done the rack and stack and all the cabling beforehand. It started around 9 PM and went on till 1 AM. All set for the testing. We just cross checked whether every IP is reachable or not. Viola, the wan IP was not pinging. It was on a different VLAN and was not in the firewall rules.
Called up firewall engineer, didn't picked up the call; another same result. Escalated to manager, his phone was busy. called up project manager, he was furious (had to be). Escalated to Director, Information security and he confirmed will do something. Never ending wait started, finally around 3 AM it started working. Again stared with the EMC guy, he was really helpful and very cool person. He explained us the basics and how to proceed. He had to go for another assignment so we tested with bare minimum stuff tried to do forward replication and reverse with just 10gb of data and it worked as per our expectation. It's 5:00 AM in the morning. What a Friday night rocking....
Come Saturday, well we are in Saturday; project manager calls and asked about the status of data transfer. What??? we are human right and he was too. There was simply too much pressure on him and he was just doing his job. Mammoth task was still there to configure close to 100 luns from production to DR in different consistency group. Started at 1:00 PM, verified and we missed something. Few of the production data were on other SAN and we didn't had the license for replication from the other SAN. Not a problem, always trust Veritas storage foundation; had never let me down.
Zoned the server to the production VNX and assigned storage and did vxevac. I started around 02:00 PM and scheduled it, let it run I will do for the rest. The Unisphere GUI is a pain for configuring large number of luns only helps when you have to show it to someone (less technical :), no offense please. Unix Solaris Unix Unix Unix... the best thing ever created. Thanks Dennis Ritchie, you are not with us anymore but my sincere respect to you from my heart.
Scripted everything in just 30 minutes and it took another 10 minutes to create the luns at DR site. By 3 PM, I was set to configure CG's in Recoverpoint.
I have already written in my earlier post how I moved one of the file server to SAN. This was also a part of Recoverpoint setup. Started CG configuration and by 08 PM, all done. Wait, let me check my vxevac, it is still running and will take another 1 hour. Not to wait for it to finish, I started the first initial sync and its still going on Monday 02:00 AM. It will take another 5 days as the data is approx 14.8 TB. Finally by 10:00 PM Saturday, vxevac got over and that was also put up on RP.
Long tiring day and night. But I enjoyed it, tapped my back and said effort worth spend and you should be rewarded. A glass of Laphroaig with ice and jazz on my samsung note was the best way to end my marathon.
Getting ready for the next one and another glass to celebrate.
Waiting for the EMC engineer to do a webex and configure our Recoverpoint appliance. Normally, if you are planning for DR site (big project indeed) for the last 6 months; how much time will you give for the lifeline to work yes, data replication. 1 month would be a decent guess, we have to do in 2 days. Don't laugh, very serious.
We had done the rack and stack and all the cabling beforehand. It started around 9 PM and went on till 1 AM. All set for the testing. We just cross checked whether every IP is reachable or not. Viola, the wan IP was not pinging. It was on a different VLAN and was not in the firewall rules.
Called up firewall engineer, didn't picked up the call; another same result. Escalated to manager, his phone was busy. called up project manager, he was furious (had to be). Escalated to Director, Information security and he confirmed will do something. Never ending wait started, finally around 3 AM it started working. Again stared with the EMC guy, he was really helpful and very cool person. He explained us the basics and how to proceed. He had to go for another assignment so we tested with bare minimum stuff tried to do forward replication and reverse with just 10gb of data and it worked as per our expectation. It's 5:00 AM in the morning. What a Friday night rocking....
Come Saturday, well we are in Saturday; project manager calls and asked about the status of data transfer. What??? we are human right and he was too. There was simply too much pressure on him and he was just doing his job. Mammoth task was still there to configure close to 100 luns from production to DR in different consistency group. Started at 1:00 PM, verified and we missed something. Few of the production data were on other SAN and we didn't had the license for replication from the other SAN. Not a problem, always trust Veritas storage foundation; had never let me down.
Zoned the server to the production VNX and assigned storage and did vxevac. I started around 02:00 PM and scheduled it, let it run I will do for the rest. The Unisphere GUI is a pain for configuring large number of luns only helps when you have to show it to someone (less technical :), no offense please. Unix Solaris Unix Unix Unix... the best thing ever created. Thanks Dennis Ritchie, you are not with us anymore but my sincere respect to you from my heart.
Scripted everything in just 30 minutes and it took another 10 minutes to create the luns at DR site. By 3 PM, I was set to configure CG's in Recoverpoint.
I have already written in my earlier post how I moved one of the file server to SAN. This was also a part of Recoverpoint setup. Started CG configuration and by 08 PM, all done. Wait, let me check my vxevac, it is still running and will take another 1 hour. Not to wait for it to finish, I started the first initial sync and its still going on Monday 02:00 AM. It will take another 5 days as the data is approx 14.8 TB. Finally by 10:00 PM Saturday, vxevac got over and that was also put up on RP.
Long tiring day and night. But I enjoyed it, tapped my back and said effort worth spend and you should be rewarded. A glass of Laphroaig with ice and jazz on my samsung note was the best way to end my marathon.
Getting ready for the next one and another glass to celebrate.
Comments
Post a Comment