My world of Unix and Storage

Wow, looks challenging and it's indeed. So, we have finally decided to move our DR data center from India to US. The location I cannot disclose however its almost 50% of the original data center at Atlanta. The designing phase is over except for the data center power and cabling and I believe it happens given the short time frame of 45 days (don't laugh, it got extended by 6 months and I feel another 3 months more it deserves). There are few things which I don't know why it happens every time we try to do something big. 1. Specification keeps on changing, regarding the list of applications for which we are building our DR. 2. RPO and RTO is not decided and simply we are doing an infrastructure DR not true DR. There is no BCP. 3. We are trying new technology Cisco UCS vs discrete Dell servers and going for full virtualization at DR. We are simply doing too many things and rather experimenting new technologies at data center. 4. How the data will get replicated was decided po...

If you have enabled performance logging on EMC VNX, then it is expected to display all the statistics including the SP and FAST cache. However, with the default configuration of statistics logging, you will not be able to see. We have to go to the dashboard of statistics and click customize then check Advance. The advance features of the performance statistics will be visible now.

Sometime have you came across a situation where you want to create psrset but it doesn't works. Please check if pool facility is active or not. If its active then you won't be able to create psrset. svcs -a | egrep pool. Disable pool and create psrset again.

In Linux we can find the port to pid mapping, using netstat -np or using lsof. However, this facility is not available in Solaris. We will have to write a small perl/shell script to find the mapping. Script : ##################### #!/bin/bash export PORT=0 for name in `ps -aef | awk '{print $2}'` do export PORT=`pfiles $name | grep sockname | awk '{print $5}' 2>/dev/null` if [ -n "$PORT" ]; then echo "PID = $name PORT= $PORT" fi done unset PORT exit ####################

We had the scheduled power maintenance activity in our data center. Every team was notified and we powered doen the servers through cron job. The servers were powered down first then storage (CX-380). And the reverse was followed during startup. After the power shutdown, the SAN was booted up. we forgot to disable the the snapshots. They were still active. Once it came up, the sessions were not there, but snapshot was active. Tried to deactivate the snapshot, failed with the error the "Failing Command: K10SnapCopyAdmin DBid 0 Op 1051. 71 00 80 43" Tried to create a session and attach the snapshot to the session, still unable to deactivate it, neither was able to attach to any other session. Looked at powerlink: It says the command issued to the snapshot is not owned by the SP. I trespassed all the snapshot source lun to SPA, and then tried to deactivate, nothing happened. Rebooted the SP's one by one through the management console. (http:/SP-IP/setup) Then I was able to...

We came across a weird problem in Nagios, after it was working for almost an year. Searched the web, but didn't got any head way for the problem. Every body were complaining about the issue but the resolution were not clear. I started looking the source code, and found the function in util.c, there were different error conditions but got a hint what the problem could be. So, all set i started the nagios service and ran strace against the pid. Soon after about 40 seconds, it vanished. Forgot to mention we were running nagios on RHEL 5.5 32 bit linux. I checked the various options for the nagios process. There is one option for running it in the foreground. The syntax check was fine. As soon, it was started on the foreground, it dumped with the error message "File limit ....". Couldn't remember the exact output. There are 2 files which could be the problem, one is nagios.log or nagios.debug. Both were OK. Their is another file perfcheck.pipe or perfmon.pipe (something o...

The thought process has gone into the management of moving the production database from Solaris running on Oracle M8000 to IBM AIX. The initial cost of providing the clustered solution and the DR is coming to $2.5 million. Oracle were taking us for a ride with the discounts on M8000. The 50% agressive discount offering on M8000 comes at the same cost of M8000 that we bought last year. Along with the Veritas storage foundation and the support cost, its too expensive as a solution. We have the quotes from IBM for their p750, however we feel this is not an apples to apple comparison as the p750 lacks the RAS features. Th better would be the comparison with p595 or p785. We have to get the POC done and get the actual working configuration. Its a long way to go, since our aplication compatibility is yet to be established with AIX 6.1. Also, we are in the process of moving from 32 bit progress database to OpenEdge 64 bit. The dynamics will change with respect to cpu and memory utilization on...

A tough task is in my hands to decide between the 3 different storage vendors for our storage requirement. We have been EMC customers for last 10 yrs, but now would like go around and see what others are offering. The requirement is not huge ~260 Tb storage and ~80000 IOPS, which I believe can be meet by any mid-tier storage offering. We have VNX7500 from EMC with FAST and FAST-vp loaded with SSD's and 48GB ram (shared) and single socket CPU (Quad core) and 600 GB of SSD as L2 cache blah blah. The backend is SAS, that is a shift from CX line where we have FC. This also means we cannot re-use the existing hardware from EMC. The IOPS pattern hovers between 8000 to 40000 currently and will soon scale upto 80000 IOPS. Hitachi has a offering AMS2500, which is also SAS with 32 GB cache and Active/Active controller (difference from EMC). The maintenance cost is attractive almost 6-8% of the array cost. Hitachi also does a buy back. IBM has V7000, still to get some input on this. Will upda...