Boot archive corruption with solaris on v490

We are running our Sybase production database on SunFire v490 with Solaris 10. We recently applied the cluster patch 141414 on the system. The root disk is encapsulated in raid1 with SVM with 3 metadb replicas on each disk.

After running for a week the system ran out of max number of processes that a user can fork out. We were unable to log into the system from the RSC console also. The system was frozen and as the last option the system had to be hard rebooted. Eventually, this led to the boot archive corruption and the system did not allowed us to boot from the default disk.

We had to boot failsafe mode, to recover from the boot archive failure. This also did not worked out, so we booted of from the cdrom and mounted the first disk in /a.

We commented out the rootdev section in /etc/system and also removing the svm disk from vfstab and replacing it with ctds format disk.

Also, we put a new parameter for maxusers in /etc/system as 4096, as the default value of 2048 was not sufficient.

echo "set maxusers = 4096" >> /etc/system

After that we booted of from the first disk and it worked. Then, we again encapsulated the disk with svm.

Comments

Post a Comment

Popular posts from this blog

zpool and power path partial compatibility

Is power path fully supported with zpools, no. It's partial. You can have the pseudo devices assigned to a pool but it uses the native ctd disk only. Recently I lost 2 paths from the vnx array to m9000 server running Solaris 11.1. When I ran the zpool status -x command some of the pools were online while some of them were un available. Power path was reporting 2 paths only but it was active. So ideally the zpools should be online. But it's not. I enabled the remaining 2 paths and all became online. Internally the pools were pointing to one of the ctd devices that belonged to dead path and it was not coming online. Also few of the emcpower devices were renamed, so I had to do zpool import -d with the alternate disk. It seems power path is not completely supported now but may become in future.
Read more

Recoverpoint replication reporting with vmax splitter

Guys, Have you ever thought of automating the RP replication status and send status via email or post it to any reporting platform. Since the splitter impacts the VMAX FA's and there is an overhead of 13% on the utilization its good idea to see if the replication is driving the utilization of vmax FA. There are ways of pulling iops/read/write using symstat but its utilization (Director CPU utilization etc) which plays a significant role. As it crosses 80% we start seeing the increase in device response time. However, to measure FA utilization we will have to go through SMAS UI then pull out the reports. Its manual effort. But there is a way to automate it through REST API. You can get the details what type of objects are supported by pointing to : https://smc server :8443/univmax/restapi Coming to the point, getting recoverpoint consistency group status fully automated doesn't seems to be tough. Lets get through the steps. 1. Hash your SSH keys of the unix host to ...
Read more

Weird problem post power maintenance, no Powerpath pseudo name for Luns

We had the scheduled power maintenance activity in our data center. Every team was notified and we powered doen the servers through cron job. The servers were powered down first then storage (CX-380). And the reverse was followed during startup. After the power shutdown, the SAN was booted up. we forgot to disable the the snapshots. They were still active. Once it came up, the sessions were not there, but snapshot was active. Tried to deactivate the snapshot, failed with the error the "Failing Command: K10SnapCopyAdmin DBid 0 Op 1051. 71 00 80 43" Tried to create a session and attach the snapshot to the session, still unable to deactivate it, neither was able to attach to any other session. Looked at powerlink: It says the command issued to the snapshot is not owned by the SP. I trespassed all the snapshot source lun to SPA, and then tried to deactivate, nothing happened. Rebooted the SP's one by one through the management console. (http:/SP-IP/setup) Then I was able to...
Read more