If you’re a follower of mine on Twitter (@eczerwin), you know that about 8 months ago I was transferred by my company from Chicago, USA to Zurich, Switzerland. This mission has been exciting and a lot of fun – but it’s not easy.
The mission – Move roughly 12TB of SAP data over the ocean into two new datacenters running on new iron, with one weekend of downtime. In the future keep your eyes open for another blog post on exactly how we completed this data move.
My personal goal – Virtualize every possible server that comes over (or at least what the app owners will allow without having heart attacks).
After having a chat with the SAP app owners and trying to soothe their concerns we moved forward like this; all SAP application servers and test DBs will be virtualized, while production DBs stay physical (it’s only 2 servers). Oh well, a bit of a compromise but we are bringing the physical server count down to more than half of the current numbers. For all you hardware geeks out there let me give you a quick rundown of what we are putting this environment on …
- 2 EMC VMax- 1 for full production the other for test/dev and SRDF/A Target roughly 70TB usable
- 4 DELL R910s for Datacenter 1 – 128 total cores and 1.2 TBs total memory – Production Cluster
- 4 DELL R910s for Datacenter 2 – 128 total cores and 1.2 TBs total memory – Test/Dev/Failover cluster
The Hypervisor and utilities
- ESXi 4.1
- vCenter 4.1 running on a Windows 2008 VM
- Powerpath/VE (Just listed as supporting ESXi 4.1 about a week ago)
In addition to all this new gear, I still have not mentioned the infrastructure for the backups which will include a couple more SANs and more replication. I don’t want to be too longwinded about that here. After all, we are here to talk about virtualizing SAP.
While doing all the design for slot size and etc, I was researching and found there were some special requirements to virtualize SAP in a supported fashion. Our SAP environment is running on Windows 2003 x64, so most of these requirements will relate directly to running SAP on Windows. (Also big thanks to @BasRaayman for good insight into a lot of this.)
First and foremost, SAP absolutely requires memory reservations. Most of these servers have 48GB of memory so we needed a very large cluster design to have the proper amount of slot sizes (in this scenario with such strong hardware, I chose not to use resource pools). The way SAP works is the app fully allocates the memory and doesn’t free it up as long as it is running. If these servers don’t have the memory guaranteed to them 100% of the time they will most likely perform very poorly.
Secondly, there is some cloudy information out there about CPU reservations in SAP. I spent a while reading through many SAPNotes (which led to some confusion) and chatting with colleagues about it. In the end I decided to trust in what I had designed, and let the Hypervisor deal with the CPU scheduling. The important part in this is that you plan for capacity and design your cluster(s) very carefully. In my opinion, CPU reservations are not necessary.
Now, next up is something you will need to prod your SAP Basis Administrators about. There is a memory model setting inside of the newer versions of SAP that is important to set when running in a virtual environment. If you are running Windows 2003, it should always be set to classical. However, if you are running windows 2008
there are differences to take into account. If you have a CPU bound system with plenty of memory resources, go with the Flat Memory model as recommended by SAP. If you are memory bound, go with Classical. This can be tricky and require some monitoring and tuning after it is already live.
For the particularly high I/O VMs, I decided to use RDMs for the data volumes (these are mostly housing large SQL DBs). Now I know this is not required as VMware stated the speed difference is next to nothing. However, if I see later the performance is fine I can always convert them to VMDKs. Also by using the PowerPath/VE plugin we have true load balancing and multipathing back to the VMax Arrays. When I get around to testing the performance with and without PowerPath/VE I may write another blog post about just that!
In my opinion SAP is like any other tier 1 application – just with a few additional special requirements. It can absolutely be virtualized, don’t listen to all the naysayers. My biggest word of advice is to plan carefully and keep your capacity in check, then any other extra bumps in the road should not be too bad.
— Ed Czerwin