Friday, December 23, 2011

Troubleshooting ESX virtual machine performance issues

VMware have created a KB that has a list of other KB's that help in troubleshooting virtual environments.

Tuning ESX/ESXi for better storage performance by modifying the maximum I/O block size
Balloon driver retains hold on memory causing virtual machine guest operating system performance issues
ESX has performance issues due to IRQ sharing
Enabling IOAT and Jumbo frames
Troubleshooting network performance issues
Analyzing SCSI Reservation conflicts on VMware Infrastructure 3.x and vSphere 4.x
Determining if multiple virtual CPUs are causing performance issues
Slow performance caused by misconfigured local storage or SAN array
Slow performance caused by out of date firmware on a RAID controller or HBA
Testing virtual machine storage I/O performance
[Internal] ESX and EMC storage performance tips
iSCSI and Jumbo Frames configuration on ESX 3.x, ESX 4.x and ESXi 5.x
Using esxtop to identify storage performance issues
Configuring Jumbo Frames on a vNetwork Distributed Switch
Verifying correct storage settings on ESX 4.x and ESXi 4.x
Network I/O Resource Management in vSphere 4.1 with vDS
Impact of virtual machine memory and CPU resource limits
Checking your firmware and BIOS levels to ensure compatibility with ESX/ESXi
Enabling Jumbo Frames for VMkernel ports in a virtual distributed switch
Changing the Queue Depth for QLogic and Emulex HBAs
High co-stop (%CSTP) values seen during virtual machine snapshot activities

Thursday, December 22, 2011

VDS Best Practices -cont

Continuing  from VMware Networking blog. The final two parts are up.
  • Blade Server Deployments
  • Operational best practices
Part 5 - Blade Server Deployments
  • Blade Server with Two 10 Gigabit Ethernet network adapters
  • Design Option 1 – Static Configuration
  • Design Option 2 – Dynamic Configuration with NIOC and LBT
  • Blade Server with Hardware assisted Logical network adapters (HP-Flex10 like deployment)

Part 6 - Operational best practices

  • Command Line Interface
  • vCenter API
  • Virtual Network Monitoring and Troubleshooting
  • Conclusion

Thursday, December 8, 2011

I/O Analyzer from VMware Labs

I/O Analyzer is a virtual appliance solution by VMware that provides a simple and standardized way of measuring storage performance in a VMware vSphere virtualized environments. I/O Analyzer automates the traditional storage performance analysis cycle and reduces the performance diagnosis time from days to hours.


  • Integrated framework for storage performance testing
  • Readily deployable virtual appliance
  • Easily configure and launch I/O tests on multiple hosts
  • View integrated performance results at both guest and host levels
  • Easily export performance data for off-site analysis

You need to have the following system requirements:
  • ESX 3.5 above (4.0 and 4.1 tested)
  • 350 MB storage space
  • 1 vcpu and 512 MB virtual memory
  • Firefox and Chrome browsers supported

Wednesday, December 7, 2011

VDS Best Practices

November 16 VMware Networking blog started 6 part blog about vDS Best Practice.

"These guidelines or best practices are discussed with reference to a sample example deployment. It is important to note that the suggestions/guidelines discussed here are based on the assumptions of the example deployment. Every customer environment is different as well as customer needs are different, and customers have to take that into account while designing the virtual network infrastructure using VDS. VDS is a flexible platform that can support different requirements from different customers. "

Some great information

Part 1 - Design Considerations
  • Design Considerations
  • Infrastructure design Goals
  • Infrastructure component configurations
  • Virtual Infrastructure traffic
  • Example Deployment components
  • Network Infrastructure
  • Virtual Infrastructure Traffic Types

Virtual Infrastructure Traffic Types

Part 2 - Virtual and Physical Switch Parameters

  • VDS parameters
  • Physical Network switch parameters

Part 3 - Rack Server Deployment with Eight 1 Gigabit adapters

  • Rack Server in Example Deployment
  • Rack Server with Eight 1 Gigabit Ethernet network adapters
  • Design Option 1 – Static configuration
  • Design Option 2 – Dynamic configuration with NIOC and LBT
 Rack Server with eight 1 Gigabit Ethernet network adapters

Part 4 - Rack Server Deployment with Two 10 Gigabit adapters

  • Rack Server with Two 10 Gigabit Ethernet network adapters
  • Design Option 1 – Static Configuration
  • Design Option 2 – Dynamic Configuration with NIOC and LBT

Rack server with 2 – 10 Gig NICs

So far really good information and makes good reading. looking forward to Parts 5 and 6.

Thanks Venky.

Sunday, December 4, 2011

vCenter Chargeback Manager 2.0

VMware have released vCenter Chargeback Manager 2.0.

New in this Release
Automatic Report Scheduler
Charge thin and thick provisioned virtual machines differently
Cost variance and cost optimization
Showback Report
Apply fixed cost based on virtual machine state
Tier-based storage costing
Support for raw device mapping
Complete support for vSphere 5.0 and vCloud Director 1.5
New cost models and billing policies for vCloud Director

You can the find download here

VMware also offer a free Online self paced course.

Title  VMware vCenter Chargeback Fundamentals
Summary  - Format:
  • Self-Paced
- Length: 1 Hours
Overview  This self paced course discusses how to install, configure and manage vCenter Chargeback to implement an effective cost accounting system for the virtualized environment.
Objectives  After completing this course, learners should be able to:
•  Describe the need for chargeback
•  Discuss Chargeback methods
•  Install and manage vCenter Chargeback application
•  Deploy vCenter Chargeback as a virtual appliance
•  Configure vCenter Chargeback application with common administration settings
•  Create and manage chargeback hierarchies
•  Create and manage cost models and templates
•  Generate and manage reports
•  Monitor System health and Perform common troubleshooting
Title  VMware vCenter Chargeback Fundamentals
Summary  - Format:
  • Self-Paced
- Length: 1 Hours
Overview  This self paced course discusses how to install, configure and manage vCenter Chargeback to implement an effective cost accounting system for the virtualized environment.
Objectives  After completing this course, learners should be able to:
•  Describe the need for chargeback
•  Discuss Chargeback methods
•  Install and manage vCenter Chargeback application
•  Deploy vCenter Chargeback as a virtual appliance
•  Configure vCenter Chargeback application with common administration settings
•  Create and manage chargeback hierarchies
•  Create and manage cost models and templates
•  Generate and manage reports
•  Monitor System health and Perform common troubleshooting

Tuesday, November 29, 2011

Local Area Connection #2

During my time working with VMware vSphere and virtualising Datacenters. I have come across some weird events and one of them is this Local Area Connection # 2 after cloning VM's. This was highlighted again to me when I was "preping" for VCP5 in my LAB.

 The clone powers on and you login and start to customize.Then try to add IP to this Interface and you get a warning Networking Error: IP address already assigned to anther adapter.

To fix this I used to go into Device Manager and after enabling Show Hidden Devices ( KB here and see and old Ethernet Network Interface and delete it..You may also need to go into registry and delete the corresponding key for Local Area Connection.Then add the IP address without error.

Below are links that help in resolving this behavior.

BTW , same was commented here at Scott Lowe Blog.

VCP 5 Experience

Its been some time since my last blog.....

Finally have my feet on the ground. 

So I started by building my LAB on my laptop ( IBM X201) and working my way threw the VCP 5 Exam Blueprint LINK.

Some good stuff here

 Having a number of Cert's and familiar with the testing process and felt ready for the exam. I sat the exam on 5th November and passed!!!!

Compared to other exams didn't feel that it was any easier or harder but needed to know your stuff.

So my recommendation is build a LAB and follow the blueprint.

Friday, September 16, 2011

Auto Deploy PowerPack

My favorite tool  PowerGUI.

Alan Renouf  over at Virtu-Al has the Auto Deploy PowerPack.

"The Image Builder and Auto Deploy Powerpack is a PowerGUI Powerpack which enables us as VMware admins to have a graphical interface to Image Builder and Auto Deploy which are currently released as PowerShell snap-ins only."

He has also created a video on the basics how to use and import PowerPack.

Here is the Link Auto Deploy PowerPack

Wednesday, September 14, 2011

ESXi 5 Log Files

VMware have new locations for ESXi logs. Below are the logs, locations and a brief description.
Note: ESXi logs are in memory and syslog server is required if logs need to be retained.
VMware Log KB.
Also here is a link for to various VMware product log locations.
ESXi 5.0 Host Log Files
  • /var/log/auth.log: ESXi Shell authentication success and failure.
  • /var/log/dhclient.log: DHCP client service, including discovery, address lease requests and renewals.
  • /var/log/esxupdate.log: ESXi patch and update installation logs.
  • /var/log/hostd.log: Host management service logs, including virtual machine and host Task and Events, communication with the vSphere Client and vCenter Server vpxa agent, and SDK connections.
  • /var/log/shell.log: ESXi Shell usage logs, including enable/disable and every command entered.
  • /var/log/sysboot.log: Early VMkernel startup and module loading.
  • /var/log/syslog.log: Management service initialization, watchdogs, scheduled tasks and DCUI use.
  • /var/log/usb.log: USB device arbitration events, such as discovery and pass-through to virtual machines.
  • /var/log/vob.log: VMkernel Observation events, similar to vob.component.event.
  • /var/log/vmkernel.log: Core VMkernel logs, including device discovery, storage and networking device and driver events, and virtual machine startup.
  • /var/log/vmkwarning.log: A summary of Warning and Alert log messages excerpted from the VMkernel logs.
  • /var/log/vmksummary.log: A summary of ESXi host startup and shutdown, and an hourly heartbeat with uptime, number of virtual machines running, and service resource consumption.

Logs from vCenter Server Components on ESXi 5.0
When an ESXi 5.0 host is managed by vCenter Server 5.0, two components are installed, each with its own logs:
  • /var/log/vpxa.log: vCenter Server vpxa agent logs, including communication with vCenter Server and the Host Management hostd agent.
  • /var/log/fdm.log: vSphere High Availability logs, produced by the fdm service.

Tuesday, September 13, 2011

Booting ESX off USB/SD

Interesting blog at ESXi Chronicles regarding booting from USB/SD.

A snippet below.

At VMworld I was surprised at the number of questions I was asked related to booting ESXi off of USB/SD. Unfortunately, the vSphere docs largely assume you will use a local disk and don't really cover USB/SD, as such it’s not always easy to find answers to common questions. Below are some answers to several of the more popular USB/SD related questions.

1. What size USB/SD should I use?
The minimum disk size required to install ESXi is 1GB. When booting USB/SD there is little benefit to using a larger device because any space beyond the 1st GB will go unused. When choosing a USB/SD boot device, it’s not the size of the device that is important but the reliability of the device. Be sure to use good quality USB/SD devices.

You can read it here.

Wednesday, August 24, 2011


With the release of vSphere 5 coming soon, VMware have the certification path outlined.

The new exam is available to schedule from Monday 29th of August.

VMware has just released the VCP5 mock exam. The mock exam consists of 30 questions.

For more details and for requirements check here

Saturday, July 30, 2011

VMware Licensing Changes - True or Not

Is it True?

Gabrie van Zanten has up on his blog that a change is happening with the licensing of vSphere 5 he writes about a rumor, which you can read VMware changes vRAM licensing on vSphere 5 after customer feedback on vTAX

The new policy would be;
  • VMware vSphere 5 Essentials will give a 24GB vRAM entitlement
  • VMware vSphere 5 Essentials Plus will give a 32GB vRAM entitlement
  • Max vRAM in Essentials / Essentials Plus will be maxed at 192GB vRAM
  • VMware vSphere 5 Standard vRAM entitlement has changed to 32GB ( <- my assumption)
  • VMware vSphere 5 Enterprise vRAM entitlement will be doubled to 64GB
  • VMware vSphere 5 Enterprise Plus vRAM entitlement will be doubled to 96GB

You see other comments here

 Like I stated with the comment "Is it True?", Time will tell!


Wednesday, July 20, 2011

VCP 5 exam requirements


If you have your VCP 4, no need to attend a course until February 29 2012.

Sunday, July 10, 2011

Big Announcement from VMware

Cloud Infrastructure Launch Forum
Date: Tuesday, July 12, 2011
Time: 9:00 AM (PT) | 12:00 PM (ET)

Please join VMware executives Paul Maritz, CEO and Steve Herrod, CTO for the unveiling of the next major step forward in Cloud infrastructure.

We all know vSphere 5 is around the corner.
Will it be announced during this event??

You can register here.

Friday, June 24, 2011

VMware Converter 5 Beta

Some really good reasons to try Converter 5.

Disk Alignment, Increase partition size.

Ivobeerens has some good info.

VMware direct link is here

Tuesday, April 12, 2011

VMware vSphere 4.1 HA and DRS technical deepdive is HERE!!!!

Finally got my hands on a copy of Duncan Epping and Frank Denneman book. HA and DRS technical deepdive.

Now just need the time to read it...

VMware Performance and Capacity

Excellent presentation at Toronto VMUG. Bryan Semple CMO from VKernel.

You can download PDF doc.

RV Tools

New version is Released.

Some details below.

Version 3.1 (April, 2011)
  • Logon form tab order rearranged
  • Logon form will remember your last selected host / vCenter server
  • On vInfo new fields Provisioned, Used and shared storage
  • On vInfo new fields install Boot Required, number of Virtual Disks
  • On vInfo new fields Fault Tolerance State,  FT Latency Status, FT Band width and FT Secondary Latency
  • On vInfo new field 128-bit SMBIOS UUID of the virtual machine
  • On vDatastore new fields Total provisioned, Used and shared storage
  • On vDatastore new fields SIOC enabled flag and congested threshold value
  • On vDisk new field disk persistence mode
  • On vNetwork all IP addresses of adapter are now visible
  • On vMemory new field distributed Memory Entitlement
  • On vCPU new fields static Cpu Entitlement and field distributed Cpu Entitlement
  • On vHost new fields Current EVC mode and Max EVC mode
  • New batch command line parameters -u user and -p password
  • Bugfix: custom fields not always visible on vSnapshot tab
  • Bugfix: Export to Excel, some numeric columns are saved as text instead of numbers
  • RVToolsBatch.cmd with send by email example deployed in RVTools program file directory

Thursday, March 24, 2011

VCAP-DCA Section 4 -4.1


Section 4 – Manage Business Continuity and Protect Data
Objective 4.1: Implement and Maintain Complex VMware HA Solutions

Skills and Abilities
·         Calculate host failure requirements
·         Configure customized isolation response settings
·         Configure HA redundancy in a mixed ESX/ESXi environment
·         Configure HA related alarms and monitor an HA cluster
·         Create a custom slot size configuration
·         Understand interactions between DRS and HA
·         Create an HA solution that ensures primary node distribution across sites
·         Analyze vSphere environment to determine appropriate HA admission control policy
·         Analyze performance metrics to calculate host failure requirements
·         Analyze Virtual Machine workload to determine optimum slot size
·         Analyze HA cluster capacity to determine optimum cluster size

Configure Advance Cluster Settings using vMA

Here is another way to configure Advance Cluster Settings.

We can use vMA detailed here

Once you have the script and config file. Run the command;

./ --server XXXXXX --cluster XXXXXX --file ha_configs

Changing IP vMA

As I was in my LAB for VCAP-DCA study needed to change IP addres on my vMA box, here are two options which are detailed by Ray Heffer @ ET. Link below.

Here is one Option.
 Change the working directory to /opt/vmware/vma/bin/
# cd /opt/vmware/vma/bin/
Run the script:
# sudo ./

Wednesday, March 23, 2011

VCAP-DCA Section 6-6.1

Section 6 – Perform Advanced Troubleshooting
Objective 6.1 – Configure, Manage and Analyze vSphere Log Files

Skills and Abilities
·         Generate vCenter Server and ESX/ESXi log bundles
·         Use vicfg-syslog to configure centralized logging on ESX/ESXi Hosts
·         Test centralized logging configuration
·         Configure the vMA appliance as a log host
·         Use vilogger to enable/disable log collection on the vMA appliance
·         Use vilogger to configure log rotation and retention
·         Analyze log entries to obtain configuration information
·         Analyze log entries to identify and resolve issues

Tuesday, March 22, 2011

VCAP-DCA Section 3 -3.5


Section 3 – Deploy DRS Clusters and Manage Performance
Objective 3.5 – Utilize Advanced vSphere Performance Monitoring Tools

Skills and Abilities
·         Configure esxtop/resxtop custom profiles
·         Determine use cases for and apply esxtop/resxtop Interactive, Batch and Replay modes
·         Use vscsiStats to gather storage performance data
·         Given esxtop/resxtop output, identify relative performance data for capacity planning purposes
·         Use esxtop/resxtop to collect performance data

Thursday, March 17, 2011

Objective 8.2 -- Administer vCenter Orchestrator – Part 2

Objective 8.2 -- Administer vCenter Orchestrator
This is covered in this video.
·          Run a Workflow
·          Administer Actions, Tasks, Workflows and Policies
·          Administer Packages
·          Identify appropriate Workflow for a given management activity

Objective 8.2 -- Administer vCenter Orchestrator - Setup Lab

Objective 8.2 -- Administer vCenter Orchestrator

This video is just going over how to setup Orchestrator. As it wastnt a black and white as I thought it would be as I have never used this before.

The issue was around the database config.

Hope it helps others out there.

Wednesday, March 16, 2011

VCAP-DCA Blueprint Update

There has been an updated release of the Exam Blueprint ver 1.7. Link Below

Nort sure what has been change. Will update soon.

Monday, March 14, 2011

VCAP-DCA Section 4 - 4.4

Section 4 Manage Business Continuity and Protect Data
Objective 4.4 - Deploy and Maintain vCenter Server Heartbeat

A video on vCenter Heatbeat going over the skills and abilities

The PDF document used in the video can be found here
 PDF Document 1

PDF Document 2

The HD Video is here.

Section 4 – Manage Business Continuity and Protect Data from Carlos Vargas on Vimeo.

Saturday, March 12, 2011

VCAP DCA Lab BrownBag @

Lab Resheduled.

If you  are looking at the VCAP-DCA Cert would suggest registering and join this Brownbag.


Date/Time: Friday, March 11, 2011 @ 7PM CST

The Lab

Your company has purchased 3 servers and an iSCSI SAN as an additional production cluster. As administrator you must complete the following tasks:
  • Configure the hosts for availability and resource distribution
  • Enable future growth for the hosts
  • Enable power savings
  • Ensure the configuration of all hosts is consistent and auditable
  • Configure iSCSI to support production work loads
  • In addition to iSCSI, you must also support an HA Web network segment
Direct Link:

Thursday, March 10, 2011

VCAP-DCA Section 5 - 5.1

Section 5 Perform Operational Maintenance
VCAP-DCA Objective 5.1 : Implement and Maintain Host Profiles

Skills and Abilities
Create sub-profiles
Use Profile Editor to edit and/or disable policies
Use Host Profiles to deploy vDS

The PDF document used in the video can be found here PDF Document.
The HD Video is here.


VCAP-DCA Section 5 Objective 5.1 from Carlos Vargas on Vimeo.

Wednesday, March 9, 2011

Saturday, March 5, 2011

VCAP-DCA Section 8 - 8.3

Section 8 Perform Scripting and Automation
VCAP-DCA Objective 8.3 : Administer vSphere Using the VSphere Management Assitant

•Install and Configure vMA
•Add/Remove target servers
•Perform updates to the vMA
•Use vmkfstools to manage VMFS datastores
•Use vmware-cmd to manage VMs
•Use esxcli to manage Storage Multipathing
•Troubleshoot common vMA errors and conditions
The PDF document used in the video can be found here PDF Document.

The HD Video is here.

VCAP-DCA Section 8 Objective 8.3 from Carlos Vargas on Vimeo.

Monday, February 28, 2011

VCAP-DCA Section 2 - 2.2

VCAP-DCA Objective 2.2 : Configure and Maintain VLANs, PVLANs and VLAN settings

The video contains information on how to create VLANs and settings. Also has information on the LAB enviroment created and the componets used.

The PDF document used in the video can be found here PDF Document.

The Video is here.

VCAP-DCA Section 2 - Objective 2.2 from Carlos Vargas on Vimeo.

Friday, February 25, 2011

PowerCLI Book Heads Up

PowerCLI a must for any adminstrator of a  virtualised environment.

Here is new book dedicated to PowerCLI.

Tuesday, February 22, 2011

Wednesday, February 16, 2011

Configure SNMP ESX

Here is my first video.

VCAP-DCA -SNMP from Carlos Vargas on Vimeo.

How to configure SNMP on ESX. Link at

Sunday, February 13, 2011

VCAP-DCA Certification journey

Thought I would document my certification journey.....


Where to start..... Google search "VCAP-DCA", "VCAP-DCA Certification"
Scrolling over the results it was clear that one of the most popular and one that suited me was Sean Crookston's Blog.
With a well constructed study guide ( PDF Here ) and with most of the information that would be probably need to get at  level to achieve a PASS mark.
As he states in his blog he will no longer update the  study guide but lucky for us who are going to take this journey, Damin Karison at is stepping up and taking over with the updating of the Guide.
Apart from the blogs above and one who likes watching videos. there was one link that started me off searching for training videos and it was on Sean Crookston's VCAP-DCA blog page under the section "Other Great Resources for studying for the VCAP_DCA exam"
I'm a member of a group in LinkedIn called VMware Certified Professional (VCP) and noticed a Discussion by Eric Sloof titled VCAP-DCA Prep Video - Configure SNMP for ESX/ESXi.
With this I'm now inspired to try to emulate Erics video on some of the other sections. My thinking is as I do these videos I'm burning the steps in my head and be in readiness for the exam labs. So that's the theory!
I really don't want to duplicate all the good material out there but will concentrate more on the technical steps of  each section.
So with information above, it's time to get studying.
Starting with Section 1;
Section 1 – Implement and Manage Storage
VCAP-DCA Objective 1.1 : Implement and Manage Complex Storage Solutions
VCAP-DCA Objective 1.2 : Manage Storage Capacity in a vSphere Environment
VCAP-DCA Objective 1.3 : Configure and manage Complex Multipathing and PSA Plug-ins
Section 2 – Implement and Manage Networking
VCAP-DCA Objective 2.3 : Deploy and Maintain Scalable Virtual Networking
VCAP-DCA Objective 2.4 : Administer vNetwork Distributed Switch Settings
Section 3 – Deploy DRS Clusters and manage Performance
VCAP-DCA Objective 3.1 : Tune and Optimize vSphere Performance
VCAP-DCA Objective 3.2 : Optimize Virtual machine Resources
VCAP-DCA Objective 3.3 : Implement and Maintain Complex DRS Solutions
VCAP-DCA Objective 3.4 : Perform Capacity Planning in a vSphere Environment
Section 4 – Manage Business Continuity and Protect Data
VCAP-DCA Objective 4.2 : Deploy and Test VMware FT
VCAP-DCA Objective 4.3 : Configure a vSphere Environment to support MSCS Clustering
Section 5 – Perform Operational Maintenance
VCAP-DCA Objective 5.2 : Deploy and Manage Complex Update Manager Environments
Section 6 – Perform Advanced Troubleshooting
VCAP-DCA Objective 6.2 : Troubleshoot CPU and Memory Performance
VCAP-DCA Objective 6.3 : Troubleshoot Network Performance and Connectivity
VCAP-DCA Objective 6.4 : Troubleshoot Storage Performance and Connectivity
VCAP-DCA Objective 6.5 : Troubleshoot vCenter Server and ESX/ESXi Host Management
Section 7 – Secure a vSphere Environment
VCAP-DCA Objective 7.1 : Secure ESX/ESXi hosts
VCAP-DCA Objective 7.2 : Configure and Maintain the ESX Firewall
VCAP-DCA Objective 7.3 : Deploy and Administer vShield zones
Section 8 – Perform Scripting and Automation
VCAP-DCA Objective 8.1 : Execute VMware Cmdlets and Customize Scripts using PowerCLI
Section 9 – Perform Advanced vSphere Installations and Configurations
VCAP-DCA Objective 9.1 : Install ESX Server with custom Settings
VCAP-DCA Objective 9.2 : Plan and Execute Scripted Installations
VCAP-DCA Objective 9.3 : Configure vCenter Server Linked Mode