SQL Server CPU Pressure on VMware: When Low CPU Ready Means the Problem Is Yours

SQL Server CPU Pressure on VMware: When Low CPU Ready Means the Problem Is Yours – SQLYARD

SQL Server CPU Pressure on VMware: When Low CPU Ready Means the Problem Is Yours


SQL Server 2016 and Later · VMware vSphere · All Editions

SQL Server is showing 90 percent CPU in Task Manager. The first call goes to the VMware team. The VMware team pulls up vCenter, shows CPU Ready time at 2 percent, and closes the ticket. The problem is not theirs. They are right. The problem is inside SQL Server and the DBA team now owns it with no clear starting point.

This scenario happens constantly in virtualized SQL Server environments. CPU pressure and VMware CPU Ready are related but separate concepts, and conflating them is one of the most common diagnostic wrong turns in a production incident. This article explains exactly what CPU Ready measures, why low CPU Ready definitively rules out hypervisor oversubscription, and then provides the complete diagnostic pathway inside SQL Server to find the actual cause when the hypervisor is not the problem.

The one-sentence version: Low CPU Ready rules out host oversubscription. The problem is a guest-level workload, plan, or scheduler issue and the tools to diagnose it are all inside SQL Server.

1 What VMware CPU Ready Actually Measures Beginner

CPU Ready time is a VMware performance metric that measures the percentage of time a virtual machine was ready to run but could not because no physical CPU was available to service it. In simple terms: the VM had work to do, the SQLOS had a thread that wanted to execute, but the hypervisor scheduler could not hand it a physical CPU immediately because all available physical cores were already servicing other VMs or workloads on the same host.

High CPU Ready time means the ESXi host is oversubscribed. Too many vCPUs have been allocated across VMs relative to the physical cores available, and VMs are queuing at the hypervisor level waiting for physical CPU time. This manifests in SQL Server as erratic response times, inconsistent query durations, and CPU stalls that are invisible inside SQL Server’s own wait statistics.

The standard guideline from VMware documentation is that CPU Ready time above 5 percent sustained warrants investigation and above 10 percent represents genuine contention that will degrade guest performance. Below 5 percent, particularly below 2 to 3 percent, the hypervisor is delivering CPU access to the VM with minimal delay. The physical host is not the bottleneck.

CPU Ready is measured as a percentage over a 20-second sample interval in vCenter. A value of 2,000 milliseconds of CPU Ready in a 20-second window equals 10 percent CPU Ready. Most vCenter dashboards display the percentage directly. The raw millisecond value divided by the sample interval in milliseconds gives the percentage. Anything under 5 percent during peak hours indicates a healthy host from a CPU availability standpoint.

2 Why Low CPU Ready Closes the VMware Investigation Beginner

When CPU Ready is consistently below 5 percent, the hypervisor is granting the VM near-immediate access to physical CPU time whenever a thread is ready to run. The physical host is not oversubscribed relative to this VM’s workload. Adding vCPUs or migrating to a less loaded host will not materially change the performance of SQL Server, because the VM already receives the CPU cycles it requests without waiting for them at the hypervisor level.

The corollary is direct: if SQL Server is showing high CPU utilization and the guest OS is reporting high CPU usage, but CPU Ready is low, SQL Server is consuming that CPU genuinely. Something inside SQL Server is requesting and receiving a high volume of CPU work. The diagnostic questions shift completely from “is the hypervisor giving us enough CPU?” to “why is SQL Server consuming so much CPU?”

The most expensive diagnostic mistake in a virtualized SQL Server environment is spending days in vCenter and ESXi configuration while the root cause sits in a poorly-parameterized query, a stale statistics plan regression, or a CTFP setting of 5 that forces every query to run parallel. Low CPU Ready is the signal to stop looking at VMware and start looking at SQL Server internals.

3 Signal Wait Time vs Resource Wait Time: The First SQL Server Test Intermediate

The first query to run when investigating CPU pressure in SQL Server is not a query for top CPU consumers. It is the signal wait analysis against sys.dm_os_wait_stats. This single ratio determines whether SQL Server is genuinely CPU-bound at the scheduler level or whether high CPU usage is being driven by lock contention, I/O, or other resource waits that happen to consume CPU while resolving.

SQL Server workers exist in one of three states. Running: the worker has a scheduler quantum and is actively executing. Runnable: the resource the worker needed is available, but the scheduler is busy and the worker is waiting for its turn to execute. Suspended: the worker is waiting for a resource (a lock, an I/O completion, a latch) and has voluntarily left the scheduler queue.

Signal wait time is the time a worker spends in the Runnable state waiting for a CPU scheduler quantum after a resource became available. It is pure CPU scheduling wait. Resource wait time is total wait time minus signal wait time: the time spent waiting for locks, I/O, latches, and other non-CPU resources.

-- Signal wait analysis: the first CPU pressure test
-- Signal waits above 10-15% of total waits = concerning per Microsoft
-- Signal waits above 25% = high CPU pressure per Microsoft documentation
-- Run against the instance, not a specific database

SELECT
    SUM(signal_wait_time_ms)                                AS TotalSignalWaitMs,
    SUM(wait_time_ms)                                       AS TotalWaitMs,
    SUM(wait_time_ms - signal_wait_time_ms)                 AS TotalResourceWaitMs,
    CAST(100.0 * SUM(signal_wait_time_ms)
         / NULLIF(SUM(wait_time_ms), 0) AS DECIMAL(5,2))   AS SignalWaitPct,
    CASE
        WHEN CAST(100.0 * SUM(signal_wait_time_ms)
             / NULLIF(SUM(wait_time_ms), 0) AS DECIMAL(5,2)) > 25
            THEN 'HIGH: Significant CPU scheduler pressure'
        WHEN CAST(100.0 * SUM(signal_wait_time_ms)
             / NULLIF(SUM(wait_time_ms), 0) AS DECIMAL(5,2)) > 10
            THEN 'ELEVATED: Investigate CPU consumers'
        ELSE 'LOW: CPU scheduling is not the primary bottleneck'
    END                                                     AS Interpretation
FROM sys.dm_os_wait_stats
WHERE wait_type NOT IN (
    -- Exclude background and benign waits that inflate total wait time
    'SLEEP_TASK', 'SLEEP_SYSTEMTASK', 'SLEEP_DBSTARTUP', 'SLEEP_DBTASK',
    'SLEEP_TEMPDBSTARTUP', 'SLEEP_MASTERDBREADY', 'SLEEP_MASTERMDREADY',
    'SLEEP_MASTERUPGRADED', 'SLEEP_MSDBSTARTUP', 'SLEEP_TEMPDBSTARTUP',
    'WAITFOR', 'BROKER_TO_FLUSH', 'BROKER_TASK_STOP', 'CLR_AUTO_EVENT',
    'DISPATCHER_QUEUE_SEMAPHORE', 'FT_IFTS_SCHEDULER_IDLE_WAIT',
    'HADR_WORK_QUEUE', 'HADR_FILESTREAM_IOMGR_IOCOMPLETION',
    'LAZYWRITER_SLEEP', 'LOGMGR_QUEUE', 'ONDEMAND_TASK_QUEUE',
    'REQUEST_FOR_DEADLOCK_SEARCH', 'RESOURCE_QUEUE', 'SERVER_IDLE_CHECK',
    'SLEEP_DBSTARTUP', 'SLEEP_DBTASK', 'SLEEP_TEMPDBSTARTUP',
    'SNI_HTTP_ACCEPT', 'SP_SERVER_DIAGNOSTICS_SLEEP', 'SQLTRACE_BUFFER_FLUSH',
    'SQLTRACE_INCREMENTAL_FLUSH_SLEEP', 'WAIT_XTP_OFFLINE_CKPT_NEW_LOG',
    'XE_DISPATCHER_WAIT', 'XE_TIMER_EVENT', 'SOS_WORK_DISPATCHER'
);

Why this ratio matters more than Task Manager CPU percentage: A SQL Server instance can show 80 percent CPU in Task Manager with a signal wait percentage of only 3 percent. This means CPU is high but waits are driven by resource contention (locks, I/O) where CPU is consumed during retry cycles and buffer operations, not because the SQL Server scheduler queue is backed up. The right fix in that scenario is lock optimization or I/O improvement, not adding CPU. The signal wait percentage tells the DBA which direction to look before any other query runs.

4 sys.dm_os_schedulers: Is SQL Server CPU-Starved at the Scheduler Level? Intermediate

SQL Server’s SQLOS assigns each logical processor a scheduler. Worker threads execute on a specific scheduler. When more worker threads want to execute than the scheduler can service, they queue in the runnable state. The runnable_tasks_count column in sys.dm_os_schedulers shows how many tasks are currently queued waiting for their turn on each scheduler.

This is the SQL Server-internal equivalent of VMware CPU Ready, and it is distinct from VMware CPU Ready. A VM can have low VMware CPU Ready (the hypervisor grants physical CPU immediately) while simultaneously having high runnable_tasks_count (SQL Server’s internal scheduler queue is backed up because too many threads want to execute at the same time). The two are independent measurements at different layers of the stack.

-- Check runnable queue depth across all SQL Server schedulers
-- runnable_tasks_count > 1 on multiple schedulers = SQL Server is scheduler-bound
-- This is genuine CPU starvation at the SQLOS level, separate from VMware

SELECT
    scheduler_id,
    cpu_id,
    status,
    is_online,
    runnable_tasks_count,       -- tasks queued waiting for a CPU quantum
    current_tasks_count,        -- total tasks on this scheduler
    work_queue_count,           -- tasks waiting to be picked up
    pending_disk_io_count,      -- pending I/O completions on this scheduler
    load_factor,                -- SQL Server load balancing metric
    CASE
        WHEN runnable_tasks_count > 3
            THEN 'HIGH: Significant scheduler queue - CPU starvation at SQLOS level'
        WHEN runnable_tasks_count > 1
            THEN 'ELEVATED: Multiple tasks queued'
        ELSE 'OK'
    END                         AS SchedulerAssessment
FROM sys.dm_os_schedulers
WHERE status = 'VISIBLE ONLINE'   -- only active schedulers
ORDER BY runnable_tasks_count DESC;

-- Summary: total runnable tasks across all schedulers
-- Snapshot this every 5-10 seconds during peak load for trend
SELECT
    SUM(runnable_tasks_count)   AS TotalRunnableTasks,
    MAX(runnable_tasks_count)   AS MaxOnAnyScheduler,
    COUNT(*)                    AS TotalSchedulers,
    CASE
        WHEN SUM(runnable_tasks_count) > COUNT(*) * 2
            THEN 'CRITICAL: Average runnable queue > 2 per scheduler'
        WHEN SUM(runnable_tasks_count) > COUNT(*)
            THEN 'HIGH: Average runnable queue > 1 per scheduler'
        WHEN SUM(runnable_tasks_count) > 0
            THEN 'MODERATE: Some scheduler queuing observed'
        ELSE 'CLEAN: No scheduler queuing'
    END                         AS OverallAssessment
FROM sys.dm_os_schedulers
WHERE status = 'VISIBLE ONLINE';

5 SOS_SCHEDULER_YIELD: What the Wait Type Tells You Intermediate

SQL Server uses cooperative scheduling through SQLOS. Threads do not run until the operating system preempts them. Instead, threads voluntarily yield the scheduler after approximately 4 milliseconds to allow other runnable tasks to execute. This voluntary yield is recorded as an SOS_SCHEDULER_YIELD wait.

On a system with adequate CPU, the wait time for SOS_SCHEDULER_YIELD is near zero because the yielding thread is immediately scheduled back. On a CPU-pressured system, the yielding thread joins the runnable queue and waits for its next quantum. A high SOS_SCHEDULER_YIELD wait time combined with high signal wait percentage is a reliable indicator of genuine CPU pressure at the SQL Server scheduler level.

-- Check SOS_SCHEDULER_YIELD in context of total wait statistics
-- High SOS_SCHEDULER_YIELD + high signal_wait_time = CPU pressure confirmed

SELECT TOP 20
    wait_type,
    waiting_tasks_count,
    wait_time_ms,
    signal_wait_time_ms,
    wait_time_ms - signal_wait_time_ms      AS resource_wait_time_ms,
    -- For SOS_SCHEDULER_YIELD, nearly all of wait_time IS signal wait
    -- because the thread already has its resource, just needs CPU time
    CAST(100.0 * signal_wait_time_ms
         / NULLIF(wait_time_ms, 0) AS DECIMAL(5,1))  AS PctSignalWait,
    CAST(wait_time_ms / 1000.0
         / NULLIF(waiting_tasks_count, 0) AS DECIMAL(10,3)) AS AvgWaitSec
FROM sys.dm_os_wait_stats
WHERE wait_type NOT IN (
    'SLEEP_TASK', 'WAITFOR', 'BROKER_TO_FLUSH', 'LAZYWRITER_SLEEP',
    'LOGMGR_QUEUE', 'REQUEST_FOR_DEADLOCK_SEARCH', 'RESOURCE_QUEUE',
    'SERVER_IDLE_CHECK', 'XE_DISPATCHER_WAIT', 'XE_TIMER_EVENT',
    'SOS_WORK_DISPATCHER', 'HADR_WORK_QUEUE'
)
  AND wait_time_ms > 0
ORDER BY wait_time_ms DESC;

-- Specifically check SOS_SCHEDULER_YIELD position in overall wait profile
SELECT
    wait_type,
    CAST(wait_time_ms / 1000.0 AS DECIMAL(10,1))       AS TotalWaitSec,
    CAST(signal_wait_time_ms / 1000.0 AS DECIMAL(10,1)) AS SignalWaitSec,
    waiting_tasks_count,
    CAST(wait_time_ms * 100.0
         / SUM(wait_time_ms) OVER ()  AS DECIMAL(5,2))  AS PctOfTotalWaits
FROM sys.dm_os_wait_stats
WHERE wait_type = 'SOS_SCHEDULER_YIELD';

SOS_SCHEDULER_YIELD is a symptom, not a root cause. According to Microsoft documentation, SOS_SCHEDULER_YIELD appearing as a top wait type should prompt checking Performance Monitor for sustained high Processor Time and Processor Queue Length, and validating the runnable state of workers via sys.dm_os_schedulers. After confirming genuine CPU pressure, the investigation moves to finding which queries are consuming that CPU, covered in the next section.

6 sys.dm_exec_query_stats: Finding the Top CPU Consumers Right Now Intermediate

Once signal waits confirm CPU pressure and sys.dm_os_schedulers shows a backed-up runnable queue, the next step is identifying which queries are consuming the CPU. sys.dm_exec_query_stats accumulates CPU consumption since the plan entered the cache. sys.dm_exec_requests shows what is actively running right now.

-- TOP 20 CPU consumers in the plan cache (cumulative since SQL Server start or plan compilation)
-- Sort by total_worker_time DESC for all-time top consumers
-- Sort by total_worker_time / execution_count for worst per-execution consumers

SELECT TOP 20
    qs.total_worker_time / 1000                 AS TotalCPUms,
    qs.execution_count,
    qs.total_worker_time / qs.execution_count / 1000
                                                AS AvgCPUms,
    qs.total_elapsed_time / qs.execution_count / 1000
                                                AS AvgElapsedMs,
    qs.total_logical_reads / qs.execution_count AS AvgLogicalReads,
    qs.total_physical_reads / qs.execution_count AS AvgPhysicalReads,
    qs.total_worker_time * 100.0
        / SUM(qs.total_worker_time) OVER ()     AS PctOfTotalCPU,
    qs.creation_time                            AS PlanCompiledAt,
    DB_NAME(CAST(pa.value AS INT))              AS DatabaseName,
    SUBSTRING(st.text,
        (qs.statement_start_offset / 2) + 1,
        ((CASE qs.statement_end_offset
            WHEN -1 THEN DATALENGTH(st.text)
            ELSE qs.statement_end_offset END
          - qs.statement_start_offset) / 2) + 1
    )                                           AS QueryText
FROM sys.dm_exec_query_stats              qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st
CROSS APPLY sys.dm_exec_plan_attributes(qs.plan_handle) pa
WHERE pa.attribute = 'dbid'
ORDER BY qs.total_worker_time DESC;

-- Active sessions RIGHT NOW sorted by CPU time consumed in this request
-- Use this during a live CPU spike to see what is running
SELECT TOP 20
    r.session_id,
    r.status,
    r.wait_type,
    r.wait_time / 1000                          AS WaitTimeSec,
    r.cpu_time                                  AS CPUms,
    r.total_elapsed_time / 1000                 AS ElapsedSec,
    r.logical_reads,
    r.reads                                     AS PhysicalReads,
    r.writes,
    r.dop                                       AS DegreeOfParallelism,
    DB_NAME(r.database_id)                      AS DatabaseName,
    LEFT(st.text, 200)                          AS QueryText,
    qp.query_plan
FROM sys.dm_exec_requests                 r
CROSS APPLY sys.dm_exec_sql_text(r.sql_handle)  st
CROSS APPLY sys.dm_exec_query_plan(r.plan_handle) qp
WHERE r.session_id != @@SPID
  AND r.status != 'background'
ORDER BY r.cpu_time DESC;

The DOP column on dm_exec_requests is the fastest way to spot runaway parallel queries. A DOP of 16 on an 8-core server means that single query is consuming 16 worker threads. If multiple such queries run simultaneously, the runnable queue fills immediately and every other query on the server feels the CPU pressure. High DOP values on non-trivial queries should trigger a MAXDOP and CTFP review.

7 Parallelism: MAXDOP and Cost Threshold as CPU Multipliers Intermediate

Parallelism is the most common unrecognized CPU amplifier in SQL Server environments. A single parallel query consuming 8 logical CPUs is indistinguishable from eight single-threaded queries in terms of total CPU consumption, but it creates a very different scheduling pattern. All 8 threads run simultaneously, competing with each other for scheduler time and leaving fewer schedulers available for other sessions.

Two settings control parallelism behavior. MAXDOP (Max Degree of Parallelism) sets the maximum number of threads a single query can use. The Cost Threshold for Parallelism (CTFP) sets the minimum estimated query cost at which SQL Server considers a parallel plan at all.

The default CTFP is 5. According to SQL Server community best practice (discussed extensively at SQLYARD), a CTFP of 5 means almost every non-trivial query is eligible for a parallel plan because the SQL Server query optimizer assigns an estimated cost of 5 or more to most joins against tables with more than a few thousand rows. The result is that the query optimizer considers and often chooses parallel plans for queries that would run perfectly well in serial with lower total CPU consumption. On an 8-core server with CTFP of 5, the entire server is frequently running in high-DOP mode even when the workload does not warrant it.

-- Check current MAXDOP and CTFP settings
SELECT
    name,
    value_in_use,
    CASE name
        WHEN 'max degree of parallelism'
            THEN CASE
                WHEN value_in_use = 0 THEN 'WARNING: MAXDOP 0 = unlimited parallelism'
                WHEN value_in_use = 1 THEN 'INFO: No parallelism. May be too restrictive on multi-core.'
                WHEN value_in_use <= 8 THEN 'OK: Standard MAXDOP setting'
                ELSE 'REVIEW: High MAXDOP may allow excessive thread consumption per query'
            END
        WHEN 'cost threshold for parallelism'
            THEN CASE
                WHEN value_in_use <= 5  THEN 'WARNING: Default of 5 causes over-parallelism on OLTP workloads'
                WHEN value_in_use < 35  THEN 'ELEVATED: Consider raising to 35-50 for OLTP'
                WHEN value_in_use BETWEEN 35 AND 50 THEN 'GOOD: In recommended OLTP range'
                ELSE 'REVIEW: High CTFP may prevent beneficial parallelism on large analytical queries'
            END
        ELSE ''
    END                     AS Assessment
FROM sys.configurations
WHERE name IN ('max degree of parallelism', 'cost threshold for parallelism')
ORDER BY name;

-- Find queries currently running in parallel and their thread counts
SELECT
    r.session_id,
    r.dop                   AS DegreeOfParallelism,
    r.cpu_time              AS CPUms,
    r.status,
    r.wait_type,
    DB_NAME(r.database_id)  AS DatabaseName,
    LEFT(st.text, 150)      AS QueryText
FROM sys.dm_exec_requests         r
CROSS APPLY sys.dm_exec_sql_text(r.sql_handle) st
WHERE r.dop > 1
  AND r.session_id != @@SPID
ORDER BY r.dop DESC, r.cpu_time DESC;

-- Check CXPACKET and CXCONSUMER wait balance
-- High CXPACKET alone is not a problem if CXCONSUMER is close
-- CXPACKET >> CXCONSUMER = skewed parallelism, possible parallel plan issue
SELECT
    wait_type,
    waiting_tasks_count,
    wait_time_ms,
    signal_wait_time_ms
FROM sys.dm_os_wait_stats
WHERE wait_type IN ('CXPACKET', 'CXCONSUMER')
ORDER BY wait_type;

8 Plan Regressions Presenting as CPU Pressure Intermediate

One of the most deceptive CPU pressure patterns is a plan regression that drives a previously fast query into a CPU-intensive execution path. The query has not changed. The data volume has not changed dramatically. But stale statistics or parameter sniffing has caused the optimizer to choose a scan-heavy plan that reads millions of rows where a previous seek-based plan read thousands. The result is a query that appears CPU-heavy but is actually doing unnecessary logical I/O that translates to CPU consumption in the buffer pool layer.

The diagnostic signature is a query with high total_worker_time relative to total_logical_reads in sys.dm_exec_query_stats, combined with a plan compilation date that is recent (within the last few hours or since the last statistics update). Plan Store data from Query Store makes this comparison straightforward: compare the current plan’s runtime statistics against the plan’s historical performance.

-- Find recent plan compilations with high CPU - potential plan regressions
-- Plans compiled recently that are consuming disproportionate CPU
SELECT TOP 20
    qs.creation_time                            AS PlanCompiledAt,
    qs.total_worker_time / 1000                 AS TotalCPUms,
    qs.execution_count,
    qs.total_worker_time / qs.execution_count / 1000 AS AvgCPUms,
    qs.total_logical_reads / qs.execution_count AS AvgLogicalReads,
    -- High logical reads per CPU unit = scan-heavy plan likely
    CAST(qs.total_logical_reads * 1.0
         / NULLIF(qs.total_worker_time, 0) AS DECIMAL(10,2))
                                                AS ReadsPerCPUUnit,
    LEFT(st.text, 200)                          AS QueryText
FROM sys.dm_exec_query_stats              qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st
WHERE qs.creation_time > DATEADD(HOUR, -6, GETDATE())  -- plans compiled in last 6 hours
  AND qs.execution_count > 5                            -- must have run enough to matter
ORDER BY qs.total_worker_time DESC;

-- Query Store: find queries where current plan is worse than historical best
-- Run in the target database
SELECT TOP 20
    qsp.query_id,
    qt.query_sql_text,
    qsp.plan_id,
    rs.avg_cpu_time / 1000                      AS AvgCPUms,
    rs.avg_logical_io_reads                     AS AvgLogicalReads,
    rs.count_executions,
    qsp.last_compile_start_time                 AS PlanCompiledAt,
    -- Compare to the best historical plan for the same query
    best_plan.best_avg_cpu_ms
FROM sys.query_store_plan               qsp
JOIN sys.query_store_query              qsq ON qsp.query_id = qsq.query_id
JOIN sys.query_store_query_text         qt  ON qsq.query_text_id = qt.query_text_id
JOIN sys.query_store_runtime_stats      rs  ON qsp.plan_id = rs.plan_id
JOIN (
    SELECT
        qsq2.query_id,
        MIN(rs2.avg_cpu_time) / 1000 AS best_avg_cpu_ms
    FROM sys.query_store_plan           qsp2
    JOIN sys.query_store_query          qsq2 ON qsp2.query_id = qsq2.query_id
    JOIN sys.query_store_runtime_stats  rs2  ON qsp2.plan_id  = rs2.plan_id
    GROUP BY qsq2.query_id
) best_plan ON qsp.query_id = best_plan.query_id
WHERE rs.avg_cpu_time / 1000 > best_plan.best_avg_cpu_ms * 3  -- current plan 3x worse than best
  AND rs.count_executions > 10
ORDER BY rs.avg_cpu_time DESC;

9 vNUMA and NUMA Alignment: The Subtle CPU Tax Intermediate

Even with low VMware CPU Ready, a NUMA alignment problem can make SQL Server work harder per unit of logical work. This is the most nuanced CPU pressure source on the list because it does not show up as a single obvious wait type. It manifests instead as slightly higher wait times across multiple wait categories and slightly higher CPU consumption per query than expected for the logical work being done.

Modern multi-socket servers have NUMA (Non-Uniform Memory Access) architecture where each CPU socket has faster access to its local memory than to memory controlled by another socket. SQL Server is NUMA-aware and tries to keep memory allocations local to the scheduler processing the query. On physical hardware this works naturally because the BIOS exposes the correct NUMA topology to the OS and SQL Server.

On VMware, the virtual NUMA (vNUMA) topology that the VM sees may not match the physical NUMA topology of the underlying host, particularly when a VM is configured with more vCPUs than exist on a single physical NUMA node, or when a VM is sized without regard for physical NUMA boundaries. SQL Server then makes scheduling decisions based on incorrect NUMA information, leading to cross-NUMA memory access patterns that consume more CPU cycles per memory operation.

-- Check SQL Server NUMA configuration
SELECT
    memory_node_id,
    online_scheduler_count,
    active_worker_count,
    avg_load_balance,
    memory_object_count
FROM sys.dm_os_memory_nodes
WHERE memory_node_id != 64;   -- exclude the DAC node

-- Check for NUMA imbalance: if schedulers are unequally loaded across NUMA nodes,
-- SQL Server may be making suboptimal scheduling decisions due to vNUMA mismatch
SELECT
    parent_node_id          AS NUMANode,
    COUNT(*)                AS SchedulerCount,
    SUM(runnable_tasks_count) AS TotalRunnableTasks,
    AVG(load_factor)        AS AvgLoadFactor,
    MAX(load_factor)        AS MaxLoadFactor
FROM sys.dm_os_schedulers
WHERE status = 'VISIBLE ONLINE'
GROUP BY parent_node_id
ORDER BY parent_node_id;

-- If one NUMA node shows significantly higher runnable_tasks_count
-- or load_factor than others, this warrants a VMware team conversation
-- about vNUMA alignment with the physical host topology

NUMA misalignment is a VMware configuration problem, not a SQL Server problem, but it appears as a guest-level workload issue in isolation. The fix requires the VMware team to set numa.nodeAffinity or adjust vCPU socket and cores-per-socket configuration to align with the physical NUMA topology of the ESXi host. Raising this finding requires showing the NUMA imbalance data from the queries above and pairing it with the physical host topology from vCenter.

10 The Complete Diagnostic Workflow Beginner

Run these steps in order during a CPU pressure incident. Each step either identifies the root cause or narrows the scope for the next step.

  1. Confirm VMware CPU Ready is low (<5%). If high, escalate to VMware team to address host oversubscription. If low, proceed to SQL Server diagnosis.
  2. Run the signal wait analysis (Section 3). If signal wait percentage is below 10 percent, CPU utilization is driven by resource waits not scheduler pressure. Investigate the top resource wait types instead. If signal wait percentage is above 10 to 15 percent, confirm SQL Server scheduler pressure and proceed.
  3. Check sys.dm_os_schedulers runnable queue (Section 4). Confirm that tasks are queuing at the SQL Server scheduler level. This confirms genuine CPU starvation, not just high CPU utilization from a single large query.
  4. Check SOS_SCHEDULER_YIELD in wait statistics (Section 5). Confirm it appears in the top wait types. High signal wait component on SOS_SCHEDULER_YIELD confirms the yield wait is CPU-bound, not resource-bound.
  5. Identify top CPU consumers (Section 6). Find the specific queries or batches responsible. Check DOP on active requests for runaway parallel queries.
  6. Check MAXDOP and CTFP settings (Section 7). If CTFP is at the default of 5 and multiple queries are running parallel on what should be OLTP workload, raise CTFP to 35 to 50 and monitor the effect.
  7. Check for plan regressions (Section 8). If a top CPU consumer has a recently compiled plan and high logical reads per CPU unit, investigate statistics freshness and parameter sniffing.
  8. Check NUMA alignment (Section 9). If runnable tasks concentrate on specific NUMA nodes, flag the imbalance and engage the VMware team with the specific configuration data.

11 Workshop: CPU Pressure Diagnosis Script Set Intermediate

The following script combines all diagnostic queries into a single execution that produces a complete CPU pressure picture. Run this against the SQL Server instance during or immediately after a CPU pressure incident.

-- =====================================================================
-- SQL Server CPU Pressure Diagnosis Script Set
-- Run on the affected SQL Server instance during or after a CPU incident
-- =====================================================================

-- Section 1: Signal wait ratio
PRINT '=== 1. SIGNAL WAIT ANALYSIS ==========';
SELECT
    CAST(100.0 * SUM(signal_wait_time_ms) / NULLIF(SUM(wait_time_ms),0) AS DECIMAL(5,2))
        AS SignalWaitPct,
    CASE
        WHEN CAST(100.0 * SUM(signal_wait_time_ms) / NULLIF(SUM(wait_time_ms),0) AS DECIMAL(5,2)) > 25
            THEN 'HIGH CPU PRESSURE: Signal waits above 25% of total'
        WHEN CAST(100.0 * SUM(signal_wait_time_ms) / NULLIF(SUM(wait_time_ms),0) AS DECIMAL(5,2)) > 10
            THEN 'ELEVATED: Signal waits above 10% - investigate CPU consumers'
        ELSE 'LOW: CPU scheduling not the primary bottleneck'
    END AS Assessment
FROM sys.dm_os_wait_stats
WHERE wait_type NOT IN (
    'SLEEP_TASK','WAITFOR','BROKER_TO_FLUSH','LAZYWRITER_SLEEP',
    'LOGMGR_QUEUE','REQUEST_FOR_DEADLOCK_SEARCH','RESOURCE_QUEUE',
    'SERVER_IDLE_CHECK','XE_DISPATCHER_WAIT','XE_TIMER_EVENT',
    'SOS_WORK_DISPATCHER','HADR_WORK_QUEUE'
);

-- Section 2: Scheduler runnable queue
PRINT '=== 2. SCHEDULER RUNNABLE QUEUE ======';
SELECT
    SUM(runnable_tasks_count)   AS TotalRunnableTasks,
    MAX(runnable_tasks_count)   AS MaxOnAnyScheduler,
    COUNT(*)                    AS TotalSchedulers,
    GETDATE()                   AS Snapshot
FROM sys.dm_os_schedulers
WHERE status = 'VISIBLE ONLINE';

-- Section 3: Top 5 wait types right now
PRINT '=== 3. TOP 5 WAIT TYPES ==============';
SELECT TOP 5
    wait_type,
    wait_time_ms,
    signal_wait_time_ms,
    waiting_tasks_count,
    CAST(100.0 * signal_wait_time_ms / NULLIF(wait_time_ms,0) AS DECIMAL(5,1)) AS PctSignalWait
FROM sys.dm_os_wait_stats
WHERE wait_type NOT IN (
    'SLEEP_TASK','WAITFOR','BROKER_TO_FLUSH','LAZYWRITER_SLEEP',
    'LOGMGR_QUEUE','REQUEST_FOR_DEADLOCK_SEARCH','RESOURCE_QUEUE',
    'SERVER_IDLE_CHECK','XE_DISPATCHER_WAIT','XE_TIMER_EVENT',
    'SOS_WORK_DISPATCHER','HADR_WORK_QUEUE'
)
  AND wait_time_ms > 0
ORDER BY wait_time_ms DESC;

-- Section 4: Top 10 CPU consumers from cache
PRINT '=== 4. TOP 10 CPU CONSUMERS ==========';
SELECT TOP 10
    qs.total_worker_time / 1000             AS TotalCPUms,
    qs.execution_count,
    qs.total_worker_time / qs.execution_count / 1000 AS AvgCPUms,
    qs.total_logical_reads / qs.execution_count AS AvgLogicalReads,
    qs.creation_time                        AS PlanCompiledAt,
    LEFT(st.text, 150)                      AS QueryText
FROM sys.dm_exec_query_stats          qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st
ORDER BY qs.total_worker_time DESC;

-- Section 5: Active parallel requests
PRINT '=== 5. ACTIVE PARALLEL REQUESTS ======';
SELECT
    r.session_id,
    r.dop,
    r.cpu_time              AS CPUms,
    r.status,
    r.wait_type,
    DB_NAME(r.database_id)  AS Database_Name,
    LEFT(st.text, 100)      AS QueryText
FROM sys.dm_exec_requests         r
CROSS APPLY sys.dm_exec_sql_text(r.sql_handle) st
WHERE r.dop > 1
  AND r.session_id != @@SPID
ORDER BY r.dop DESC;

-- Section 6: MAXDOP and CTFP
PRINT '=== 6. PARALLELISM SETTINGS ==========';
SELECT name, value_in_use
FROM sys.configurations
WHERE name IN ('max degree of parallelism', 'cost threshold for parallelism')
ORDER BY name;

-- Section 7: NUMA alignment check
PRINT '=== 7. NUMA ALIGNMENT ================';
SELECT
    parent_node_id          AS NUMANode,
    COUNT(*)                AS Schedulers,
    SUM(runnable_tasks_count) AS RunnableTasks,
    AVG(load_factor)        AS AvgLoad
FROM sys.dm_os_schedulers
WHERE status = 'VISIBLE ONLINE'
GROUP BY parent_node_id
ORDER BY parent_node_id;

References


Discover more from SQLYARD

Subscribe to get the latest posts sent to your email.

Leave a Reply

Discover more from SQLYARD

Subscribe now to keep reading and get access to the full archive.

Continue reading