SQL Server Error 833 and Long Sync IO in Always On Availability Groups
Two messages in the SQL Server error log signal I/O problems that become significantly more complex in Always On Availability Group environments:
In a standalone SQL Server instance, these messages mean storage is slow. In an AG environment with synchronous commit mode, the consequences extend further: secondary replica I/O slowness directly increases primary commit latency, and primary I/O problems cascade to secondary redo queues. Understanding the AG-specific amplification is what this article covers.
Foundation article: For a complete explanation of Long Sync IO mechanics, nonpreemptive mode, and the standalone diagnostic workflow see the SQLYARD Long Sync IO guide. This article focuses specifically on the Always On context.
- Error 833: What It Means
- Long Sync IO: The Early Warning
- Why AG Synchronous Commit Makes I/O Slowness Worse
- Step 1: Find the Pattern in the Error Log
- Step 2: File-Level Latency
- Step 3: Wait Statistics
- Step 4: AG Queue Health
- Step 5: Autogrowth Events
- Step 6: Scheduler Pressure
- Step 7: Power Plan Check
1 Error 833: What It Means Beginner
Error 833 is logged when a disk read or write operation takes more than 15 seconds to complete. This is not a slow query. It is a storage stall at the I/O subsystem level. SQL Server’s scheduler monitor detects the pending I/O, waits, and logs the message when the 15-second threshold is crossed. The server continues operating but the affected operations are severely delayed.
| File Type | Healthy Latency | Concerning Latency | Error 833 Territory |
|---|---|---|---|
| Data files (.mdf/.ndf) | Under 20 ms | 20 to 50 ms | Over 50 ms sustained |
| Log files (.ldf) | Under 5 ms | 5 to 15 ms | Over 15 ms sustained |
| TempDB files | Under 10 ms | 10 to 30 ms | Over 30 ms sustained |
Error 833 is a serious signal, not background noise. A single occurrence during a one-time maintenance event may be acceptable. Repeated occurrences during normal production workload mean the storage subsystem cannot keep pace with the I/O demand. Ignoring it leads to transaction timeouts, blocking escalation, and in AG environments, synchronization failures.
2 Long Sync IO: The Early Warning Beginner
The Long Sync IO message fires at one second, well before the 833 threshold of 15 seconds. It is SQL Server’s early warning that synchronous I/O is stalling the scheduler. A worker thread entered nonpreemptive mode to perform a file operation, and that operation has not returned from the operating system within 1,000 ms. The scheduler cannot dispatch other work until the call returns.
Repeated Long Sync IO messages during normal workload indicate the system is approaching the conditions that produce Error 833. Treat them as a pre-833 signal that requires investigation, not as an informational message to acknowledge and move on from.
3 Why AG Synchronous Commit Makes I/O Slowness Worse Intermediate
In synchronous commit mode, the primary replica cannot acknowledge a transaction commit to the application until the secondary replica has hardened the log records to its own transaction log. The commit chain is:
- Primary writes the log block to its transaction log
- Primary sends the log block to the secondary over the network
- Secondary hardens the log block to its own transaction log and sends an acknowledgment
- Primary receives the acknowledgment and notifies the application that the commit succeeded
- Separately, the secondary’s redo thread applies the changes to its data files
This chain has two distinct I/O paths where problems surface differently:
| I/O Problem Location | What Slows Down | Wait Types on Primary | What to Check |
|---|---|---|---|
| Secondary log file I/O slow | Primary commit latency increases. Every transaction waits longer for the secondary ACK. | HADR_SYNC_COMMITWRITELOG |
Log file latency on the secondary replica |
| Secondary data file I/O slow | Redo queue grows on secondary. Data files lag behind log. Secondary becomes increasingly stale. | PAGEIOLATCH_* on secondary |
Data file latency on the secondary replica |
| Primary log file I/O slow | Primary commit latency increases directly. Log send queue may grow if log send is also affected. | WRITELOG |
Log file latency on the primary replica |
| Primary data file I/O slow | Query performance degrades. Redo on secondary may lag if workload drives heavy data reads on primary. | PAGEIOLATCH_* on primary |
Data file latency on the primary replica |
The secondary’s log I/O speed directly limits primary commit throughput in synchronous mode. A secondary replica with slow log file storage forces the primary to wait on every single commit. This is a frequently missed cause of primary performance degradation that has nothing to do with the primary’s own hardware.
4 Step 1: Find the Pattern in the Error Log Beginner
Start by pulling both message types from the error log and noting the timestamps. Matching them to AG queue snapshots and storage activity windows reveals the correlation.
-- Check current error log for 833 and Long Sync IO messages
EXEC xp_readerrorlog 0, 1, N'833';
EXEC xp_readerrorlog 0, 1, N'Long Sync IO';
-- Check the previous error log (after a service restart)
EXEC xp_readerrorlog 1, 1, N'833';
EXEC xp_readerrorlog 1, 1, N'Long Sync IO';
-- Note the timestamps and whether they cluster during peak hours
-- Then use the same time window for all subsequent diagnostic steps
5 Step 2: File-Level Latency Intermediate
Run this on both the primary and secondary replicas. The secondary’s log write latency is especially important in synchronous commit mode.
-- Run on PRIMARY and SECONDARY replicas separately
SELECT
DB_NAME(vfs.database_id) AS DatabaseName,
mf.type_desc AS FileType,
mf.physical_name,
vfs.num_of_reads,
vfs.io_stall_read_ms / NULLIF(vfs.num_of_reads, 0) AS AvgReadMs,
vfs.num_of_writes,
vfs.io_stall_write_ms / NULLIF(vfs.num_of_writes, 0) AS AvgWriteMs
FROM sys.dm_io_virtual_file_stats(NULL, NULL) vfs
JOIN sys.master_files mf
ON mf.database_id = vfs.database_id
AND mf.file_id = vfs.file_id
ORDER BY (vfs.io_stall_read_ms + vfs.io_stall_write_ms) DESC;
-- On the secondary, focus especially on LOG file AvgWriteMs
-- Values consistently above 5ms indicate the secondary is struggling to harden log records
-- which directly increases primary commit latency in synchronous mode
6 Step 3: Wait Statistics Intermediate
-- Top wait types: run on the primary to understand what the primary is waiting on
SELECT TOP 15
wait_type,
waiting_tasks_count,
wait_time_ms,
wait_time_ms / NULLIF(waiting_tasks_count, 0) AS AvgWaitMs
FROM sys.dm_os_wait_stats
WHERE wait_type NOT LIKE 'SLEEP%'
AND wait_type NOT LIKE 'BROKER_%'
AND waiting_tasks_count > 0
ORDER BY wait_time_ms DESC;
| Wait Type | What It Means in an AG Context |
|---|---|
PAGEIOLATCH_* | Slow data file reads on the replica running this query. On the secondary, indicates redo thread I/O pressure. |
WRITELOG | Slow log writes. On the primary, the primary’s own log I/O is slow. May also reflect secondary ACK wait indirectly. |
HADR_SYNC_COMMIT | Primary is waiting for secondary to acknowledge log hardening. Points to secondary log I/O or network latency. |
HADR_DATABASE_FLOW_CONTROL | Primary is throttling log send rate because secondary cannot keep up with redo. Secondary is falling behind. |
7 Step 4: AG Queue Health Intermediate
-- AG queue sizes and throughput rates for all replicas
SELECT
ar.replica_server_name,
drs.database_id,
DB_NAME(drs.database_id) AS DatabaseName,
drs.is_primary_replica,
drs.synchronization_state_desc,
drs.log_send_queue_size AS LogSendQueueKB,
drs.redo_queue_size AS RedoQueueKB,
drs.log_send_rate AS LogSendRateKBps,
drs.redo_rate AS RedoRateKBps
FROM sys.dm_hadr_database_replica_states drs
JOIN sys.availability_replicas ar
ON drs.replica_id = ar.replica_id
ORDER BY DatabaseName, drs.is_primary_replica DESC;
| Column | What to Look For |
|---|---|
LogSendQueueKB growing | Log is being generated faster than it is being sent. Check primary log I/O and network bandwidth. |
RedoQueueKB growing | Log is arriving faster than the secondary can apply it to data files. Check secondary data file I/O. |
LogSendRateKBps low | Network bandwidth or primary disk throughput may be limiting log send speed. |
RedoRateKBps low | Secondary redo thread is constrained. May be I/O bound on data files or CPU constrained. |
8 Step 5: Autogrowth Events Intermediate
Autogrowth events cause I/O stalls during file initialization. In an AG environment they affect both replicas because the secondary must also accommodate the same file sizes. Frequent autogrowth is a common hidden cause of intermittent 833 and Long Sync IO messages.
-- Find recent autogrowth events from the default trace
DECLARE @tracepath NVARCHAR(260);
SELECT @tracepath = path FROM sys.traces WHERE is_default = 1;
SELECT
te.name AS EventName,
t.StartTime,
t.DatabaseName,
t.FileName,
t.IntegerData / 128.0 AS GrowthMB,
t.Duration / 1000 AS DurationMs
FROM sys.fn_trace_gettable(@tracepath, DEFAULT) t
JOIN sys.trace_events te ON t.EventClass = te.trace_event_id
WHERE t.EventClass IN (92, 93) -- 92 = Data File Auto Grow, 93 = Log File Auto Grow
ORDER BY t.StartTime DESC;
9 Step 6: Scheduler Pressure Intermediate
-- Check scheduler load: high runnable_tasks_count indicates workers piling up behind slow I/O
SELECT
scheduler_id,
current_tasks_count,
runnable_tasks_count,
work_queue_count,
pending_disk_io_count
FROM sys.dm_os_schedulers
WHERE scheduler_id < 255
ORDER BY runnable_tasks_count DESC;
-- High runnable_tasks_count on one scheduler during the problem window
-- correlates with a Long Sync IO stall on that specific scheduler
10 Step 7: Power Plan Check Beginner
The Windows power plan affects disk I/O responsiveness. The Balanced power plan throttles CPU frequency under low-utilization conditions, which increases I/O latency. SQL Server and its storage should always run on the High Performance plan.
-- Check the active Windows power plan
EXEC xp_cmdshell 'powercfg /GETACTIVESCHEME';
-- Expected output contains: High performance
-- If it shows Balanced or Power Saver, escalate to the infrastructure team immediately
-- This is a common cause of Long Sync IO on VMs and physical servers alike
On VMware and Hyper-V, the power plan must be set at the guest OS level. The hypervisor power policy also matters. VMware recommends setting the host power policy to High Performance in vSphere for SQL Server VMs. A VM set to High Performance at the OS level but running on a host with Balanced power policy still experiences CPU frequency throttling that contributes to I/O latency.
11 Quick Wins: Apply Immediately Beginner
These can be applied without a maintenance window and provide immediate relief while the lasting fixes are planned.
Switch to asynchronous commit temporarily
If the secondary's slow log I/O is blocking primary commits and users are experiencing timeouts, switching to asynchronous commit removes the secondary's hardening requirement from the primary commit path. This eliminates the HADR_SYNC_COMMIT wait immediately. It also means the secondary is no longer a zero-data-loss target until synchronous commit is restored.
-- Switch secondary to asynchronous commit (removes secondary ACK from primary commit path)
-- Use during active incident to restore primary throughput immediately
ALTER AVAILABILITY GROUP [YourAGName]
MODIFY REPLICA ON N'YourSecondaryServerName'
WITH (AVAILABILITY_MODE = ASYNCHRONOUS_COMMIT);
-- After storage is fixed, restore synchronous commit
ALTER AVAILABILITY GROUP [YourAGName]
MODIFY REPLICA ON N'YourSecondaryServerName'
WITH (AVAILABILITY_MODE = SYNCHRONOUS_COMMIT);
Asynchronous commit means the secondary is no longer a zero-data-loss failover target. If a failover occurs while in asynchronous mode, committed transactions on the primary that have not yet been hardened on the secondary may be lost. Coordinate with the business before making this change and document that RPO is temporarily relaxed until synchronous commit is restored.
Additional quick wins
- Enable Instant File Initialization for SQL Server data files to eliminate initialization stalls during autogrowth
- Pre-size data and log files to eliminate autogrowth events during production hours
- Set autogrowth in fixed MB increments rather than percentages to make growth events predictable in duration
- Exclude SQL Server files from antivirus scanning: all .mdf, .ndf, .ldf, .bak files and the TempDB path on all replicas
- Separate data and log files onto different disks if they currently share a volume
- Set NTFS allocation size to 64 KB on SQL Server volumes for better I/O alignment
- Verify High Performance power plan on all replicas including the secondary
12 Lasting Fixes: Storage and Infrastructure Intermediate
- Upgrade the storage tier. If file-level latency from Step 2 consistently exceeds the thresholds in Section 1, the storage cannot meet the workload's I/O demand. Increase IOPS and throughput allocation on the SAN or switch to NVMe for log files.
- Update firmware and drivers. HBA, RAID controller, and multipath driver firmware is a surprisingly common source of I/O stalls. Confirm all components on all replicas are on current stable firmware.
- Enable write caching with battery backup. A write-back cache on the storage controller dramatically reduces log write latency. Confirm the cache is battery-backed or flash-backed before enabling write-back to avoid data loss on power failure.
- Move redo and data files to faster storage on the secondary. If the secondary's redo queue is growing (Section 7), the secondary's data file storage cannot keep pace. Moving those files to faster storage is the structural fix.
- Increase RAM. Higher buffer pool memory reduces physical data file reads by keeping more pages in memory, reducing
PAGEIOLATCH_*waits. - Index tuning. Reducing full table scans through proper index coverage reduces physical I/O demand, which reduces the frequency of both PAGEIOLATCH and Long Sync IO events.
13 Optional: Tuning Parallel Redo on the Secondary Advanced
Since SQL Server 2016, AG secondaries use parallel redo threads to apply log records to data files faster. In most environments parallel redo improves redo throughput. In some specific scenarios, particularly where redo thread coordination overhead exceeds the parallelism benefit, disabling it may help.
-- Check if parallel redo waits are dominating on the secondary
SELECT TOP 10
wait_type,
waiting_tasks_count,
wait_time_ms
FROM sys.dm_os_wait_stats
WHERE wait_type LIKE 'PARALLEL_REDO%'
ORDER BY wait_time_ms DESC;
-- If PARALLEL_REDO waits dominate and redo throughput is poor,
-- test disabling parallel redo with Trace Flag 3459 (secondary only)
-- This is a test, not a permanent recommendation without measurement
DBCC TRACEON(3459, -1);
DBCC TRACESTATUS(3459);
-- Measure redo_rate in sys.dm_hadr_database_replica_states before and after
-- If redo_rate improves: consider keeping TF 3459 active
-- If redo_rate is unchanged or worse: re-enable parallel redo
DBCC TRACEOFF(3459, -1);
DBCC TRACESTATUS(3459);
Test parallel redo changes during a controlled window. Some older SQL Server builds require a service restart to re-enable parallel redo after disabling it with TF 3459. Patch to the current cumulative update before testing this change. Measure redo_rate from sys.dm_hadr_database_replica_states for at least 30 minutes before and after to confirm the effect.
14 Extended Events Session for Long Sync IO Advanced
For targeted capture of I/O events during the problem window without the overhead of a full trace.
-- Create an Extended Events session to capture Long Sync IO and slow file I/O
IF EXISTS (SELECT * FROM sys.server_event_sessions WHERE name = N'Track_LongSyncIO')
DROP EVENT SESSION [Track_LongSyncIO] ON SERVER;
GO
CREATE EVENT SESSION [Track_LongSyncIO] ON SERVER
ADD EVENT sqlserver.database_file_size_change,
ADD EVENT sqlserver.file_read_completed (
ACTION (sqlserver.database_name)
WHERE duration > 1000 -- reads taking more than 1 second
),
ADD EVENT sqlserver.file_write_completed (
ACTION (sqlserver.database_name)
WHERE duration > 1000 -- writes taking more than 1 second
),
ADD EVENT sqlserver.error_reported (
ACTION (sqlserver.database_name)
WHERE message LIKE N'%Long Sync IO%'
)
ADD TARGET package0.event_file (
SET filename = N'C:\XE\longsyncio.xel',
max_file_size = 100,
max_rollover_files = 5
);
GO
-- Start the session
ALTER EVENT SESSION [Track_LongSyncIO] ON SERVER STATE = START;
-- Stop when done
-- ALTER EVENT SESSION [Track_LongSyncIO] ON SERVER STATE = STOP;
-- DROP EVENT SESSION [Track_LongSyncIO] ON SERVER;
Correlating XEvent output with AG queue snapshots: Run the Extended Events session during the problem window while also capturing sys.dm_hadr_database_replica_states every 30 seconds via a SQL Agent job. The combination shows exactly which file I/O events correspond to spikes in log_send_queue_size or redo_queue_size, making it possible to prove whether the I/O problem is on the primary or secondary.
References
- Microsoft Docs: MSSQLSERVER_833 Database Engine Error
- Microsoft Docs: sys.dm_hadr_database_replica_states
- Microsoft Docs: sys.dm_io_virtual_file_stats
- Microsoft Docs: Availability Modes in Always On Availability Groups
- Microsoft SQL Server Team: How It Works: Sync IOs in Nonpreemptive Mode
- SQLYARD: Understanding Long Sync IO in SQL Server
- SQLYARD: Always On Availability Groups Complete Guide
- SQLYARD: When SQL Server Is Slow: Triage Playbook
Discover more from SQLYARD
Subscribe to get the latest posts sent to your email.


