IPC:BgWorkerShutdown
This is part of Parallel query execution. There will be a main backend process and few worker processes.
The IPC:BgWorkerShutdown wait event occurs when a main backend process is waiting for a background worker process to cleanly shut down and terminate.
Common Problems
Hanging Sessions
When a main backend process is waiting for a background worker to shut down, it may lead to hanging sessions if the worker does not terminate as expected. This can occur due to various reasons such as resource contention, deadlocks, or bugs in the worker process.
Sometimes the main backend process may become uninterruptible while waiting for the worker to shut down, making it difficult to terminate the session using pg_terminate_backend().
Investigation
Check whether workers are visible
Mouseover the process id of the main backend process and check whether there are any worker processes associated with it.
If there are no worker processes visibile, There is something wrong with the worker process.
Use top
capture the top command output by repated execution
$ sudo top -b -c -n 24 -d 5 > top
grep the output for the main backend process id and check whether there are any worker processes associated with it.
$ grep "1726716" top
---- ouptput truncated ----
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
1726716 postgres 20 0 17.0g 92796 20936 S 0.0 0.1 0:42.59 postgres: dbuser1 db1 192.168.0.78(58946) SELECT
1757088 postgres 20 0 16.9g 5160 4516 S 0.0 0.0 0:00.00 postgres: parallel worker for PID 1726716
if the workers process is visible at OS level, but sleeping (Status 'S') and not consuming any CPU, Then there is something wrong with the worker process.
Check PostgreSQL logs
Look for errors like "out of memory" "terminating background worker" "parallel worker failed to initialize" "lost connection to parallel worker" and "No space left on device"
Following is an example
$ grep -E 'ERROR:|FATAL:|PANIC:' postgres.log-20251104.txt | sed -E 's/.*(ERROR:|FATAL:|PANIC:)/\1/' | sort | uniq -c | sort -nr
---- ouptput truncated ----
243 ERROR: out of memory
38 FATAL: terminating background worker "parallel worker" due to administrator command
15 ERROR: syntax error at or near ")" at character 3322
15 ERROR: parallel worker failed to initialize
4 FATAL: out of memory
2 FATAL: terminating connection due to idle-session timeout
2 ERROR: lost connection to parallel worker
2 ERROR: could not resize shared memory segment "/PostgreSQL.1098917120" to 4194304 bytes: No space left on device
1 ERROR: out of memory at character 25278
General suggessions
Essential Tuning
Make sure that the parameters, especially the parallelism related parameters are properly tuned according to your system resources and workload.
Avoid excessive parallelism which can lead to resource contention and instability.
Update PostgreSQL
Ensure that you are running the latest stable version of PostgreSQL, as updates often include bug fixes and performance improvements related to parallel query execution.
PostgreSQL minor versions will be updated in every 3 months. It is mandatory to keep your PostgreSQL minor version up to date for every compliance and security reason.
Carefull about extensions
Extensions are the most frequent cause of instability in PostgreSQL, especially those that interact with query execution.
Avoid using extensions that are not widely used or not well-maintained or have known issues with parallelism.