How to limit a number of PowerShell Jobs running simultaneously

Recently, I received a task which required me to run a particular command on a several thousands of servers. Since execution of this command takes some time, it is just logical to run them in a parallel mode. And PowerShell Background Jobs are the first thing comes in mind.
But resources of my PC are limited — it cannot run more than a 100 jobs simultaneously. Unfortunately, PowerShell doesn’t have a built-in functionality for limiting background jobs yet.

Though, I’m not the first one who stuck with the same problem: official “Hey, Scripting Guy!” blog has introduced us a queue based on .NET Framework objects. But I couldn’t manage this solution to work and needed something simpler. After all, all we need are:

  • a loop
  • a counter, for how much jobs are active and running
  • a variable, allowing next job in queue to start

Eventually, I came up with a piece of code like this:

Whereas my version is similar to a solution proposed at StackOverflow (which I only found AFTER completing my own version, of course), the SO version suffers from a bug where some items in queue may be skipped.

While PS Jobs are so easy to play with, Boe Prox claims that runspaces work much and much faster. Go, check it out, how you can use them to accelerate your job queue in his blog.

Some other queueing technics in PowerShell: