Azure DevOps Server 2019 release stage stuck "In Progress" with deployment job never starting

Leoh Venter 25 Reputation points
2026-06-24T06:44:31.2433333+00:00

Issue

I have a classic Azure DevOps release pipeline (Pipeline A) where Stage B is configured to deploy at 02:00 after an earlier stage (Stage A) has completed successfully.

On multiple occasions across multiple releases, Stage B transitions to “In Progress” / “1 pending” at the scheduled time, but the deployment job never actually starts on the target agent. The stage then remains stuck indefinitely until I manually intervene later in the morning. If I manually cancel/redeploy the same Stage B deployment in the same release at around 07:30, it runs successfully on the same target agent without any pipeline/task changes.

Symptoms

When the issue occurs:

  • Pipeline A shows Stage B as In Progress
  • Stage B shows 1 pending
  • The job view shows “The job has not yet started”
  • No task logs are produced for Stage B because the job never actually starts

Query

How can I determine the root cause of this behavior, and are there any known issues in Azure DevOps Server 2019 Release Management / Deployment Groups that could cause a scheduled stage deployment to remain pending indefinitely without ever being handed to the target agent?

Extra Notes / Observations

  • Stage B’s trigger fires correctly at 02:00. The issue is not that the trigger fails to fire; the stage enters an “in progress / pending” state as expected.
  • Stage B uses a Deployment Group job (not a standard Agent job).
  • Only one deployment target / agent is intended to run this stage:
    • Deployment target: Agent A
    • Targeting is done via deployment group tags configured on Stage B.
  • There are no other release pipelines using the same deployment group / target agent at the time the trigger fires.
  • Agent A remains online and healthy during the failure window.
  • Agent A’s _diag logs show only normal polling / keep-alive traffic from Azure DevOps Server during the failure window, e.g.:
[2026-06-11 23:59:46Z INFO MessageListener] Sent GetAgentMessage to keep alive agent [REDACTED], session [REDACTED]
[2026-06-12 00:00:10Z INFO MessageListener] Sleeping for 7.412 seconds before retrying.
  • During the failure window, there is no log entry indicating that Stage B’s job was assigned to Agent A.
  • If I manually cancel/redeploy the stuck Stage B deployment later in the morning, it succeeds immediately using the same deployment target / agent and same release definition.
  • Because the same stage can be manually redeployed successfully without any definition changes, this appears to be a problem with how the original scheduled deployment is being queued/handed off rather than with the tasks themselves.

Additional context

  • I created a stripped-down test release pipeline using the same general stage/deployment-group structure but without the actual deployment tasks, to try to reproduce the issue safely. So far I have not reproduced the issue in that stripped pipeline.

Environment

  • Azure DevOps Server 2019 Update 1.1 (17.153.29522.3)
  • Agent version: 3.255.0
  • Agent OS: Ubuntu Server

Questions

  1. Is there a known Azure DevOps Server 2019 issue where a scheduled Deployment Group stage can enter an In Progress / 1 pending state but never actually hand the job to the deployment target agent?
  2. Are there specific server-side Release Management / Deployment Group logs I should inspect to determine why the deployment was never assigned?
  3. Is there a way to tell from Azure DevOps Server whether the deployment is stuck:
    • before entering the deployment/agent queue,
    • waiting on a deployment-group lock,
    • or waiting on some internal Release Management queue state?

Images

Image 1 (Pipeline A pipeline view): pipeline-view

Image 2 (Stage B stage view): stage-view

Image 3 (Stage B job view): job-view

Azure DevOps
0 comments
Answer accepted by question author
Rakesh Mishra 11,340 Reputation points Microsoft External Staff Moderator
2026-06-24T11:59:52.4233333+00:00

Hi Leoh,

Thank you for reaching out and providing such detailed context. Based on the symptoms described where the scheduled Deployment Group stage enters the "In Progress / 1 pending" state but the job view says "The job has not yet started" and no agent logs are generated. The issue is occurring on the server side before the job is officially handed off to the agent queue. Because a manual redeployment succeeds, we can rule out agent health or task misconfiguration.

Here is how you can determine the root cause, based on official Microsoft troubleshooting guidelines:

1. Check Parallel Jobs and Concurrency Limits Even if no other releases are running in this specific pipeline, your Azure DevOps Server might be hitting a global concurrency limit at 02:00 due to other nightly builds or releases running across the collection. As stated in the official documentation for Troubleshoot pipeline failure to start:

"If you're currently running other pipelines, you might not have any remaining parallel jobs, or you might have hit your free limits... All available agents are in use. Jobs might wait if all your agents are currently busy."

To verify this, when the issue occurs at 02:00, navigate to Project settings > Parallel jobs and check the Agent pools tab to see if there are other jobs holding the concurrency slots.

2. Enable Verbose Logging and Inspect Server Logs Since the agent's _diag logs only show keep-alive polling, you must inspect the server-side routing.

  • Enable debug logging on the pipeline. According to the Review logs to diagnose pipeline issues documentation: "To configure verbose logs for all runs, you can add a variable named system.debug and set its value to true."
  • Inspect the Azure DevOps Server Application Tier logs. Look in the Windows Event Viewer (Application log) and the server's _diag folder for the TfsJobAgent service. If the scheduled job is failing to enqueue internally, the TfsJobAgent logs around 02:00 will show if it crashed, timed out, or failed to assign the job.

3. Use Supported REST APIs for Visibility Instead of querying the SQL database directly (which is unsupported), use the Azure DevOps Server 2019 REST APIs to check the queue status. Call the following endpoint around the failure window to see if the job is listed and inspect its queuePosition and status: GET https://{yourServer}/{collection}/_apis/distributedtask/pools/{poolId}/jobs?api-version=5.1

Workaround / Next Step: If the TfsJobAgent service is silently dropping the scheduled task due to a hung internal thread, restarting the Azure DevOps Server Background Job Agent (TfsJobAgent) service from services.msc on the server can clear any orphaned tasks and prevent the queue from hanging.

Please let me know in comments if it works are any other questions.

Note: This response is generated with the help of AI systems.

Was this answer helpful?

1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.