Unofficial SQS DLQ Redrive API Documentation

Update 2023-06-08: These APIs are officially available and documented now. See the announcement blog post: https://aws.amazon.com/blogs/aws/a-new-set-of-apis-for-amazon-sqs-dead-letter-queue-redrive/

In December 2021, AWS announced dead-letter queue redrive to source queues for SQS. This feature meant that SQS can now manage moving messages from a dead-letter queue to another queue (usually the source queue to which they were originally sent).

The announcement says that “you redrive the messages using the Amazon SQS console” and goes on to give a good explanation of the ClickOps involved. At a glance, there seem to be no API actions for DLQ redrive, but there are: they just aren’t included in the SDKs are are undocumented (until now).

This post aims to serve as the missing documentation for the DLQ redrive actions until such time as AWS provides official documentation.

🫵 If you have any suggestion on how to improve this documentation (typo fixes, corrections, rewording) then please let me know!

Table of Contents

• Assumptions
• How to call these API actions
• 📝 Action: CreateMoveTask
  - Request Syntax
  - Request Parameters
  - Response Syntax
  - Response Elements
  - Errors
  - Error Examples
• 📝 Action: ListMoveTasks
  - Request Syntax
  - Request Parameters
  - Response Syntax
  - Response Elements
• 📝 Action: CancelMoveTask
  - Request Syntax
  - Request Parameters
  - Response Syntax
  - Response Elements
  - Errors
  - Error Examples
• ⚙️ Data Type: MoveTask
  - Syntax
  - Contents
• Conclusion

This is what DALL·E thinks dead-letter queues look like.

Assumptions

I’m assuming that you have a good understanding of SQS, dead-letter queues, and even the console UI version of the DQL redrive feature. If that’s not the case, I suggest reading the DLQ redrive announcement blog post first (perhaps other documentation too).

How to call these API actions

You’ll need to sign your requests to SQS using SigV4. Thankfull, there are plenty of open source libraries (such as this) that will do the signing for you. There’s also a CLI tool which you might find useful to try these out.

In case it isn’t obvious, I’ve left out the SQS endpoint from the URLs below. These can be found here and will be something like https://sqs.us-east-1.amazonaws.com/.

📝 Action: CreateMoveTask

Creates a new move task to redrive messages from a dead-letter queue.

Notes

• Set the Accept header to application/json otherwise you will get XML.

Request Syntax

POST /?Action=CreateMoveTask&Version=2012–11–05&SourceArn=SourceArn&TaskName=TaskName

Request Parameters

These go in the query string.

• SourceArn
  The ARN of the dead-letter queue to redrive messages from.
  Type: String
  Required: Yes
  Example: arn:aws:sqs:us-east-1:888888888888:example-dlq
• DestinationArn
  The ARN of the queue to which SQS should send messages. If omitted, SQS will redrive messages to their respective source queues.
  Type: String
  Required: No
• TaskName
  Your name for the move task. The SQS console uses a version 4 UUID, but it can be anything. This name is used later to cancel a running task.
  Type: String
  Valid Length: 1 to ??
  Required: Yes
• MaxNumberOfMessagesPerSecond
  The maximum number of messages you want SQS to redrive per second. If omitted, SQS will use attempt to “optimize” the number.
  Type: Integer
  Required: No

Response Syntax

{
    "CreateMoveTaskResponse": {
        "CreateMoveTaskResult": {
            "ApproximateNumberOfMessagesMoved": number,
            "CreateTime": number,
            "DestinationArn": "string",
            "FailureReason": "string",
            "MaxNumberOfMessagesPerSecond": number,
            "MaxNumberOfMessagesToMove": number,
            "SourceArn": "string",
            "Status": "string",
            "TaskHandle": "string",
            "TaskName": "string",
            "UnprocessedArn": "string"
        },
        "ResponseMetadata": {
          "RequestId": "string"
        }
    }
}

Response Elements

• CreateMoveTaskResult
The new move tasks current state.
  Status should always be RUNNING.
  Type: MoveTask object

Errors

• InvalidParameterValue
  Status Code: 400
  One of the parameters in the request is not valid.

Error Examples

The following is returned when either a task with the same name exists or there is already another running task:

{
    "Error": {
        "Code": "InvalidParameterValue",
        "Message": "There is already a task running. Only one active task is allowed for a source queue arn at a given time.",
        "Type": "Sender"
    },
    "RequestId": "a41f24c7-1de8-5e1e-bc38-7877a3cc4a9b"
}

📝 Action: ListMoveTasks

Returns a list of move tasks for a dead-letter queue.

Notes

• Set the Accept header to application/json otherwise you will get XML.

Request Syntax

POST /?Action=ListMoveTasks&Version=2012–11–05&SourceArn=SourceArn

Request Parameters

These go in the query string.

• SourceArn
  The ARN of a dead-letter queue.
  Type: String
  Required: Yes
  Example: arn:aws:sqs:us-east-1:888888888888:example-dlq
• MaxResults
  Maximum number of results to include in the response.
  Type: Integer
  Valid Value Range: 1 to 10
  Required: No
• NextToken
  Pagination token to request the next set of results.
  Type: String
  Required: No

Response Syntax

{
    "ListMoveTasksResponse": {
        "ListMoveTasksResult": {
            "NextToken": "string",
            "Results": [
                {
                    "ApproximateNumberOfMessagesMoved": number,
                    "CreateTime": number,
                    "DestinationArn": "string",
                    "FailureReason": "string",
                    "MaxNumberOfMessagesPerSecond": number,
                    "MaxNumberOfMessagesToMove": number,
                    "SourceArn": "string",
                    "Status": "string",
                    "TaskHandle": "string",
                    "TaskName": "string",
                    "UnprocessedArn": "string"
                }
            ]
        },
        "ResponseMetadata": {
          "RequestId": "string"
        }
    }
}

Response Elements

• NextToken
  Pagination token to include in the next request. Token value is null if there are no additional results to request, or if you did not set MaxResults in the request.
  Type: String
  Required: No
• Results.N
A list of move tasks.
  Type: Array of MoveTask objects.

📝 Action: CancelMoveTask

Cancels a running move task.

Will return an error if the task status is not RUNNING, CANCELLING, or CANCELLED.

Notes

• Set the Accept header to application/json otherwise you will get XML.

Request Syntax

POST /?Action=CancelMoveTask&Version=2012–11–05&SourceArn=SourceArn&TaskName=TaskName

Request Parameters

These go in the query string.

• SourceArn
  The ARN of the task’s source dead-letter queue.
  Type: String
  Required: Yes
  Example: arn:aws:sqs:us-east-1:888888888888:example-dlq
• TaskName
  The name of the move task to cancel.
  Type: String
  Required: Yes
  Example: debfe408-b32c-44bc-b52b-52c93cb30d27

Response Syntax

{
    "CancelMoveTaskResponse": {
        "CancelMoveTaskResult": {
            "ApproximateNumberOfMessagesMoved": number,
            "CreateTime": number,
            "DestinationArn": "string",
            "FailureReason": "string",
            "MaxNumberOfMessagesPerSecond": number,
            "MaxNumberOfMessagesToMove": number,
            "SourceArn": "string",
            "Status": "string",
            "TaskHandle": "string",
            "TaskName": "string",
            "UnprocessedArn": "string"
        },
        "ResponseMetadata": {
          "RequestId": "string"
        }
    }
}

Response Elements

• CancelMoveTaskResult
The cancelled move tasks current state.
  Status should always be CANCELLING or CANCELLED.
  Type: MoveTask object

Errors

• InvalidParameterValue
  Status Code: 400
  One of the parameters in the request is not valid.

Error Examples

{
    "Error": {
        "Code": "InvalidParameterValue",
        "Message": "Only active tasks can be cancelled.",
        "Type": "Sender"
    },
    "RequestId": "91109d4b-655c-5b36-8e46-c1fbd2d1229e"
}

⚙️ Data Type: MoveTask

Details about a move task.

Syntax

{
    "ApproximateNumberOfMessagesMoved": number,
    "CreateTime": number,
    "DestinationArn": "string",
    "FailureReason": "string",
    "MaxNumberOfMessagesPerSecond": number,
    "MaxNumberOfMessagesToMove": number,
    "SourceArn": "string",
    "Status": "string",
    "TaskHandle": "string",
    "TaskName": "string",
    "UnprocessedArn": "string"
}

Contents

• ApproximateNumberOfMessagesMoved
The approximate number of messages that have been moved by the task.
  Type: Integer
  Required: Yes
• CreateTime
Unix/epoch timestamp of when the task was created. Includes milliseconds.
  Type: Integer
  Required: Yes
  Example: 1681303603034
• DestinationArn
The ARN of the target queue the task sends messages to. Will be null when messages are being sent back to their source queue.
  Type: String
  Required: No
• FailureReason
The reason the task failed if Status is FAILED.
  Type: String
  Required: No
  Example: CouldNotDetermineMessageSource
• MaxNumberOfMessagesPerSecond
The rate at which SQS should redrive messages. Will be null when messages are being sent using the “system optimized” setting.
  Type: Integer
  Valid Value Range: 1 to 500
  Required: No
• MaxNumberOfMessagesToMove
Seems to be set by SQS based on the “approximate number of messages available” when the task is created and used in the UI to calculate “percent processed”.
  Type: Integer
  Required: Yes
• SourceArn
The ARN of the dead-letter queue the task is for. This should always be the same as your request SourceArn.
  Type: String
  Required: Yes
• Status
Status of the task.
  Type: String
  Required: Yes
  Value Values: COMPLETED, RUNNING, FAILED, CANCELLING, CANCELLED.
• TaskHandle
Uniquley identifies the move task. Is derived from the TaskName and SourceArn (see Value Structure below).
  Type: String
  Required: Yes
  Value Structure: Base64-encoded JSON object containing taskId (same as TaskName) and sourceArn (same as SourceArn). Example:eyJ0YXNrSWQiOiI4M2RkMzY3YS02MDZmLTRiODctOWM4OS03MmMxMDY3YTMzMWUiLCJzb3VyY2VBcm4iOiJhcm46YXdzOnNxczp1cy1lYXN0LTE6ODg4ODg4ODg4ODg4OmV4YW1wbGUtZGxxIn0=
• TaskName
The name of the task given when it was created. The SQS console uses a version 4 UUID.
  Type: String
  Required: Yes
  Example: 83dd367a-606f-4b87–9c89–72c1067a331e
• UnprocessedArn
I’m not sure what this is used for or what the ARN looks like because it seems to be null regardless of the task’s Status. I assume it may be set in certain error scenarios.
  Type: String
  Required: No

Conclusion

Should you use these undocumented actions?

These API actions are part of the SQS API and the console depends on them (you’re indirectly already using them), but as far as I know they aren’t officially supported.

Having said that, I see no reason not to use them. If you build something based on these APIs and they break, you could always go back to using the SQS console. Just ensure you have that fallback process in place and, of course, don’t use them for anything mission critical.

For more like this, please follow me on Medium and Twitter.