Now Available: SQS FIFO Queue Support and Enhanced Message Control

Today we're announcing support for Amazon SQS FIFO and Fair queues in Proxylity UDP Gateway, along with enhanced message formatting options for all queue types. These new capabilities enable ordered packet processing, fair message distribution, exactly-once delivery semantics, and greater control over how UDP packet data is formatted in queue messages.

FIFO and Fair Queue Support

SQS FIFO queues preserve the order of messages and provide exactly-once processing, making them ideal for scenarios where packet ordering matters. AWS Fair queues provide fair processing distribution across message groups with improved throughput. UDP Gateway now provides specialized configuration options for working with both queue types, including dynamic message group IDs and deduplication identifiers extracted directly from packet payloads.

Dynamic Message Group IDs

Both FIFO and Fair queues require a message group ID. For FIFO queues, the message group ID provides ordering guarantees within each group. For Fair queues, it enables balanced processing distribution across groups. UDP Gateway allows you to extract message group IDs dynamically from packet data, enabling ordering within logical groups (FIFO) or fair processing distribution (Fair) while maintaining parallel processing across different groups:

Configuration example (JSON):

{
  "DestinationArn": "arn:aws:sqs:us-east-1:123456789012:device-data.fifo",
  "Arguments": {
    "MessageGroupIdExpression": "[0:4]",
    "MessageGroupIdFormatter": "hex"
  }
}

This configuration extracts the first 4 bytes from each packet payload and uses them as the message group ID, formatted as hexadecimal. For FIFO queues, packets with the same identifier maintain strict ordering, while packets with different identifiers can be processed in parallel. For Fair queues, this ensures balanced processing across different message groups.

Dynamic Deduplication IDs

FIFO queues support content-based deduplication to prevent duplicate message processing. UDP Gateway allows you to dynamically extract deduplication IDs from packet payloads, ideal when your UDP protocol includes unique transaction or sequence identifiers:

Configuration example (JSON):

{
  "DestinationArn": "arn:aws:sqs:us-east-1:123456789012:transactions.fifo",
  "Arguments": {
    "MessageGroupIdExpression": "[0:4]",
    "MessageGroupIdFormatter": "hex",
    "MessageDeduplicationIdExpression": "[4:8]",
    "MessageDeduplicationIdFormatter": "hex"
  }
}

This extracts bytes 0-4 as the message group ID and bytes 4-8 as the deduplication ID. SQS will automatically reject duplicate messages within the deduplication interval (5 minutes), ensuring exactly-once processing.

Enhanced Message Formatting

In addition to FIFO queue support, we've added new message formatting options for all SQS destinations:

Raw Message Format

The new SendRaw option allows you to send raw binary UDP payloads as message bodies without JSON wrapping. This is ideal when downstream systems expect native protocol formats or when minimizing message size is important:

{
  "DestinationArn": "arn:aws:sqs:us-east-1:123456789012:raw-packets",
  "Arguments": {
    "SendRaw": true
  }
}

Message Attributes

The UseMessageAttributes option moves packet metadata (source IP/port, listener domain/port, timestamp, etc.) from the JSON message body to SQS message attributes. This simplifies message processing when you need metadata for routing or filtering but want a cleaner message body:

{
  "DestinationArn": "arn:aws:sqs:us-east-1:123456789012:device-telemetry",
  "Arguments": {
    "UseMessageAttributes": true
  }
}

Use Cases

These new capabilities enable several powerful use cases:

• Ordered Transaction Processing - Use FIFO queues with dynamic message group IDs to maintain ordering per device or session while enabling parallel processing across different devices
• Exactly-Once Financial Transactions - Extract transaction IDs from packets for deduplication, ensuring each payment or order is processed exactly once
• Protocol-Native Processing - Send raw binary payloads to maintain protocol integrity for downstream systems that expect native formats
• Efficient Metadata Filtering - Use message attributes for SQS message filtering and routing without parsing JSON message bodies

Getting Started

All SQS destination configurations support these new features immediately. Update your listener configurations to add the new Arguments properties, and UDP Gateway will automatically use the enhanced functionality.

For FIFO queues, ensure your queue name ends with .fifo and configure appropriate message group ID and deduplication settings based on your protocol requirements. Fair queues also use the .fifo suffix and support message group IDs for fair processing distribution. For Standard queues, experiment with SendRaw and UseMessageAttributes to optimize for your specific use case.

Complete documentation is available in the SQS Destinations guide, including detailed examples, formatter options, and best practices for Standard, FIFO, and Fair queue configurations.

A ready-to-deploy example demonstrating both Standard and FIFO queues with various configuration options is available in the SQS example in our GitHub examples repository.