Now Available: IoT Core Data Destinations

The new destination option for AWS IoT Core enables UDP Gateway to publish messages to AWS IoT Core's data plane, allowing integration with IoT device ecosystems, IoT Core Rules, and Thing Shadows. This integration is ideal for bridging UDP-based device protocols into AWS IoT Core's MQTT infrastructure, where powerful Rules Engine capabilities can route, transform, and process device data.

Operating Modes

The IoT Data adapter supports two distinct operating modes that determine how UDP packets are published to MQTT topics:

Static Mode (Default)

In static mode, all UDP packets are published to a single MQTT topic with metadata wrapped in a JSON packet envelope. The envelope includes source IP/port, timestamp, batch tag, and the payload encoded according to the destination's formatter setting.

When to use static mode:

• Centralized processing of all device traffic
• Analytics and monitoring scenarios requiring full metadata
• When you need context about the source (IP address, port, timestamp)
• Debugging and protocol development

Configuration: No special configuration required. Static mode is the default when DynamicTopicSuffix is not set or is false.

Dynamic Topic Suffix Mode

In dynamic topic suffix mode, each UDP packet is published to a unique MQTT topic based on extracted payload data or source network information. The raw binary payload is sent without JSON wrapping, preserving the original device protocol format.

When to use dynamic mode:

• Per-device processing pipelines (device ID extracted from payload)
• IoT Core Rules that route by topic pattern
• When downstream systems expect raw device protocols
• Thing Shadow updates or device registry integrations

Configuration options:

• DynamicTopicSuffix (boolean, default: false) - Set to true to enable dynamic topic suffix mode
• TopicSuffixExpression (string, optional) - A binary range expression that extracts bytes from the UDP payload (e.g., "[0:4]" extracts the first 4 bytes). If not specified, the topic suffix defaults to {source_ip}/{source_port}
• TopicSuffixFormatter (string, default: "hex") - Controls how extracted bytes are converted to a string for the topic. Options: "hex", "ascii", "utf8", "base64"

Topic construction:

{base_topic_from_arn}/{dynamic_suffix}

Where the suffix is either:

• The formatted result of TopicSuffixExpression evaluated against the payload, or
• {source_ip}/{source_port} if no expression is provided

Example configuration (JSON):

{
  "DestinationArn": "arn:aws:iot:us-east-1:123456789012:topic/devices",
  "DynamicTopicSuffix": true,
  "TopicSuffixExpression": "[0:4]",
  "TopicSuffixFormatter": "hex"
}

This configuration would publish packets to topics like devices/A1B2C3D4, where A1B2C3D4 is the hex representation of the first 4 bytes of each UDP payload.

Integration Patterns

The following patterns demonstrate common use cases for IoT Data destinations, leveraging the strengths of both UDP Gateway and AWS IoT Core.

Pattern 1: Per-Device Topics

Route messages to device-specific topics based on a device identifier extracted from the payload. This enables per-device processing pipelines and IoT Core Rules that operate on individual devices.

Configuration (YAML):

DestinationArn: arn:aws:iot:us-east-1:123456789012:topic/devices
DynamicTopicSuffix: true
TopicSuffixExpression: "[0:4]"  # 4-byte device ID from payload start
TopicSuffixFormatter: hex

Behavior: A UDP packet with bytes [0xA1, 0xB2, 0xC3, 0xD4, ...] is published to devices/A1B2C3D4 with the raw binary payload preserved.

Use cases:

• Device-specific processing pipelines
• Per-device metrics and monitoring
• Device registry lookup and validation via IoT Core Rules

IoT Core Rule example:

SELECT * FROM 'devices/+'
WHERE get_thing_shadow(topic(2)).thingName IS NOT NULL

Pattern 2: Source-Based Routing

Route by source IP address and port when devices don't include identifiers in the payload. This is useful for legacy devices or network-based device tracking.

Configuration (YAML):

DestinationArn: arn:aws:iot:us-east-1:123456789012:topic/sources
DynamicTopicSuffix: true
# No TopicSuffixExpression - defaults to {source_ip}/{source_port}

Behavior: A UDP packet from 192.168.1.100:51234 is published to sources/192.168.1.100/51234.

Use cases:

• Network-based device tracking
• IP allowlist validation
• Troubleshooting connectivity issues
• Legacy devices without protocol-level identification

Pattern 3: Centralized Collection

Aggregate all UDP traffic to a single topic for analytics, monitoring, or debugging. This uses static mode to preserve full metadata about each packet.

Configuration (YAML):

DestinationArn: arn:aws:iot:us-east-1:123456789012:topic/udp/inbound
# Default static mode - no dynamic suffix configuration

Behavior: All packets are published to udp/inbound with a JSON metadata envelope that includes source IP/port, timestamp, batch tag, and base64-encoded payload.

Use cases:

• Centralized logging and auditing
• Traffic analysis and anomaly detection
• Protocol debugging and development
• Analytics requiring full context

Topic Structure Constraints

The dynamic suffix is appended directly to the base topic from the ARN. This focused approach keeps UDP Gateway simple while leveraging IoT Core's powerful Rules Engine for complex routing scenarios.

What's Supported

Single dynamic segment appended to base topic:

• ✅ devices/{extracted_id} - Device identifier from payload
• ✅ sources/{ip}/{port} - Source network information (default)
• ✅ sensors/{sensor_type} - Single extracted field

Topic construction pattern: {base_topic_from_arn}/{dynamic_suffix}

What's Not Supported

Multiple dynamic segments or suffix insertion:

• ❌ devices/{id}/shadow/update - Cannot append literal text after dynamic part
• ❌ sensors/{building}/{floor}/{type} - Multiple dynamic segments not supported
• ❌ $aws/things/{id}/shadow/update - Thing Shadow direct updates require workaround

Design Rationale

This suffix-based approach provides clear separation of concerns:

• UDP Gateway handles: Protocol translation and device identification
• IoT Core handles: Complex routing, transformation, and fan-out

Complex topic hierarchies and specialized patterns (like Thing Shadows) are better implemented using IoT Core Rules, which can access device registries, transform payloads, and republish to any topic structure. This keeps UDP Gateway configuration simple while enabling flexible evolution of routing logic without gateway redeployment.

Batching Behavior

Like other Destinations, IoT Data respects the destination's batching configuration:

• Count - Maximum messages per batch
• Size - Maximum total bytes per batch
• Time - Maximum wait time before sending

Important: Each message in the batch is published as a separate MQTT message (not as a single batched message). The publications occur concurrently for efficiency, but IoT Core receives individual messages on their respective topics.

Batching recommendations:

• Configure batch size based on your traffic patterns and latency requirements
• Smaller batches reduce latency but may increase costs
• Larger batches improve throughput and reduce per-message overhead
• Consider IoT Core's rate limits when configuring batch sizes for high-volume deployments

Regional Considerations

The IoT Data adapter uses the region specified in the destination ARN to connect to the appropriate AWS IoT Data endpoint. Ensure that your UDP Gateway deployment has network access to the target region's endpoint.

Ensure your IAM role has iot:Publish permissions in all target regions.

Things to Know About IoT Data Destinations

• The ARN must use the iot service identifier and specify a topic path
• Dynamic mode preserves raw binary payloads, while static mode wraps them in JSON
• Topic suffixes are automatically sanitized to comply with MQTT restrictions
• Complex topic patterns (multiple dynamic segments, literal suffixes) require IoT Core Rules
• All messages are published with QoS 0 (at most once)
• Batching controls how many messages are published concurrently, not combined into single messages
• The service adapter uses the region from the ARN, ensure IAM permissions match
• Consider using composite destinations to publish to both IoT Data and other services simultaneously

Destinations for AWS IoT Data are available now for all customers.