Select Availability of Packet Sourcing

Proxylity UDP Gateway excels at receiving UDP packets from the edge and routing them to serverless backends — Lambda functions, S3, SQS queues, Kinesis streams, and more. That model covers the majority of use cases we've seen, but a class of real-world scenarios needs something more: the ability for an application to originate UDP packets outside of any inbound request and have them delivered to a connected client.

This article explains why that capability matters, what the requirements look like, and how we are building support for it into UDP Gateway.

The Request Path Isn't Always Enough

The existing request path supports two patterns:

Write-only delivery: Write-only delivery of packet data for processing and archival using services like Kinesis, SQS and DDB that store the information without generating a response.
Request/Response handling: Request/Response handling of packets where the protocol includes outcome or acknowledgement responses using Lambda or Step Functions.

Since Destinations support multiplexing these patterns can be combined. For example, using a Lambda Destination to generate a response alongside a Firehose Destination for archival.

The limitation of these patterns is that they can only generate egress (outbound) packets during a direct synchronous call to Lambda or Step Functions.

We added support for Lambda Response Streaming, which allows sending more than one response on a timed cadence, but since the Lambda execution must run for the full duration of the response stream it can be expensive and is limited to the maximum Lambda execution time (currently 15 minutes).

For some customers this limitation was a barrier. Some examples:

Push notifications to IoT sensors
Sending game state
Over the Air updates
Periodic server-initiated pings (availability testing)
Async responses to work that outlives the original request
Fan-out from an event to multiple connected peers

We could see the pattern, but the implementation was challenging.

Requirements for Reliable Out-of-Band Packet Sourcing

As background, a Destination receives packets from UDP Gateway as JSON objects containing information about the source, the destination, the protocol and the payload. In addition, every packet has a unique Tag property that identifies it.

The packet Tag serves two purposes:

Tracability of each packet as it traverses the system. This tags allows tracking performance, diagnosing issues, and correlating events across different components.
Reply routing by identifying the original path taken by the ingress packet, allowing the replies to egress vie the same external IP and port. This is critical for firewall and NAT compatibility.

Packet handlers don't need to use the client IP or port to generate responses. By using the Tag responses egress via the Listener's domain/IP and port.

Handlers could theoretically send UDP packets directly, but using the Tag ensures proper routing and compliance with network policies. Sending packets to the same client directly would mean the client would see a different source IP and port, which could cause issues with firewalls and NAT.

Operating with a global footprint adds additional constraints to avoid unnecessary cross-region latency while preserving our flexible global deployment options. It wouldn't be acceptable to source all packets from a single region. Nor would it be acceptable to use different egress endpoints for different regions.

We needed a design that could allow packet sourcing in any region while maintaining consistent egress endpoints and minimizing cross-region latency.

We also needed a design that could scale. Our customer's solutions range from small deployments with a trickle of packets to large-scale deployments with substantial deluges of packets. We didn't want packet sourcing to fall short on that axis of performance either, it needed to handle both ends of the spectrum efficiently and economically.

Oh, and did we mention security? Support for WireGuard Listeners in addition to plain UDP needed to be there from day 1.

We set a high bar. Clearing it wasn't (and perhaps still isn't) a given.

The Design We've Landed On

The primary customer-facing artifact for packet sourcing is an aptly named new regional resource: PacketSource. For now it's only available as an IaC resource and will be surfaced in the UI once we reach General Availability.

A PacketSource binds an SNS topic in your account to a UDP Gateway Listener. The Listener provides the egress endpoint; the SNS topic is how your code sends packets to the Listener.

How and when packets are sent to the SNS topic is up to you, allowing for flexible integration with your existing workflows. And, of course, you have complete control over permission to use the topic via IAM. Your account, your rules.

Unlike the request path, packets sent via a PacketSource don't make use of the Tag from a request. Rather, egress packets are sent to the client by IP and port directly. This design choice was motivated by use cases that aren't triggered by requests at all.

The message to SNS includes the Data field, which contains the payload to be sent, and the Remote object which contains the destination IP, port and peer key (when applicable).

Egress packets generated by a PacketSource are billed at the regular packet rate or batch tiers based on your Subscription.

Here is a minimal CloudFormation snippet showing the customer-side resources. The SNS topic stays in your account; you register it with Proxylity when creating the PacketSource.

"ReplySource": {
  "Type": "Custom::ProxylityUdpGatewayPacketSource",
  "Properties": {
    "Enabled": true,
    "ListenerName": {
      "Ref": "DecoderListener"
    },
    "SnsTopicSource": {
      "Role": {
        "Fn::GetAtt": [
          "RoleForProxylity",
          "Arn"
        ]
      },
      "TopicArn": {
        "Fn::GetAtt": [
          "ReplyTopic",
          "TopicArn"
        ]
      }
    }
  }
},

Publishing a sourced packet from a Lambda function looks like this:

import boto3, json, base64
sns = boto3.client("sns")
payload = b"\x01\x02\x03\x04"  # your UDP payload
sns.publish(
  TopicArn="arn:aws:sns:us-east-1:123456789012:my-packet-source-topic",
  Message=json.dumps({
    "Data": base64.b64encode(payload).decode(),
    "Remote": {
      "IP": "203.0.113.42",
      "Port": 4567
      # "PeerKey": "<WireGuard public key, if applicable>"
    }
  })
)

Operational Considerations

We only support SNS for now. Likewise packets sources are available via IaC only, and not present in the app UI. We don't support formatters other than base64. Access to PacketSource requires review and approval. But we want your input, so please reach out.

Sourced packet throughput scales automatically — there is no separate rate limit today, though we reserve the right to throttle accounts that send bursts inconsistent with their stated use case during the select availability period.

If the gateway cannot deliver a sourced packet (the remote address is unreachable, the Remote object is malformed, or other errors) the SNS message is discarded in alignment with the semantics of UDP. We recommend attaching a dead-letter queue to your SNS topic to capture and inspect failures. Structured delivery errors are also emitted to CloudWatch Logs for the Listener, alongside inbound packet events, so a single log group covers the full picture.

WireGuard Listeners are supported from day one. Set the PeerKey field in Remote to the WireGuard public key of the target peer; the gateway handles encapsulation transparently.

We Want Your Feedback

Your feedback is invaluable to us as we work toward General Availability. If you would like to get an early look at the Packet Source feature or provide feedback, please reach out to us by email or LinkedIn.

What's Next?

The PacketSource feature is still under developement. We plan to introduce batched sends, TTL hints, and an app user experience prior to General Availability. We also plan to listen carefully to your feedback and iterate based on what we hear.