AWS Kinesis

Amazon Kinesis connector for reading from and writing to Kinesis streams.

Connection Profile

No connection profile needed. Uses AWS IAM credentials from environment.

Connection Table

stream_name (string, required): Kinesis stream name
aws_region (string, optional): AWS region
type (required): One of:
- Source:
  - offset (string, required): "latest" | "earliest"
- Sink:
  - records_per_batch (integer, optional): Records to batch before writing (max 500)
  - batch_max_buffer_size (integer, optional): Max batch size in bytes (max 4000000)
  - batch_flush_interval_millis (integer, optional): Milliseconds to wait before flushing

JSON Schema Reference

Connection Table Schema

{
  "type": "object",
  "properties": {
    "stream_name": {"type": "string"},
    "aws_region": {"type": "string"},
    "type": {
      "oneOf": [
        {
          "title": "Source",
          "required": ["offset"],
          "properties": {
            "offset": {"type": "string", "enum": ["latest", "earliest"]}
          }
        },
        {
          "title": "Sink",
          "properties": {
            "records_per_batch": {"type": "integer", "maximum": 500},
            "batch_max_buffer_size": {"type": "integer", "maximum": 4000000},
            "batch_flush_interval_millis": {"type": "integer"}
          }
        }
      ]
    }
  },
  "required": ["stream_name", "type"]
}