LogoLogo
ProductsDevelopersAPI ReferenceContact Us
Developers
Developers
  • KORE Developer Experience Overview
  • API Management
    • API Clients
    • Authorization
    • API Responses
  • Webhooks
    • Connection Settings
    • Verifying Callbacks from KORE
    • Idempotency
  • Event Streams
    • Destinations
    • Streaming Rules
    • Events
  • Get Started
    • APIs
    • Webhooks
    • Event Streams
      • AWS Kinesis Quick Start
        • Script to Create a Kinesis Stream
      • Webhook Quick Start
  • How To ...
    • Invite and collaborate with developers
    • APIs
      • Refresh API Access Token
    • Webhooks
      • Validate Webhook Signatures
      • Rotate my Webhook Secret
      • Test and Troubleshoot my Webhook Integration
    • Event Streams
      • Test My Destination
      • Modify my Destination
      • Modify my Streaming Rule
On this page
  • Create a Kinesis stream for your Account
  • Create a Kinesis Destination at KORE
  • (Optional) Test your Destination
  • Create a Streaming Rule
  • Read and Parse the Data

Was this helpful?

  1. Get Started
  2. Event Streams

AWS Kinesis Quick Start

PreviousEvent StreamsNextScript to Create a Kinesis Stream

Last updated 8 months ago

Was this helpful?

Create a Kinesis stream for your Account

We have prepared a for you that sets up a Kinesis stream with a shard count. Copy the script and save it to a file named create_kinesis_stream.sh. Make it executable with chmod 755. The script will set up the policies and roles KORE needs to write to your stream. The only thing you need to do is run the script and pass the stream name and shard count as arguments.

You will need to ensure that your terminal has before executing both this script and the script you will use later in this quick start.

Use the name kore-events for the Kinesis stream. You can use any name, but if you use a different one than kore-events, you will need to change the permission instructions later on with your custom name.

Set the number of shards to 1. This makes initial validation easier. It can be modified after the Destination has been set up.

Here's an example:

./create_kinesis_stream.sh kore-events 1 | jq . > kore-destination.json

The script uses the to create the Kinesis stream with the specific shard count. It will return a JSON payload with the details you need to create a Destination. In the above example, the received data will be written to a file named kore-destination.json.

The stream is created in the region specified by the environment variable AWS_DEFAULT_REGION.

You can update the shard count of the Kinesis stream after it has been validated by using this script:

Copy code block
if [ $# -ne 2 ]; then
  echo
  echo "usage: $0 <stream_name> <shard_count> <scaling_type>"
  echo "<scaling_type> possible values UNIFORM_SCALING”
  echo
  exit 1
fi

# update stream
STREAM_NAME=$1
SHARD_COUNT=$2
SCALING_TYPE=$3

aws kinesis update-shard-count --stream-name $STREAM_NAME --scaling-type $SCALING_TYPE --target-shard-count $SHARD_COUNT

Here is an example showing how the Kinesis stream shard count is increased to 6:

./update_kinesis_shard_count.sh kore-events 6 UNIFORM_SCALING

Create a Kinesis Destination at KORE

To create a new Kinesis destination, go to the navigation bar on the left side of the screen and click on the Destinations option. This will open the Destinations page. Select the +Create button to add.

Provide a friendly name and select the destination type "AWS Kinesis" then click on continue.

Provide your kinesis configuration by filling in the following inputs

  • ARN: is a Unique identifier for your Amazon Kinesis resource. It typically follows this format: arn:aws:kinesis:region:account-id:stream/stream-name

    • You can find your Kinesis ARN in AWS Management Console :

      • Navigate to Amazon Kinesis => Data Streams

  • Role ARN: is a unique identifier for an IAM (Identity and Access Management). This identifier typically follows this format arn:aws:iam::account-id:role/role-name

    • You can find the ARN of an IAM role in the AWS Management Console :

      • Navigate to IAM service => Roles

  • External ID: is an additional security measure for stream connections. This unique identifier serves as a secret token that authorizes KORE to assume the provided IAM Role, granting secure access to your AWS Kinesis resources.

Once you have completed all required fields in the form, click the 'Finish' button . This action will finalize the setup process and create a new Kinesis destination.

(Optional) Test your Destination

This step is optional but encouraged because it ensures KORE and your destination can successfully communicate before trying to stream events to your destinations.

Create a Streaming Rule

A streaming rule that tells KORE which events you streamed to which destination

Provide your rule configuration, filling in the required fields for the rule configuration:

  • Rule Name: A unique identifier for the rule you are creating.

  • Destination: The endpoint where the events will be sent.

  • Product: Select the product group relevant to your events.

After selecting a product group, you'll have a list of available event types and their associated schemas. To configure your rule:

  • Review the displayed event types relevant to the chosen product group.

  • Select the appropriate schema version for each event type you wish to include in your rule.

  • Your selections will determine which event types and schema versions will be applied to your streaming rule.

Once you have completed all required fields in the form, click the Create button. This action will finalize the setup process and create a new streaming rule. You should start receiving events sent to the destination when they happen.

Read and Parse the Data

The data you receive will follow the formats as designed:

Review our on how you can test your destination.

To create a new rule, go to the navigation bar on the left side of the screen and click on the option. This will open the Rules page. Select the +Create button.

Review our to see an example of this.

script
valid AWS credentials
AWS CLI
guide
Streaming Rules
guide
Event Schema
Event Data Object Schema