create-pipe
Description
Create a pipe. Amazon EventBridge Pipes connect event sources to targets and reduces the need for specialized knowledge and integration code.
See also: AWS API Documentation
Synopsis
create-pipe
--name <value>
[--description <value>]
[--desired-state <value>]
--source <value>
[--source-parameters <value>]
[--enrichment <value>]
[--enrichment-parameters <value>]
--target <value>
[--target-parameters <value>]
--role-arn <value>
[--tags <value>]
[--log-configuration <value>]
[--kms-key-identifier <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]
Options
--name (string)
--description (string)
--desired-state (string)
The state the pipe should be in.
Possible values:
RUNNINGSTOPPED
--source (string)
--source-parameters (structure)
The parameters required to set up a source for your pipe.
FilterCriteria -> (structure)
The collection of event patterns used to filter events.
To remove a filter, specify a FilterCriteria object with an empty array of Filter objects.
For more information, see Events and Event Patterns in the Amazon EventBridge User Guide .
Filters -> (list)
The event patterns.
(structure)
Filter events using an event pattern. For more information, see Events and Event Patterns in the Amazon EventBridge User Guide .
Pattern -> (string)
KinesisStreamParameters -> (structure)
The parameters for using a Kinesis stream as a source.
BatchSize -> (integer)
DeadLetterConfig -> (structure)
Define the target queue to send dead-letter queue events to.
Arn -> (string)
The ARN of the specified target for the dead-letter queue.
For Amazon Kinesis stream and Amazon DynamoDB stream sources, specify either an Amazon SNS topic or Amazon SQS queue ARN.
OnPartialBatchItemFailure -> (string)
AUTOMATIC_BISECT halves each batch and retry each half until all the records are processed or there is one failed message left in the batch.MaximumBatchingWindowInSeconds -> (integer)
MaximumRecordAgeInSeconds -> (integer)
MaximumRetryAttempts -> (integer)
ParallelizationFactor -> (integer)
StartingPosition -> (string)
StartingPositionTimestamp -> (timestamp)
StartingPosition set to AT_TIMESTAMP , the time from which to start reading, in Unix time seconds.DynamoDBStreamParameters -> (structure)
The parameters for using a DynamoDB stream as a source.
BatchSize -> (integer)
DeadLetterConfig -> (structure)
Define the target queue to send dead-letter queue events to.
Arn -> (string)
The ARN of the specified target for the dead-letter queue.
For Amazon Kinesis stream and Amazon DynamoDB stream sources, specify either an Amazon SNS topic or Amazon SQS queue ARN.
OnPartialBatchItemFailure -> (string)
AUTOMATIC_BISECT halves each batch and retry each half until all the records are processed or there is one failed message left in the batch.MaximumBatchingWindowInSeconds -> (integer)
MaximumRecordAgeInSeconds -> (integer)
MaximumRetryAttempts -> (integer)
ParallelizationFactor -> (integer)
StartingPosition -> (string)
SqsQueueParameters -> (structure)
The parameters for using a Amazon SQS stream as a source.
BatchSize -> (integer)
MaximumBatchingWindowInSeconds -> (integer)
ActiveMQBrokerParameters -> (structure)
The parameters for using an Active MQ broker as a source.
Credentials -> (tagged union structure)
The credentials needed to access the resource.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:BasicAuth.BasicAuth -> (string)
QueueName -> (string)
BatchSize -> (integer)
MaximumBatchingWindowInSeconds -> (integer)
RabbitMQBrokerParameters -> (structure)
The parameters for using a Rabbit MQ broker as a source.
Credentials -> (tagged union structure)
The credentials needed to access the resource.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:BasicAuth.BasicAuth -> (string)
QueueName -> (string)
VirtualHost -> (string)
BatchSize -> (integer)
MaximumBatchingWindowInSeconds -> (integer)
ManagedStreamingKafkaParameters -> (structure)
The parameters for using an MSK stream as a source.
TopicName -> (string)
StartingPosition -> (string)
BatchSize -> (integer)
MaximumBatchingWindowInSeconds -> (integer)
ConsumerGroupID -> (string)
Credentials -> (tagged union structure)
The credentials needed to access the resource.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:SaslScram512Auth, ClientCertificateTlsAuth.SaslScram512Auth -> (string)
ClientCertificateTlsAuth -> (string)
SelfManagedKafkaParameters -> (structure)
The parameters for using a self-managed Apache Kafka stream as a source.
A self managed cluster refers to any Apache Kafka cluster not hosted by Amazon Web Services. This includes both clusters you manage yourself, as well as those hosted by a third-party provider, such as Confluent Cloud , CloudKarafka , or Redpanda . For more information, see Apache Kafka streams as a source in the Amazon EventBridge User Guide .
TopicName -> (string)
StartingPosition -> (string)
AdditionalBootstrapServers -> (list)
An array of server URLs.
(string)
BatchSize -> (integer)
MaximumBatchingWindowInSeconds -> (integer)
ConsumerGroupID -> (string)
Credentials -> (tagged union structure)
The credentials needed to access the resource.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:BasicAuth, SaslScram512Auth, SaslScram256Auth, ClientCertificateTlsAuth.BasicAuth -> (string)
SaslScram512Auth -> (string)
SaslScram256Auth -> (string)
ClientCertificateTlsAuth -> (string)
ServerRootCaCertificate -> (string)
Vpc -> (structure)
This structure specifies the VPC subnets and security groups for the stream, and whether a public IP address is to be used.
Subnets -> (list)
Specifies the subnets associated with the stream. These subnets must all be in the same VPC. You can specify as many as 16 subnets.
(string)
SecurityGroup -> (list)
Specifies the security groups associated with the stream. These security groups must all be in the same VPC. You can specify as many as five security groups.
(string)
JSON Syntax:
{
"FilterCriteria": {
"Filters": [
{
"Pattern": "string"
}
...
]
},
"KinesisStreamParameters": {
"BatchSize": integer,
"DeadLetterConfig": {
"Arn": "string"
},
"OnPartialBatchItemFailure": "AUTOMATIC_BISECT",
"MaximumBatchingWindowInSeconds": integer,
"MaximumRecordAgeInSeconds": integer,
"MaximumRetryAttempts": integer,
"ParallelizationFactor": integer,
"StartingPosition": "TRIM_HORIZON"|"LATEST"|"AT_TIMESTAMP",
"StartingPositionTimestamp": timestamp
},
"DynamoDBStreamParameters": {
"BatchSize": integer,
"DeadLetterConfig": {
"Arn": "string"
},
"OnPartialBatchItemFailure": "AUTOMATIC_BISECT",
"MaximumBatchingWindowInSeconds": integer,
"MaximumRecordAgeInSeconds": integer,
"MaximumRetryAttempts": integer,
"ParallelizationFactor": integer,
"StartingPosition": "TRIM_HORIZON"|"LATEST"
},
"SqsQueueParameters": {
"BatchSize": integer,
"MaximumBatchingWindowInSeconds": integer
},
"ActiveMQBrokerParameters": {
"Credentials": {
"BasicAuth": "string"
},
"QueueName": "string",
"BatchSize": integer,
"MaximumBatchingWindowInSeconds": integer
},
"RabbitMQBrokerParameters": {
"Credentials": {
"BasicAuth": "string"
},
"QueueName": "string",
"VirtualHost": "string",
"BatchSize": integer,
"MaximumBatchingWindowInSeconds": integer
},
"ManagedStreamingKafkaParameters": {
"TopicName": "string",
"StartingPosition": "TRIM_HORIZON"|"LATEST",
"BatchSize": integer,
"MaximumBatchingWindowInSeconds": integer,
"ConsumerGroupID": "string",
"Credentials": {
"SaslScram512Auth": "string",
"ClientCertificateTlsAuth": "string"
}
},
"SelfManagedKafkaParameters": {
"TopicName": "string",
"StartingPosition": "TRIM_HORIZON"|"LATEST",
"AdditionalBootstrapServers": ["string", ...],
"BatchSize": integer,
"MaximumBatchingWindowInSeconds": integer,
"ConsumerGroupID": "string",
"Credentials": {
"BasicAuth": "string",
"SaslScram512Auth": "string",
"SaslScram256Auth": "string",
"ClientCertificateTlsAuth": "string"
},
"ServerRootCaCertificate": "string",
"Vpc": {
"Subnets": ["string", ...],
"SecurityGroup": ["string", ...]
}
}
}
--enrichment (string)
--enrichment-parameters (structure)
The parameters required to set up enrichment on your pipe.
InputTemplate -> (string)
Valid JSON text passed to the enrichment. In this case, nothing from the event itself is passed to the enrichment. For more information, see The JavaScript Object Notation (JSON) Data Interchange Format .
To remove an input template, specify an empty string.
HttpParameters -> (structure)
Contains the HTTP parameters to use when the target is a API Gateway REST endpoint or EventBridge ApiDestination.
If you specify an API Gateway REST API or EventBridge ApiDestination as a target, you can use this parameter to specify headers, path parameters, and query string keys/values as part of your target invoking request. If you’re using ApiDestinations, the corresponding Connection can also have these values configured. In case of any conflicting keys, values from the Connection take precedence.
PathParameterValues -> (list)
The path parameter values to be used to populate API Gateway REST API or EventBridge ApiDestination path wildcards (“*”).
(string)
HeaderParameters -> (map)
The headers that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.
key -> (string)
value -> (string)
QueryStringParameters -> (map)
The query string keys/values that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.
key -> (string)
value -> (string)
Shorthand Syntax:
InputTemplate=string,HttpParameters={PathParameterValues=[string,string],HeaderParameters={KeyName1=string,KeyName2=string},QueryStringParameters={KeyName1=string,KeyName2=string}}
JSON Syntax:
{
"InputTemplate": "string",
"HttpParameters": {
"PathParameterValues": ["string", ...],
"HeaderParameters": {"string": "string"
...},
"QueryStringParameters": {"string": "string"
...}
}
}
--target (string)
--target-parameters (structure)
The parameters required to set up a target for your pipe.
For more information about pipe target parameters, including how to use dynamic path parameters, see Target parameters in the Amazon EventBridge User Guide .
InputTemplate -> (string)
Valid JSON text passed to the target. In this case, nothing from the event itself is passed to the target. For more information, see The JavaScript Object Notation (JSON) Data Interchange Format .
To remove an input template, specify an empty string.
LambdaFunctionParameters -> (structure)
The parameters for using a Lambda function as a target.
InvocationType -> (string)
Specify whether to invoke the function synchronously or asynchronously.
-
REQUEST_RESPONSE(default) - Invoke synchronously. This corresponds to theRequestResponseoption in theInvocationTypeparameter for the Lambda Invoke API. -
FIRE_AND_FORGET- Invoke asynchronously. This corresponds to theEventoption in theInvocationTypeparameter for the Lambda Invoke API.
For more information, see Invocation types in the Amazon EventBridge User Guide .
StepFunctionStateMachineParameters -> (structure)
The parameters for using a Step Functions state machine as a target.
InvocationType -> (string)
Specify whether to invoke the Step Functions state machine synchronously or asynchronously.
-
REQUEST_RESPONSE(default) - Invoke synchronously. For more information, see StartSyncExecution in the Step Functions API Reference .
Note
REQUEST_RESPONSE is not supported for STANDARD state machine workflows.-
FIRE_AND_FORGET- Invoke asynchronously. For more information, see StartExecution in the Step Functions API Reference .
For more information, see Invocation types in the Amazon EventBridge User Guide .
KinesisStreamParameters -> (structure)
The parameters for using a Kinesis stream as a target.
PartitionKey -> (string)
EcsTaskParameters -> (structure)
The parameters for using an Amazon ECS task as a target.
TaskDefinitionArn -> (string)
TaskCount -> (integer)
TaskDefinition . The default is 1.LaunchType -> (string)
FARGATE value is supported only in the Regions where Fargate with Amazon ECS is supported. For more information, see Fargate on Amazon ECS in the Amazon Elastic Container Service Developer Guide .NetworkConfiguration -> (structure)
Use this structure if the Amazon ECS task uses the awsvpc network mode. This structure specifies the VPC subnets and security groups associated with the task, and whether a public IP address is to be used. This structure is required if LaunchType is FARGATE because the awsvpc mode is required for Fargate tasks.
If you specify NetworkConfiguration when the target ECS task does not use the awsvpc network mode, the task fails.
awsvpcConfiguration -> (structure)
Use this structure to specify the VPC subnets and security groups for the task, and whether a public IP address is to be used. This structure is relevant only for ECS tasks that use the awsvpc network mode.
Subnets -> (list)
Specifies the subnets associated with the task. These subnets must all be in the same VPC. You can specify as many as 16 subnets.
(string)
SecurityGroups -> (list)
Specifies the security groups associated with the task. These security groups must all be in the same VPC. You can specify as many as five security groups. If you do not specify a security group, the default security group for the VPC is used.
(string)
AssignPublicIp -> (string)
ENABLED only when LaunchType in EcsParameters is set to FARGATE .PlatformVersion -> (string)
Specifies the platform version for the task. Specify only the numeric portion of the platform version, such as 1.1.0 .
This structure is used only if LaunchType is FARGATE . For more information about valid platform versions, see Fargate Platform Versions in the Amazon Elastic Container Service Developer Guide .
Group -> (string)
CapacityProviderStrategy -> (list)
The capacity provider strategy to use for the task.
If a capacityProviderStrategy is specified, the launchType parameter must be omitted. If no capacityProviderStrategy or launchType is specified, the defaultCapacityProviderStrategy for the cluster is used.
(structure)
The details of a capacity provider strategy. To learn more, see CapacityProviderStrategyItem in the Amazon ECS API Reference.
capacityProvider -> (string)
weight -> (integer)
base -> (integer)
EnableECSManagedTags -> (boolean)
EnableExecuteCommand -> (boolean)
PlacementConstraints -> (list)
An array of placement constraint objects to use for the task. You can specify up to 10 constraints per task (including constraints in the task definition and those specified at runtime).
(structure)
An object representing a constraint on task placement. To learn more, see Task Placement Constraints in the Amazon Elastic Container Service Developer Guide.
type -> (string)
expression -> (string)
distinctInstance . To learn more, see Cluster Query Language in the Amazon Elastic Container Service Developer Guide.PlacementStrategy -> (list)
The placement strategy objects to use for the task. You can specify a maximum of five strategy rules per task.
(structure)
The task placement strategy for a task or service. To learn more, see Task Placement Strategies in the Amazon Elastic Container Service Service Developer Guide.
type -> (string)
field -> (string)
PropagateTags -> (string)
TagResource API action.ReferenceId -> (string)
Overrides -> (structure)
The overrides that are associated with a task.
ContainerOverrides -> (list)
One or more container overrides that are sent to a task.
(structure)
The overrides that are sent to a container. An empty container override can be passed in. An example of an empty container override is {"containerOverrides": [ ] } . If a non-empty container override is specified, the name parameter must be included.
Command -> (list)
The command to send to the container that overrides the default command from the Docker image or the task definition. You must also specify a container name.
(string)
Cpu -> (integer)
cpu units reserved for the container, instead of the default value from the task definition. You must also specify a container name.Environment -> (list)
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the task definition. You must also specify a container name.
(structure)
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the task definition. You must also specify a container name.
name -> (string)
value -> (string)
EnvironmentFiles -> (list)
A list of files containing the environment variables to pass to a container, instead of the value from the container definition.
(structure)
A list of files containing the environment variables to pass to a container. You can specify up to ten environment files. The file must have a .env file extension. Each line in an environment file should contain an environment variable in VARIABLE=VALUE format. Lines beginning with # are treated as comments and are ignored. For more information about the environment variable file syntax, see Declare default environment variables in file .
If there are environment variables specified using the environment parameter in a container definition, they take precedence over the variables contained within an environment file. If multiple environment files are specified that contain the same variable, they’re processed from the top down. We recommend that you use unique variable names. For more information, see Specifying environment variables in the Amazon Elastic Container Service Developer Guide .
This parameter is only supported for tasks hosted on Fargate using the following platform versions:
- Linux platform version
1.4.0or later. - Windows platform version
1.0.0or later.
type -> (string)
s3 .value -> (string)
Memory -> (integer)
MemoryReservation -> (integer)
Name -> (string)
ResourceRequirements -> (list)
The type and amount of a resource to assign to a container, instead of the default value from the task definition. The only supported resource is a GPU.
(structure)
The type and amount of a resource to assign to a container. The supported resource types are GPUs and Elastic Inference accelerators. For more information, see Working with GPUs on Amazon ECS or Working with Amazon Elastic Inference on Amazon ECS in the Amazon Elastic Container Service Developer Guide
type -> (string)
GPU or InferenceAccelerator .value -> (string)
The value for the specified resource type.
If the GPU type is used, the value is the number of physical GPUs the Amazon ECS container agent reserves for the container. The number of GPUs that’s reserved for all containers in a task can’t exceed the number of available GPUs on the container instance that the task is launched on.
If the InferenceAccelerator type is used, the value matches the deviceName for an InferenceAccelerator specified in a task definition.
Cpu -> (string)
EphemeralStorage -> (structure)
The ephemeral storage setting override for the task.
Note
This parameter is only supported for tasks hosted on Fargate that use the following platform versions:
- Linux platform version
1.4.0or later. - Windows platform version
1.0.0or later.
sizeInGiB -> (integer)
21 GiB and the maximum supported value is 200 GiB.ExecutionRoleArn -> (string)
InferenceAcceleratorOverrides -> (list)
The Elastic Inference accelerator override for the task.
(structure)
Details on an Elastic Inference accelerator task override. This parameter is used to override the Elastic Inference accelerator specified in the task definition. For more information, see Working with Amazon Elastic Inference on Amazon ECS in the Amazon Elastic Container Service Developer Guide .
deviceName -> (string)
deviceName specified in the task definition.deviceType -> (string)
Memory -> (string)
TaskRoleArn -> (string)
Tags -> (list)
The metadata that you apply to the task to help you categorize and organize them. Each tag consists of a key and an optional value, both of which you define. To learn more, see RunTask in the Amazon ECS API Reference.
(structure)
A key-value pair associated with an Amazon Web Services resource. In EventBridge, rules and event buses support tagging.
Key -> (string)
Value -> (string)
BatchJobParameters -> (structure)
The parameters for using an Batch job as a target.
JobDefinition -> (string)
name , name:revision , or the Amazon Resource Name (ARN) for the job definition. If name is specified without a revision then the latest active revision is used.JobName -> (string)
ArrayProperties -> (structure)
The array properties for the submitted job, such as the size of the array. The array size can be between 2 and 10,000. If you specify array properties for a job, it becomes an array job. This parameter is used only if the target is an Batch job.
Size -> (integer)
RetryStrategy -> (structure)
The retry strategy to use for failed jobs. When a retry strategy is specified here, it overrides the retry strategy defined in the job definition.
Attempts -> (integer)
RUNNABLE status. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.ContainerOverrides -> (structure)
The overrides that are sent to a container.
Command -> (list)
The command to send to the container that overrides the default command from the Docker image or the task definition.
(string)
Environment -> (list)
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the task definition.
Note
Environment variables cannot start with “Batch “. This naming convention is reserved for variables that Batch sets.(structure)
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the task definition.
Note
Environment variables cannot start with “Batch “. This naming convention is reserved for variables that Batch sets.Name -> (string)
Value -> (string)
InstanceType -> (string)
The instance type to use for a multi-node parallel job.
Note
This parameter isn’t applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn’t be provided.ResourceRequirements -> (list)
The type and amount of resources to assign to a container. This overrides the settings in the job definition. The supported resources include GPU , MEMORY , and VCPU .
(structure)
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
Type -> (string)
GPU , MEMORY , and VCPU .Value -> (string)
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn’t exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren’t available for jobs that are running on Fargate resources.
type=”MEMORY”
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you’re trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
VCPU = 0.25
VCPU = 0.25 or 0.5
VCPU = 0.25, 0.5, or 1
VCPU = 0.5, or 1
VCPU = 0.5, 1, or 2
VCPU = 1 or 2
VCPU = 1, 2, 4, or 8
VCPU = 2 or 4
VCPU = 2, 4, or 8
VCPU = 4
VCPU = 4 or 8
VCPU = 8
VCPU = 8 or 16
VCPU = 16
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
MEMORY = 512, 1024, or 2048
MEMORY = 1024, 2048, 3072, or 4096
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
DependsOn -> (list)
A list of dependencies for the job. A job can depend upon a maximum of 20 jobs. You can specify a SEQUENTIAL type dependency without specifying a job ID for array jobs so that each child array job completes sequentially, starting at index 0. You can also specify an N_TO_N type dependency with a job ID for array jobs. In that case, each index child of this job must wait for the corresponding index child of each dependency to complete before it can begin.
(structure)
An object that represents an Batch job dependency.
JobId -> (string)
Type -> (string)
Parameters -> (map)
Additional parameters passed to the job that replace parameter substitution placeholders that are set in the job definition. Parameters are specified as a key and value pair mapping. Parameters included here override any corresponding parameter defaults from the job definition.
key -> (string)
value -> (string)
SqsQueueParameters -> (structure)
The parameters for using a Amazon SQS stream as a target.
MessageGroupId -> (string)
MessageDeduplicationId -> (string)
This parameter applies only to FIFO (first-in-first-out) queues.
The token used for deduplication of sent messages.
HttpParameters -> (structure)
These are custom parameter to be used when the target is an API Gateway REST APIs or EventBridge ApiDestinations.
PathParameterValues -> (list)
The path parameter values to be used to populate API Gateway REST API or EventBridge ApiDestination path wildcards (“*”).
(string)
HeaderParameters -> (map)
The headers that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.
key -> (string)
value -> (string)
QueryStringParameters -> (map)
The query string keys/values that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.
key -> (string)
value -> (string)
RedshiftDataParameters -> (structure)
These are custom parameters to be used when the target is a Amazon Redshift cluster to invoke the Amazon Redshift Data API BatchExecuteStatement.
SecretManagerArn -> (string)
Database -> (string)
DbUser -> (string)
StatementName -> (string)
WithEvent -> (boolean)
Sqls -> (list)
The SQL statement text to run.
(string)
SageMakerPipelineParameters -> (structure)
The parameters for using a SageMaker pipeline as a target.
PipelineParameterList -> (list)
List of Parameter names and values for SageMaker Model Building Pipeline execution.
(structure)
Name/Value pair of a parameter to start execution of a SageMaker Model Building Pipeline.
Name -> (string)
Value -> (string)
EventBridgeEventBusParameters -> (structure)
The parameters for using an EventBridge event bus as a target.
EndpointId -> (string)
abcde.veo .DetailType -> (string)
Source -> (string)
Resources -> (list)
Amazon Web Services resources, identified by Amazon Resource Name (ARN), which the event primarily concerns. Any number, including zero, may be present.
(string)
Time -> (string)
CloudWatchLogsParameters -> (structure)
The parameters for using an CloudWatch Logs log stream as a target.
LogStreamName -> (string)
Timestamp -> (string)
TimestreamParameters -> (structure)
The parameters for using a Timestream for LiveAnalytics table as a target.
TimeValue -> (string)
EpochTimeUnit -> (string)
The granularity of the time units used. Default is MILLISECONDS .
Required if TimeFieldType is specified as EPOCH .
TimeFieldType -> (string)
The type of time value used.
The default is EPOCH .
TimestampFormat -> (string)
How to format the timestamps. For example, yyyy-MM-dd'T'HH:mm:ss'Z' .
Required if TimeFieldType is specified as TIMESTAMP_FORMAT .
VersionValue -> (string)
64 bit version value or source data field that represents the version value for your data.
Write requests with a higher version number will update the existing measure values of the record and version. In cases where the measure value is the same, the version will still be updated.
Default value is 1.
Timestream for LiveAnalytics does not support updating partial measure values in a record.
Write requests for duplicate data with a higher version number will update the existing measure value and version. In cases where the measure value is the same, Version will still be updated. Default value is 1 .
Note
Version must be 1 or greater, or you will receive a ValidationException error.DimensionMappings -> (list)
Map source data to dimensions in the target Timestream for LiveAnalytics table.
For more information, see Amazon Timestream for LiveAnalytics concepts
(structure)
Maps source data to a dimension in the target Timestream for LiveAnalytics table.
For more information, see Amazon Timestream for LiveAnalytics concepts
DimensionValue -> (string)
DimensionValueType -> (string)
DimensionName -> (string)
SingleMeasureMappings -> (list)
Mappings of single source data fields to individual records in the specified Timestream for LiveAnalytics table.
(structure)
Maps a single source data field to a single record in the specified Timestream for LiveAnalytics table.
For more information, see Amazon Timestream for LiveAnalytics concepts
MeasureValue -> (string)
MeasureValueType -> (string)
MeasureName -> (string)
MultiMeasureMappings -> (list)
Maps multiple measures from the source event to the same record in the specified Timestream for LiveAnalytics table.
(structure)
Maps multiple measures from the source event to the same Timestream for LiveAnalytics record.
For more information, see Amazon Timestream for LiveAnalytics concepts
MultiMeasureName -> (string)
MultiMeasureAttributeMappings -> (list)
Mappings that represent multiple source event fields mapped to measures in the same Timestream for LiveAnalytics record.
(structure)
A mapping of a source event data field to a measure in a Timestream for LiveAnalytics record.
MeasureValue -> (string)
MeasureValueType -> (string)
MultiMeasureAttributeName -> (string)
JSON Syntax:
{
"InputTemplate": "string",
"LambdaFunctionParameters": {
"InvocationType": "REQUEST_RESPONSE"|"FIRE_AND_FORGET"
},
"StepFunctionStateMachineParameters": {
"InvocationType": "REQUEST_RESPONSE"|"FIRE_AND_FORGET"
},
"KinesisStreamParameters": {
"PartitionKey": "string"
},
"EcsTaskParameters": {
"TaskDefinitionArn": "string",
"TaskCount": integer,
"LaunchType": "EC2"|"FARGATE"|"EXTERNAL",
"NetworkConfiguration": {
"awsvpcConfiguration": {
"Subnets": ["string", ...],
"SecurityGroups": ["string", ...],
"AssignPublicIp": "ENABLED"|"DISABLED"
}
},
"PlatformVersion": "string",
"Group": "string",
"CapacityProviderStrategy": [
{
"capacityProvider": "string",
"weight": integer,
"base": integer
}
...
],
"EnableECSManagedTags": true|false,
"EnableExecuteCommand": true|false,
"PlacementConstraints": [
{
"type": "distinctInstance"|"memberOf",
"expression": "string"
}
...
],
"PlacementStrategy": [
{
"type": "random"|"spread"|"binpack",
"field": "string"
}
...
],
"PropagateTags": "TASK_DEFINITION",
"ReferenceId": "string",
"Overrides": {
"ContainerOverrides": [
{
"Command": ["string", ...],
"Cpu": integer,
"Environment": [
{
"name": "string",
"value": "string"
}
...
],
"EnvironmentFiles": [
{
"type": "s3",
"value": "string"
}
...
],
"Memory": integer,
"MemoryReservation": integer,
"Name": "string",
"ResourceRequirements": [
{
"type": "GPU"|"InferenceAccelerator",
"value": "string"
}
...
]
}
...
],
"Cpu": "string",
"EphemeralStorage": {
"sizeInGiB": integer
},
"ExecutionRoleArn": "string",
"InferenceAcceleratorOverrides": [
{
"deviceName": "string",
"deviceType": "string"
}
...
],
"Memory": "string",
"TaskRoleArn": "string"
},
"Tags": [
{
"Key": "string",
"Value": "string"
}
...
]
},
"BatchJobParameters": {
"JobDefinition": "string",
"JobName": "string",
"ArrayProperties": {
"Size": integer
},
"RetryStrategy": {
"Attempts": integer
},
"ContainerOverrides": {
"Command": ["string", ...],
"Environment": [
{
"Name": "string",
"Value": "string"
}
...
],
"InstanceType": "string",
"ResourceRequirements": [
{
"Type": "GPU"|"MEMORY"|"VCPU",
"Value": "string"
}
...
]
},
"DependsOn": [
{
"JobId": "string",
"Type": "N_TO_N"|"SEQUENTIAL"
}
...
],
"Parameters": {"string": "string"
...}
},
"SqsQueueParameters": {
"MessageGroupId": "string",
"MessageDeduplicationId": "string"
},
"HttpParameters": {
"PathParameterValues": ["string", ...],
"HeaderParameters": {"string": "string"
...},
"QueryStringParameters": {"string": "string"
...}
},
"RedshiftDataParameters": {
"SecretManagerArn": "string",
"Database": "string",
"DbUser": "string",
"StatementName": "string",
"WithEvent": true|false,
"Sqls": ["string", ...]
},
"SageMakerPipelineParameters": {
"PipelineParameterList": [
{
"Name": "string",
"Value": "string"
}
...
]
},
"EventBridgeEventBusParameters": {
"EndpointId": "string",
"DetailType": "string",
"Source": "string",
"Resources": ["string", ...],
"Time": "string"
},
"CloudWatchLogsParameters": {
"LogStreamName": "string",
"Timestamp": "string"
},
"TimestreamParameters": {
"TimeValue": "string",
"EpochTimeUnit": "MILLISECONDS"|"SECONDS"|"MICROSECONDS"|"NANOSECONDS",
"TimeFieldType": "EPOCH"|"TIMESTAMP_FORMAT",
"TimestampFormat": "string",
"VersionValue": "string",
"DimensionMappings": [
{
"DimensionValue": "string",
"DimensionValueType": "VARCHAR",
"DimensionName": "string"
}
...
],
"SingleMeasureMappings": [
{
"MeasureValue": "string",
"MeasureValueType": "DOUBLE"|"BIGINT"|"VARCHAR"|"BOOLEAN"|"TIMESTAMP",
"MeasureName": "string"
}
...
],
"MultiMeasureMappings": [
{
"MultiMeasureName": "string",
"MultiMeasureAttributeMappings": [
{
"MeasureValue": "string",
"MeasureValueType": "DOUBLE"|"BIGINT"|"VARCHAR"|"BOOLEAN"|"TIMESTAMP",
"MultiMeasureAttributeName": "string"
}
...
]
}
...
]
}
}
--role-arn (string)
--tags (map)
The list of key-value pairs to associate with the pipe.
key -> (string)
value -> (string)
Shorthand Syntax:
KeyName1=string,KeyName2=string
JSON Syntax:
{"string": "string"
...}
--log-configuration (structure)
The logging configuration settings for the pipe.
S3LogDestination -> (structure)
The Amazon S3 logging configuration settings for the pipe.
BucketName -> (string)
BucketOwner -> (string)
OutputFormat -> (string)
How EventBridge should format the log records.
EventBridge currently only supports json formatting.
Prefix -> (string)
Specifies any prefix text with which to begin Amazon S3 log object names.
You can use prefixes to organize the data that you store in Amazon S3 buckets. A prefix is a string of characters at the beginning of the object key name. A prefix can be any length, subject to the maximum length of the object key name (1,024 bytes). For more information, see Organizing objects using prefixes in the Amazon Simple Storage Service User Guide .
FirehoseLogDestination -> (structure)
The Amazon Data Firehose logging configuration settings for the pipe.
DeliveryStreamArn -> (string)
CloudwatchLogsLogDestination -> (structure)
The Amazon CloudWatch Logs logging configuration settings for the pipe.
LogGroupArn -> (string)
Level -> (string)
The level of logging detail to include. This applies to all log destinations for the pipe.
For more information, see Specifying EventBridge Pipes log level in the Amazon EventBridge User Guide .
IncludeExecutionData -> (list)
Specify ALL to include the execution data (specifically, the payload , awsRequest , and awsResponse fields) in the log messages for this pipe.
This applies to all log destinations for the pipe.
For more information, see Including execution data in logs in the Amazon EventBridge User Guide .
By default, execution data is not included.
(string)
Shorthand Syntax:
S3LogDestination={BucketName=string,BucketOwner=string,OutputFormat=string,Prefix=string},FirehoseLogDestination={DeliveryStreamArn=string},CloudwatchLogsLogDestination={LogGroupArn=string},Level=string,IncludeExecutionData=string,string
JSON Syntax:
{
"S3LogDestination": {
"BucketName": "string",
"BucketOwner": "string",
"OutputFormat": "json"|"plain"|"w3c",
"Prefix": "string"
},
"FirehoseLogDestination": {
"DeliveryStreamArn": "string"
},
"CloudwatchLogsLogDestination": {
"LogGroupArn": "string"
},
"Level": "OFF"|"ERROR"|"INFO"|"TRACE",
"IncludeExecutionData": ["ALL", ...]
}
--kms-key-identifier (string)
The identifier of the KMS customer managed key for EventBridge to use, if you choose to use a customer managed key to encrypt pipe data. The identifier can be the key Amazon Resource Name (ARN), KeyId, key alias, or key alias ARN.
If you do not specify a customer managed key identifier, EventBridge uses an Amazon Web Services owned key to encrypt pipe data.
For more information, see Managing keys in the Key Management Service Developer Guide .
--cli-input-json | --cli-input-yaml (string) Reads arguments from the JSON string provided. The JSON string follows the format provided by --generate-cli-skeleton. If other arguments are provided on the command line, those values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally. This may not be specified along with --cli-input-yaml.
--generate-cli-skeleton (string) Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the value input, prints a sample input JSON that can be used as an argument for --cli-input-json. Similarly, if provided yaml-input it will print a sample input YAML that can be used with --cli-input-yaml. If provided with the value output, it validates the command inputs and returns a sample output JSON for that command. The generated JSON skeleton is not stable between versions of the AWS CLI and there are no backwards compatibility guarantees in the JSON skeleton generated.
Global Options
--debug (boolean)
Turn on debug logging.
--endpoint-url (string)
Override command’s default URL with the given URL.
--no-verify-ssl (boolean)
By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.
--no-paginate (boolean)
Disable automatic pagination. If automatic pagination is disabled, the AWS CLI will only make one call, for the first page of results.
--output (string)
The formatting style for command output.
- json
- text
- table
- yaml
- yaml-stream
--query (string)
A JMESPath query to use in filtering the response data.
--profile (string)
Use a specific profile from your credential file.
--region (string)
The region to use. Overrides config/env settings.
--version (string)
Display the version of this tool.
--color (string)
Turn on/off color output.
- on
- off
- auto
--no-sign-request (boolean)
Do not sign requests. Credentials will not be loaded if this argument is provided.
--ca-bundle (string)
The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings.
--cli-read-timeout (int)
The maximum socket read time in seconds. If the value is set to 0, the socket read will be blocking and not timeout. The default value is 60 seconds.
--cli-connect-timeout (int)
The maximum socket connect time in seconds. If the value is set to 0, the socket connect will be blocking and not timeout. The default value is 60 seconds.
--cli-binary-format (string)
The formatting style to be used for binary blobs. The default format is base64. The base64 format expects binary blobs to be provided as a base64 encoded string. The raw-in-base64-out format preserves compatibility with AWS CLI V1 behavior and binary values must be passed literally. When providing contents from a file that map to a binary blob fileb:// will always be treated as binary and use the file contents directly regardless of the cli-binary-format setting. When using file:// the file contents will need to properly formatted for the configured cli-binary-format.
- base64
- raw-in-base64-out
--no-cli-pager (boolean)
Disable cli pager for output.
--cli-auto-prompt (boolean)
Automatically prompt for CLI input parameters.
--no-cli-auto-prompt (boolean)
Disable automatically prompt for CLI input parameters.
Examples
Note
To use the following examples, you must have the AWS CLI installed and configured. See the Getting started guide in the AWS CLI User Guide for more information.
Unless otherwise stated, all examples have unix-like quotation rules. These examples will need to be adapted to your terminal’s quoting rules. See Using quotation marks with strings in the AWS CLI User Guide .
To Create a pipe
The following create-pipe example creates a Pipe named Demo_Pipe with SQS as the source and CloudWatch Log Group as the target for the Pipe.
aws pipes create-pipe \
--name Demo_Pipe \
--desired-state RUNNING \
--role-arn arn:aws:iam::123456789012:role/service-role/Amazon_EventBridge_Pipe_Demo_Pipe_28b3aa4f \
--source arn:aws:sqs:us-east-1:123456789012:Demo_Queue \
--target arn:aws:logs:us-east-1:123456789012:log-group:/aws/pipes/Demo_LogGroup
Output:
{
"Arn": "arn:aws:pipes:us-east-1:123456789012:pipe/Demo_Pipe",
"Name": "Demo_Pipe",
"DesiredState": "RUNNING",
"CurrentState": "CREATING",
"CreationTime": "2024-10-08T12:33:59-05:00",
"LastModifiedTime": "2024-10-08T12:33:59.684839-05:00"
}
For more information, see Amazon EventBridge Pipes concepts in the Amazon EventBridge User Guide.
Output
Arn -> (string)
Name -> (string)
DesiredState -> (string)
CurrentState -> (string)
CreationTime -> (timestamp)
LastModifiedTime -> (timestamp)
© Copyright 2025, Amazon Web Services. Created using Sphinx.