Kafka Operations

Available Operations

Scale Cluster

What is it?

It enables user to modify the capacity of the cluster by changing the count of instances that are already part of the cluster. User can also modify the instance type of the cluster. The scaling operation supports both vertical & horizontal scale up and scale down.

When to use?

This operation can be run only when the cluster is in running state or when the previous operation is in success status. Users need to wait if any existing operation is already running on the same cluster and can check its status via odin provided Jenkins.

Limitations

Scale operation cannot be stopped.
If the previous operation on the cluster was unsuccessful , then user would not be able to scale cluster.

How to use?

Specify the desired number of instances for the cluster in desired_instance_count . Please note that you need to provide the number of instances that you require at the end state. E.g: If yo u have 5 instances and you want to add 5 more, the value of desired_instance_count will be 10 as the same represents end state.

Options

Option

Type

Description

Example

desiredInstanceCount

integer

The no. of instances

required by the user at the end state.

--options '{

"desiredInstanceCount": 12,

"instanceType":

"m5.4xlarge"}'

instanceType

text

The instance type to be provided here.

--options '{

"desiredInstanceCount": 12,

"instanceType":

"m5.4xlarge"}'

If the user wants to scale up horizontally, user can pass only desired_instance_count in the options.
If the user wants to scale vertically, user can pass only instance_type in the options.

When scaling a cluster, if the user specifies a different instance type than the current one, all existing instances in the cluster will also be changed to the new instance type.

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation scale 
--options '{"desired_instance_count": NO_OF_INSTANCE_AT_END_STATE,
"instance_type": "DESIRED_INSTANCE_TYPE"}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation scale
--options '{"desired_instance_count": 12,
"instance_type": "m5.4xlarge"}'
--env myEnv

Restart Cluster

What is it?

This operation helps the users to restart a cluster.

When to use?

This operation can be run only when the cluster is in running state or when an existing operation is success . User need to wait if any existing operation is already running on the same cluster and can check its status via odin provided Jenkins.

Option

Type

Description

Example

nodes

list

The list of IPs that needs to be restarted should be provided in comma separated format . If no list is provided, all nodes will be restarted.

--options

'{ "nodes":

["10.10.10.1"]

}'

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation restart 
--options '{"nodes": ["IP_1","IP_2"]}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation restart 
--options '{"nodes": ["10.10.10.1","168.122.90.1"]}'
--env sr001

Replace Node

What is it?

Replace node replaces the node that has concerns (such as hardware issues, disk corruption or any AWS event related to h/w issues) with the node of the same instance type.

When to use?

Option

Type

Description

Example

nodes

list

The list of IPs that needs to be replaced to be provided in comma separated format .

--options

'{ "nodes":

["10.10.10.1"]

}'

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation replace_node 
--options '{"nodes": ["IP_1","IP_2"]}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation replace_node 
--options '{"nodes": ["10.10.10.1","168.122.90.1"]}'
--env sr001

Pause

What is it?

This operation enables users to pause a cluster. This can be used to optimise cost, especially in load environment during idle hours. Post pause operation, all the nodes of the specific cluster will be in 'stopped' state.

When to use?

Option

This command does not take any input in the form of a option. A blank json to be passed for options here.

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation pause 
--options '{}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation pause --options '{}'
--env sr001

Resume

What is it?

This operation enables users to resume the normal operation of an already paused cluster.

When to use?

This operation can be run only when the cluster is in successful pause state.

Option

This command does not take any input in the form of a option. A blank json to be passed for options here.

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation resume 
--options '{}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation resume --options '{}'
--env sr001

Modify Volume

What is it?

This operation enables users to change the disk size of each node in the cluster.

When to use?

This operation can be run when extra disk size is required for any event and come back to original disk size. This can be run only when the cluster is in healthy state state , migration of data is possible on the requested volume size and replication factor is more than the availability zone count.

Option

Type

Description

Example

type

list

The list of IPs that needs to be replaced to be provided in comma separated format .

--options

'{ "nodes":

["10.10.10.1"]

}'

size

integer

The size of volume in GBs

--options

'{ "size": 1000 }'

iops

integer

The count of IOPS for the volume

--options

'{ "iops": 1000 }'

throughput

integer

The amount of throughput ( MiB/s)

--options

'{"throughput": 500 }'

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation modify-volume 
--options '{}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation modify-volume --options '{"volume":{"size":1000}}'
--env sr001

Config Update

What is it?

This operation enables users to change configuration of a cluster after its creation.

When to use?

This operation can be run only when the cluster is in healthy state.

Option

Type

Description

Example

serverProperties

object

This specifices which server configurations to update

--options=‘{"serverProperties":{}}’

server_properties options -

Option

Type

Description

Example

numPartitions

integer

The list of IPs that needs to be replaced to be provided in comma separated format.

--options

'{ "serverProperties":{"numPartitions": 6}}'

logRetentionHours

integer

The duration to retain log segments.

--options

'{ "serverProperties":{"logRetentionHours": 100 }}'

logRetentionCheckIntervalMs

integer

The interval to trigger log segments cleanup.

--options

'{ "serverProperties":{"logRetentionCheckIntervalMs": 1000 }}'

defaultReplicationFactor

integer

The default replication factor for cluster topics.

--options

'{"serverProperties":{"defaultReplicationFactor": 500 }}'

deleteTopic

boolean

To enable immediate deletion of topics marked for deletion.

--options

'{"serverProperties":{"deleteTopic": true }}'

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation config-update
--options '{"serverProperties":{...}}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation config-update --options '{"serverProperties":{"numPartitions":6, "deleteTopic": true}}'
--env sr001

PreviousElasticache Operations NextRDS Operations

Last updated 23 days ago