Kafka Operations

Available Operations

1. Scale Cluster

2. Restart Cluster

3. Replace Node

4. Pause

5. Resume

Scale Cluster

What is it?

It enables user to modify the capacity of the cluster by changing the count of instances that are already part of the cluster. User can also modify the instance type of the cluster. The scaling operation supports both vertical & horizontal scale up and scale down.

When to use?

This operation can be run only when the cluster is in running state or when the previous operation is in success status. User need to wait if any existing operation is already running on the same cluster and can check its status via odin provided Jenkins.

Limitations

1. Scale operation cannot be stopped.

2. If the previous operation on the cluster was unsuccessful , then user would not be able to scale cluster.

How to use?

Insert the desired instance count for the cluster in desired_instance_count . Please note that you need to provide the number of instances that you require at the end state. E.g: If yo u have 5 instances and you want to add 5 more, the value of desired_instance_count will be 10 as the same represents end state.

Options

Option	Type	Description	Example
desired_instance_count	integer	The no. of instances required by the user at the end state.	--options '{ "desired_instance_count": 12, "instance_type": "m5.4xlarge"}'
instance_type	text	The instance type to be provided here.	--options '{ "desired_instance_count": 12, "instance_type": "m5.4xlarge"}'

1. If the user wants to scale up horizontally, user can pass only desired_instance_count in the options.

2. Horizontal scale up operation will NOT rebalance old topic partitions on newer nodes. If the data needs to be rebalanced across newer nodes, the user needs to trigger rebalancing.

3. If the user wants to scale vertically, user can pass only instance_type in the options.

When scaling a cluster, if the user specifies a different instance type than the current one, all existing instances in the cluster will also be changed to the new instance type.

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation scale 
--options '{"desired_instance_count": NO_OF_INSTANCE_AT_END_STATE,
"instance_type": "DESIRED_INSTANCE_TYPE"}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation scale
--options '{"desired_instance_count": 12,
"instance_type": "m5.4xlarge"}'
--env myEnv

Restart Cluster

What is it?

This operation helps the users to restart a cluster.

When to use?

This operation can be run only when the cluster is in running state or when an existing operation is success . User need to wait if any existing operation is already running on the same cluster and can check its status via odin provided Jenkins.

Option

Option	Type	Description	Example
nodes	list	The list of IPs that needs to be restarted to be provided in comma separated format . If no list is provided, all nodes will be restarted.	--options '{ "nodes": ["10.10.10.1"] }'

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation restart 
--options '{"nodes": ["IP_1","IP_2"]}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation restart 
--options '{"nodes": ["10.10.10.1","168.122.90.1"]}'
--env sr001

Replace Node

What is it?

Replace node replaces the node that has concerns (such as hardware issues, disk corruption or any AWS event related to h/w issues) with the node of the same instance type.

When to use?

This operation can be run only when the cluster is in running state or when an existing operation is success . User need to wait if any existing operation is already running on the same cluster and can check its status via odin provided Jenkins.

Option

Option	Type	Description	Example
nodes	list	The list of IPs that needs to be replaced to be provided in comma separated format .	--options '{ "nodes": ["10.10.10.1"] }'

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation replace_node 
--options '{"nodes": ["IP_1","IP_2"]}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation replace_node 
--options '{"nodes": ["10.10.10.1","168.122.90.1"]}'
--env sr001

Pause

What is it?

This operation enables users to pause a cluster. This can be used to optimise cost, especially in load environment during idle hours. Post pause operation, all the nodes of the specific cluster will be in 'stopped' state.

When to use?

This operation can be run only when the cluster is in running state or when an existing operation is success . User need to wait if any existing operation is already running on the same cluster and can check its status via odin provided Jenkins.

Option

This command does not take any input in the form of a option. A blank json to be passed for options here.

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation pause 
--options '{}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation pause --options '{}'
--env sr001

Resume

What is it?

This operation enables users to resume the normal operation of an already paused cluster.

When to use?

This operation can be run only when the cluster is in successful pause state.

Option

This command does not take any input in the form of a option. A blank json to be passed for options here.

Command

odin operate component --name [YOUR_COMPONENT_NAME] --service [YOUR_SERVICE_NAME]
--operation resume 
--options '{}'
--env [YOUR_ENV_NAME]

Example

odin operate component --name deadpool-kafka --service deadpool-service
--operation resume --options '{}'
--env sr001