How to delete a Pods from a Kubernetes Node

 How to delete a Pods from a Kubernetes Node

when Managing the kubernetes cluster, you may situation to delete pods from one of your nodes.

 There are specific steps to minimise disruption of your application

First, confirm the pod located node using kubectl get nodes command.

Node should configured as unscheduled by using kubectl cordon command. This ensure that no new pods will get scheduled to the node while you are preparing it for removal or maintenance.

Now you can work on moving pods off of the node. For any pods that are controlled by a Deployment or ReplicaSet (not a StatefulSet), you can increase the replica count by the number of pods that are about to be moved off of the node and then manually delete the pods that were running on the node. If you are not concerned with having one less replica at a time, you can skip the scaling step and just delete the necessary pods one at a time.

For example, if you had a Deployment called “web” with 3 replicas, and 1 of them is running on the node that will be deleted, and it is not acceptable to only run 2 replicas at any time, you would scale the Deployment up to 4 replicas, wait for the new pod to become Ready, and then delete the old pod using kubectl delete pod.  You can then scale back down to 3 replicas to get back to your original configuration.

Once you have finished maintenance on a node, you can use the kubectl uncordon command to allow scheduling on the node again. Then, as pods need to be scheduled, they will appear back on that node.

Add Disk-shelf in Netapp

1)physically mount the disk shelf in filer rack.
2)break the cluster between head01 and head02 using "cf disable" command
3)power down the head02 filer using "halt -t" command
4)set the shelf id in new disk shelf and set the switch to 2GB position.also connect the cable in the backside of the disk shelf.
5) First,Turn on the all the disk shelf, After 3 minutes, Turn on the filer head.
6)log in to the filer and check the filer ststus using "sysconfig " command,
7) add those disk in exsiting aggr and we keep 2 or 3 disk as a spare disk.
8) enable the cluster between head01 using "cf enable " command.
9)check the cluster status using "cf status" command.

Netapp to VFiler Migration Steps

Filer Migration steps: 

Ø  Validate Back ups and Snaps for DR purposes

Ø  DNS Validation an hour before the migration

Ø   Run cifs shares on the host filer.

Ø  Copy cifsconfig_share.cfg from filer/etc/ to local of the machine and send this attachment to  who is performing this activity.

Ø  Copy “quota” and “exports” files to a server from host filer “etc” folder. Save the whole ‘c: \etc’ folder on a server if possible.

Ø  Stop cifs shares on source filer, type “cifs terminate” cmd

Ø  From vFiler: run the last snapmirror for each qtree

Ø  From vFiler: Break the snapmirror for each qtree

Ø  On source filer: break any legacy (DR) snapmirror(s)

Ø  Disable the cluster

Ø   From source filer: run “setup” cmd to rename the source filer to “–old” as suffix

Ø   Wait for 15 minutes until the AD replication is done.

Ø  From vFiler: run “setup” cmd to rename the destination filer to the same as source filer Login to nearest Domain Controller to change the DNS records.

Follow the below steps to change the DNS records:

Ø  Login to nearest DC.

Ø  Go to Administrator Tools  DNS  on right pane expand the domain controller  click on the domain.

Ø  Search for the filer IP address and delete that record.

Ø  Right click on domain  click on New Host (A)

Ø  Add the record with new   hostname and new IP address.

Ø  For each vfiler: Log onto MMC and create cifs shares for each qtree

Ø  Verify the share permission and security permission for each share on vfiler with CIFS shares backed up on old filer

Ø  Rename the quota file on vFiler, Copy the quota file from the source to vFiler and edit the path of qtree, e.g. /vol/data/qtree –actual=/vol/vfiler-name/qtree

Ø  From vFiler: to initialize quota, run “quota off ” and “quota on ” amd

Ø  For home directory filers only: Configure cifs_homedir.cfg on vFiler with the home directory path, e.g. /vol/vFiler-name/users

Ø  For home directory filers only: Run “cifs homedir load” cmd on vFiler

Ø  Rename the exports file on vFiler, Copy the exports file from the source to vFiler and edit the path of qtree or volume, e.g. /vol/data/qtree –actual=/vol/vfiler-name/qtree

Ø  Run exportfs –a and than run exportfs.

Ø  Update the DNS for the Host Filer and vfiler.

Ø  Client validation – test to make sure vFiler shares are accessible

Ø  Create an ontrack backup request for the new vfiler.

Ø  Inform management and helpdesk the status of the migration

Filer Back out Steps TimeLine: Within filer migration timeline:        

Ø  DNS Validation (AIM = NetNOCIA), Change DNS back to original filer name

Ø  Stop cifs shares on vFiler

Ø  From vFiler: run “setup” cmd to rename the source filer to “–a” as suffix

Ø  On the original filer, run “setup” cmd to rename the name to the same as original filer

Ø  Wait 15 minutes until the AD replication is done

Ø  Client validation

Ø  Inform management and helpdesk the status of the migration

Data Domain Commands

system
Commands matching "system":
system option reset {login-banner} -Reset (to default) system options
system option set {login-banner } -Set login banner file
system option show - Show system options
system poweroff - Shutdown and turn off the power
system reboot -Shutdown and restart the server
system set date Set the system clock
system show all - Show all the system information
system show date -Show the system clock
system show detailed-stats [stop start ([interval ] [count] )] -> Report more detailed system statistics
system show detailed-version - Show version and release information
system show hardware - Show hardware information
system show modelno - Show the hardware model number
system show nvram - Show system nvram
system show performance [raw fsop] [ {hr min sec} [ {hr min sec}]] -> Report system performance summary
system show ports - Show port information
system show serialno - Show the hardware serial number
system show stats [stop start ([interval] [count] )] -> Report system statistics
system show uptime - Show the time since a reboot
system show version - Show the system version
system status - Show system hardware status
system upgrade - Upgrade the software from the specified in the /ddvar/releases directory

Commands matching "disk":
disk add enclosure Add a disk or an enclosure of disks to expand the filesystem
disk beacon . - Flash a disk identification LED
disk expand - Expand DD510 and DD530 storage capacity
disk fail . - Fail a disk and force reconstruction
disk multipath option show - Show multipath options
disk multipath option set monitor {enabled disabled} - Enable or disable multipath monitoring
disk multipath option reset monitor -Reset multipath monitoring
disk multipath reset stats - Clear statistics for all paths of all disks
disk multipath show history - Show multi-path events history
disk multipath show stats - Show statistics for all paths of all disks
disk multipath status [] - Show multi-path configurations and run time status
disk port show {stats summary} - Show disk port information
disk rescan - Rescan all disks to look for newly removed/inserted disks or luns
disk reset performance - Clear the disk performance statistics
disk show detailed-raid-info - Show detailed disk RAID configuration
disk show failure-history - Show disk failure history
disk show hardware - Show disk hardware
disk show performance - Show disk performance statistics
disk show raid-info -Show RAID configuration
disk show reliability-data - Show disk reliability information
disk status - Show disk status
disk unfail . - Unfail a disk and spare it