Linux performance Tuning

Performance Tuning is not just setting up parameters.Tuning is to achieve balance among different subsystems. Those sub systems are

1) CPU
2) Memory
3) Network
4) I/O

The subsystems are highly dependable on each other ,high resource usage of one subsystem cause degradation in other subsystem .

To identify root cause first we need to determine what type of application it is .The application stack of any system broken down in to two

1) IO Bound : These type of applications use high memory and less cpu.CPU is just useful to process IO rrequests.IO applications process large volumes of data,usually DB applications are cosider as IO bound applications.

2) CPU Bound : Heavy calculations and batch processing requires high cpu . High volume web servers and mail servers consider as CPU bound applications.

Example of Server stats taken by VM Stat.


In first one table cpu is idle most of the time and in second table cpu is not available. we can determine this by seeing last column (id - which represents idle time).

CPU utilization depends on what resources are attempting to access. The kernel has a scheduler for scheduling two kind of resources threads and interrupts. Below is the priority given for resources from highest to lowest.

1) Hardware Interrupts 2) Software Interrupts 3) Real Threads 4) Kernel Threads 5) User Threads

1) Context Switches : The kerenel can process from 50 to 50000 process threads.Based on priority kerenel change the process thread sequence. every time it change the sequence it is called context switches.

2) The run queue : The threads which are in queue for processing .The higher number of threads in waiting  the run queue is high and kernel takes long time to process those.

3) CPU utilization : Usage of the CPU.Most performance monitoring tools categorized cpu in following ways.

1) User Time : The percentage of time CPU spends  executing threads at user space.

2) System Time: The percentage of time CPU spends  executing kernel threads and interrupts.

3) Wait IO : The percentage of time CPU spends  idle all process threads are waiting IO request to complete.

4) Idle  : CPU is completely idle and all threads and interrupts are completed.

Time slicing : Time slicing is a numeric value which denotes how long a task can run until it is executed.Too long time slice cause the poor interactive performance , too short will cause wastage of resources as it requires context switching .

5) Priorities : Common type of algorithm used  is scheduling based algorithm.

CPU Performance Monitoring :

1) Run queues :  Run queue should not have more than 1-3 threads per processor.For dual core processor 6 threads in run queue are acceptable

2) CPU utilization:

           65 to 70% - user time , 30 -35 % system time , 0-5%  idle time

Best tool for linux monitoring are TOP,VMSTAT

Using VMSTAT:it is low overhead tool, it has to be placed in each server where we need to monitor.
it runs in two modes 1) Average 2) Sample mode

Sample mode runs for specific interval .