Thank you! Your feedback has been delivered
Thank you! Your feedback has been sent

Ubuntu 12.04 Server config optimization advice

Hi everybody.

I ran into some CPU consumption issues. Need an advice for server configuration.

Below is my current server configuration. Right now the server's CPU usage is around 300%-320% I would like to lower it to 200%. I have 25-30k page views per day.

In near future I'll migrate to Ubuntu 14.04

Switch from apache+passenger to nginx+unicorn

Rails to 4.1.x

Until then, I need some tuning to reach best performance.

I suppose there's a lot that can be done to reduce the load of my sever just with changing the configuration, but I totally don't understand how to properly calculate with my hardware and environment

Thank you for your help!

My current stack:

Hosted on linode 6GB plan with 4 cpu cores
Ubuntu 12.04.4 LTS (GNU/Linux 3.14.1-x86_64-linode39 x86_64)

Mysql as mariadb-5.5.33a-linux-i686
Ruby 2.1.2p95-github (development) [x86_64-linux]
Rails 4.0.5
Memcached (dalli) - uses 1gb of ram
Passenger 4.0.42 as apache module

Sphinx 2.1.4-release (rel21-r4421)
**For Sphinx search I'm using both RT and Delta index. With default memory usage config.
workers = threads**

apachectl -V
Server version: Apache/2.2.22 (Ubuntu)
Server built:   Mar 19 2014 21:10:46
Server's Module Magic Number: 20051115:30
Server loaded:  APR 1.4.6, APR-Util 1.3.12
Compiled using: APR 1.4.6, APR-Util 1.3.12
Architecture:   32-bit
Server MPM:     Worker
  threaded:     yes (fixed thread count)
    forked:     yes (variable process count)

PassengerMaxPoolSize 10

<IfModule mpm_worker_module>
    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadLimit          64
    ThreadsPerChild      30
    MaxClients          150
    MaxRequestsPerChild   3000
</IfModule>


MYSQL config
###
# MyISAM #
key-buffer-size                = 32M
myisam-recover                 = FORCE,BACKUP

# SAFETY #
max-allowed-packet             = 16M
max-connect-errors             = 1000000

# DATA STORAGE #
#datadir                        = /usr/local/mysql/

# BINARY LOGGING #
#log-bin                        = /usr/local/mysql/mysql-bin
#expire-logs-days               = 14
#sync-binlog                    = 1

# CACHES AND LIMITS #
tmp-table-size                 = 32M
max-heap-table-size            = 32M
query-cache-type               = 0
query-cache-size               = 0
max-connections                = 500
thread-cache-size              = 50
open-files-limit               = 65535
table-definition-cache         = 4096
table-open-cache               = 4096

# INNODB #
innodb-flush-method            = O_DIRECT
innodb-log-files-in-group      = 2
innodb-log-file-size           = 64M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table          = 1
innodb-buffer-pool-size        = 592M
### End of mysql config


uptime
15:22:02 up 19 days, 12:50,  2 users,  load average: 4.85, 5.15, 5.18

top - 15:25:54 up 19 days, 12:54,  1 user,  load average: 4.18, 4.60, 4.95
Tasks: 211 total,   1 running, 210 sleeping,   0 stopped,   0 zombie
Cpu(s): 71.3%us,  3.1%sy,  0.0%ni, 25.1%id,  0.0%wa,  0.0%hi,  0.2%si,  0.3%st
Mem:   6152748k total,  5724320k used,   428428k free,   234604k buffers
Swap:   262140k total,    42328k used,   219812k free,  1657704k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
7741 deployer  20   0  109m  64m  11m S   49  1.1   9938:00 searchd
2978 mysql     20   0 2041m 852m 6372 S   43 14.2   2671:41 mysqld
13380 deployer  20   0  136m 109m 3804 S   40  1.8   6:24.64 ruby
13355 deployer  20   0  137m 109m 3720 S   29  1.8   6:15.88 ruby
13458 deployer  20   0  137m 109m 3740 S   27  1.8   6:26.07 ruby
13411 deployer  20   0  137m 109m 3816 S   23  1.8   5:45.98 ruby
13518 deployer  20   0  138m 109m 3788 S   15  1.8   6:16.41 ruby
13266 deployer  20   0  136m 109m 3804 S   14  1.8   5:37.68 ruby
13949 u1003     20   0  114m  25m  20m S    8  0.4   0:27.39 php5
13444 deployer  20   0  138m 109m 3760 S    7  1.8   4:28.21 ruby
13499 deployer  20   0  138m 109m 3796 S    7  1.8   4:52.35 ruby
13275 deployer  20   0  137m 109m 3756 S    6  1.8   4:13.02 ruby
13334 deployer  20   0  135m 108m 3724 S    6  1.8   4:50.12 ruby
13366 deployer  20   0  139m 110m 3728 S    6  1.8   5:12.73 ruby
13483 deployer  20   0  139m 109m 3804 S    5  1.8   5:14.59 ruby
1210 www-data  20   0  280m  11m 1996 S    2  0.2   0:08.26 apache2
3638 www-data  20   0  279m  10m 2000 S    2  0.2   0:03.44 apache2
15201 memcache  20   0  600m 556m  868 S    2  9.3   3:19.70 memcached
2928 www-data  20   0  280m  11m 1996 S    2  0.2   0:05.10 apache2
13294 deployer  20   0  137m 109m 3816 S    1  1.8   6:02.40 ruby
13344 deployer  20   0  136m 108m 3716 S    1  1.8   5:29.71 ruby
12464 root      20   0 44868 4556 2716 S    1  0.1   2:08.12 PassengerHelper
13674 u1003     20   0  116m  29m  22m S    1  0.5   0:44.37 php5
    7 root      20   0     0    0    0 S    0  0.0  36:31.73 rcu_sched
2384 root      20   0  6684 2004 1888 S    0  0.0   0:13.76 sshd
5532 root      20   0  2852 1212  884 R    0  0.0   0:00.06 top
5653 root      20   0  8292 2840 2260 S    0  0.0   0:00.01 sshd
13426 deployer  20   0  138m 109m 3724 S    0  1.8   5:48.60 ruby
25377 deployer  20   0 50396  32m  11m S    0  0.5   1:58.64 searchd
    1 root      20   0  3512 1512 1056 S    0  0.0   0:04.24 init
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
User Gravatar

Paprikas

Posted May 27 2014 2:45 UTC

$30


  • Expired
  • ruby
    rails
    performance
  • 1218 Views

8 Replies


For the MySQL/Maria part, you can try using these tuners:

They should give you an initial idea of what things can be tweak on your database configuration.

You could also try enabling the Slow Query Log temporarily to locate queries that can be optimized.

User Gravatar

yorch

Posted May 27 2014 4:50 UTC

@yorch, thank you for replying, but I'm looking for more complex solution. Not just info like "do it by yourself".

I'm here for getting complete solution.

Anyway, thanks for the info.

User Gravatar

Paprikas

Posted May 27 2014 6:00 UTC

Caching queries for 3 minutes redused cpu usage to 220-250%. But I'm still believe that my configuration needs adjustments.

User Gravatar

Paprikas

Posted May 27 2014 6:32 UTC

Unfortunately there is no one size fit all configuration values for databases, they need to be continuously tweaked according to your application and stack specific needs. Those two tools are a great way do it.

User Gravatar

yorch

Posted May 27 2014 7:21 UTC

As you showed, the main applications that are consuming the CPU are Sphinx, MySQL and Ruby/Rails; Apache is not something to worry in your specific case and goal (reduce CPU).

So, if you want to keep reducing your CPU usage, you will have to start optimizing at the application level (Rails and Sphinx), not just your configuration files.

User Gravatar

yorch

Posted May 27 2014 7:37 UTC

query_cache_type=OPTION

0 : Don't cache results in or retrieve results from the query cache.

1 : Cache all query results except for those that begin with SELECT S_NO_CACHE.

2 : Cache results only for queries that begin with SELECT SQL_CACHE



Don't cache? If you disable caching, mysql (mariasql) would consume many resources.
To optimize database and search, you need to check if you indexed database and sphinx well. And also check if queries use that indexes.



When cpu usage from 'Ruby' is high, it means many client or inefficient code.

User Gravatar

ceram1

Posted May 30 2014 16:47 UTC

2ceram1, I have sensitive data, so I don't use Mysql caching. I'm using cache in my backend. So if configs is ok - need to optimize sql queries instead.

User Gravatar

Paprikas

Posted Jun 1 2014 13:44 UTC

You can use somthing like SQL_NO_CACHE instead of disabling whole cache.


And if you want to optimize query, https://tools.percona.com/query-advisor this may help you

User Gravatar

ceram1

Posted Jun 1 2014 17:24 UTC

Add a reply

By posting a reply on CodersClan you agree to our Terms & Conditions