Puppet uses Nginx multi-port to achieve load balancing

As the company’s application requirements increase, continuous expansion is required, and the number of servers also increases. When the number of servers continues to increase, we will find that a puppetmaster is under great pressure, slow in parsing, and “time
out” and other errors, is there any way to optimize it at this time? We searched for solutions on the Puppet official website and found that puppetmaster can be configured with multiple ports, combined with WEB proxy (Nginx is recommended), so that the puppetmaster’s capacity can be increased by at least several times The above is equivalent to optimizing the processing capabilities of puppet to a large extent.

1. Following the previous environment settings, our server environment and software version here are:

Server system: CentOS5.8 x86_64

Ruby version: ruby-1.8.5

Puppet version: puppet-2.7.9

Nginx version: nginx-0.8.46

2. Mongrel installation

To use puppet multi-port configuration, you need to specify the mongrel type, which is not installed by default and needs to be installed:

yum install -y rubygem-mongrel

3. Configure puppet master

Add the following two lines at the end of the /etc/sysconfig/puppetmaster file, representing multi-port and mongrel types respectively, and the contents are as follows:

PUPPETMASTER_PORTS=(8141 8142 8143 8144 8145)
PUPPETMASTER_EXTRA_OPTS=”–servertype=mongrel –ssl_client_header=HTTP_X_SSL_SUBJECT”

4. Install Nginx service

Before installation, please ensure that the pcre-devel regular library has been installed in the system, and then compile and install Nginx. SSL module parameter support needs to be added. The installation process of Nginx is as follows:

yum -y install pcre-devel
cd /usr/local/src
wget http://nginx.org/download/nginx-0.8.46.tar.gz
tar zxvf nginx-0.8.46.tar.gz
cd nginx-0.8.46
./configure –prefix=/usr/local/nginx –with-http_ssl_module
make && make install && cd ../

Add the www user group and users, the command is as follows:

groupadd www
useradd -g www www

5. We modify the configuration file nginx.conf according to the requirements of puppet. The content is as follows:

user www;

worker_processes 8;

events {

worker_connections 65535;

}

http {

include mime.types;

default_type application/octet-stream;

sendfile on;

tcp_nopush on;

keepalive_timeout 65;

#Define puppet client access puppet-server log format

log_format main ‘$remote_addr – $remote_user [$time_local] “$request” $request_length $request_time $time_local’

‘$status $body_bytes_sent $bytes_sent $connection $msec “$http_referer”‘

‘”$http_user_agent” $http_x_forwarded_for $upstream_response_time $upstream_addr $upstream_status’;

access_log /usr/local/nginx/logs/access.log main;

upstream puppetmaster {

server 127.0.0.1:8141;

server 127.0.0.1:8142;

server 127.0.0.1:8143;

server 127.0.0.1:8144;

server 127.0.0.1:8145;

}

server {

listen 8140;

root /etc/puppet;

ssl on;

ssl_session_timeout 5m;

#The following is the puppetmaster server-side certificate address

ssl_certificate /var/lib/puppet/ssl/certs/server.cn7788.com.pem;

ssl_certificate_key /var/lib/puppet/ssl/private_keys/server.cn7788.com.pem;

ssl_client_certificate /var/lib/puppet/ssl/ca/ca_crt.pem;

ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;

ssl_verify_client optional;

#File sections

location /production/file_content/files/ {

types { }

default_type application/x-raw;

#Define puppet push path alias

alias /etc/puppet/files/;

}

# Modules files sections

location ~ /production/file_content/modules/.+/ {

root /etc/puppet/modules;

types { }

default_type application/x-raw;

rewrite ^/production/file_content/modules/(.+)/(.+)$ /$1/files/$2 break;

}

location / {

## set jump�� to puppetmasterload balancing

proxy_pass http://puppetmaster;

proxy_redirect off;

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Client-Verify $ssl_client_verify;

proxy_set_header X-SSL-Subject $ssl_client_s_dn;

proxy_set_header X-SSL-Issuer $ssl_client_i_dn;

proxy_buffer_size 10m;

proxy_buffers 1024 10m;

proxy_busy_buffers_size 10m;

proxy_temp_file_write_size 10m;

proxy_read_timeout 120;

}

}

}

6. After modifying the nginx.conf file, we need to start nginx and puppet-server, how should we operate at this time?

1. We first shut down the puppetmaster process, and then start nginx first, otherwise nginx will fail to start. The command is as follows:

/usr/local/nginx/sbin/nginx

After nginx occupies the default port 8140 of puppetmaster, we can use the following command to check whether port 8140 is taken over by nginx, as follows:

lsof -i:8140

This command shows the result that 8140 is taken over by the nginx process as follows:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
nginx 4121 root 6u IPv4 20668 0t0 TCP *:8140 (LISTEN)
nginx 4122 www 6u IPv4 20668 0t0 TCP *:8140 (LISTEN)

Let’s start puppetmaster again, the command is as follows:

service puppetmaster start

If the ruby ​​version is 1.8.5, there will be the following warning after running puppetmaster, as follows:

The meaning of this warning value is:

It’s just a warning. Mongrel wants a Ruby version of at least
1.8.6.

But it still runs just fine with previous versions. Just ignore
the warning.

Translated into Chinese means:

Mongrel requires at least ruby ​​version 1.8.6 or later, but it still runs on the current version, please ignore the current warning. In order to ensure the stability of the entire puppet operating environment, I choose to use the 1.8.5 version of ruby ​​here.