級Web服務集群)
1. 企業(yè)級Web服務集群部署概述在當今互聯(lián)網(wǎng)服務架構(gòu)中高可用、可擴展的Web服務集群已成為企業(yè)級應用的標配。通過Ansible這一自動化運維工具我們可以實現(xiàn)從單機部署到集群化管理的平滑過渡將Nginx、PHP和MySQL的部署過程標準化、自動化。這種方案特別適合需要快速部署、統(tǒng)一管理多臺服務器的場景比如電商大促期間的臨時擴容、開發(fā)測試環(huán)境的快速搭建或是生產(chǎn)環(huán)境的標準化部署。我曾在一次金融系統(tǒng)升級項目中用Ansible Playbook在2小時內(nèi)完成了原本需要2天的手工部署工作。這個Playbook不僅部署了NginxPHPMySQL的基礎(chǔ)環(huán)境還實現(xiàn)了配置文件的版本化管理、服務狀態(tài)的自動校驗以及部署后的基礎(chǔ)性能測試。這種效率提升在緊急故障恢復時尤為寶貴。2. 環(huán)境規(guī)劃與準備工作2.1 基礎(chǔ)設(shè)施規(guī)劃在開始編寫Playbook前我們需要明確集群架構(gòu)。一個典型的企業(yè)級Web集群包含2臺Nginx做負載均衡keepalived實現(xiàn)VIP高可用3臺PHP應用服務器根據(jù)業(yè)務壓力可橫向擴展1主2從的MySQL集群建議使用Galera Cluster或主從復制1臺Ansible控制機也可用開發(fā)者本地電腦提示生產(chǎn)環(huán)境建議將數(shù)據(jù)庫服務器與Web服務器物理隔離避免資源爭用。測試環(huán)境可以合并部署。2.2 Ansible基礎(chǔ)配置在控制機上安裝Ansible后需配置/etc/ansible/hosts文件定義服務器分組。以下是示例[loadbalancer] lb1 ansible_host192.168.1.101 lb2 ansible_host192.168.1.102 [webserver] web1 ansible_host192.168.1.111 web2 ansible_host192.168.1.112 web3 ansible_host192.168.1.113 [database] db1 ansible_host192.168.1.201 db2 ansible_host192.168.1.202 db3 ansible_host192.168.1.203測試連接性ansible all -m ping -u root2.3 系統(tǒng)統(tǒng)一化處理企業(yè)環(huán)境中服務器可能存在不同的初始狀態(tài)建議在Playbook開頭添加預處理任務- name: 基礎(chǔ)系統(tǒng)標準化 hosts: all tasks: - name: 配置統(tǒng)一yum源 copy: src: files/company.repo dest: /etc/yum.repos.d/ - name: 安裝基礎(chǔ)工具 yum: name: [vim,wget,net-tools,telnet,lsof,tree] state: present - name: 統(tǒng)一時間同步 yum: name: chrony state: present service: name: chronyd state: started enabled: yes3. Nginx集群部署實現(xiàn)3.1 Nginx安裝與基礎(chǔ)配置創(chuàng)建nginx.ymlPlaybook- name: 部署Nginx集群 hosts: loadbalancer vars: nginx_version: 1.25.3 worker_connections: 4096 tasks: - name: 安裝依賴 yum: name: [gcc,pcre-devel,openssl-devel,zlib-devel] state: present - name: 下載Nginx源碼包 get_url: url: http://nginx.org/download/nginx-{{nginx_version}}.tar.gz dest: /tmp/nginx-{{nginx_version}}.tar.gz - name: 解壓安裝 unarchive: src: /tmp/nginx-{{nginx_version}}.tar.gz dest: /tmp/ remote_src: yes register: nginx_src - name: 編譯安裝 command: | ./configure --prefix/usr/local/nginx \ --with-http_ssl_module \ --with-http_stub_status_module \ --with-threads \ --with-file-aio make make install args: chdir: /tmp/nginx-{{nginx_version}} - name: 創(chuàng)建系統(tǒng)服務文件 copy: content: | [Unit] Descriptionnginx service Afternetwork.target [Service] Typeforking ExecStart/usr/local/nginx/sbin/nginx ExecReload/usr/local/nginx/sbin/nginx -s reload ExecStop/usr/local/nginx/sbin/nginx -s quit PrivateTmptrue [Install] WantedBymulti-user.target dest: /usr/lib/systemd/system/nginx.service - name: 啟動并設(shè)置開機自啟 systemd: name: nginx state: started enabled: yes3.2 負載均衡配置在templates/nginx.conf.j2創(chuàng)建配置模板user nginx; worker_processes {{ansible_processor_vcpus}}; events { worker_connections {{worker_connections}}; } http { upstream backend { {% for host in groups[webserver] %} server {{hostvars[host].ansible_host}}:8000 weight1 max_fails3 fail_timeout30s; {% endfor %} } server { listen 80; server_name {{domain_name}}; location / { proxy_pass http://backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } location /nginx_status { stub_status on; access_log off; allow 127.0.0.1; deny all; } } }在Playbook中添加任務- name: 配置負載均衡 template: src: templates/nginx.conf.j2 dest: /usr/local/nginx/conf/nginx.conf notify: reload nginx handlers: - name: reload nginx systemd: name: nginx state: reloaded3.3 Keepalived高可用配置對于生產(chǎn)環(huán)境需要為Nginx負載均衡器配置VIP高可用- name: 配置Keepalived hosts: loadbalancer tasks: - name: 安裝keepalived yum: name: keepalived state: present - name: 配置MASTER節(jié)點 template: src: templates/keepalived-master.conf.j2 dest: /etc/keepalived/keepalived.conf when: inventory_hostname lb1 notify: restart keepalived - name: 配置BACKUP節(jié)點 template: src: templates/keepalived-backup.conf.j2 dest: /etc/keepalived/keepalived.conf when: inventory_hostname lb2 notify: restart keepalived handlers: - name: restart keepalived systemd: name: keepalived state: restarted4. PHP應用服務器部署4.1 PHP-FPM環(huán)境搭建創(chuàng)建php.ymlPlaybook- name: 部署PHP環(huán)境 hosts: webserver vars: php_version: 8.2 tasks: - name: 安裝EPEL倉庫 yum: name: epel-release state: present - name: 安裝Remi倉庫 yum: name: https://rpms.remirepo.net/enterprise/remi-release-7.rpm state: present - name: 啟用PHP模塊 command: yum-config-manager --enable remi-php{{php_version}} - name: 安裝PHP及擴展 yum: name: [php,php-fpm,php-mysqlnd,php-opcache,php-gd,php-mbstring,php-xml] state: present - name: 配置PHP-FPM template: src: templates/www.conf.j2 dest: /etc/php-fpm.d/www.conf notify: restart php-fpm - name: 啟動PHP-FPM systemd: name: php-fpm state: started enabled: yesPHP-FPM配置模板templates/www.conf.j2關(guān)鍵參數(shù)[www] user nginx group nginx listen 0.0.0.0:8000 listen.allowed_clients 127.0.0.1,{{ groups[loadbalancer] | map(extract, hostvars, ansible_host) | join(,) }} pm dynamic pm.max_children 50 pm.start_servers 5 pm.min_spare_servers 5 pm.max_spare_servers 10 pm.max_requests 5004.2 Nginx與PHP聯(lián)動配置在Web服務器上需要配置Nginx處理PHP請求- name: 配置Web服務器Nginx hosts: webserver tasks: - name: 創(chuàng)建網(wǎng)站目錄 file: path: /data/www state: directory owner: nginx group: nginx - name: 部署測試PHP文件 copy: content: ?php phpinfo(); ? dest: /data/www/index.php - name: 配置Nginx template: src: templates/webserver-nginx.conf.j2 dest: /usr/local/nginx/conf/nginx.conf notify: reload nginxWeb服務器Nginx配置模板示例server { listen 8000; server_name localhost; root /data/www; location / { index index.php index.html; } location ~ \.php$ { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; } }5. MySQL集群部署5.1 MySQL主從復制配置對于中小規(guī)模集群可采用主從復制方案- name: 部署MySQL主庫 hosts: db1 tasks: - name: 安裝MySQL yum: name: [mysql-community-server,mysql-community-client] state: present - name: 啟動MySQL systemd: name: mysqld state: started enabled: yes - name: 設(shè)置root密碼 mysql_user: name: root password: {{mysql_root_password}} host: localhost priv: *.*:ALL,GRANT - name: 創(chuàng)建復制用戶 mysql_user: name: repl password: {{mysql_repl_password}} host: % priv: *.*:REPLICATION SLAVE - name: 配置主庫my.cnf template: src: templates/my-master.cnf.j2 dest: /etc/my.cnf notify: restart mysql - name: 部署MySQL從庫 hosts: db2,db3 tasks: - name: 安裝MySQL yum: name: [mysql-community-server,mysql-community-client] state: present - name: 配置從庫my.cnf template: src: templates/my-slave.cnf.j2 dest: /etc/my.cnf notify: restart mysql - name: 設(shè)置復制 mysql_replication: mode: changemaster master_host: {{hostvars[db1].ansible_host}} master_user: repl master_password: {{mysql_repl_password}} master_log_file: {{master_status.File}} master_log_pos: {{master_status.Position}} vars: master_status: {{ lookup(mysql, SHOW MASTER STATUS, wantlistTrue)[0] }} notify: start slave handlers: - name: restart mysql systemd: name: mysqld state: restarted - name: start slave mysql_replication: mode: startslave5.2 數(shù)據(jù)庫初始化建議將數(shù)據(jù)庫初始化腳本納入版本控制- name: 初始化業(yè)務數(shù)據(jù)庫 hosts: db1 tasks: - name: 上傳SQL腳本 copy: src: files/init_db.sql dest: /tmp/ - name: 執(zhí)行初始化 mysql_db: name: {{item}} state: import target: /tmp/init_db.sql with_items: - app_db - report_db6. 集群驗證與調(diào)優(yōu)6.1 服務連通性測試創(chuàng)建驗證Playbookverify.yml- name: 驗證集群狀態(tài) hosts: localhost connection: local tasks: - name: 測試負載均衡 uri: url: http://{{vip_address}}/index.php return_content: yes register: web_test - name: 檢查PHP信息頁 assert: that: - PHP Version in web_test.content - name: 測試數(shù)據(jù)庫連接 mysql_query: login_host: {{hostvars[db1].ansible_host}} login_user: root login_password: {{mysql_root_password}} query: SHOW DATABASES register: db_test - name: 驗證數(shù)據(jù)庫 assert: that: - app_db in db_test.results[0]6.2 性能調(diào)優(yōu)建議根據(jù)實際業(yè)務需求調(diào)整以下參數(shù)Nginx調(diào)優(yōu)worker_processes: CPU核心數(shù)worker_connections: 根據(jù)內(nèi)存調(diào)整每個連接約占用256KBkeepalive_timeout: 長連接超時時間PHP-FPM調(diào)優(yōu)pm.max_children (總內(nèi)存 - 系統(tǒng)預留) / 單個PHP進程內(nèi)存 pm.start_servers CPU核心數(shù) × 2 pm.max_spare_servers CPU核心數(shù) × 4MySQL調(diào)優(yōu)innodb_buffer_pool_size 總內(nèi)存的50-70% innodb_log_file_size buffer_pool_size的25% max_connections 根據(jù)應用需求設(shè)置7. 生產(chǎn)環(huán)境注意事項安全加固為MySQL root賬戶設(shè)置強密碼并限制訪問IPNginx關(guān)閉server_tokens顯示版本信息PHP禁用危險函數(shù)exec, system等監(jiān)控方案Nginx: 通過stub_status模塊收集指標PHP-FPM: 啟用status頁面MySQL: 部署Prometheus exporter備份策略- name: MySQL每日備份 cron: name: MySQL backup minute: 0 hour: 2 job: mysqldump -uroot -p{{mysql_root_password}} --all-databases | gzip /backup/mysql-$(date \%Y\%m\%d).sql.gz灰度發(fā)布方案通過Ansible的--limit參數(shù)分批發(fā)布在負載均衡器上配置健康檢查部署前自動創(chuàng)建數(shù)據(jù)庫備份在實際企業(yè)部署中我通常會先在一個測試節(jié)點完整運行Playbook確認無誤后再通過--limit參數(shù)分批部署到生產(chǎn)環(huán)境。對于配置變更使用--tags功能只執(zhí)行相關(guān)任務減少影響范圍。