分类目录归档：Linux

beanstalkd 安装和配置

安装
安装以centos为例

yum  install beanstalkd

配置

使用centos yum安装，通过查看服务脚本发现有这个配置文件

cat /etc/sysconfig/beanstalkd

主要修改几个地方
ADDR=-l 10.1.3.16 # 修改为内网或本地地址，否则会开放外网非常危险
PORT=-p 11111 # 修改下端口
BINLOG_DIR=-b /var/lib/beanstalkd/binlog # 启用二进制日志防止意外断电或重启导致数据丢失

 

客户端连接问题：

Connection error 113: No route to host

解决办法：防火墙开放对应端口

性能：

$start = microtime(true);
for ($i = 0; $i < 100000; $i ++) {
    $beansClient->useTube('myAwesomeTube')->put("哈哈哈job's payload呵呵呵呵");
}
echo microtime(true) - $start, "\n";

插入一万条数据一秒(本地连接)

插入十万条数据十秒(本地连接)

插入一万条数据14秒(内网连接)

本地取出1w条数据要1.5秒

内网取出1w条数据要33秒

本地取出 1173359条数据 要 200秒

结论：吃内存，快还是慢你觉得呢？

upstream timed out (110: Connection timed out) while reading upstream

2018/04/26 23:34:39 [error] 10872#0: *268906 upstream timed out (110: Connection timed out) while reading upstream, client: 119.103.223.107, server: ****, request: “GET /**** HTTP/1.1”, upstream: “****”, host: “****”

#fastcgi_connect_timeout 600;
#fastcgi_read_timeout 600;
#fastcgi_send_timeout 600;
proxy_connect_timeout 600;
proxy_read_timeout 600;
proxy_send_timeout 600;

upstream timed out (110: Connection timed out) while reading upstream,

upstream timed out (110: Connection timed out) while reading response header from upstream

(24: Too many open files)

2018/04/26 23:32:58 [crit] 1457#0: *259351 open() “/usr/share/nginx/html/50x.html” failed (24: Too many open files), client: 112.17.247.117, server: ***, request: “GET /*** HTTP/1.1”, upstream: “***”, host: “***”

worker_rlimit_nofile 65535;

while read line无法读取最后一行的问题

while read line读取文件时，如果文件最后一行之后没有换行符\n,则read读取最后一行时遇到文件结束符EOF,循环终止,虽然此时$line内存有最后一行,但程序已经没有机会再处理此行,因此可以通过以下代码来解决此问题:

while read line || [[ -n ${line} ]]; do
…
done
这样当文件没有结束时不会测试-n $line,当遇到文件结束时,仍然可以通过测试$line是否有内容来进行继续处理。

http://ju.outofmemory.cn/entry/86710

MySQL自动备份到亚马逊S3


#!/bin/sh

# Updates etc at: https://github.com/woxxy/MySQL-backup-to-Amazon-S3
# Under a MIT license

# change these variables to what you need
MYSQLROOT=XXX
MYSQLPASS=XXXX
S3BUCKET=xx-mysql-backup
# DATABASE='--all-databases'
DATABASE=${1-default}
FILENAME=${DATABASE}
HOST=master
DUMPOPTION='--quick --single-transaction'
# the following line prefixes the backups with the defined directory. it must be blank or end with a /
S3PATH=mysql-backup/${DATABASE}
# when running via cron, the PATHs MIGHT be different. If you have a custom/manual MYSQL install, you should set this manually like MYSQLDUMPPATH=/usr/local/mysql/bin/
MYSQLDUMPPATH=
#tmp path.
TMP_PATH=/tmp/

DATESTAMP=$(date +".%Y-%m-%d.%H:%M:%S")
DELDATESTAMP=$(date -d"15 day ago 2017-04-16" +".%Y-%m-%d.%H:%M:%S")
DAY=$(date +"%d")
DAYOFWEEK=$(date +"%A")

PERIOD=${2-day}
if [ ${PERIOD} = "auto" ]; then
if [ ${DAY} = "01" ]; then
PERIOD=month
elif [ ${DAYOFWEEK} = "Sunday" ]; then
PERIOD=week
else
PERIOD=day
fi
fi

echo "Selected period: $PERIOD."

echo "Starting backing up the database to a .gz file..."

# dump all databases
${MYSQLDUMPPATH}mysqldump -h${HOST} ${DUMPOPTION} --user=${MYSQLROOT} --password=${MYSQLPASS} ${DATABASE} | gzip > ${TMP_PATH}${FILENAME}.gz

echo "Done backing up the database to a file."
# echo "Starting compression..."

# tar czf ${TMP_PATH}${FILENAME}${DATESTAMP}.tar.gz ${TMP_PATH}${FILENAME}.sql
mv ${TMP_PATH}${FILENAME}.gz ${TMP_PATH}${FILENAME}${DATESTAMP}.gz

# echo "Done compressing the backup file."

# upload all databases
echo "Uploading the new backup..."
s3cmd put -f --check-md5 -s --continue-put ${TMP_PATH}${FILENAME}${DATESTAMP}.gz s3://${S3BUCKET}/${S3PATH}${PERIOD}/
echo "New backup uploaded."
if [ $? -ne 0 ]
then
echo "Re uploading the backup file..."
s3cmd put -f --check-md5 -s --continue-put ${TMP_PATH}${FILENAME}${DATESTAMP}.gz s3://${S3BUCKET}/${S3PATH}${PERIOD}/
echo "Re upload backup file done."
fi

echo "Moving the backup from past $PERIOD to another folder..."
s3cmd mv --recursive s3://${S3BUCKET}/${S3PATH}${PERIOD}/${FILENAME}${DELDATESTAMP} s3://${S3BUCKET}/${S3PATH}previous_${PERIOD}/
echo "Past backup moved."

# we want at least two backups, two months, two weeks, and two days
echo "Removing old backup (15 ${PERIOD}s ago)..."
s3cmd del --recursive s3://${S3BUCKET}/${S3PATH}previous_${PERIOD}/
echo "Old backup removed."

# remove databases dump
# rm ${TMP_PATH}${FILENAME}.sql
echo "Removing the gz files..."
rm ${TMP_PATH}${FILENAME}${DATESTAMP}.gz
echo "Files removed."
echo "All done."

nginx: [emerg] unknown directive “proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for” in

$remote_addr 可信时才能信任 X-Forwarded-For、$proxy_add_x_forwarded_for

nginx配置代理转发，部分配置如下

        location / {
            # index index.html index.htm index.php;
            # try_files $uri $uri/ /index.php$request_uri;
            proxy_set_header Host $host;
            #proxy_set_header X-Real-IP $remote_addr;
            #proxy_set_header X-Real-Port $remote_port;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass https://10.29.185.61:8063;
        }

$proxy_add_x_forwarded_forthe “X-Forwarded-For” client request header field with the $remote_addr variable appended to it, separated by a comma. If the “X-Forwarded-For” field is not present in the client request header, the $proxy_add_x_forwarded_for variable is equal to the $remote_addr variable.

“ X-Forward-For”客户端请求头字段，后面附加 $remote _ addr 变量，中间用逗号分隔。如果客户端请求头中没有“ X-Forward-For”字段，则 $proxy_add_x_forwarded_for 变量等于 $remote_addr 变量。