月度归档:2017年08月

ELK的启动和停止脚本

#!/bin/sh

path_base=`pwd`
path_elastic="$path_base/elasticsearch-5.5.1/bin/elasticsearch"
path_kibana="$path_base/kibana-5.5.1/bin/kibana"
path_logstash="$path_base/logstash-5.5.1/bin/logstash"

mode=$1

kibana_process=`ps -ef | grep "kibana"| grep -v grep`
elastic_process=`ps -ef | grep "elasticsearch"| grep -v grep|grep -v controller`

case "$mode" in
   'start')
	echo "it's ready to start op...."
	if test -n "$elastic_process"; then
		echo "---the elasticsearch had already started.."
	else
		`$path_elastic -d`
	fi

	if test -n "$kibana_process"; then
		echo "---the kibana had already started.."
	else
		`nohup $path_kibana > /dev/null 2>&1 &`
	fi
	echo 'success to start.'
	echo '---1--you can test logstash by way bellow----'
	echo 'cd logstash-5.5.1 && bin/logstash -f ./config/test.conf && cd ..'
	echo 'or cd logstash-5.5.1 && bin/logstash -f ./config/h5.api.vip.com.conf && cd ..'
	;;
   'stop')
	echo "it's ready to check process..."
	if test -n "$kibana_process"; then
		echo "had find kibana process.."
		`echo $kibana_process | awk '{print ($2)}' | xargs kill -9`
	fi
	if test -n "$elastic_process"; then
		echo "had find elasticsearch process.."
		`echo $elastic_process | awk '{print ($2)}' | xargs kill -9`
	fi
	echo 'success to kill.'
	;;
    *)
	basename=`basename "$0"`
	echo "Usage: $basename  {start|stop}  [ ELK server options ]"
	exit 1
	;;
esac
exit 1

数据库性能压测

mysqltest]$ mysqlslap -S /tmp/mysqltest/mysql.sock -uroot --create=/tmp/mysqltest/user_cart.sql --create-schema=test --query=/tmp/mysqltest/cart1.sql --concurrency=1024 --iterations=3
Benchmark
Average number of seconds to run all queries: 59.774 seconds
Minimum number of seconds to run all queries: 59.570 seconds
Maximum number of seconds to run all queries: 60.040 seconds
Number of clients running queries: 1024
Average number of queries per client: 954
QPS=16343

机器学习的一些库

Gensim是一个相当专业的计算相似度的Python工具包。
在文本处理中,比如商品评论挖掘,有时需要了解每个评论分别和商品的描述之间的相似度,以此衡量评论的客观性。
评论和商品描述的相似度越高,说明评论的用语比较官方,不带太多感情色彩,比较注重描述商品的属性和特性,角度更客观。
http://radimrehurek.com/gensim/

————————————-
图像识别类库
https://github.com/tesseract-ocr/tesseract

原本由惠普开发的图像识别类库tesseract-ocr已经更新到2.04, 就是最近Google支持的那个OCR。原先是惠普写的,现在Open source了。