分类目录归档:Linux开发

Linux开发,只要在Linux环境下的一切相关开发,包括c++\goLang\Web,shell命令等。

常用netstat命令

统计某个IP时的连接数
netstat -nalp|grep 127.0.0.1:9006|wc
查看某个连接的创建进程
netstat -nalp|grep 120.92.21.108|grep ESTABLISHED|more
统计TIME_WAIT的数量
netstat -nalp|grep 120.92.21.108|grep TIME_WAIT|wc

netstat -nalp|grep 127.0.0.1:9006|grep ESTABLISHED|more && netstat -nalp|grep 114.112.66.246|grep ESTABLISHED|more&& netstat -nalp|grep 114.112.66.248|grep ESTABLISHED|more&& netstat -nalp|grep 114.112.66.44|grep ESTABLISHED|more&& netstat -nalp|grep 114.112.66.46|grep ESTABLISHED|more&& netstat -nalp|grep 120.131.1.61|grep ESTABLISHED|more&& netstat -nalp|grep 120.92.201.25|grep ESTABLISHED|more&& netstat -nalp|grep 120.92.20.174|grep ESTABLISHED|more&& netstat -nalp|grep 120.92.209.115|grep ESTABLISHED|more&& netstat -nalp|grep 120.92.21.108|grep ESTABLISHED|more&& netstat -nalp|grep 127.0.0.1|grep ESTABLISHED|more

借用msys2搭建linux子系统

1.安装msys2,它自带了pacman包管理工具,可以像yum或apt-get一样轻松添加软件。
http://www.msys2.org/,安装最新版本。如果此处没有标示最新版,也可以去https://sourceforge.net/projects/msys2/下载。正常情况下,两个站点的最新版是一样的。

2.使用pacman包管理工具,安装git,它自动解决包依赖问题。
pacman -S git

ssh指令定制及自动补全

1.利用Git的git-bash来做ssh的交互终端
不再使用git-bash进行交互了,统一使用msys2工具包进行管理。它可以实现linux的子系统。
注:参考这文章http://www.kxtry.com/archives/2334
2.在git-bash终端中执行【vim .bashrc】,添加以下指令

complete -W "$(echo $(grep '^Host ' F:/tools/myssh/config  | sort -u | sed 's/^Host //'))" remote
complete -W "$(echo $(grep '^Host ' F:/tools/myssh/config  | sort -u | sed 's/^Host //'))" xscp

3.编辑ssh的config文件

Host logkaf67
    HostName 10.0.0.67
    Port 22
    User heguowen
    IdentityFile c:\tools\ssh\id_rsa
    ProxyJump jumpServer   #since ssh 7.3 supports.
    #ProxyCommand ssh abc@10.2.16.210 nc %h %p

4.编辑remote脚本

#!/bin/sh

path_current=`pwd`
path_script=$(cd "$(dirname "$0")"; pwd)
cfg=$path_script/config

if [ $# -lt 0 ]; then
   awk '{if($1 == "Host"){print $2}}' $cfg
else
   ssh -F $cfg $*
fi

5.编辑xscp脚本

#!/bin/sh

path_current=`pwd`
path_script=$(cd "$(dirname "$0")"; pwd)
cfg=$path_script/config
if [ $# -lt 1 ]; then
   awk '{if($1 == "Host"){print $2}}' $cfg
else
   scp -F $cfg $*
fi

SSH from A through B to C, using private key on B



https://serverfault.com/questions/337274/ssh-from-a-through-b-to-c-using-private-key-on-b/701884#701884?tdsourcetag=s_pctim_aiomsg

Host jumpServer
        HostName x.y.z.h
        Port 22
        User abc
        IdentityFile ~/.ssh/id_rsa

Host A74
    HostName 10.1.0.1
    Port 22
    User zyx
    IdentityFile ~/.ssh/a73.id_rsa
    ProxyJump jumpServer    #ProxyJump是从ssh7.3开始支持。#ProxyCommand需要nc支持
    #ProxyCommand ssh -o 'ForwardAgent yes' jumpServer 'ssh-add && nc %h %p'

使用匿名函数,简易地把同步转异步

使用匿名函数,简易地把同步转异步。
大概思路,已经省去了线程池的相关说明。

#define MYASSERT(x)
 
typedef std::function FunAsyncTask;
typedef std::function FunSuccessCallBack;
typedef std::function FunErrorCallBack;
  
//从KAsyncTask派生
class AsyncTask     
{
private:
	const FunAsyncTask task;
	const FunSuccessCallBack ok_cb;
	const FunErrorCallBack err_cb;
        const QPointer callerThiz; //外部调用者的This指针,目的利用它来检验当前匿名函数所捕获的参数是否仍有效。
	QVariant result;
	QVariant err;
        //ToDo保存调用线程,调用线程与成功及错误的处理是同一个线程,也即是调用成功及错误之前先检查线程是否相同。
	int threadCallId; //任务请求的线程。
	int threadDoTaskId; //任务处理的线程
public:
	static void post_aync_task(QObject *thiz, FunAsyncTask t, FunSuccessCallBack ok, FunErrorCallBack err)
	{
		AsyncTask *task = new AsyncTask(thiz, t, ok, err);
		//ToDo推送至多线程任务队列。
	}
private:
	AsyncTask(QObject *thiz, FunAsyncTask t, FunSuccessCallBack ok, FunErrorCallBack err)
		: callerThiz(thiz), task(t), ok_cb(ok), err_cb(err)
	{
		threadCallId = GetCurrentThreadId();
	}
 
	void run()
	{
		// AnyncThread do the job.
		threadDoTaskId = GetCurrentThreadId();
		try
		{
			result = task();
			//ToDo Save Ok
		}
		catch (...)
		{
			//TODO Save Error;
		}
	}
 
	void handleResult()
	{
		//ToDo MainThread to do the job.
                MYASSERT(callerThiz);
		MYASSERT(threadCallId == GetCurrentThreadId());
		ok_cb(result);
	}
 
	void handleError()
	{
		//ToDo MainThread to do the Error
                MYASSERT(callerThiz);
		MYASSERT(threadCallId == GetCurrentThreadId());
		err_cb(err);
	}
};
 
void assertValidTaskLambda(char* txt)
{
	//ToDo提取lamda表达式的中括号的参数进行检查,不允许使用"=&"的特殊字符,有该字符则输出错误及抛出异常。
        //提取括号外及大括号前的字符串,不允许"mutable"参数使用。
	printf("\r\n%s", txt);
}

void assertValidCallBackLambda(char* ok, char* err)
{
        //ToDo提取lamda表达式的中括号的参数进行检查,不允许使用"&"的特殊字符,有该字符则输出错误及抛出异常。
        //提取括号外及大括号前的字符串,不允许"mutable"参数使用。
	printf("\r\n%s-%s", ok, err);
}
 
#define POST_AYNC_TASK(task, ok_cb, err_cb) \
{\
        assertValidQObject(this);  \
	assertValidTaskLambda(#task); \
        assertValidCallBackLambda(#ok_cb, #err_cb) \
	AsyncTask::post_aync_task(this, task, ok_cb, err_cb); \
}
 
int MyClass::execute()
{
	int x = 1;
	int y = 2;
	int z = 3;
	int m = 4;

 	FunAsyncTask task = [x, y, z]()->int{
		return QVariant(123);
	};

 	FunSuccessCallBack ok = [x, m](QVariant r){
		printf("ok...ok...%d", r);
	};

	FunErrorCallBack err = [z, m](QVariant err){
		printf("failed...failed...%d", err);
	}; 

	POST_AYNC_TASK(task, ok, err); //这种使用,报错。
 
	// 推荐以下方式使用,目的是语法检查,减少多线程出错。
        // POST_AYNC_TASK((task_exp),(ok_exp),(err_exp))
        // 如果表达式不用括号包裹起来的话,可能会导致编译出错。
	POST_AYNC_TASK(([x, y, z]()->QVariant{
		int v1 = x;
		int v2 = v1*2;
		Sleep(1000);
		return QVariant(v2 + z);
	}), ([=](QVariant r){
		printf("ok...ok...%d", r);
	}), ([=](QVariant err){
		printf("failed...failed...%d", err);
	}));
 
	getchar();
}

javacv相关

javacv是一款基于opencv的java版视觉处理库,同时也集成了ffmpeg,非常适合java服务端处理数据。当然c++服务端,依旧推荐使用opencv+ffmpeg了。
以下是javacv的rtmp/rtsp的直播例子,也可搜索关键字“javacv开发详解”。
https://blog.csdn.net/eguid_1/article/details/51659578
https://blog.csdn.net/column/details/12986.html
顺便说一下,直播方案之一是:nginx+nginx-rtmp-module+ffmpeg可快速拾搭建直播流媒体服务器。

cAdvisor的单机监控

cAdvisor是谷哥提供的一款单机docker运行监测工具,包括CPU、内存、网络等,缺点是监控的时间段只在几分钟,没有历史查询及持久化的能力,简单查看一下没有问题,如果要用于企业级监控还是使用grafana+elk这类。

简单的测试命令如下,可以直接看到输出log,结束后,仍可以docker start mytest直接启动:
docker run --name mytest  -p 8080:8080 -v /var/run:/var/run:rw -v /:/rootfs:ro -v /sys:/sys:ro -v /var/lib/docker/:/var/lib/docker:ro  google/cadvisor:latest
-------
也可以使用docker-compose.yml配置来启动,如下:。
version: '3'
services:
  cadvisor:
    ## Image
    build: cadvisor
    ##image: google/cadvisor:v0.27.1
 
    ## Base
    privileged: false
    restart: always
    container_name: cadvisor
    hostname: ${ACADVISOR_HOSTNAME}
    command: -storage_driver=influxdb -storage_driver_db=${ACADVISOR_DB} -storage_driver_host=${INFLUXBD_SERVER_IP}:${INFLUXDB_SERVER_PORT}  
 
    ## Network
    #network_mode: host  
    #expose:
    ports:
      - "8080:8080"
    #depends_on:
    #link:
 
    ## Storage
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro 
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
 
    ## Env
    environment:
      - SERVICE_IGNORE=true

如果遇到无法启动,则可以直接用以下命令,检查日志输出结果。

docker logs mytest

————————
如果报错如下:

[abc@localhost ~]$ docker logs mytest
I0501 02:01:54.006051       1 storagedriver.go:50] Caching stats in memory for 2m0s
I0501 02:01:54.006344       1 manager.go:151] cAdvisor running in container: "/sys/fs/cgroup/cpuacct,cpu"
I0501 02:01:54.037279       1 fs.go:139] Filesystem UUIDs: map[]
I0501 02:01:54.037305       1 fs.go:140] Filesystem partitions: map[shm:{mountpoint:/dev/shm major:0 minor:79 fsType:tmpfs blockSize:0} tmpfs:{mountpoint:/dev major:0 minor:82 fsType:tmpfs blockSize:0} /dev/mapper/cl-root:{mountpoint:/var/lib/docker major:253 minor:0 fsType:xfs blockSize:0} /dev/sda1:{mountpoint:/rootfs/boot major:8 minor:1 fsType:xfs blockSize:0}]
I0501 02:01:54.039141       1 manager.go:225] Machine: {NumCores:2 CpuFrequency:2807689 MemoryCapacity:3958325248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:fef8b2542f22422eaf1adab44a0cd5f2 SystemUUID:F91D4D56-9CDF-2AB7-E229-52E12085A954 BootID:7a90ab7b-3915-4615-ab38-3df3c3824a4e Filesystems:[{Device:overlay DeviceMajor:0 DeviceMinor:78 Capacity:39700664320 Type:vfs Inodes:19394560 HasInodes:true} {Device:tmpfs DeviceMajor:0 DeviceMinor:82 Capacity:1979162624 Type:vfs Inodes:483194 HasInodes:true} {Device:/dev/mapper/cl-root DeviceMajor:253 DeviceMinor:0 Capacity:39700664320 Type:vfs Inodes:19394560 HasInodes:true} {Device:/dev/sda1 DeviceMajor:8 DeviceMinor:1 Capacity:1063256064 Type:vfs Inodes:524288 HasInodes:true} {Device:shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:483194 HasInodes:true}] DiskMap:map[253:0:{Name:dm-0 Major:253 Minor:0 Size:39720058880 Scheduler:none} 253:1:{Name:dm-1 Major:253 Minor:1 Size:2147483648 Scheduler:none} 8:0:{Name:sda Major:8 Minor:0 Size:42949672960 Scheduler:deadline}] NetworkDevices:[{Name:br-00027b44406b MacAddress:02:42:58:af:8f:6a Speed:0 Mtu:1500} {Name:ens33 MacAddress:00:0c:29:85:a9:54 Speed:1000 Mtu:1500}] Topology:[{Id:0 Memory:4294365184 Cores:[{Id:0 Threads:[0] Caches:[]} {Id:1 Threads:[1] Caches:[]}] Caches:[{Size:6291456 Type:Unified Level:3}]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
I0501 02:01:54.040527       1 manager.go:231] Version: {KernelVersion:3.10.0-514.el7.x86_64 ContainerOsVersion:Alpine Linux v3.4 DockerVersion:1.13.1 DockerAPIVersion:1.26 CadvisorVersion:v0.28.3 CadvisorRevision:1e567c2}
I0501 02:01:54.057139       1 factory.go:356] Registering Docker factory
I0501 02:01:56.058709       1 factory.go:54] Registering systemd factory
I0501 02:01:56.060014       1 factory.go:86] Registering Raw factory
I0501 02:01:56.061514       1 manager.go:1178] Started watching for new ooms in manager
W0501 02:01:56.061593       1 manager.go:313] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory
I0501 02:01:56.078795       1 manager.go:329] Starting recovery of all containers
I0501 02:01:56.156028       1 manager.go:334] Recovery completed
F0501 02:01:56.156054       1 cadvisor.go:156] Failed to start container manager: inotify_add_watch /sys/fs/cgroup/cpuacct,cpu: no such file or directory


这是谷歌的新版bug,需要如下解决

mount -o remount,rw '/sys/fs/cgroup'
ln -s /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/cpuacct,cpu

————————-
附上使用截图


具体至某一容器

环境变量的修改

docker-compose的文件是不支持环境变量的使用,故为了适应在不同环境中配置不同的变量值如IP地址,则需要利用envsubst这个程序。
1.创建一个与环境变量相关的docker模板,命名为base-compose.yml,内容大概如下:。

version: '3'
services:
  cadvisor:
    ## Image
    build: cadvisor
    ##image: google/cadvisor:v0.27.1

    ## Base
    privileged: false
    restart: always
    container_name: cadvisor
    hostname: ${ACADVISOR_HOSTNAME}
    command: -storage_driver=influxdb -storage_driver_db=${ACADVISOR_DB} -storage_driver_host=${INFLUXBD_SERVER_IP}:${INFLUXDB_SERVER_PORT}  

    ## Network
    #network_mode: host  
    #expose:
    ports:
      - "9090:8080"
    #depends_on:
    #link:

    ## Storage
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro 
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro

    ## Env
    environment:
      - SERVICE_IGNORE=true

2.具体的环境变量如下,文件如下:

#!/bin/bash
export PRIVATE_IP=$(cat /etc/hosts | grep `hostname` |awk '{print $1}')
ret=$(curl https://ipapi.co/json| grep -v org)
ret=`echo $ret | sed 's/ip/public_ip/g'`
IP_INFO=`echo $ret | sed 's/"//g'| sed 's/{//g'| sed 's/}//g'| sed '/^$/g'| sed 's/[[:space:]]//g'`

export PUBLIC_IP=`echo $IP_INFO | awk -F ',' '{print $1}' | awk -F ':' '{print $2}'`

export MONITOR_VOLUME='/home/xxx/volume/monitor'
export KMS_VOLUME='/home/xxx/volume/kms'

export ACADVISOR_HOSTNAME=`hostname`
export ACADVISOR_DB='cadvisor'

export INFLUXBD_SERVER_IP=${PUBLIC_IP}
export INFLUXDB_SERVER_PORT='8086'

export GRAFANA_SERVER_IP=${PUBLIC_IP}
export GRAFANA_SERVER_PORT='13000'

export GRAPHITE_SERVER_IP=${PUBLIC_IP}
export GRAPHITE_SERVER_PORT='12003'

export ESEARCH_SERVER_URL='http://'${PUBLIC_IP}':9200'

3.替换指令

#!/bin/bash
source env.sh
envsubst  docker-compose.yml

cef的Qt的封装库及webrtc的测试

1.webrtc已经在cef3.x中支持。
2.由于cef的编译复杂性,有一些第三方机构专门编译一些全功能的二进制库,并提供了相应的例子,很方便地直接引入到第三方应用中。
3.在window中,简述其编译测试如下:
A-首先下载cef的库文件。下载地址如下:
http://opensource.spotify.com/cefbuilds/index.html【记住要下载32位的版本,不要下载64位的】
下载完成,先阅读其CMakeLists.txt文件,它说明了在各个平台该如何二次编译,如window的编译截图如下:

注:————–一定要用cmake -G “Visual Studio 14″【也即是vs2015,社区版就可以了编译了】的命令,令其生成工程vc的工程文件。———
注:————–不要用cmake的gui方式生成工程,因为可能会失败,而产生困扰——————————-
4.用vs2015打开工程,选择全编译。
5.调试cefclient工程,在调试器中添加–enable-media-stream选项。

6.打开webrtc的测试页面如下:
https://webrtc.github.io/samples/
7.截图如下:经验检,摄像头及声音都正常。


8.

—————————–
有人针对这场景写了一个QT的封装库,可以方便直接被Qt使用,在github随便搜索的结果有如下,可以逐个测试。
https://tishion.github.io/QCefView/
—————————
https://github.com/GreatTux/CefMinGWQt,这个是mingw版本。
———————————–
https://github.com/Dax89/QtCEF
——————-

带动态链接库的启动脚本

#!/bin/sh

path_current=`pwd`
path_script=$(cd "$(dirname "$0")"; pwd)
mode=$1

exe_name=NetSpeedServer

export PATH=$PATH:$path_script
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$path_script

app_process=`ps -ef | grep "$exe_name"| grep -v grep`

case "$mode" in
   'start')
        echo "it's ready to start op...."
        if test -n "$app_process"; then
                echo ""
                echo "$app_process"
                echo ""
        else
		cd $path_script
                nohup $path_script/$exe_name host=39.105.7.201 port=9513 > /dev/null 2>&1 &
		cd $path_current

        fi

        echo 'success to start.'
        ;;
   'stop')
        echo "it's ready to check process..."
        if test -n "$app_process"; then
                echo "had find app process informaton"
                echo $app_process | awk '{print ($2)}' | xargs kill -9
        fi
        echo 'success to kill.'
        ;;
    *)
        basename=`basename "$0"`
        echo "Usage: $basename  {start|stop}  [ server options ]"
        exit 1
        ;;
esac
exit 1