IO

定义

I/O(英语:Input/Output),即输入/输出,通常指数据在存储器(内部和外部)或其他周边设备之间的输入和输出,是信息处理系统(例如计算机)与外部世界(可能是人类或另一信息处理系统)之间的通信。输入是系统接收的信号或数据,输出则是从其发送的信号或数据。

冯•诺伊曼计算机的基本思想中有提到计算机硬件组成应为五大部分:控制器,运算器,存储器,输入和输出。这里的输入和输出就指IO。

输入设备

  • 键盘
  • 定点设备
  • 指点杆
  • 鼠标
  • 触控板
  • 轨迹球
  • 扫描仪
  • 麦克风
  • 相机

输出设备

  • 屏幕、投影机
  • 打印机、
  • 扬声器、耳机
  • 闪光灯

双向设备

  • 存储设备,如RAM
  • 一部分USB和火线设备
  • 触摸屏
  • 刻录机
  • ADSL
  • ATM
  • USB

应用程序角度分析

换句话说应用程序发起的一次IO操作实际包含两个阶段:

  1. IO调用阶段:应用程序进程向内核发起系统调用
  2. IO执行阶段:内核执行IO操作并返回

    a. 准备数据阶段:内核等待I/O设备准备好数据

    b. 拷贝数据阶段:将数据从内核缓冲区拷贝到用户空间缓冲区

IO-1

内核空间角度图示

IO-2

应用程序中进程在发起IO调用后至内核执行IO操作返回结果之前,若发起系统调用的线程一直处于等待状态,则此次IO操作为阻塞IO。阻塞IO简称BIO,Blocking IO。

BIO带来了一个问题:如果内核数据需要耗时很久才能准备好,那么用户进程将被阻塞,浪费性能。

那解决方案自然也容易想到,将阻塞变为非阻塞,那就是用户进程在发起系统调用时指定为非阻塞,内核接收到请求后,就会立即返回,然后用户进程通过轮询的方式来拉取处理结果。非阻塞IO简称NIO,Non-Blocking IO。 IO-3

NIO带来了一个问题:就是频繁轮询导致的无效系统调用。

解决NIO的思路就是降解无效的系统调用。

  • IO多路复用之select/poll

select是内核提供的系统调用,它支持一次查询多个系统调用的可用状态,当任意一个结果状态可用时就会返回,用户进程再发起一次系统调用进行数据读取。换句话说,就是NIO中N次的系统调用,借助select,只需要发起一次系统调用就够了。

IO-4

select/poll 虽然解决了NIO重复无效系统调用用的问题,但同时又引入了新的问题。问题是:

  1. 用户空间和内核空间之间,大量的数据拷贝
  2. 内核循环遍历IO状态,浪费CPU时间。

针对select/poll引入的问题,我们把解决问题的思路转回到内核上,如何减少内核重复无效的循环遍历呢?变主动为被动,基于事件驱动来实现。

IO-6

epoll,已经大大优化了IO的执行效率,但在IO执行的第一阶段:数据准备阶段都还是被阻塞的。所以这是一个可以继续优化的点。

异步IO真正实现了IO全流程的非阻塞。用户进程发出系统调用后立即返回,内核等待数据准备完成,然后将数据拷贝到用户进程缓冲区,然后发送信号告诉用户进程IO操作执行完毕。 IO-5

所以,之所以称为异步IO,取决于IO执行的第二阶段是否阻塞。因此前面讲的BIO,NIO均为同步IO。

参考

  • https://zh.wikipedia.org/wiki/I/O

排队系统设计

问题描述

请设计一个排队系统,能够让每个进入队伍的用户都能看到自己在队列中所处的位置和变化,队伍可能随时有人加入和退出;当有人退出影响到用户的位置排名时需要及时反馈到用户。

这是一道系统设计题。

我们先分析一下题干。就是我们平常去吃饭,前面排老长队,取一个号等着叫号类似。

假设用户ID为数字:1,3,4,8, 10。 模拟题干的排队行为。 一开始排队系统为空。 此时,第一个用户1进入,排队系统1 第二个用户进入,排队系统1,3

以此类推,1,3,4,8,10已进入排队系统。 我们需要知道排队系统的长度,也就是下一个加入用户的位置。 需要通过用户ID查到该用户的位置,此时,我们很容易想到hashtable,用户ID当做key,位置当做值。

用户8退出,此时,查询用户8所在的位置,需要清空hashtable中用户8的记录,一次遍历用户8以后的节点,并更新对应的hashtable中位置信息,最后删除用户8节点。这个队列非常像我们平常使用的链表。

结论就是利用hashtable+linked list两个主要数据结构实现。

解决此类问题,与我们分析数学证明题的思路相似。

该问题的引申,比如我们去银行办理业务,一般分为普通用户和贵宾用户,普通用户排队时,一会来了一个贵宾用户,就插队了。

动态规划学习笔记

介绍

动态规划(dynamic programming)是运筹学的一个分支,是求解决策过程(decision process)最优化的数学方法。 [1] 20世纪50年代初美国数学家R.E.Bellman等人在研究多阶段决策过程(multistep decision process)的优化问题时,提出了著名的最优化原理(principle of optimality),把多阶段过程转化为一系列单阶段问题,利用各阶段之间的关系,逐个求解,创立了解决这类过程优化问题的新方法——动态规划。1957年出版了他的名著《Dynamic Programming》,这是该领域的第一本著作。

分类

动态规划一般可分为线性动规,区域动规,树形动规,背包动规四类。

举例:

  • 线性动规:拦截导弹,合唱队形,挖地雷,建学校,剑客决斗等;
  • 区域动规:石子合并, 加分二叉树,统计单词个数,炮兵布阵等;
  • 树形动规:贪吃的九头龙,二分查找树,聚会的欢乐,数字三角形等;
  • 背包问题:01背包问题,完全背包问题,分组背包问题,二维背包,装箱问题,挤牛奶(同济ACM第1132题)等;

概念的意义

动态规划问世以来,在经济管理、生产调度、工程技术和最优控制等方面得到了广泛的应用。例如最短路线、库存管理、资源分配、设备更新、排序、装载等问题,用动态规划方法比用其它方法求解更为方便。

思想与性质

动态规划最重要的是掌握他的思想,动态规划的核心思想是把原问题分解成子问题进行求解,也就是分治的思想。

实现套路

我们实现动态规划算法,常用的是2个实现套路,一个是自底向上,另外一个是自顶向下。无论是何种方式,我们都要明确动态规划的过程,把状态表示、状态转移、边界都考虑好。

问题

换硬币问题

想兑换100元钱,有1,2,5,10四种钱,问总共有多少兑换方法。

解: 推到过程,也可以用于写完代码将其当成测试用例。

  • f(0) = 1 设置一个初始值
  • f(1) = f(0)
  • f(2) = f(1) + f(0)
  • f(3) = f(2) + f(1)
  • ……
  • f(n) = f(n-1) + f(n-2) + f(n-5) + f(n-10)
  • f(n),n >= 0

边界 n >= 0

状态表示 f(n)

状态转移方程 f(n) = f(n-1) + f(n-2) + f(n-5) + f(n-10)

转化成代码:

<?php

function coin($inputValue, $coins)
{
    $middle = [];
    $len = count($coins);
    $middle[0] = 1;

    for ($n = 1; $n <= $inputValue; $n++) {
        for ($i = 0; $i < $len; $i++) {
            if (empty($middle[$n])) {
                $middle[$n] = 0;
            }
            if ($n - $coins[$i] >= 0) {
                $middle[$n] += $middle[$n-$coins[$i]];
            }
        }
    }

    return $middle[$inputValue];
}

$coins = [1, 2, 5, 10];
$inputValue = 6;

echo coin($inputValue, $coins);

最大连续子数组和

输入一个整形数组,数组里有正数也有负数。数组中连续的一个或多个整数组成一个子数组,每个子数组都有一个和。 求所有子数组的和的最大值,要求时间复杂度为O(n)。

例如输入的数组为1, -2, 3, 10, -4, 7, 2, -5,和最大的子数组为3, 10, -4, 7, 2, 因此输出为该子数组的和18。

推到过程:

g(1) = g(0) + 1
f(1) = max(g(1), f(0)) = 1

g(2) = max(g(1) + (-2), 0) = 0
f(2) = max(g(2), f(1)) = 1

g(3) = max(g(2) + 3, 0) = 3
f(3) = max(g(3), f(2)) = 3

g(4) = max(g(3) + 10, 0) = 13
f(4) = max(g(4), f(3)) = 13

g(5) = max(g(4) + (-4), 0) = 9
f(5) = max(g(5), f(4)) = 13

g(6) = max(g(5) + 7, 0) = 16
f(6) = max(g(6), f(5)) = 16

g(7) = max(g(6) + 2, 0) = 18
f(7) = max(g(7), f(6)) = 18

g(8) = max(g(7) + (-5), 0) = 13
f(8) = max(g(8), f(7)) = 18

...

g(n) = max(g(n-1) + arr[n], 0)
f(n) = max(g(n), f(n-1))

边界:

n > 0

初始值定义:

g(0) = 0; f(0) = -0xFFFFFFFF;

状态表示 g(n) 表示累积到当前的和,和小于0,从下一个数字重新计算 f(n) 表示当前和最大值

状态转移方程 g(n) = max(g(n-1) + arr[n], 0) f(n) = max(g(n), f(n-1))

代码如下:

<?php

function maxSum($arr)
{
    $g = 0;
    $f = -0xFFFFFFFF;
    $l = count($arr);

    for ($i = 0; $i < $l; $i++) {
        if ($g <= 0) {
            $g = $arr[$i];
        } else {
            $g += $arr[$i];
        }

        if ($f < $g) {
            $f = $g;
        }
    }

    return $f;
}

echo maxSum([
    1, -2, 3, 10, -4, 7, 2, -5
]);

交替字符串

输入三个字符串s1、s2和s3,判断第三个字符串s3是否由前两个字符串s1和s2交错而成,即不改变s1和s2中各个字符原有的相对顺序,例如当s1 = “aabcc”,s2 = “dbbca”,s3 = “aadbbcbcac”时,则输出true,但如果s3=“accabdbbca”,则输出false。

推到过程: s1字符索引用i表示,s2字符索引用j表示,s3字符索引用k表示。

i = 1, j = 0, k = 1;
i = 2, j = 0, k = 2;
i = 2, j = 1, k = 3;
i = 3, j = 1, k = 4;
i = 3, j = 2, k = 5;
i = 4, j = 2, k = 6;
i = 4, j = 3, k = 7;
<组成k的情况可能有两种,可能来自s1,也能可能来自于s2>
i = 4, j = 4, k = 8; i = 5, j = 3, k = 8;
i = 4, j = 5, k = 9;
i = 5, j = 5, k = 10;

由此可以知:k=i+j-1 进而用dp[i][j]表示状态,用一维数组无法表示两个状态,因为s3字符来源由s1或s2,所以是两种状态。

状态表示: dp[i][j] 代表s3[0...i+j-1]是否由s1[0..i-1]s2[0...j-1]的字符组成

状态转移方程: dp[i][j] = ((s1[i-1] == s3[i+j-1]) && dp[i-1][j] || (s2[j-1] == s3[i+j-1]) && dp[i][j-1])

边界:

空串可以由空串组成

代码如下:

<?php

function alternate_str($s1, $s2, $s3)
{
    $l1 = strlen($s1);
    $l2 = strlen($s2);
    $l3 = strlen($s3);

    if ($l1 + $l2 !== $l3) {
        return false;
    }

    $dp = [];
    $dp[0][0] = true;

    for ($i = 0; $i <= $l1; $i++) {
        for ($j = 0; $j <= $l2; $j++) {
            if ($dp[$i][$j] ||
                ($i - 1 >= 0 && $dp[$i-1][$j] == true && $s1[$i-1] == $s3[$i+$j - 1]) ||
                ($j - 1 >= 0 && $dp[$i][$j-1] == true && $s2[$j-1] == $s3[$i+$j - 1])
            ) {
                $dp[$i][$j] = true;
            } else {
                $dp[$i][$j] = false;
            }
        }
    }
    echo json_encode($dp, 128);

    return $dp[$l1-1][$l2-1];
}

$s1 = 'aabcc';
$s2 = 'dbbca';
$s3 = 'aadbbcbcac';

echo alternate_str($s1, $s2, $s3);

参考

  • https://baike.baidu.com/item/%E5%8A%A8%E6%80%81%E8%A7%84%E5%88%92/529408?fr=aladdin
  • https://baijiahao.baidu.com/s?id=1631319141857419948&wfr=spider&for=pc
  • https://github.com/julycoding/The-Art-Of-Programming-By-July/blob/master/ebook/zh/02.05.md
  • https://github.com/julycoding/The-Art-Of-Programming-By-July/blob/master/ebook/zh/05.04.md

携程酒店SDK封装

最近做的分销业务,携程只提供了HTTP API,没有提供php sdk。这方面,淘宝做的不错(飞猪)直接点击下载sdk。

虽然提供了两个客户端sdk,跟现有框架融合是一个问题。淘宝的sdk直接利用composer包的方式加载:

╰─$ ls -lh .
total 744
-rw-r--r--   1 martin  staff   6.3K  5 23 16:48 README.md
....
-rw-r--r--   1 martin  staff   229B  5  8 12:02 codeception.yml
-rw-r--r--   1 martin  staff   3.0K  5 26 17:33 composer.json
-rw-r--r--   1 martin  staff   327K  5 26 17:33 composer.lock
-rw-r--r--   1 martin  staff    15K  5 26 17:33 symfony.lock
lrwxr-xr-x   1 martin  staff    43B  5  8 12:02 taobao-sdk -> taobao-sdk-PHP-auto_1568088312300-20191114/
drwxr-xr-x  11 martin  staff   352B  5  8 12:02 taobao-sdk-PHP-auto_1568088312300-20191114
drwxr-xr-x  53 martin  staff   1.7K  5 26 17:33 vendor

composer.json再加入仓库引入源:

    },
    "repositories": [
        {
            "type": "path",
            "url": "./taobao-sdk"
        }
    ],
    "minimum-stability":"dev"
}

这样就纳入了包管理。

淘宝的sdk,需要增加一个composer.json文件。

{
    "name": "martin/taobao-sdk",
    "authors": [
        {
            "name": "liujingyu",
            "email": "liujingyu@xiaozhu.com"
        }
    ],
    "require": {},
    "autoload":{
        "classmap": [
            "top/",
            "aliyun/",
            "dingtalk/",
            "QimenCloud/"
        ]
    },
    "minimum-stability":"dev"
}

安装可。

composer install martin/taobao-sdk

接下来聊聊,携程sdkhttps://dev.ctrip.com/#/document?id=83。 基于携程接口特点设计如下:


class CtripConfig
{
    const SOURCE = CommonSetting::CTRIP;
    const MODE = 1;
    const FORMAT = 'json';
    const REFRESH_TOKEN_EXPIRE = 900; // expire 15min

    // 携程API列表
    const API_LIST = [
        // api_name
        // @see ....
        //'api_name' => [
            //'path' => [
                //'prod' => '',
                //'test' => '',
                //'dev' => '',
                //'*' => '',
            //],
        //],

        // 获取全量城市信息
        // @see https://dev.ctrip.com/#/document?id=121
        'getAllCityInfo' => [
            'path' => [
                'prod' => 'f321ca3356b643ec8e32db0aec6a4b70',
                'test' => 'da97859659134aab9bfea1afbfa96d83',
                'dev' => 'da97859659134aab9bfea1afbfa96d83',
                '*' => 'da97859659134aab9bfea1afbfa96d83',
            ],
            'rate' => 200, // 每分钟200次
        ],

        // 获取酒店清单
        // @see https://dev.ctrip.com/#/document?id=144
        'getHotelList' => [
            'path' => [
                'prod' => '3a46e1d3100d46d8a46f52c3dc6b0273',
                'test' => '4c81467f1e5843fbb0a5eaba23906d70',
                'dev' => '4c81467f1e5843fbb0a5eaba23906d70',
                '*' => '4c81467f1e5843fbb0a5eaba23906d70',
            ],
            'rate' => 200, // 每分钟200次
        ],

        // 获取酒店静态信息
        // @see https://dev.ctrip.com/#/document?id=145
        'getHotelStaticInfo' => [
            'path' => [
                'prod' => 'becf5b5008064a26be77cac065688ca7',
                'test' => '62a8bb573582435bbe8b361334efe36f',
                'dev' => '62a8bb573582435bbe8b361334efe36f',
                '*' => '62a8bb573582435bbe8b361334efe36f',
            ],
            'rate' => 600, //每分钟600
        ],

        // 获取房型静态信息
        // @see https://dev.ctrip.com/#/document?id=146
        'getRoomTypeStaticInfo' => [
            'path' => [
                'prod' => '39122604bf0e4b33a9569658cf273161',
                'test' => 'c04d1d7b7af449999dba88896fcc0811',
                'dev' => 'c04d1d7b7af449999dba88896fcc0811',
                '*' => 'c04d1d7b7af449999dba88896fcc0811',
            ],
            'rate' => 600, //每分钟600
        ],

        // 监测静态信息变化
        // @see https://dev.ctrip.com/#/document?id=56
        'listenStaticInfo' => [
            'path' => [
                'prod' => '66fd1e3089fb448e912808cccbe017c0',
                'test' => 'd0e65061aef24f88854031c60cabfd91',
                'dev' => 'd0e65061aef24f88854031c60cabfd91',
                '*' => 'd0e65061aef24f88854031c60cabfd91',
            ],
            'filter' => 'ChangeInfos',
            'rate' => 1000, // 每分钟1000次
        ],

        // 监测房价、房量、房态增量变化接口
        // @see https://dev.ctrip.com/#/document?id=98
        'listenRoomPriceNumberStatus' => [
            'path' => [
                'prod' => 'a8f721b9716a41ca9df8b326f10655df',
                'test' => '68c959f3d5a94d7ca4ddd97cbe97d424',
                'dev' => '68c959f3d5a94d7ca4ddd97cbe97d424',
                '*' => '68c959f3d5a94d7ca4ddd97cbe97d424',
            ],
            'filter' => 'ChangeInfos',
            'rate' => 1000, // 每分钟1000次
        ],

        // 监测房型上下线
        // @see https://dev.ctrip.com/#/document?id=161
        'listenRoomTypeOnAndOffLine' => [
            'path' => [
                'prod' => '4536a4178961443a937423c1eb8fa2ba',
                'test' => 'ef89b1f6ec8e49d39d5debac065a9a4a',
                'dev' => 'ef89b1f6ec8e49d39d5debac065a9a4a',
                '*' => 'ef89b1f6ec8e49d39d5debac065a9a4a',
            ],
            'filter' => 'RoomStatusChangeItems',
            'rate' => 100, // 每分钟100次
        ],

        // 每日价格拉取接口(仅国内酒店)<落地价>
        // @see https://dev.ctrip.com/#/document?id=95
        'fetchLandPrice' => [
            'path' => [
                'prod' => '',
                'test' => '99503c56b9fa4417a8bc77b92ea56d41',
                'dev' => '99503c56b9fa4417a8bc77b92ea56d41',
                '*' => '99503c56b9fa4417a8bc77b92ea56d41',
            ],
        ],

        // 报价实时查询接口(国内酒店+海外酒店) <直连价>
        // @see https://dev.ctrip.com/#/document?id=94
        'fetchDirectPrice' => [
            'path' => [
                'prod' => '4c15cadb05f14aa9836260c53ab74526',
                'test' => '14a14763194648f5b9d2427f00b3ca04',
                'dev' => '14a14763194648f5b9d2427f00b3ca04',
                '*' => '14a14763194648f5b9d2427f00b3ca04',
            ],
            'rate' => 6000, //每分钟6000次
        ],

        // 可预订检查
        // @see https://dev.ctrip.com/#/document?id=62
        'checkBookAble' => [
            'path' => [
                'prod' => '419f7a688e3c45f481d295708b870323',
                'test' => 'd67d24cfdc2940cd858f506c708cf0d4',
                'dev' => 'd67d24cfdc2940cd858f506c708cf0d4',
                '*' => 'd67d24cfdc2940cd858f506c708cf0d4',
            ]
        ],

        // 订单保存
        // @see https://dev.ctrip.com/#/document?id=63
        'saveOrderBook' => [
            'path' => [
                'prod' => '2b0c7e0eebe2467a97be3d284313a129',
                'test' => 'acf5c44749234668ba3c539cc1d26c87',
                'dev' => 'acf5c44749234668ba3c539cc1d26c87',
                '*' => 'acf5c44749234668ba3c539cc1d26c87',
            ]
        ],

        // 订单提交
        // @see https://dev.ctrip.com/#/document?id=64
        'submitOrderBook' => [
            'path' => [
                'prod' => 'f40c5c71f84745cbbc73238252d9d43c',
                'test' => '5f3c81bc0c90458ba94bfe2dafea8459',
                'dev'  => '5f3c81bc0c90458ba94bfe2dafea8459',
                '*'    => '5f3c81bc0c90458ba94bfe2dafea8459',
            ]
        ],

        // 获取订单详情
        // @see https://dev.ctrip.com/#/document?id=67
        'getOrderBookDetail' => [
            'path' => [
                'prod' => 'e4a2a8ac057f4b378c9e153ef3f35249',
                'test' => '5db9d7e805ef4b36a8cf0e57900e118a',
                'dev'  => '5db9d7e805ef4b36a8cf0e57900e118a',
                '*'    => '5db9d7e805ef4b36a8cf0e57900e118a',
            ]
        ],

        // 取消订单
        // @see https://dev.ctrip.com/#/document?id=68
        'cancelBookOrder' => [
            'path' => [
                'prod' => 'f40a36ac2dfe42518fb883d33fa33391',
                'test' => '519114d791b94babba31f32eb5d559eb',
                'dev'  => '519114d791b94babba31f32eb5d559eb',
                '*'    => '519114d791b94babba31f32eb5d559eb',
            ]
        ],

        // 检测订单状态变化
        // @see https://dev.ctrip.com/#/document?id=66
        'listenOrderStatusChange' => [
            'path' => [
                'prod' => 'e5e981e2cf9340f99f9cbf79c82668ac',
                'test' => '61aebb32ae1347c5a274fc9fb4b4f962',
                'dev'  => '61aebb32ae1347c5a274fc9fb4b4f962',
                '*'    => '61aebb32ae1347c5a274fc9fb4b4f962',
            ]
        ],

        //查询某城市下各酒店的起价
        // @see https://dev.ctrip.com/#/document?id=93
        'fetchLowerPrice' => [
            'path' => [
                'prod' => '4c6f5dcc4663443b8d8ae044ad8bee10',
                'test' => 'c659a717bf23422b907ff7f9c75022ed',
                'dev'  => 'c659a717bf23422b907ff7f9c75022ed',
                '*'    => 'c659a717bf23422b907ff7f9c75022ed',
            ],
            'rate' => 200, // 每分钟200次
        ]
    ];

    /**
     * 根据key获取携程api icode
     *
     * @param $api
     * @return bool|string
     */
    public static function getCtripIcode($api)
    {
        $env = getenv('APP_ENV');
        $default = '*';
        if (isset(CtripConfig::API_LIST[$api]['path'][$env])) {
            return CtripConfig::API_LIST[$api]['path'][$env];
        } else {
            if (isset(CtripConfig::API_LIST[$api])) {
                return CtripConfig::API_LIST[$api]['path'][$default];
            } else {
                return false;
            }
        }
    }

    /**
     * 获取结果过滤器
     */
    public static function getApiFilter($api)
    {
        if (empty(CtripConfig::API_LIST[$api]['filter'])) {
            return false;
        }

        return CtripConfig::API_LIST[$api]['filter'];
    }
}

class CtripToken
{

    private $aid;
    private $sid;
    private $key;
    private $container;
    private $redis;
    private $client;
    private $logger;

    public function __construct(
        ContainerInterface $container,
        ClientInterface $redis,
        LoggerInterface $logger
    ) {
        $this->container = $container;
        $this->redis = $redis;
        $this->client = $this->container->get('bl_guzzle.client.ctrip');
        $this->logger = $logger;

        $this->aid = getenv('CTRIP_AID');
        $this->sid = getenv('CTRIP_SID');
        $this->key = getenv('CTRIP_KEY');
    }


    /**
     * (支持分布式获取Token)获取 token.
     */
    protected function getToken()
    {
        $token = $this->redis->get(self::SOURCE."_token");
        $refreshToken = $this->redis->get(self::SOURCE."_refresh_token");

        if (!empty($token)) {
            return $token;
        }

        // 加锁,避免重复刷新token
        $lock = function ($key) {
            return $this->redis->set("ctrip:lock", $key, "nx", "ex", 10);
        };

        $unlock = function ($key) {
            $script = <<<LUA
if redis.call("get",KEYS[1]) == ARGV[1]
then
    return redis.call("del",KEYS[1])
else
    return 0
end
LUA;
            return $this->redis->eval($script, 1, "ctrip:lock", $key);
        };

        $key = rand(1, 100000);

        token:
        if ($lock($key)) {
            if (empty($refreshToken)) {
                // token 初始化
                $res = $this->client->get(
                    '/openserviceauth/authorize.ashx?' .
                    http_build_query([
                        'AID' => $this->aid,
                        'SID' => $this->sid,
                        'KEY' => $this->key,
                    ])
                );
            } else {
                // token 更新
                $res = $this->client->get(
                    '/openserviceauth/refresh.ashx?'.
                    http_build_query([
                        'AID' => $this->aid,
                        'SID' => $this->sid,
                        'refresh_token' => $refreshToken,
                    ])
                );
            }

            // 解锁失败,说明锁超时,返回null,上层进行重试
            if (!$unlock($key)) {
                $this->logger->warning('unlock failure');
                return null;
            }
        } else {
            $this->logger->warning('lock failure');
            // 未取得锁,上层进行重试
            return null;
        }
        // 加锁结束

        if (200 != $res->getStatusCode()) {
            throw new Exception('CTRIP GET Token Http Client error');
        }

        $data = json_decode($res->getBody(), true);
        if (!empty($data['ErrCode']) && $data['ErrCode'] == 1005) {
            // 重置刷新Token
            $refreshToken = null;
            goto token;
        }
        if (empty($data['Expires_In']) || empty($data['Access_Token']) || empty($data['Refresh_Token'])) {
            throw new Exception('CTRIP GET TOKNE Http Client error, response data format error:'.json_encode($data));
        }

        $this->redis->setex(self::SOURCE."_token", $data['Expires_In'], $data['Access_Token']);
        $this->redis->setex(self::SOURCE."_refresh_token", self::REFRESH_TOKEN_EXPIRE, $data['Refresh_Token']);

        return $data['Access_Token'];
    }

    protected function removeCtripAccessToken()
    {
        return $this->redis->del(self::SOURCE."_token");
    }
}

class CtripClient
{

    private $aid;
    private $sid;
    private $key;
    private $container;
    private $redis;
    private $client;
    private $messageControl;
    private $logger;
    private $apiRateControl;

    public function __construct(
        ContainerInterface $container,
        ClientInterface $redis,
        MessageControl $messageControl,
        LoggerInterface $logger,
        APIRateControl $apiRateControl,
        Token $token
    ) {
        $this->container = $container;
        $this->redis = $redis;
        $this->client = $this->container->get('bl_guzzle.client.ctrip');
        $this->messageControl = $messageControl;
        $this->logger = $logger;
        $this->apiRateControl = $apiRateControl;
        $this->token = $token;
        $this->aid = getenv('CTRIP_AID');
        $this->sid = getenv('CTRIP_SID');
        $this->key = getenv('CTRIP_KEY');
    }

    /**
     * 支持 API_LIST 里面的所有接口.
     *
     * @param $name
     * @param $arguments
     * @return array
     * @throws FatalException
     */
    public function __call($name, $arguments)
    {
        if (isset(CtripConfig::API_LIST[$name])) {
            if ($arguments[0] instanceof HotelRequest) {
                return $this->getDetailData($name, $arguments[0]->toArray());
            }
            throw new \InvalidArgumentException();
        }

        throw new \BadMethodCallException();
    }

    /**
     * @param $response
     * @return array|mixed
     */
    protected function getContent($response)
    {
        if (200 == $response->getStatusCode()) {
            return \GuzzleHttp\json_decode($response->getBody(), true);
        } else {
            return [];
        }
    }

    /**
     * 获取数据接口.
     *
     * @param string $api
     * @param array $dataParam
     *
     * @return array
     **/
    protected function getDetailData($api, $dataParam)
    {
        try {
            if (!($icode = CtripConfig::getCtripIcode($api))) {
                throw new \InvalidArgumentException(sprintf("icode find by api, not found the api:%s", $api));
            }

            // api 接口测试限制
            if (($coolingSecond = $this->apiRateControl->getCoolingSecondForAPI($api))) {
                sleep($coolingSecond);
            }

token:
            $token = Util::try(3, function () {
                    return $this->token->getToken();
                    }, true);

            if (null == $token) {
                throw new \Exception('get token retry 3 failure');
            }

            $params = http_build_query([
                    'AID'   => $this->aid,
                    'SID'   => $this->sid,
                    'ICODE' => $icode,
                    'UUID'  => Util::getUUID(),
                    'Token' => $token,
                    'Mode'  => self::MODE,
                    'Format'=> self::FORMAT
            ]);

            $response = $this->client->post(
                    '/openservice/serviceproxy.ashx?'.$params,
                    [
                    'json' => $dataParam
                    ]
                    );

            $content = $this->getContent($response);
            if (!empty($content['ErrCode']) && $content['ErrCode'] == 232) {
                // clear store redis token and retry
                $this->token->removeCtripAccessToken();
                goto token;
            }

            if (($field = CtripConfig::getApiFilter($api)) && !empty($content[$field])) {
                // 去除重复调用的数据
                $content[$field] = $this->messageControl->filterMessage($content[$field]);
                return $content;
            } else {
                return $content;
            }
        } catch (Throwable $e) {
            $this->logger->error('CtripClient Throwable', [
                    'api' => $api,
                    'data' => $dataParam,
                    'message' => $e->getMessage(),
            ]);
        }

        return null;
    }

    public static function successAndReturnLastRecordId($response)
    {
        $successAndReturnLastRecordId = function ($response) {
            if (self::success($response)) {
                if (!empty($response['PagingInfo']['LastRecordID'])) {
                    return $response['PagingInfo']['LastRecordID'];
                }
            }
            return false;
        };
        return $successAndReturnLastRecordId($response);
    }

    public static function success($response)
    {
        $success = function ($response) {
            if (!empty($response['ResponseStatus']['Ack']) && $response['ResponseStatus']['Ack'] == 'Success') {
                return true;
            }
            return false;
        };
        return $success($response);
    }
}

// 工具类封装
class Util
{
	// 重试逻辑,正常逻辑和异常逻辑分开
	public static function retry($retries, callable $fn, $sleep = false)
    {
        $base = $retries * 2;

        beginning:
        try {
            return $fn();
        } catch (\Exception $e) {
            if (!$retries) {
                return null;
            }

            if ($sleep) {
                // 休眠时间递增(单位s), 1, 2, 5
                $sleepTime = ($base - $retries) / $retries;
                sleep($sleepTime);
            }

            $retries--;
            goto beginning;
        }
    }

    const LOOP_RUN_CONTINUE = 0;
    const LOOP_NORMAL_STOP = 1;
    const LOOP_API_EXCEPTION_STOP = 2;

    /**
     * 任务循环器, 可复用,用于分页数据获取.
     *
     * @param $run
     * @param mixed ...$args
     */
    public static function loop($run, ...$args)
    {
        $loop = function () use ($run, $args) {
            do {
                // status: 0 循环, 1 正常结束, 2 接口异常结束
                $status = $run(...$args);
                if ($status != 0) {
                    break;
                }
            } while (true);
        };
        $loop();
    }
}

// 携程接口API调用频次记录,并且返回冷却时间
class APIRateControl
{
    private $redis;

    const KEY_PREFIX = 'ctrip:rate:';

    public function __construct(ClientInterface $redis)
    {
        $this->redis = $redis;
    }

    /**
     * 获取API冷却时间.
     *
     * @param string $api
     * @param string $second
     * @param int $times
     *
     * @return int
     * @throws \Exception
     */
    public function getAPICoolingSecond($api, $second, $times)
    {
        $lockedTimes = function ($api, $expire, $times) {
            $script = <<<LUA
local times = redis.call('incr',KEYS[1])

-- 第一次访问的时候加上过期时间60秒(60秒过后从新计数)
if times == 1 then
    redis.call('expire',KEYS[1], ARGV[1])
end

-- 注意,从redis进来的默认为字符串,lua同种数据类型只能和同种数据类型比较
if times > tonumber(ARGV[2]) then
    return 0
end
return 1
LUA;
            return $this->redis->eval($script, 1, self::KEY_PREFIX.$api, $expire, $times);
        };

        if ($lockedTimes($api, $second, $times)) {
            return 0;
        }

        $sleep = $this->redis->ttl(self::KEY_PREFIX.$api);
        if ($sleep > 0) {
            return $sleep;
        } else {
            return 0;
        }
    }

    /**
     * 冷却时间.
     *
     * @param string $api
     *
     * @return int
     */
    public function getCoolingSecondForAPI($api)
    {
        if (empty(CtripConfig::API_LIST[$api]['rate'])) {
            return 0;
        }

        return $this->getAPICoolingSecond(
            $api,
            60,
            CtripConfig::API_LIST[$api]['rate']
        );
    }
}


<?php

use Adbar\Dot;

/**
 *
 * 使用如下:
 *  // 测试HotelRequest toArray
 * $r = new HotelRequest();
 * $r->PagingSettings_LastRecordID = 100;
 * $r->PagingSettings_PageSize = 200;
 * $r->Settings_ExtendedNodes = [ 'HotelStaticInfo.GeoInfo' ];
 * $r->Settings_PrimaryLangID = 'zh_cn';
 * echo $r->toJson(),PHP_EOL;
 * var_dump($r->toArray());
 */

class HotelRequest
{
    // 格式     'SearchCandidate_HotelID' => 'SearchCandidate.HotelID',
    const RULES = [
        // 酒店
        'SearchCandidate_HotelID' => 'SearchCandidate.HotelID',
        'SearchCandidate_RoomIDs' => 'SearchCandidate.RoomIDs',
        'Settings_PrimaryLangID' => 'Settings.PrimaryLangID',
        'Settings_ExtendedNodes' => 'Settings.ExtendedNodes',
        'Settings_SearchTags' => 'Settings.SearchTags',
        'PagingSettings_PageSize' => 'PagingSettings.PageSize',
        'PagingSettings_LastRecordID' => 'PagingSettings.LastRecordID',
        'SearchCandidate_DateTimeRange_Start' => 'SearchCandidate.DateTimeRange.Start',
        'SearchCandidate_DateTimeRange_End' => 'SearchCandidate.DateTimeRange.End',
        'SearchCandidate_StartTime' => 'SearchCandidate.StartTime',
        'SearchCandidate_Duration' => 'SearchCandidate.Duration',
        'Settings_ShowDataRange' => 'Settings.ShowDataRange',
        'Settings_IsShowChangeDetails' => 'Settings.IsShowChangeDetails',
        'Settings_DataCategory' => 'Settings.DataCategory',
        'SearchCandidate_SearchByType_SearchType' => 'SearchCandidate.SearchByType.SearchType',
        'SearchCandidate_SearchByType_IsHaveHotel' => 'SearchCandidate.SearchByType.IsHaveHotel',
        'SearchCandidate_SearchByCityID_CityID' => 'SearchCandidate.SearchByCityID.CityID',

        // 订单部分 //
        // 可预订检查
        'AvailRequestSegments_Version' => 'AvailRequestSegments.Version',
        'AvailRequestSegments_TransactionIdentifier' => 'AvailRequestSegments.TransactionIdentifier',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_HotelRef_HotelCode' =>
        'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.HotelRef.HotelCode',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_StayDateRange_Start' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.StayDateRange.Start',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_StayDateRange_End' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.StayDateRange.End',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_RoomID' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.RatePlanCandidates.RatePlanCandidate.RoomID',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_RatePlanID' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.RatePlanCandidates.RatePlanCandidate.RatePlanID',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_RatePlanCategory' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.RatePlanCandidates.RatePlanCandidate.RatePlanCategory',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_PrepaidIndicator' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.RatePlanCandidates.RatePlanCandidate.PrepaidIndicator',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RoomStayCandidates_RoomStayCandidate_GuestCounts_GuestCount_Count' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.RoomStayCandidates.RoomStayCandidate.GuestCounts.GuestCount.Count',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RoomStayCandidates_RoomStayCandidate_Quantity' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.RoomStayCandidates.RoomStayCandidate.Quantity',
        'AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_TPA_Extensions_LateArrivalTime' => 'AvailRequestSegments.AvailRequestSegment.HotelSearchCriteria.Criterion.TPA_Extensions.LateArrivalTime',
        // 保存订单
        'Version' => 'Version',
        'TransactionIdentifier' => 'TransactionIdentifier',
        'UniqueID' => 'UniqueID',
        'UniqueID_Type' => 'UniqueID.Type',
        'UniqueID_ID' => 'UniqueID.ID',
        'HotelReservations_HotelReservation_RoomStays_RoomStay_RoomTypes_RoomType_NumberOfUnits' => 'HotelReservations.HotelReservation.RoomStays.RoomStay.RoomTypes.RoomType.NumberOfUnits',
        'HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_RoomID' => 'HotelReservations.HotelReservation.RoomStays.RoomStay.RatePlans.RatePlan.RoomID',
        'HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_RatePlanCategory' => 'HotelReservations.HotelReservation.RoomStays.RoomStay.RatePlans.RatePlan.RatePlanCategory',
        'HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_PrepaidIndicator' => 'HotelReservations.HotelReservation.RoomStays.RoomStay.RatePlans.RatePlan.PrepaidIndicator',
        'HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_RatePlanID' => 'HotelReservations.HotelReservation.RoomStays.RoomStay.RatePlans.RatePlan.RatePlanID',
        'HotelReservations_HotelReservation_RoomStays_RoomStay_BasicPropertyInfo_HotelCode' => 'HotelReservations.HotelReservation.RoomStays.RoomStay.BasicPropertyInfo.HotelCode',
        'HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_PersonName' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.Profiles.ProfileInfo.Profile.Customer.PersonName',
        'HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_PersonName_Name' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.Profiles.ProfileInfo.Profile.Customer.ContactPerson.PersonName.Name',
        'HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_Telephone_PhoneTechType' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.Profiles.ProfileInfo.Profile.Customer.ContactPerson.Telephone.PhoneTechType',
        'HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_Telephone_PhoneNumber' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.Profiles.ProfileInfo.Profile.Customer.ContactPerson.Telephone.PhoneNumber',

        'HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_ContactType' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.Profiles.ProfileInfo.Profile.Customer.ContactPerson.ContactType',
        'HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.Profiles.ProfileInfo.Profile.Customer.ContactPerson',
        'HotelReservations_HotelReservation_ResGuests_ResGuest_TPA_Extensions_LateArrivalTime' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.TPA_Extensions.LateArrivalTime',
        'HotelReservations_HotelReservation_ResGuests_ResGuest_ArrivalTime' => 'HotelReservations.HotelReservation.ResGuests.ResGuest.ArrivalTime',
        'HotelReservations_HotelReservation_ResGlobalInfo_GuestCounts_GuestCount_Count' => 'HotelReservations.HotelReservation.ResGlobalInfo.GuestCounts.GuestCount.Count',
        'HotelReservations_HotelReservation_ResGlobalInfo_TimeSpan_Start' => 'HotelReservations.HotelReservation.ResGlobalInfo.TimeSpan.Start',
        'HotelReservations_HotelReservation_ResGlobalInfo_TimeSpan_End' => 'HotelReservations.HotelReservation.ResGlobalInfo.TimeSpan.End',
        //'HotelReservations_HotelReservation_ResGlobalInfo_DepositPayments' => 'HotelReservations.HotelReservation.ResGlobalInfo.DepositPayments',
        'HotelReservations_HotelReservation_ResGlobalInfo_Total_ExclusiveAmount' => 'HotelReservations.HotelReservation.ResGlobalInfo.Total.ExclusiveAmount',
        'HotelReservations_HotelReservation_ResGlobalInfo_Total_InclusiveAmount' => 'HotelReservations.HotelReservation.ResGlobalInfo.Total.InclusiveAmount',
        'HotelReservations_HotelReservation_ResGlobalInfo_Total_CurrencyCode' => 'HotelReservations.HotelReservation.ResGlobalInfo.Total.CurrencyCode',
        'HotelReservations_HotelReservation_ResGlobalInfo_TPA_Extensions_TotalCost_ExclusiveAmount' => 'HotelReservations.HotelReservation.ResGlobalInfo.TPA_Extensions.TotalCost.ExclusiveAmount',
        'HotelReservations_HotelReservation_ResGlobalInfo_TPA_Extensions_TotalCost_InclusiveAmount' => 'HotelReservations.HotelReservation.ResGlobalInfo.TPA_Extensions.TotalCost.InclusiveAmount',
        'HotelReservations_HotelReservation_ResGlobalInfo_TPA_Extensions_TotalCost_CurrencyCode' => 'HotelReservations.HotelReservation.ResGlobalInfo.TPA_Extensions.TotalCost.CurrencyCode',

        // 订单提交
        'ReservationPayment_ReservationID_Type' => 'ReservationPayment.ReservationID.Type',
        'ReservationPayment_ReservationID_ID' => 'ReservationPayment.ReservationID.ID',
        'ReservationPayment_PaymentDetail' => 'ReservationPayment.PaymentDetail',
        'ReservationPayment_Invoice' => 'ReservationPayment.Invoice',

        // 监测订单状态变化
        'HotelReservations' => 'HotelReservations',
        'SearchTypeEntity_SearchType' => 'SearchTypeEntity.SearchType',
        'SearchTypeEntity_HotelCount' => 'SearchTypeEntity.HotelCount',
        'SearchTypeEntity_PageIndex' => 'SearchTypeEntity.PageIndex',
        'PublicSearchParameter_City' => 'PublicSearchParameter.City',
        'PublicSearchParameter_CheckInDate' => 'PublicSearchParameter.CheckInDate',
        'PublicSearchParameter_CheckOutDate' => 'PublicSearchParameter.CheckOutDate',
        'PublicSearchParameter_HotelList' => 'PublicSearchParameter.HotelList',
        'PublicSearchParameter_FilterRoomByPerson' => 'PublicSearchParameter.FilterRoomByPerson',
        'PublicSearchParameter_OnlyPPPrice' => 'PublicSearchParameter.OnlyPPPrice',
        'PublicSearchParameter_OnlyFGPrice' => 'PublicSearchParameter.OnlyFGPrice',
        'PublicSearchParameter_IsJustifyConfirm' => 'PublicSearchParameter.IsJustifyConfirm',
        'FacilityEntity_ChildrenAgeList' => 'FacilityEntity.ChildrenAgeList',
        'FacilityEntity_priceRange_lowPrice' => 'FacilityEntity.priceRange.lowPrice',
        'FacilityEntity_priceRange_highPrice' => 'FacilityEntity.priceRange.highPrice',
        'OrderBy_OrderName' => 'OrderBy.OrderName',
        'OrderBy_OrderType' => 'OrderBy.OrderType',
        'SearchCandidate_DateRange_Start' => 'SearchCandidate.DateRange.Start',
        'SearchCandidate_DateRange_End' => 'SearchCandidate.DateRange.End',
        'SearchCandidate_Occupancy_Adult' => 'SearchCandidate.Occupancy.Adult',
        'SearchCandidate_Occupancy_Child' => 'SearchCandidate.Occupancy.Child',
        'SearchCandidate_Occupancy_ChildInfo' => 'SearchCandidate.Occupancy.ChildInfo',
        'SearchCandidate_SearchTags' => 'SearchCandidate.SearchTags',
        'Settings_DisplayCurrency' => 'Settings.DisplayCurrency',
        'Settings_IsShowNonBookable' => 'Settings.IsShowNonBookable',
    ];


    public $Version = null;
    public $TransactionIdentifier = null;
    public $SearchCandidate_HotelID = null;
    public $SearchCandidate_RoomIDs = null;
    public $Settings_PrimaryLangID = null;
    public $Settings_ExtendedNodes = null;
    public $Settings_SearchTags = null;
    public $PagingSettings_PageSize = null;
    public $PagingSettings_LastRecordID = null;
    public $SearchCandidate_DateTimeRange_Start = null;
    public $SearchCandidate_DateTimeRange_End = null;
    public $SearchCandidate_StartTime = null;
    public $SearchCandidate_Duration = null;
    public $Settings_ShowDataRange = null;
    public $Settings_IsShowChangeDetails = null;
    public $Settings_DataCategory = null;
    public $SearchCandidate_SearchByType_SearchType = null;
    public $SearchCandidate_SearchByType_IsHaveHotel = null;
    public $SearchCandidate_SearchByCityID_CityID = null;
    public $SearchTypeEntity_SearchType = null;
    public $SearchTypeEntity_HotelCount = null;
    public $SearchTypeEntity_PageIndex = null;
    public $PublicSearchParameter_City = null;
    public $PublicSearchParameter_CheckInDate = null;
    public $PublicSearchParameter_CheckOutDate = null;
    public $PublicSearchParameter_HotelList = null;
    public $PublicSearchParameter_FilterRoomByPerson = null;
    public $PublicSearchParameter_OnlyPPPrice = null;
    public $PublicSearchParameter_OnlyFGPrice = null;
    public $PublicSearchParameter_IsJustifyConfirm = null;
    public $FacilityEntity_ChildrenAgeList = null;
    public $FacilityEntity_priceRange_lowPrice = null;
    public $FacilityEntity_priceRange_highPrice = null;
    public $OrderBy_OrderName = null;
    public $OrderBy_OrderType = null;
    public $SearchCandidate_DateRange_Start = null;
    public $SearchCandidate_DateRange_End = null;
    public $SearchCandidate_Occupancy_Adult = null;
    public $SearchCandidate_Occupancy_Child = null;
    public $SearchCandidate_Occupancy_ChildInfo = null;
    public $SearchCandidate_SearchTags = null;
    public $Settings_DisplayCurrency = null;
    public $Settings_IsShowNonBookable = null;

    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_HotelRef_HotelCode = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_StayDateRange_Start = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_StayDateRange_End = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_RoomID = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_RatePlanID = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_RatePlanCategory = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RatePlanCandidates_RatePlanCandidate_PrepaidIndicator = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RoomStayCandidates_RoomStayCandidate_GuestCounts_GuestCount_Count = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_RoomStayCandidates_RoomStayCandidate_Quantity = null;
    public $AvailRequestSegments_AvailRequestSegment_HotelSearchCriteria_Criterion_TPA_Extensions_LateArrivalTime = null;
    public $AvailRequestSegments_Version;
    public $AvailRequestSegments_TransactionIdentifier;
    public $HotelReservations = [];
    //public $HotelReservations_HotelReservation_ResGlobalInfo_DepositPayments = [];
    public $UniqueID = null;
    public $UniqueID_Type = null;
    public $UniqueID_ID = null;
    public $HotelReservations_HotelReservation_RoomStays_RoomStay_RoomTypes_RoomType_NumberOfUnits = null;
    public $HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_RoomID = null;
    public $HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_RatePlanCategory = null;
    public $HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_PrepaidIndicator = null;
    public $HotelReservations_HotelReservation_RoomStays_RoomStay_RatePlans_RatePlan_RatePlanID = null;
    public $HotelReservations_HotelReservation_RoomStays_RoomStay_BasicPropertyInfo_HotelCode = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_PersonName = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_PersonName_Name = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_Telephone_PhoneTechType = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_Telephone_PhoneNumber = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson_ContactType = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_Profiles_ProfileInfo_Profile_Customer_ContactPerson = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_TPA_Extensions_LateArrivalTime = null;
    public $HotelReservations_HotelReservation_ResGuests_ResGuest_ArrivalTime = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_GuestCounts_GuestCount_Count = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_TimeSpan_Start = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_TimeSpan_End = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_Total_ExclusiveAmount = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_Total_InclusiveAmount = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_Total_CurrencyCode = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_TPA_Extensions_TotalCost_ExclusiveAmount = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_TPA_Extensions_TotalCost_InclusiveAmount = null;
    public $HotelReservations_HotelReservation_ResGlobalInfo_TPA_Extensions_TotalCost_CurrencyCode = null;
    public $ReservationPayment_ReservationID_Type = null;
    public $ReservationPayment_ReservationID_ID = null;
    public $ReservationPayment_PaymentDetail = null;
    public $ReservationPayment_Invoice = null;

    protected $dot;

    const HOTEL_STATIC_INFO_EXTENDED_NODES = [
        "HotelStaticInfo.GeoInfo",//重要
        "HotelStaticInfo.NearbyPOIs",//已废弃请下线处理
        "HotelStaticInfo.TransportationInfos",
        "HotelStaticInfo.Brand",
        "HotelStaticInfo.Group",
        "HotelStaticInfo.Ratings",//重要
        "HotelStaticInfo.Policies",//重要
        "HotelStaticInfo.AcceptedCreditCards",//重要
        "HotelStaticInfo.ImportantNotices",//重要
        "HotelStaticInfo.Facilities", //已废弃,请切换至FacilitiesV2节点
        "HotelStaticInfo.FacilitiesV2",//重要
        "HotelStaticInfo.Pictures",//重要
        "HotelStaticInfo.Descriptions",//重要
        "HotelStaticInfo.Themes",
        "HotelStaticInfo.ContactInfo",
        "HotelStaticInfo.HotelTags.IsBookable",//重要
        "HotelStaticInfo.HotelTags.IsHeCheng",
        "HotelStaticInfo.ArrivalTimeLimitInfo",//重要
        "HotelStaticInfo.DepartureTimeLimitInfo",//重要
        "HotelStaticInfo.ExternalFacility.Parking",
        "HotelStaticInfo.ExternalFacility.ChargingPoint",
        "HotelStaticInfo.BossInfos",
        "HotelStaticInfo.NormalizedPolicies.ChildAndExtraBedPolicy",//重要
        "HotelStaticInfo.HotelDiscount",
        "HotelStaticInfo.SellerShowInfos",
        "HotelStaticInfo.VideoItems",
        "HotelStaticInfo.NormalizedPolicies.MealsPolicy", //已废弃,请切换至MealsPolicyV2
        "HotelStaticInfo.NormalizedPolicies.MealsPolicyV2",//重要
        "HotelStaticInfo.HotelTags.ReservedData",
        "HotelStaticInfo.HotelPromotions",
        "HotelStaticInfo.HotelTaxRuleInfos",//重UniqueID要
        "HotelStaticInfo.SepecialServiceForChinese",
        "HotelStaticInfo.HotelFeatures",
        "HotelStaticInfo.HotelUsedNames",
        "HotelStaticInfo.BnBHotel", // 获取民宿酒店专有信息
        "HotelStaticInfo.ContactPersonInfos",
        "HotelStaticInfo.HotelCloseTime",  //酒店不营业时段
    ];

    const ROOM_TYPE_STATIC_EXTENDED_NODES = [
        //物理房型相关扩展节点
        "RoomTypeInfo.Facilities",
        "RoomTypeInfo.Pictures",
        "RoomTypeInfo.Descriptions",
        "RoomTypeInfo.ChildLimit",
        "RoomTypeInfo.BroadNet",
        "RoomTypeInfo.RoomBedInfos",
        "RoomTypeInfo.BnBHotel",    // 获取民宿房型专有信息
        "RoomTypeInfo.Smoking",//已废弃,请使用售卖房型吸烟信息
        "RoomTypeInfo.RoomTypeTags.ReservedData",
        //售卖房型相关扩展节点
        "RoomInfo.ApplicabilityInfo",
        "RoomInfo.Smoking",
        "RoomInfo.BroadNet",//已废弃,请使用物理房型层宽带信息,售卖房型层宽带信息暂停使用
        "RoomInfo.RoomBedInfos",
        "RoomInfo.RoomFGToPPInfo",
        "RoomInfo.ChannelLimit",
        "RoomInfo.ExpressCheckout",
        "RoomInfo.RoomTags",
        "RoomInfo.RoomGiftInfos",
        "RoomInfo.AreaApplicabilityInfo",
        "RoomInfo.RoomPromotions",
        "RoomInfo.HotelPromotions",
        "RoomInfo.MaskCampaignInfos",//Mask房型获取
        "RoomInfo.BookingRules",
        "RoomInfo.RoomTags.HotelDiscount",
        "RoomInfo.IsNeedCustomerTelephone",
    ];

    const ROOM_TYPE_STATIC_SEARCH_TAGS = [
        [
            "Code" => "IsOutputHiddenMaskRoom", //异地Mask房型获取
        ],
        [
            "Code"=> "IsOutputLimitDestinationRoom", //目的地Mask房型获取
        ]
    ];

    public function __construct()
    {
        $this->dot = new Dot;
    }

    protected function convertData()
    {
        foreach (self::RULES as $underlineRule => $dotRule) {
            if (null === $this->{$underlineRule}) {
                continue;
            }
            $this->dot->add($dotRule, $this->{$underlineRule});
        }
    }

    public function toArray()
    {
        $this->convertData();
        return $this->dot->all();
    }

    public function toJson()
    {
        $this->convertData();
        return $this->dot->toJson();
    }
}


class Test
{
    public function __construct($ctripClient)
	{
		$this->ctripClient = $ctripClient;
	}

    public function main()
    {
        $fetchOnePage = function (&$lastRecordId) {
            $request = new HotelRequest();
            $request->SearchCandidate_SearchByType_SearchType = self::SEARCH_TYPE;
            $request->SearchCandidate_SearchByType_IsHaveHotel = self::IS_HAVE_HOTEL;
            $request->PagingSettings_PageSize = self::PAGE_SIZE;
            $request->PagingSettings_LastRecordID = $lastRecordId;

            $response = $this->ctripClient->getAllCityInfo($request);

            if (!CtripClient::success($response) || empty($response['CityInfos']['CityInfo'])) {
                return Util::LOOP_API_EXCEPTION_STOP;
            }

            foreach ($response['CityInfos']['CityInfo'] as $city) {
				var_dump($city);
            }

            $lastRecordId = CtripClient::successAndReturnLastRecordId($response);
            if (empty($lastRecordId)) {
                unset($request, $response);
                return Util::LOOP_NORMAL_STOP;
            }

            unset($request, $response);
            return Util::LOOP_RUN_CONTINUE;
        };

        $lastRecordId = '';
        Util::loop($fetchOnePage, $lastRecordId);
    }
}

$test = new Test(new CtripClient());
$test->main();
  • CtripConfig配置信息类
  • CtripToken维护携程token类支持多进程并发获取token。
  • CtripClient对外提供的客户端类
  • HotelRequest对外提供的请求类
  • Util工具类
  • Test演示类

总结:支持分布式获取token;支持HotelRequest请求类参数配置;支持接口配置;支持接口限速配置;封装重试方法;封装分页循环获取方法。

二叉堆笔记

介绍

一个小顶堆(或大顶堆)是一个完整的二叉树。其中每个节点都小于其子节点。因此,根是树中的最小元素。

图-2

在最小堆中有两个关键操作:insertextract_min

插入操作

当我们向一个最小堆插入元素时,总是从底部开始。从最右边的节点开始插入操作以保持树的完整性。然后,通过与其祖先节点进行交换来“修复”树,直到找到新元素的适当位置。我们基本上是在向上传递最小的元素

图-1

提取最小元素

找到小顶堆的最小元素是小菜一碟:它总是在顶部。颇为棘手的是如何删除该元素(其实也不是那么棘手)。首先,删除最小元素并将其与堆中的最后一个元素(位于最底层、最右边的元素)进行交换。然后,向下传递这个元素,不断使其与自身子节点之一进行交换,直到小顶堆的属性得以恢复。是和左边的孩子节点还是右边的孩子节点进行交换取决于它们的值。左右元素之间没有固定的顺序,但是为了保持小顶堆的元素有序,你需要选择两者中较小的元素。

图-3

堆的基本操作

堆的几个基本操作都依赖于两个重要的函数siftUpsiftDown,堆的insert通常是在堆尾插入新元素并siftUp调整堆,而extractMin是在删除堆顶元素,然后将最后一个元素放置堆顶并调用siftDown调整堆。 堆得基本操作的时间复杂度如下表所示:

  • heapify O(N)
  • insert O(logN)
  • extractMin O(logN)
  • delete O(logN)
  • peek O(1)

总结

  • (1)堆是一颗完全二叉树;
  • (2)小(大)顶堆中的每一个节点都不小于(不大于)它的父节点;
  • (3)堆的插入、删除元素的时间复杂度都是O(log n);
  • (4)建堆的时间复杂度是O(n);
  • (5)堆排序的时间复杂度是O(nlog n);
  • (6)堆排序的空间复杂度是O(1);

代码

最小堆代码示例:

<?php

class Heap
{

    private $items;
    private $size;
    private $length;

    public function __construct($length)
    {
        $this->items = [];
        $this->size = 0;
        $this->length = $length;
    }

    public function peek()
    {
        return $this->items[0]??null;
    }

    public function insert($value)
    {
        if ($this->size == $this->length) {
            return false;
        }
        $this->items[$this->size] = $value;
        $this->shitUp($this->size);
        $this->size++;
        return true;
    }

    public function extractMin()
    {
        $min = $this->peek();
        if (null == $min) {
            return $min;
        }

        $this->items[0] = $this->items[$this->size-1];

        $this->size--;
        unset($this->items[$this->size]);

        $this->shitDown(0);

        return $min;
    }

    private function shitUp($pos)
    {
        if (0 == $pos) {
            return;
        }

        if ($pos&1) {
            $parentPos = ($pos-1)/2;
        } else {
            $parentPos = ($pos-2)/2;
        }

        if ($this->items[$pos] < $this->items[$parentPos]) {
            list($this->items[$pos], $this->items[$parentPos]) = [
                $this->items[$parentPos],
                $this->items[$pos],
            ];
            $this->shitUp($parentPos);
        }
    }

    private function shitDown($pos)
    {
        $left = $pos * 2 + 1;
        $right = $pos * 2 + 2;
        $mpos = $pos;
        if (!empty($this->items[$left]) && $this->items[$left] < $this->items[$pos]) {
            $mpos = $left;
        }
        if (!empty($this->items[$right]) && $this->items[$right] < $this->items[$mpos]) {
            $mpos = $right;
        }

        if ($mpos != $pos) {
            list($this->items[$pos], $this->items[$mpos]) = [
                $this->items[$mpos],
                $this->items[$pos],
            ];
            $this->shitDown($mpos);
        }
    }

    public function display()
    {
        $this->_display(0, 0);
    }

    private function _display($root, $layout)
    {
        if (empty($this->items[$root])) {
            return;
        }

        echo str_repeat("\t", $layout). $this->items[$root], PHP_EOL;

        $left = $root*2 +1;
        $right = $root*2 + 2;
        $layout += 1;

        if (!empty($this->items[$left])) {
            $this->_display($left, $layout);
        }

        if (!empty($this->items[$right])) {
            $this->_display($right, $layout);
        }
    }

    public static function main()
    {
        $heap = new self(10);
        foreach ([1, 3, 5, 4, 8, 9, 10] as $value) {
            $heap->insert($value);
        }

        echo '遍历显示堆', PHP_EOL;
        $heap->display();

        echo "peek", PHP_EOL;
        echo $heap->peek(), PHP_EOL;

        echo "extractMin", PHP_EOL;
        echo $heap->extractMin(), PHP_EOL;
        echo $heap->extractMin(), PHP_EOL;
        echo $heap->extractMin(), PHP_EOL;

        echo "peek", PHP_EOL;
        echo $heap->peek(), PHP_EOL;

        echo "extractMin", PHP_EOL;
        echo $heap->extractMin(), PHP_EOL;
        echo $heap->extractMin(), PHP_EOL;

        echo '遍历显示堆', PHP_EOL;
        $heap->display();
    }
}

Heap::main();

运行结果:


遍历显示堆
1
        3
                4
                8
        5
                9
                10
peek
1
extractMin
1
3
4
peek
5
extractMin
5
8
遍历显示堆
9
        10

二叉堆排序代码如下:

<?php

class HeapSort
{
    private $items;

    public function __construct(array $items)
    {
        $this->items = $items;
    }

    public function sort()
    {
        $arrLen = count($this->items);
        $this->buildMaxHeap($arrLen);

        for ($i = $arrLen-1; $i >= 0; $i--) {
            list($this->items[0], $this->items[$i]) = [
                $this->items[$i],
                $this->items[0]
            ];

            $arrLen -= 1;
            $this->heapify(0, $arrLen);
        }

        return $this->items;
    }

    private function buildMaxHeap($arrLen)
    {
        for ($i = intval($arrLen/2); $i >= 0; $i--) {
            $this->heapify($i, $arrLen);
        }
    }

    private function heapify($i, $arrLen)
    {
        $left = 2 * $i + 1;
        $right = 2 * $i + 2;
        $largest = $i;

        if ($left < $arrLen && $this->items[$left] > $this->items[$largest]) {
            $largest = $left;
        }
        if ($right < $arrLen && $this->items[$right] > $this->items[$largest]) {
            $largest = $right;
        }
        if ($largest != $i) {
            list($this->items[$i], $this->items[$largest]) = [
                $this->items[$largest],
                $this->items[$i]
            ];
            $this->heapify($largest, $arrLen);
        }
    }

    public static function main()
    {
        $items = [
            1,
            5,
            3,
            6,
            2,
            7,
            4,
        ];
        echo '排序前', PHP_EOL;
        var_dump($items);

        $self = new self($items);
        echo '排序后', PHP_EOL;
        var_dump($self->sort());

    }
}

HeapSort::main();

运行结果:

/usr/local/opt/php@7.1/bin/php /Users/martin/heapsort.php
排序前
/Users/martin/heapsort.php:70:
array(7) {
  [0] =>
  int(1)
  [1] =>
  int(5)
  [2] =>
  int(3)
  [3] =>
  int(6)
  [4] =>
  int(2)
  [5] =>
  int(7)
  [6] =>
  int(4)
}
排序后
/Users/martin/heapsort.php:74:
array(7) {
  [0] =>
  int(1)
  [1] =>
  int(2)
  [2] =>
  int(3)
  [3] =>
  int(4)
  [4] =>
  int(5)
  [5] =>
  int(6)
  [6] =>
  int(7)
}

延伸

  • 优先队列
  • 堆排序
  • 海量实数中(一亿级别以上)找到TopK(一万级别以下)的数集合。

参考

  • https://blog.csdn.net/u014532901/article/details/78573556
  • https://zhuanlan.zhihu.com/p/78146654

Trie

介绍

Trie树,又称单词查找树、字典树,是一种树形结构,是一种哈希树的变种,是一种用于快速检索的多叉树结构。典型应用是用于统计和排序大量的字符串(但不仅限于字符串),所以经常被搜索引擎系统用于文本词频统计。

它的优点是:最大限度地减少无谓的字符串比较,查询效率比哈希表高。Trie 的强大之处就在于它的时间复杂度,插入和查询的效率很高,都为O(N),其中 N 是待插入/查询的字符串的长度,而与 Trie 中保存了多少个元素无关。 Trie的核心思想是空间换时间。利用字符串的公共前缀来降低查询时间的开销以达到提高效率的目的。 Trie树也有它的缺点,Trie树的内存消耗非常大。

三个基本特性

1)根节点不包含字符,除根节点外每一个节点都只包含一个字符。  

2)从根节点到某一节点,路径上经过的字符连接起来,为该节点对应的字符串。 

3)每个节点的所有子节点包含的字符都不相同。

用PHP实现一版。


<?php

class TrieNode
{
    // <char, TrieNode>
    public $children = [];
    public $isEnfOfWord = false;
}

class Trie
{
    public $root;

    public function __construct()
    {
        $this->root = new TrieNode;
    }

    public function insert(string $word)
    {
        $current = $this->root;

        $l = strlen($word);
        for ($i = 0; $i < $l; $i++) {
            $c = $word[$i];
            if (empty($current->children[$c])) {
                $current->children[$c] = new TrieNode;
            }
            $current = $current->children[$c];
        }
        $current->isEnfOfWord = true;
    }

    public function search(string $word)
    {
        $current = $this->root;

        $l = strlen($word);
        for ($i = 0; $i < $l; $i++) {
            $c = $word[$i];
            if (empty($current->children[$c])) {
                return false;
            }
            $current = $current->children[$c];
        }

        return ($current && $current->isEnfOfWord);
    }

    private function _remove($root, string $word, int $depth)
    {
        if (!$root) {
            return null;
        }

        // If last character of key is being processed
        if ($depth == strlen($word)) {
            // This node is no more end of word after removal of given key
            if ($root->isEnfOfWord) {
                $root->isEnfOfWord = false;
            }

            // If given is not prefix of any other word
            if ($this->isEmpty($root)) {
                $root = null;
            }
            return $root;
        }

        // If not last character, recur for the child
        // obtained using ASCII value
        $root->children[$word[$depth]] = $this->_remove(
            $root->children[$word[$depth]],
            $word,
            $depth + 1
        );

        // If root does not have any child (its only child got
        // deleted), and it is not end of another word.
        if ($this->isEmpty($root) && $root->isEnfOfWord == false) {
            $root = null;
        }

        return $root;
    }

    public function remove($word)
    {
        return $this->_remove($this->root, $word, 0);
    }

    public function isEmpty($root)
    {
        foreach ($root->children as $child) {
            if (!empty($child)) {
                return false;
            }
        }
        return true;
    }

    public function display()
    {
        $words = [];
        $this->_display($this->root, '', $words);
        foreach ($words as $word) {
            echo $word, PHP_EOL;
        }
    }

    private function _display($root, $prefix, &$words)
    {
        if (empty($root->children)) {
            return;
        }
        $old = $prefix;
        foreach ($root->children as $char => $child) {
            if (empty($child)) {
                continue;
            }
            $prefix = $old . $char;
            if ($child->isEnfOfWord) {
                $words[] = $prefix;
            }
            $this->_display($child, $prefix, $words);
        }
    }

    public function wordCount()
    {
        return $this->_wordCount($this->root);
    }

    private function _wordCount($root)
    {
        $result = 0;
        if ($root->isEnfOfWord) {
            $result++;
        }
        foreach ($root->children as $char => $child) {
            $result += $this->_wordCount($child);
        }

        return $result;
    }

    public static function main()
    {
        // Input keys (use only 'a' through 'z' and lower case)
        $keys = ["the", "a", "there", "answer", "any", "by", "bye", "their"];
        $output = ["Not present in trie", "Present in trie"];

        $trie = new Trie;
        foreach ($keys as $key) {
            $trie->insert($key);
        }
        $trie->remove('by');

        $trie->display();
        echo $trie->wordCount(), PHP_EOL;

    }
}

Trie::main();

延伸

  • 使用Trie实现一个字典
    class TrieNode
    {
      // <char, TrieNode>
      public $children = [];
      public $isEnfOfWord = false;
      public $meaning = ''; // 字典的释义
    }
    
  • 词频统计
    class TrieNode
    {
      // <char, TrieNode>
      public $children = [];
      public $isEnfOfWord = false;
      public $wordFreq = 0; // 词频统计
    }
    
  • 字典序排序
  • 前缀匹配
  • 最长公共前缀

问题实例

1、一个文本文件,大约有一万行,每行一个词,要求统计出其中最频繁出现的前10个词,请给出思想,给出时间复杂度分析

提示:用trie树统计每个词出现的次数,时间复杂度是O(nle)(le表示单词的平均长度),然后是找出出现最频繁的前10个词。当然,也可以用堆来实现,时间复杂度是O(nlg10)。所以总的时间复杂度,是O(nle)与O(nlg10)中较大的哪一个。

2、寻找热门查询

原题:搜索引擎会通过日志文件把用户每次检索使用的所有检索串都记录下来,每个查询串的长度为1-255字节。假设目前有一千万个记录,这些查询串的重复读比较高,虽然总数是1千万,但是如果去除重复和,不超过3百万个。一个查询串的重复度越高,说明查询它的用户越多,也就越热门。请你统计最热门的10个查询串,要求使用的内存不能超过1G。

提示:利用trie树,关键字域存该查询串出现的次数,没有出现为0。最后用10个元素的最小推来对出现频率进行排序。

参考

  • https://blog.csdn.net/hguisu/article/details/8131559
  • https://github.com/julycoding/The-Art-Of-Programming-By-July/blob/master/ebook/zh/06.09.md

HashTable

HashTable是一种通过将键(key)映射为值(value)从而实现快速查找的数据结构。实现HashTable的方式有很多种。本次介绍2种,并给出第一种的PHP版本实现。

我们使用一个链表构成的数组与一个散列函数来实现散列表。当插入键(字符串或几乎其他所有数据类型)和值时,我们按照如下方法操作。

(1) 首先,计算键的散列值。键的散列值通常为int或者long型。请注意,不同的两个键可以有相同的散列值,因为键的数量是无穷的,而int型的总数是有限的。

(2) 之后,将散列值映射为数组的索引。可以使用类似于hash(key) % array_length的方式完成这一步骤,不同的两个散列值则会被映射到相同的数组索引。

(3) 此数组索引处存储的元素是一系列由键和值为元素组成的链表。请将映射到此索引的键和值存储在这里。由于存在冲突,我们必须使用链表:有可能对于相同的散列值有不同的键,也有可能不同的散列值被映射到了同一个索引。

通过键来获取值则需重复此过程。首先通过键计算散列值,再通过散列值计算索引。之后,查找链表来获取该键所对应的值。

如果冲突发生很多次,最坏情况下的时间复杂度是O(N),其中N是键的数量。但是,我们通常假设一个不错的实现方式会将冲突数量保持在最低水平,在此情况下,时间复杂度是O(1)。

另一种方法是通过平衡二叉搜索树来实现散列表。该方法的查找时间是O(logN)。该方法的好处是用到的空间可能更少,因为我们不再需要分配一个大数组。还可以按照键的顺序进行迭代访问,在某些时候这样做很有用

我们实现一个PHP版的HashTable, 数据结构ElementHashTable,散列函数,我们使用Times33*hash。 功能:添加(覆盖)、删除、获取,内部隐藏逻辑rehashrehash两种情况,一种是扩容、一种是缩容。基于isRehash进行判断。

<?php

class Element
{
    public $key;
    public $value;
    public $hash;

    public function __construct($key, $value, $hash)
    {
        $this->key = $key;
        $this->value = $value;
        $this->hash = $hash;
    }
}

class HashTable
{
    private $item = [];
    public $length;
    public $useElement;

    public function __construct()
    {
        $this->initHashTable();
    }

    public function get($key):?string
    {
        $e = $this->_has($key);
        if ($e instanceof Element) {
            return $e->value;
        }
        return null;
    }

    public function store($key, $value): void
    {
        if ($this->isRehash()) {
            $this->rehash();
        }

        $hash = $this->hash($key);
        $slot = $hash % $this->length;

        $element = $this->_has($key);
        if ($element instanceof Element) {
            if ($element->value !== $value) {
                foreach ($this->item[$slot] as $element) {
                    if ($element->key == $key) {
                        $element->value = $value;
                        return;
                    }
                }
            }
        } else {
            if (empty($this->item[$slot])) {
                $this->item[$slot] = [];
            }
            array_push($this->item[$slot], new Element($key, $value, $hash));
            $this->useElement++;
        }

        return;
    }

    public function del($key):?bool
    {
        if (($e = $this->_has($key)) instanceof Element) {
            $this->_del($e);
            return true;
        } else {
            return true;
        }
    }

    private function _del(Element $delElement)
    {
        foreach ($this->item[$slot = ($delElement->hash % $this->length)] as $k => $e) {
            if ($e->key == $delElement->key) {
                unset($this->item[$slot][$k]);
            }
        }

        $this->useElement--;

        if ($this->isRehash()) {
            $this->rehash();
        }
    }

    public function has($key)
    {
        if ($this->_has($key) instanceof Element) {
            return true;
        } else {
            return false;
        }
    }

    private function _has($key):?Element
    {
        $hash = $this->hash($key);
        $slot = $hash % $this->length;

        if (empty($this->item[$slot])) {
            return null;
        }

        foreach ($this->item[$slot] as $element) {
            if ($element->key == $key) {
                return $element;
            }
        }

        return null;
    }

    private function hash(string $key)
    {
        if (is_numeric($key)) {
            return $key;
        } elseif (is_string($key)) {
            $hash = 5381;
            $l = strlen($key);
            for ($i = 0; $i < $l; $i++) {
                $num = ord($key[$i]);
                $hash += $num + $hash << 5 + $hash;
            }
            return $hash;
        }
    }

    private function initHashTable()
    {
        $this->length = 8;
        $this->useElement = 0;
    }

    private function isRehash():bool
    {
        if ($this->useElement > $this->length
            || ($this->useElement > 8 && 2*$this->useElement < $this->length)) {
            $this->rehash();
            echo 'rehash',PHP_EOL;
            return true;
        } else {
            return false;
        }
    }

    private function rehash()
    {
        if ($this->useElement > $this->length) {
            $newLength = 2 * $this->length;
        } else {
            $newLength = $this->length / 2;
        }

        $newItem = [];

        foreach ($this->item as $slot => $items) {
            foreach ($items as $item) {
                $slot = ($item->hash = $this->hash($item->key)) % $newLength;

                if (empty($newItem[$slot])) {
                    $newItem[$slot] = [];
                }
                array_push($newItem[$slot], $item);
            }
        }

        $this->length = $newLength;
        $this->item = $newItem;
    }

    public function debugHashTable()
    {
        echo json_encode($this->item, 128);
    }

    public static function main() {

        $hashTable = new HashTable();

        $items = [
            'key' => '1212',
            'c' => '12311',
            122 => '1231',
            'key1' => '1212',
            'c1' => '12311',
            1221 => '1231',
            'ke' => '1212',
            'ky1' => '1212',
            'ey1' => '1212',
            'ky' => '1212',
        ];

        foreach ($items as $k => $v) {
            $hashTable->store($k, $v);
        }

        foreach ($items as $k => $v) {
            echo $k, ":", $hashTable->get($k), PHP_EOL;
        }

        echo '添加key后结果:',PHP_EOL;
        $hashTable->debugHashTable();

        foreach ($items as $k => $v) {
            $hashTable->del($k);
            if ($k == 'ey1') {
                break;
            }
        }

        echo '删除key后,结果:',PHP_EOL;
        $hashTable->debugHashTable();
        echo $hashTable->length;
    }
}

HashTable::main();

判断一个哈希算法的好坏有以下几点:

  • 一致性,等价的键必然产生相等的哈希值;
  • 高效性,计算简便;
  • 均匀性,均匀地对所有的键进行哈希。

更多延伸阅读

  • php HashTable
  • redis HashTable
  • php vs redis rehash方式为什么不一样
  • hash 函数实现方式:Times33(djb2), sdbm,lose

优化task-log索引

背景

task log data

索引表数据过大,基于场景,分析该表就是统计任务写入状态使用,便于每天汇总报告.

解决方案

利用Index Template 格式如下:

PUT _template/task_log
{
	"index_patterns": ["task-log-*"],
	"settings": {
		"number_of_shards": 3
	},
	"mappings": {
		"_source": {
			"enabled": false
		},
		"dynamic": false,
		"properties": {
			"id": {
				"type": "long"
			},
			"success": {
				"type": "boolean"
			},
			"createtime": {
				"type": "date",
				"format": "yyyy-MM-dd HH:mm:ss"
			}
		}
	},
	"aliases": {
	  "task-log": {}
	},
	"version": 2
}

按天创建索引,而且支持动态创建(task_log Template)

POST task-log-20200325/_doc
{
  "id":1011221,
  "success":false,
  "createtime":"2020-12-12 13:12:12"
}

分析数据(每天的成功个数和失败个数)

GET task-log/_search
{
  "size": 0,
  "aggs": {
    "avg_day": {
      "date_histogram": {
        "field": "createtime",
        "interval": "day"
      },
      "aggs": {
        "success": {
          "terms": {
            "field": "success",
            "size": 10
          }
        }
      }

    }
  }
}

结果:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 7,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "avg_day" : {
      "buckets" : [
        {
          "key_as_string" : "2020-12-12 00:00:00",
          "key" : 1607731200000,
          "doc_count" : 7,
          "success" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : 1,
                "key_as_string" : "true",
                "doc_count" : 5
              },
              {
                "key" : 0,
                "key_as_string" : "false",
                "doc_count" : 2
              }
            ]
          }
        }
      ]
    }
  }
}

基本功能实现了,原来的_source数据不存在了,一条省个及字节,一天几亿条,那就节省不少空间。

优化点

  1. _source enable false
  2. index template 支持version, alias, mapping
  3. 移除副本配置,缩小不必要的数据空间
  4. 一个存储日志的主分片理论值50G,三个主分片可容纳150G数据,基于现有每天的数据量按每条1KB,可容纳150亿条数据,足矣满足现在业务使用场景。

参考

https://blog.csdn.net/napoay/article/details/62233031

学习费曼学习法

费曼学习法

学习Swoole

学习Swoole需要掌握哪些基础知识

多进程/多线程

  • 了解Linux操作系统进程和线程的概念
  • 了解Linux进程/线程切换调度的基本知识
  • 了解进程间通信的基本知识,如管道、UnixSocket、消息队列、共享内存

SOCKET

  • 了解SOCKET的基本操作如accept/connect、send/recv、close、listen、bind
  • 了解SOCKET的接收缓存区、发送缓存区、阻塞/非阻塞、超时等概念

IO复用

  • 了解select/poll/epoll
  • 了解基于select/epoll实现的事件循环,Reactor模型
  • 了解可读事件、可写事件

TCP/IP网络协议

  • 了解TCP/IP协议
  • 了解TCP、UDP传输协议

调试工具

  • 使用 gdb 调试Linux程序
  • 使用 strace 跟踪进程的系统调用
  • 使用 tcpdump 跟踪网络通信过程
  • 其他Linux系统工具,如ps、lsof、top、vmstat、netstat、sar、ss等

概括一句话读《linux高性能服务器编程》

Swoole 图

类图

swoole class

运行流程图

运行流程图

进程 / 线程结构图

进程 / 线程结构图

结构图2

实践代码

可以用代码中作”基酒” -> sf-swoole-console

Docker运行环境:https://hub.docker.com/r/phpswoole/swoole

简单写了一个mapreduce 的add 加法:feature-op-add

docker run -p 9501:9501 -it -v $(pwd):/var/html phpswoole/swoole  /bin/bash
cd /var/html
php console task:server start
$ telnet 127.0.0.1 9501
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
{"op":"add", "data":[1,2]}
3

自定义协议

EOF结束符

发送者和接收者约定数据包已一个特殊的结束符(EOF)做结尾。适合协议相对简单的需求,常见的比如 redis、memcache、ftp、stmp等都是用\r\n换行符作为结束符。

固定包头+包体协议

固定包头+包体协议

<?php
// server.php
$serv = new Swoole\Server("127.0.0.1", 9501);
$serv->set(array(
    'open_length_check' => true,
    'package_length_type' => 'n',
    'package_length_offset' => 6,
    'package_body_offset' => 8,
    'package_max_length' => 2000,
));
$serv->on('Connect', function ($serv, $fd) {
    echo "Client: Connect.\n";
});
$serv->on('Receive', function ($serv, $fd, $from_id, $data) {
	$header = substr($data, 0, 8);
	$p = unpack('a6begin/nbodyLen', $header);
	if ($p['begin'] != 'SWOOLE'){
		return;
	}
	$len = $p['bodyLen'];
	$bodyPack = unpack("a{$len}body", substr($data, 8, $len));
    $serv->send($fd, "Server: ".$bodyPack['body']."\n");
});
$serv->on('Close', function ($serv, $fd) {
    echo "Client: Close.\n";
});
$serv->start();
<?php
//client.php
$client = new swoole_client(SWOOLE_SOCK_TCP);
if (!$client->connect('127.0.0.1', 9501)) {
	exit("connect failed. Error: {$client->errCode}\n");
}
$msg = 'Hello World!';
$client->send(sendMsg($msg)); // 正常发包
$client->send(sendMsg($msg.'0').sendMsg($msg.'1').sendMsg($msg.'2'));// 模拟粘包
echo $client->recv();
$client->close();
function sendMsg($msg) {
	$p = 'SWOOLE';
	$p .= pack('n', strlen($msg));
	$p .= pack('a' . strlen($msg), $msg);
	return $p;
}

抓包数据分析

13:36:39.050774 IP 127.0.0.1.33844 > 127.0.0.1.9501: Flags [P.], seq 1:21, ack 1, win 342, options [nop,nop,TS val 4907685 ecr 4907685], length 20
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  0048 a56a 4000 4006 9743 7f00 0001 7f00  .H.j@.@..C......
        0x0020:  0001 8434 251d 496d 47ce 8446 849b 8018  ...4%.ImG..F....
        0x0030:  0156 fe3c 0000 0101 080a 004a e2a5 004a  .V.<.......J...J
        0x0040:  e2a5 5357 4f4f 4c45 000c 4865 6c6c 6f20  ..SWOOLE..Hello.
        0x0050:  576f 726c 6421                           World!
13:36:39.050820 IP 127.0.0.1.9501 > 127.0.0.1.33844: Flags [.], ack 21, win 342, options [nop,nop,TS val 4907685 ecr 4907685], length 0
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  0034 663e 4000 4006 d683 7f00 0001 7f00  .4f>@.@.........
        0x0020:  0001 251d 8434 8446 849b 496d 47e2 8010  ..%..4.F..ImG...
        0x0030:  0156 fe28 0000 0101 080a 004a e2a5 004a  .V.(.......J...J
        0x0040:  e2a5                                     ..
13:36:39.050862 IP 127.0.0.1.33844 > 127.0.0.1.9501: Flags [P.], seq 21:84, ack 1, win 342, options [nop,nop,TS val 4907685 ecr 4907685], length 63
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  0073 a56b 4000 4006 9717 7f00 0001 7f00  .s.k@.@.........
        0x0020:  0001 8434 251d 496d 47e2 8446 849b 8018  ...4%.ImG..F....
        0x0030:  0156 fe67 0000 0101 080a 004a e2a5 004a  .V.g.......J...J
        0x0040:  e2a5 5357 4f4f 4c45 000d 4865 6c6c 6f20  ..SWOOLE..Hello.   //这里
        0x0050:  576f 726c 6421 3053 574f 4f4c 4500 0d48  World!0SWOOLE..H
        0x0060:  656c 6c6f 2057 6f72 6c64 2131 5357 4f4f  ello.World!1SWOO
        0x0070:  4c45 000d 4865 6c6c 6f20 576f 726c 6421  LE..Hello.World!
        0x0080:  32                                       2
13:36:39.050885 IP 127.0.0.1.9501 > 127.0.0.1.33844: Flags [.], ack 84, win 342, options [nop,nop,TS val 4907685 ecr 4907685], length 0
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  0034 663f 4000 4006 d682 7f00 0001 7f00  .4f?@.@.........
        0x0020:  0001 251d 8434 8446 849b 496d 4821 8010  ..%..4.F..ImH!..
        0x0030:  0156 fe28 0000 0101 080a 004a e2a5 004a  .V.(.......J...J
        0x0040:  e2a5                                     ..
13:36:39.051626 IP 127.0.0.1.9501 > 127.0.0.1.33844: Flags [P.], seq 1:22, ack 84, win 342, options [nop,nop,TS val 4907685 ecr 4907685], length 21
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  0049 6640 4000 4006 d66c 7f00 0001 7f00  .If@@.@..l......
        0x0020:  0001 251d 8434 8446 849b 496d 4821 8018  ..%..4.F..ImH!..
        0x0030:  0156 fe3d 0000 0101 080a 004a e2a5 004a  .V.=.......J...J
        0x0040:  e2a5 5365 7276 6572 3a20 4865 6c6c 6f20  ..Server:.Hello. // 这里
        0x0050:  576f 726c 6421 0a                        World!.


再来一个漂亮版本的代码

RPC 编码协议类


<?php

namespace Superman2014\SfSwooleConsole;

/**
 * Class Protocol
 * @package Superman2014\SfSwooleConsole
 *
 *
 * 包结构
 *
 * 字段	字节数	说明
 * 包头	定长	每一个通信消息必须包含的内容
 * 包体	不定长	根据每个通信消息的不同产生变化
 *
 * 其中包头详细内容如下:
 *
 * 字段        字节数 类型  说明
 * pkg_len	    2   ushort	整个包的长度,不超过4K
 * version	    1 	uchar	通讯协议版本号
 * command_id	2 	ushort	消息命令ID
 * result	    2 	short	请求时不起作用;请求返回时使用
 *
 */
class Protocol
{

    const VERSION = '1';

    const HEADER = 7;

    const PACKAGE_LENGTH = 4096;

    public static function partHeader($bodyLen, $version, $commandId, $result)
    {
        return pack("nCns", $bodyLen, $version, $commandId, $result);
    }

    public static function partBody($msg)
    {
        return pack("a". strlen($msg), $msg);
    }

    public static function encode($msg, $commandId, $result = 0)
    {

        return self::partHeader(strlen($msg), self::VERSION, $commandId, $result)
            . self::partBody($msg);

    }

    public static function decode($msg)
    {

        $header = substr($msg, 0, self::HEADER);
        $p = unpack('nbodyLen/Cversion/ncommandId/sresult', $header);
        $len = $p['bodyLen'];
        $bodyPack = unpack("a{$len}body", substr($msg, self::HEADER, $len));
        return array_merge($bodyPack, $p);
    }

    public static function main()
    {
        $msg = self::encode("hello", "1001", 0);

        var_dump(self::decode($msg));

        //
//        array(5) {
//        ["body"]=>
//  string(5) "hello"
//        ["bodyLen"]=>
//  int(5)
//  ["version"]=>
//  int(1)
//  ["commandId"]=>
//  int(1001)
//  ["result"]=>
//  int(0)
//    }
    }
}

//Protocol::main();

常量类

<?php

namespace Superman2014\SfSwooleConsole;

class TaskConstant
{
    const COMMAND_SET = [
        self::START,
        self::STOP,
        self::RESTART,
        self::RELOAD,
        self::PING,
        self::STATUS,
        self::USAGE,
    ];

    const USER_COMMAND = [
        self::STATUS,
        self::RELOAD,
        self::RESTART,
        self::STOP,
    ];

    const START = 'start';

    const STOP = 'stop';

    const STATUS = 'status';

    const RELOAD = 'reload';

    const RESTART = 'restart';

    const PING = 'ping';

    const USAGE = 'usage';

    const DATA = 'data';

    const COMMAND_ID_LIST = [
        1001 => self::START,
        1002 => self::STOP,
        1003 => self::RESTART,
        1004 => self::RELOAD,
        1005 => self::PING,
        1006 => self::STATUS,
        1007 => self::DATA,
    ];

    public static function commandIdByName($name)
    {
       return array_flip(self::COMMAND_ID_LIST)[$name];
    }
}

<?php

namespace Superman2014\SfSwooleConsole;

use Swoole\Server;
use Swoole\Process;
use Swoole\Client;
use Swoole\Timer;

class TaskServer
{
    const NAME = 'sf-swoole-console';

    public $config = [
        'worker_num' => 4,
        'task_worker_num' => 4,
        'daemonize' => true,
        'backlog' => 128,
        'log_file' => '/tmp/swoole.log',
        'log_level' => 0,
        'task_ipc_mode' => 3,
        'heartbeat_check_interval' => 5,
        'heartbeat_idle_time' => 10,
        'pid_file' => '/tmp/sf-swoole-console.pid',

        'open_length_check' => true,
        'package_length_type' => 'n',
        'package_length_offset' => 0,
        'package_body_offset' => Protocol::HEADER,
        'package_max_length' => Protocol::PACKAGE_LENGTH,
    ];

    const LISTEN_HOST = '0.0.0.0';

    const MANAGE_HOST = '127.0.0.1';

    const PORT = 9501;

    public function __construct($command)
    {
        switch ($command) {
            case TaskConstant::START:
                $this->start();
                break;
            case TaskConstant::STOP:
                 $this->clientSendCommand()(TaskConstant::STOP);
                break;
            case TaskConstant::STATUS:
                $recv = $this->clientSendCommand()(TaskConstant::STATUS);
                var_dump($recv['body']);
                break;
            case TaskConstant::PING:
                var_dump($this->clientSendCommand()(TaskConstant::PING));
                break;
            case TaskConstant::RELOAD:
                $recv = $this->clientSendCommand()( TaskConstant::RELOAD);
                echo $recv['body'],PHP_EOL;
                break;
            case TaskConstant::RESTART:
                $recv = $this->clientSendCommand()( TaskConstant::RESTART);
                echo $recv['body'],PHP_EOL;
                break;
        }
    }

    public function clientSendCommand()
    {
        return function ($command) {
            $client = new Client(SWOOLE_SOCK_TCP);
            if (!$client->connect(self::MANAGE_HOST, self::PORT, -1)) {
                exit("connect failed. Error: {$client->errCode}\n");
            }

            if (in_array($command, TaskConstant::COMMAND_SET)) {
                $client->send(Protocol::encode($command, TaskConstant::commandIdByName($command)));
            } else {
                $client->send(Protocol::encode($command, TaskConstant::commandIdByName(TaskConstant::DATA)));
            }

            $recv = $client->recv();
            $client->close();
            return Protocol::decode($recv);
        };
    }

    public function start()
    {
        $server = new Server(self::LISTEN_HOST, self::PORT, SWOOLE_BASE, SWOOLE_SOCK_TCP);

        $server->set($this->config);

        $server->on('Connect', [$this, 'onConnect']);
        $server->on('Close', [$this, 'onClose']);
        $server->on('Task', [$this, 'onTask']);
        $server->on('Finish', [$this, 'onFinish']);
//        $server->on('Start', [$this, 'onStart']); // SWOOLE_BASE 不存在
        $server->on('WorkerStart', [$this, 'onWorkerStart']);
        $server->on('WorkerStop', [$this, 'onWorkerStop']);
        $server->on('ManagerStart', [$this, 'onManagerStart']);
        $server->on('Shutdown', [$this, 'onShutdown']);
        $server->on('WorkerExit', [$this, 'onWorkerExit']);

        /**
         * 用户进程实现了广播功能,循环接收unixSocket的消息,并发给服务器的所有连接
         */
        $process = new Process(
            function ($process) use ($server) {
                $socket = $process->exportSocket();
                while (true) {
                    $msg = $socket->recv();

                    $command = TaskConstant::COMMAND_ID_LIST[$msg];
                    if ($command == TaskConstant::STATUS) {
                        $socket->send(json_encode($server->stats()));
                    } elseif ($command == TaskConstant::RELOAD) {
                        $server->reload(true);
                        $socket->send('reload ok');
                    } elseif ($command == TaskConstant::RESTART) {
                        Process::kill($server->manager_pid, SIGUSR1);
                        Process::kill($server->manager_pid, SIGUSR2);
                        $socket->send('restart ok');
                    } elseif ($command == TaskConstant::STOP) {
                        $server->shutdown();
                    }
                }
            },
            false,
            2,
            1
        );

        $server->addProcess($process);

        $server->on('Receive', function ($server, $fd, $reactorId, $data) use ($process) {
            $msg = Protocol::decode($data);
            if (in_array($c = TaskConstant::COMMAND_ID_LIST[$msg['commandId']], TaskConstant::USER_COMMAND)) {
                $socket = $process->exportSocket();
                $socket->send($msg['commandId']);
                $server->send($fd, Protocol::encode($socket->recv(), TaskConstant::commandIdByName(TaskConstant::DATA)));
            } else {
                $this->onReceive($server, $fd, $reactorId, $msg);
            }
        });

        $server->start();
    }

    //此回调函数在worker进程中执行
    public function onReceive(Server $server, int $fd, int $reactorId, $data)
    {
        $server->send($fd, Protocol::encode($data['body'], $data['commandId']));
    }

	// ....
}

Server 的基本基本命令stop restart ping stop 就是使用协议里面定义的commandId。 后续还会这里进行深入研究。

跟编解码类似的如编程中序列化,反序列化. 可以看看这边文档 序列化和反序列化

参考