360问答缓存缓存收敛降低维护成本

场景：
近期发现维护多个Nosql技术成本太高昂，不仅有Memcahce、Redis 而且每个Nosql 开的端口太多虽说能把业务分开管理，但是在监控和维护上带来很多工作量，所以我准备做一件能让缓存端口收敛的工作，让维护变得简单，释放人力成本,说干就干,提出申请，获得通过后就提上改造之路.

要实现的目标：

1. 减少端口数量少、便于维护
    2. 公用内存大小
    3. 借助RedisCluster来做缓存，便于横向拓展
    4. 放弃Memcache

分享下处理之路:

列出工具类代码的结构：

wd_redis.php       redis连接工具类
    wd_memcache.php    memcache 连接工具类
    wd_cacheBase.php   缓存基类,新增的，作为后续转换和切换工具类.
    wd_redisCluster.php 新增的RedisCluster类

一、准备工作,统计下问答下面使用的所有Nosql端口、以及目前使用的缓存空间大小，申请ReidsCluster资源.

![Redis.png][1]
![Memcahce.png][2]
以上为Redis 和 Memcache 两个模块的资源、以及当前承载性能、使用大小等统计。拿到以上数据后便于我们进行资源申请.

二、切换代码编写.
宗旨：
2.1 此代码类涵盖功能

2.1.1 容错
    2.1.2 便于回滚
    2.1.3 起过度作用(双写)、便于代码切换.

代码CacheBase.php在最底部，在上线过程只需要吧项目中元来调用 wd_Memcache 和 wd_redis的模块全部用来实例化 
new CacheBase 即可，其他的都不用变。控制上线、回滚、过渡都采用CacheBase中端口对应的配置来控制.

三、逐步切换上线

3.1 完成一个端口的代码改造后就把CacheBase.php中端口对应的配置属性
    codeok 和 dwrite 两个属性开启.
    3.2 上线过度后，开启 open属性、即为切换使用redis-cluster集群代替原来的 Memcache 和 Redis工作

四、观察、监控

4.1 逐个接口处理后，观察运行情况，以及项目日志，确认有误异常.
    4.2 针对新的RedisCluster端口增加监控.

五、逐步关闭过度，查看流量，下掉端口、完成任务.

5.1 逐步对端口关闭dbwrite 双写属性，断掉老的缓存使用，查看端口链接统计数，看是否还有链接数.逐步下掉端口.

应二. CacheBase.php 代码

<?php
    /**
     * @Desc:此类用户升级Cache缓存双写设计
     * @Auth:BraveDU<duchaoqun@360.cn>
     * @Date:2017-2-21 17:17:12
     */
    
    /**
     * How to use it.
     * $newConf 为新缓存所用到的config配置
     * $oldConf 为老流程中正在使用的config配置
     * $useConf = array(
     *      'old' => 'memcache',  //意思为老流程用的是memcache
     *      'new' => 'redis'      //新流程使用的缓存
     *      'use' => 'redis',     //当前在用的缓存
     *      'dwrite' => true,     //是否进行双写
     * )
     * $cache = new wd_CacheBase($oldConf)
     */
    include_once('Memcache.php');
    include_once('Redis.php');
    include_once('RedisCluster.php');
    
    class wd_CacheBase {
        private $useConf = array();
        private $cacheType = array('redis','rediscluster','memcache');
        private $oldConf;
        private $dwriteConf;
        private static $redisClsConf = array(
            'bjcc' => array('host'  =>  'xxx.xxx.153.42','port'  =>  8888,'timeout' => 0.5),
            'bjyt' => array('host'  =>  'xxx.xxx.255.223','port'  => 8889,'timeout' => 0.5),
        );
          
        function __construct($oldConf=array(),$rdsClusterConf=''){
            if(empty($oldConf)) return false;
            if($rdsClusterConf){
                $this->newConf = $rdsClusterConf;
            }else{
                $idcGroup = self::getIDCname();
                $this->newConf = self::$redisClsConf[$idcGroup];
                if(empty($this->newConf)){
                    return false;
                }
            }
            $this->oldConf = $oldConf;
            if(!isset($oldConf[0]['port'])){
                return false;
            }
            $cacheConf = self::$portArr[$oldConf[0]['port']];
    	if(empty($cacheConf)){
    		return false;
    	}
            $this->useConf = array(
                'dwrite'  => $cacheConf['dwrite'],
                'use'     => $cacheConf['open'] ? 'rediscluster' : $cacheConf['type'],
                'old'     => $cacheConf['type'],
                'new'     => 'rediscluster',
            );
            if(!isset($cacheConf['codeok']) || $cacheConf['codeok']!==true){
                return false;
            }
    
        }
    
        static private function getIDCname()
        {/*{{{*/
            $info = posix_uname();
            $hostname = $info['nodename'];
            $hostname = str_replace('.qihoo.net', '', $hostname);
            $arr = explode('.', $hostname);
            $idc = $arr[count($arr) - 1]; 
            if($idc == 'vm') $idc = 'corp';
            return $idc;
        }/*}}}*/
    
        static private $portArr  = array(
            //端口设置
            'xxx' => array(
                'type'   => 'memcache',
                'dwrite' => true,//开启双写
                'open'   => false,//开启切换
                'codeok' => false,//代码已经上线
            ),
            //用于antispam模块
            'xxx' => array(
                'type'   => 'memcache',
                'dwrite' => true,//开启双写
                'open'   => true,//开启切换
                'codeok' => true,//代码已经上线
            ),
    		//....后面省略
            
        );
    
        function __call($method,$args){
            $cacheRes = false;
            if(
                !isset($this->useConf['use']) ||
                empty($this->useConf['use']) || 
                !in_array(strtolower($this->useConf['use']),$this->cacheType)
            ){
                throw new Exception("please set cache type in userConf ~~~~");
            }
            $this->cacheConf = ($this->useConf['use'] == $this->useConf['new']) ? $this->newConf : $this->oldConf;
            //如果是元程序中用的是Memcache,更换成rediscluster,则需要对参数进行处理,防止实例化后变量污染
            $callMethod = $method;
            $callArgs   = $args;
            if($this->useConf['use'] != $this->useConf['old'] && $this->useConf['old']=='memcache'){
                switch($callMethod){
                case 'set' :
                case 'add' :
                    //判断是否设置过期时间
                    if(count($callArgs) >= 3){
                        $callMethod = 'setex';
                        $callArgs = array(
                            $callArgs[0],$callArgs[2],$callArgs[1],
                        );
                    }else{
                        $callMethod = 'set';
                    }
                    if(isset($callArgs[2]) && $callArgs[2] && is_array($callArgs[2])){
                        $callArgs[2]['cachebase_isarr']=1;
                        $callArgs[2] = json_encode($callArgs[2]);
                    }
                    break;
                case 'delete' :
                    $callMethod = 'del';
                    break;
                case 'increment' :
                    $callMethod = count($callArgs) > 1 ? 'incrBy' : 'incr';
                    break;
                case 'decrement' :
                    $callMethod = count($callArgs) > 1 ? 'decrBy' : 'decr';
                    break;
    
                }
                    
            }
            switch(strtolower($this->useConf['use'])){
                case 'redis' :
                    $cacheBuled = new wd_Redis($this->cacheConf);
                    $cacheRes = call_user_func_array(array($cacheBuled,$callMethod),$callArgs);
                    break;
                case 'rediscluster' :
                    $cacheBuled = new wd_RedisCluster($this->cacheConf);
                    $cacheRes = call_user_func_array(array($cacheBuled,$callMethod),$callArgs);
                    if(
                        in_array($callMethod,array('get')) && 
                        $this->useConf['old']=='memcache' && 
                        strpos($cacheRes,'cachebase_isarr')!==false
                    ){
                        $cacheRes = json_decode($cacheRes,true);
                        unset($cacheRes['cachebase_isarr']);
                    }
                    break;
                case 'memcache' :
                    $cacheBuled = new wd_Memcache($this->cacheConf);
                    $cacheRes = call_user_func_array(array($cacheBuled,$callMethod),$callArgs);
                    break;
            }
    
            //判断双写是否开启进行双写
            if($this->useConf['dwrite']===true && $this->useConf['old'] != $this->useConf['new']){
                 $this->dwriteConf = ($this->useConf['use'] == $this->useConf['new']) ? $this->oldConf : $this->newConf;
                 $dCacheType = ($this->useConf['use'] == $this->useConf['new']) ? $this->useConf['old'] : $this->useConf['new'];
                 if($this->disposeData($dCacheType, $method, $args)){
                     switch(strtolower($dCacheType)){
                     case 'redis' :
                         $odCacheBuled = new wd_Redis($this->dwriteConf);
                         $odcacheRes   = call_user_func_array(array($odCacheBuled,$method),$args);
                         break;
                     case 'rediscluster' :
                         $odCacheBuled = new wd_RedisCluster($this->dwriteConf);
                         $odcacheRes   = call_user_func_array(array($odCacheBuled,$method),$args);
                         break;
                     case 'memcache' :
                         $odCacheBuled = new wd_Memcache($this->dwriteConf);
                         $odcacheRes   = call_user_func_array(array($odCacheBuled,$method),$args);
                         break;
                     } 
    
                 }
                 
            }
            return $cacheRes;
        }
    
        /**
         * 数据处理只针对MC写往Redis RedisCluse中生效
         */
        private function disposeData($dCacheType, &$method, &$args){
            $res = false;
            //$mcToredisType = array('set','setex','del','add','incr','incrBy','decr','decrBy');
            //$rdsTomcType   = array('set','delete');
            //MC 往 Redis Redis-C中写
            if($this->useConf['use']=='memcache' && in_array($dCacheType,array('redis','rediscluster'))){
                    switch($method){
                    case 'set' :
                    case 'add' :
                        //判断是否设置过期时间
                        if(count($args) >= 3){
                            $method = 'setex';
                            $args = array(
                                $args[0],$args[2],$args[1],
                            );
                        }else{
                            $method = 'set';
                        }
                        if(isset($args[2]) && $args[2] && is_array($args[2])){
                            $args[2]['cachebase_isarr']=1;
                            $args[2] = json_encode($args[2]);
                        }
                        break;
                    case 'delete' :
                        $method = 'del';
                        break;
                    case 'increment' :
                        $method = count($args) > 1 ? 'incrBy' : 'incr';
                        break;
                    case 'decrement' :
                        $method = count($args) > 1 ? 'decrBy' : 'decr';
                        break;
    
                    }
                    return true; 
            }else if(in_array($this->useConf['use'],array('redis','rediscluster')) && $dCacheType=='memcache'){
                    switch($method){
                    case 'set' :
                        $args = array(
                            $args[0],
                            $args[1],
                            0
                        );
                        break;
                    case 'setex':
                        $method = 'set';
                        $args = array(
                            $args[0],
                            $args[1],
                            $args[2]
                        );
                        break;
                    case 'del':
                        $method = 'delete';
                        break;
                    }
                    return true; 
            
            }else if( in_array($this->useConf['use'],array('redis','rediscluster')) && in_array($dCacheType,array('redis','rediscluster')) ){
               return true; 
            }
    
            return false;
        }
    
    }

[1]: http://blogimg.bravedu.com/2017/05/1850808203.png
  [2]: http://blogimg.bravedu.com/2017/05/3083754394.png

BraveDu

360问答缓存缓存收敛降低维护成本

您可以选择一种方式赞助本站

发表评论