在Linux系统中将SSD当块设备缓存的方法


    原理
     写操作先缓存到ssd硬盘上,然后通过一定策略写到普通硬盘上;
     读操作热点数据可以缓存到ssd硬盘上,提高读取数据的速度。
    软件 ssd 缓存开源软件主要有bcache ,flashcache
    bcache 需要编译最新的内核,要求比较高,配置比较复杂,目前主要用于测试环境;
    flashcache 是facebook内部使用的ssd cache软件,开源出来供大家使用,目前有人编译出rhel6的rpm包,在centos6上可以直接安装使用。
    本次初步测试了flashcache
    测试环境
    sdc为 ssd 硬盘     160G*4 raid 10     用fio测试 4k 写iops为 5433
    sdb为 普通sas硬盘 146G*2 raid  1    用fio测试 4k 写iops为 447
    初步简单测试结果:
        4k 随机写可以达到 5014
        4k随机读可以达到45874
    flashcache 安装配置方法
     
    安装
    复制代码
    代码如下:
    rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org</p> <p>rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm</p> <p>yum install flashcache-utils kmod-flashcache
    配置:
    用sdc 做sdb的缓存
    复制代码
    代码如下:
    </p> <p>flashcache_create -p back cachedev /dev/sdc /dev/sdb </p> <p>cachedev cachedev, ssd_devname /dev/sdc, disk_devname /dev/sdb cache mode WRITE_BACK</p> <p>block_size 8, md_block_size 8, cache_size 0</p> <p>Flashcache metadata will use 1192MB of your 24016MB main memory
    可以用3种方式:
    复制代码
    代码如下:
    </p> <p>Writethrough - safest, all writes are cached to ssd but also written to disk</p> <p>immediately. If your ssd has slower write performance than your disk (likely</p> <p>for early generation SSDs purchased in 2008-2010), this may limit your system</p> <p>write performance. All disk reads are cached (tunable).
    数据同时写到ssd和普通硬盘
    复制代码
    代码如下:
    </p> <p>Writearound - again, very safe, writes are not written to ssd but directly to</p> <p>disk. Disk blocks will only be cached after they are read. All disk reads</p> <p>are cached (tunable).
    数据同时绕过ssd,直接写到普通硬盘
    复制代码
    代码如下:
    Writeback - fastest but less safe. Writes only go to the ssd initially, and</p> <p>based on various policies are written to disk later. All disk reads are</p> <p>cached (tunable).
    数据先写到ssd,随后写到普通硬盘
    查看
     flashstat
    ======================================================================================================
    Flashstat: a tool for flashcache status per second
    Author   : NinGoo(seaman.ning@gmail.com)
    Version  : 0.3
    ======================================================================================================
              SSD Device:   /dev/sdc         Disk Device:   /dev/sdb          Cache Mode: WRITE_BACK
                Capacity:    303998M          Block Size:         4K     Meta Block Size:      4096b
            Total Blocks:   77823488       Cached Blocks:          9      Cached Percent:          0
             Set Numbers:        512        Dirty Blocks:          0       Dirty Percent:          0
               cache_all:          1      reclaim_policy:       FIFO    dirty_thresh_pct:         20
       max_clean_ios_set:          2 max_clean_ios_total:          4     skip_seq_thresh:         0K
    ======================================================================================================
              time  read/s write/s diskr/s diskw/s  ssdr/s  ssdw/s uread/s uwrit/s metaw/s clean/s  repl/s wrepl/s   hit%  whit% dwhit%
    03-13 14:51:20       0       0       0       0       0       0       0       0       0       0       0       0   0|10    0|0    0|0 
    复制代码
    代码如下:
    fdisk -l /dev/mapper/cachedev
      可以看到一个设备
    Disk /dev/mapper/cachedev: 146.8 GB, 146778685440 bytes
    255 heads, 63 sectors/track, 17844 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x0002da0a
                   Device Boot      Start         End      Blocks   Id  System
    使用:
    可以当一个块设备来使用,划分vg lv
    复制代码
    代码如下:
    </p> <p> pvcreate /dev/mapper/cachedev</p> <p> Physical volume "/dev/mapper/cachedev" successfully created</p> <p> vgcreate ssdcachetest /dev/mapper/cachedev </p> <p> Volume group "ssdcachetest" successfully created</p> <p> lvcreate -L 100G -n ssdcache_test_lv1 ssdcachetest</p> <p> Logical volume "ssdcache_test_lv1" created
    删除
    复制代码
    代码如下:
    dmsetup remove cachedev</p> <p>flashcache_destroy /dev/sdc