在Linux系统中将SSD当块设备缓存的方法

2024.05.13

原理
 写操作先缓存到ssd硬盘上，然后通过一定策略写到普通硬盘上；
 读操作热点数据可以缓存到ssd硬盘上，提高读取数据的速度。
 软件 ssd 缓存开源软件主要有bcache ，flashcache
 bcache 需要编译最新的内核，要求比较高，配置比较复杂，目前主要用于测试环境；
 flashcache 是facebook内部使用的ssd cache软件，开源出来供大家使用，目前有人编译出rhel6的rpm包，在centos6上可以直接安装使用。
 本次初步测试了flashcache
 测试环境
 sdc为 ssd 硬盘 160G*4 raid 10 用fio测试 4k 写iops为 5433
 sdb为普通sas硬盘 146G*2 raid 1 用fio测试 4k 写iops为 447
 初步简单测试结果：
 4k 随机写可以达到 5014
 4k随机读可以达到45874
 flashcache 安装配置方法

 安装
 复制代码
 代码如下:
 rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm yum install flashcache-utils kmod-flashcache
 配置:
 用sdc 做sdb的缓存
 复制代码
 代码如下:
 flashcache_create -p back cachedev /dev/sdc /dev/sdb cachedev cachedev, ssd_devname /dev/sdc, disk_devname /dev/sdb cache mode WRITE_BACK block_size 8, md_block_size 8, cache_size 0 Flashcache metadata will use 1192MB of your 24016MB main memory
 可以用3种方式：
 复制代码
 代码如下:
 Writethrough - safest, all writes are cached to ssd but also written to disk immediately. If your ssd has slower write performance than your disk (likely for early generation SSDs purchased in 2008-2010), this may limit your system write performance. All disk reads are cached (tunable).
 数据同时写到ssd和普通硬盘
 复制代码
 代码如下:
 Writearound - again, very safe, writes are not written to ssd but directly to disk. Disk blocks will only be cached after they are read. All disk reads are cached (tunable).
 数据同时绕过ssd，直接写到普通硬盘
 复制代码
 代码如下:
 Writeback - fastest but less safe. Writes only go to the ssd initially, and based on various policies are written to disk later. All disk reads are cached (tunable).
 数据先写到ssd，随后写到普通硬盘
 查看
 flashstat
 ======================================================================================================
 Flashstat: a tool for flashcache status per second
 Author : NinGoo(seaman.ning@gmail.com)
 Version : 0.3
 ======================================================================================================
 SSD Device: /dev/sdc Disk Device: /dev/sdb Cache Mode: WRITE_BACK
 Capacity: 303998M Block Size: 4K Meta Block Size: 4096b
 Total Blocks: 77823488 Cached Blocks: 9 Cached Percent: 0
 Set Numbers: 512 Dirty Blocks: 0 Dirty Percent: 0
 cache_all: 1 reclaim_policy: FIFO dirty_thresh_pct: 20
 max_clean_ios_set: 2 max_clean_ios_total: 4 skip_seq_thresh: 0K
 ======================================================================================================
 time read/s write/s diskr/s diskw/s ssdr/s ssdw/s uread/s uwrit/s metaw/s clean/s repl/s wrepl/s hit% whit% dwhit%
 03-13 14:51:20 0 0 0 0 0 0 0 0 0 0 0 0 0|10 0|0 0|0
 复制代码
 代码如下:
 fdisk -l /dev/mapper/cachedev
 可以看到一个设备
 Disk /dev/mapper/cachedev: 146.8 GB, 146778685440 bytes
 255 heads, 63 sectors/track, 17844 cylinders
 Units = cylinders of 16065 * 512 = 8225280 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes
 Disk identifier: 0x0002da0a
 Device Boot Start End Blocks Id System
 使用：
 可以当一个块设备来使用，划分vg lv
 复制代码
 代码如下:
 pvcreate /dev/mapper/cachedev Physical volume "/dev/mapper/cachedev" successfully created vgcreate ssdcachetest /dev/mapper/cachedev Volume group "ssdcachetest" successfully created lvcreate -L 100G -n ssdcache_test_lv1 ssdcachetest Logical volume "ssdcache_test_lv1" created
 删除
 复制代码
 代码如下:
 dmsetup remove cachedev flashcache_destroy /dev/sdc