在Linux中删除超大(100-200GB)文件的实现方式

一、IO调度策略

1.1、idle

空闲磁盘调度；

该调度策略是在当前系统没有其他进程需要进行磁盘IO时，才能进行磁盘；

因此该策略对当前系统的影响基本为0；

当然，该调度策略不能带有任何优先级参数；

1.2、Best effort

是缺省的磁盘IO调度策略；   

(1)该调度策略可以指定优先级参数(范围是0~7，数值越小，优先级越高)；

(2)针对处于同一优先级的程序将采round-robin方式；   

(3)对于best effort调度策略，8个优先级等级可以说明在给定的一个调度窗口中时间片的大小；   

(4)目前，普调用户(非root用户)是可以使用该调度策略；

(5)在内核2.6.26之前，没有设置IO优先级的进程会使用“none”作为调度策略，但是这种策略使得进程看起来像是采用了best effort调度策略，因为其优先级是通过关于   cpu nice有关的公式计算得到的：io_priority = (cpu_nice + 20) /5

(6)在内核2.6.26之后，如果当前系统使用的是CFQ调度器，那么如果进程没有设置IO优先级级别，将采用与内核2.6.26之前版本同样的方式，推到出io优先级级别；

1.3、Real time

实时调度策略，如果设置了该磁盘IO调度策略，则立即访问磁盘，不管系统中其他进程是否有IO。

因此使用实时调度策略，需要注意的是，该访问策略可能会使得其他进程处于等待状态；

二、ionice命令-设置排程优先级

2.1、安装

[root@docker][19:43:12][OK] ~ #yum provides */ionice Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * epel: mirror.01link.hk * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com epel/x86_64/filelists_db | 12 MB 00:00:08 extras/7/x86_64/filelists_db | 276 kB 00:00:01 updates/7/x86_64/filelists_db | 10 MB 00:00:42 util-linux-2.23.2-65.el7.i686 : A collection of basic system utilities Repo : base Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7.x86_64 : A collection of basic system utilities Repo : base Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7_9.1.i686 : A collection of basic system utilities Repo : updates Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7_9.1.x86_64 : A collection of basic system utilities Repo : updates Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice util-linux-2.23.2-65.el7.x86_64 : A collection of basic system utilities Repo : @anaconda Matched from: Filename : /usr/bin/ionice Filename : /usr/share/bash-completion/completions/ionice

安装

yum install util-linux-2.23.2-65.el7_9.1.x86_64 -y

2.2、用法

[root@docker][19:35:01][OK] ~ #ionice –help ionice – sets or gets process io scheduling class and priority. Usage: ionice [OPTION] -p PID [PID…] ionice [OPTION] COMMAND Options: -c, –class <class> scheduling class name or number 0: none, 1: realtime, 2: best-effort, 3: idle -n, –classdata <num> scheduling class data 0-7 for realtime and best-effort classes -p, –pid=PID view or modify already running process -t, –ignore ignore failures -V, –version output version information and exit -h, –help display this help and exit

2.3、示例

—- Deleting Huge Files in Linux —– # ionice -c 3 rm /var/logs/syslog # ionice -c 3 rm -rf /var/log/apache

注意：要使用安全的方法删除大档案，我们可以使 shred、wipe 以及 secure-deletion 工具包中的不同工具，而不是 rm 命令。

三、Shred -覆盖文件以隐藏内容

3.1、安装软件包

yum install coreutils-8.22-24.el7_9.2.x86_64 -y

3.2、命令及参数选项

[root@192_168_209_128 ~]# shred –help Usage: shred [OPTION]… FILE… Overwrite the specified FILE(s) repeatedly, in order to make it harder for even very expensive hardware probing to recover the data. Mandatory arguments to long options are mandatory for short options too. -f, –force change permissions to allow writing if necessary -n, –iterations=N overwrite N times instead of the default (3) –random-source=FILE get random bytes from FILE -s, –size=N shred this many bytes (suffixes like K, M, G accepted) -u, –remove[=HOW] truncate and remove file after overwriting; See below -v, –verbose show progress -x, –exact do not round file sizes up to the next full block; this is the default for non-regular files -z, –zero add a final overwrite with zeros to hide shredding –help display this help and exit –version output version information and exit

选项说明：

-z- 添加一个带有零的最终覆盖以隐藏切碎。-v– 可以显示操作进度。-u- 覆盖后截断和删除文件。-n– 指定覆盖文件内容的次数(默认为 3)。

3.3、示例

shred覆盖文件以隐藏其内容，也可以选择删除它。

[root@docker][19:51:03][OK] ~ #shred -zvu -n 5 cuttingLog_bak shred: cuttingLog_bak: pass 1/6 (random)… shred: cuttingLog_bak: pass 2/6 (ffffff)… shred: cuttingLog_bak: pass 3/6 (random)… shred: cuttingLog_bak: pass 4/6 (000000)… shred: cuttingLog_bak: pass 5/6 (random)… shred: cuttingLog_bak: pass 6/6 (000000)… shred: cuttingLog_bak: removing shred: cuttingLog_bak: renamed to 00000000000000 shred: 00000000000000: renamed to 0000000000000 shred: 0000000000000: renamed to 000000000000 shred: 000000000000: renamed to 00000000000 shred: 00000000000: renamed to 0000000000 shred: 0000000000: renamed to 000000000 shred: 000000000: renamed to 00000000 shred: 00000000: renamed to 0000000 shred: 0000000: renamed to 000000 shred: 000000: renamed to 00000 shred: 00000: renamed to 0000 shred: 0000: renamed to 000 shred: 000: renamed to 00 shred: 00: renamed to 0 shred: cuttingLog_bak: removed

四、Wipe —在 Linux 中安全擦除文件

介绍：Linux wipe命令可以安全地从磁存储器中擦除文件，从而无法恢复。

4.1、安装软件包

rpm -ivh wipe-2.2.0-1.2.el7.rf.x86_64.rpm

4.2、命令及参数选项

[root@docker][19:58:11][OK] ~ #wipe -h Wipe v2.2.0 – released January 10, 2004 by Tom Vier <nester@users.sf.net> Usage is wipe [options] [file-list] Options: Default: wipe -ZdntVAkO -S512 -C4096 -l1 -x8 -p1 -h — help – display this screen -u — usage -c — show copyright and license -w — show warranty information -i and -I — enable (-i) or disable (-I) interaction – overrides force -f — force file wiping and override interaction -r and -R — recursion – traverse subdirectories -s — silent – disable percentage and error reporting -v — force verbose – always show percentage -V — verbose – show percentage if file is >= 25K -d and -D — delete (-d) or keep (-D) after wiping -n and -N — delete (-n) or skip (-N) special files -k and -K — lock (-k) or dont lock (-K) files -z — zero-out file – single pass of zeroes -Z — perform normal wipe passes -t and -T — enable (-t) or disable (-T) static passes -a and -A — write until out of space (-a) or dont (-A) -o[size] -O — write to stdout (-o) or use files (-O) -B(count) — block device sector count -S(size) — block device sector size – default 512 bytes or stdout write length when used with -A -C(size) — chunk size – maximum file buffer size in kilobytes (2^10) -l[0-2] — sets wipe secure level -x[1-32] -X — sets number of random passes per wipe or disables -p(1-32) — wipe file x number of times -b(0-255) — overwrite file with this value byte

4.3、示例

[root@docker][19:59:50][OK] ~ #wipe -fi cuttingLog_bak wipe: destroy file `cuttingLog_bak? y

五、Linux 安全删除工具包

Secure-delete是一个安全文件删除工具的集合，其中包含srm (secure_deletion) 工具，用于安全删除文件。

5.1、安装软件包

yum -y install srm-1.2.15-1.el7.x86_64

5.2、命令及参数选项

[root@docker][20:05:39][OK] ~ #srm –help Usage: srm [OPTION]… [FILE]… Overwrite and remove (unlink) the files. By default use the 35-pass Gutmann method to overwrite files. -d, –directory ignored (for compatability with rm(1)) -f, –force ignore nonexistant files, never prompt -i, –interactive prompt before any removal -x, –one-file-system do not cross file system boundaries -s, –simple overwrite with single pass using 0x00 (default) -P, –openbsd overwrite with three passes like OpenBSD rm -D, –dod overwrite with 7 US DoD compliant passes -E, –doe overwrite with 3 US DoE compliant passes -G, –gutmann overwrite with 35-pass Gutmann method -C, –rcmp overwrite with Royal Canadian Mounted Police passes -r, -R, –recursive remove the contents of directories -v, –verbose explain what is being done -h, –help display this help and exit -V, –version display version information and exit

5.3、示例

#srm -vf cuttingLog_bal srm: removing cuttingLog_bal

总结

以上为个人经验，希望能给大家一个参考，也希望大家多多支持。