docker镜像压缩squash的实现

目录一、docker-squash 安装二、使用介绍2.1、部分压缩指定层数2.2、完全压缩三、总结参考文档

docker build 通过Dockerfile 制作镜像会根据命令形成一层层layer,这样造成文件重复，镜像体积比较大；为了缩小层的数量及其大小，能够像压缩git commit一样压缩layer层，docker-squash能够执行此操作。Docker squash将压缩多个镜像layer层，以便删除存储在两头步骤中的所有数据，即删除掉重复数据。本文就docker-squash 使用效果展开讨论。

一、docker-squash 安装

pip install docker-squash docker-squash –help usage: docker-squash [-h] [-v] [–version] [-d] [-f FROM_LAYER] [-t TAG]                      [-m MESSAGE] [-c] [–tmp-dir TMP_DIR]                      [–output-path OUTPUT_PATH]                      image Docker layer squashing tool positional arguments:   image                 Image to be squashed optional arguments:   -h, –help            show this help message and exit   -v, –verbose         Verbose output   –version             Show version and exit   -d, –development     Does not clean up after failure for easier debugging   -f FROM_LAYER, –from-layer FROM_LAYER                         Number of layers to squash or ID of the layer (or                         image ID or image name) to squash from. In case the                         provided value is an integer, specified number of                         layers will be squashed. Every layer in the image will                         be squashed if the parameter is not provided.   -t TAG, –tag TAG     Specify the tag to be used for the new image. If not                         specified no tag will be applied   -m MESSAGE, –message MESSAGE                         Specify a commit message (comment) for the new image.   -c, –cleanup         Remove source image from Docker after squashing   –tmp-dir TMP_DIR     Temporary directory to be created and used   –output-path OUTPUT_PATH                         Path where the image should be stored after squashing.                         If not provided, image will be loaded into Docker                         daemon

二、使用介绍

2.1、部分压缩指定层数

首先docker history查看镜像层数

docker history java:3.0 IMAGE CREATED CREATED BY SIZE COMMENT 63a1b1edb376 4 weeks ago /bin/sh -c #(nop) LABEL ImageName=reg.docke… 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL BaseRootfsLayerCoun… 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL BaseLayerCount=25 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL baseOS=reg.docker.a… 0B <missing> 4 weeks ago /bin/sh -c #(nop) VOLUME [/home/admin/logs] 0B <missing> 4 weeks ago /bin/sh -c #(nop) ENTRYPOINT [“/bin/sh” “-c… 0B <missing> 4 weeks ago /bin/sh -c #(nop) ENV JVM_Xms= JVM_Xmx= JVM… 0B <missing> 4 weeks ago /bin/sh -c #(nop) ENV JAVA_HOME=/opt/taobao… 0B <missing> 4 weeks ago /bin/sh -c #(nop) ENV LANG=zh_CN.utf8 0B <missing> 4 weeks ago /bin/sh -c [[ $(uname -p) != “sw_64” ]] && r… 97MB <missing> 4 weeks ago /bin/sh -c yum clean –enablerepo=* all; … 628MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY file:cbad11e2b6b88c38… 19.9MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY file:fd3bca1e9167d639… 21MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:2761dc242787e3ab8… 39.2MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:8a9faa49606db326b… 8.49MB <missing> 4 weeks ago /bin/sh -c #(nop) LABEL ImageName=reg.docke… 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL BaseRootfsLayerCoun… 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL BaseLayerCount=10 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL baseOS=reg.docker.a… 0B <missing> 4 weeks ago /bin/sh -c #(nop) ENTRYPOINT [“”] 0B <missing> 4 weeks ago /bin/sh -c #(nop) CMD [“/sbin/init”] 0B <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:61a850de6501dac7b… 3.03kB <missing> 4 weeks ago /bin/sh -c sed -i -e s/root:[^:]*:/root:*:/… 94.6MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:ca4e72cff3459ccab… 395B <missing> 16 months ago 282MB Imported from –

使用dive 工具查看镜像效率参数

CI=true dive java:3.0 Using default CI config Image Source: docker://java:3.0 Fetching image… (this can take a while for large images) Analyzing image… efficiency: 78.5919 % UserSizeByes: 90775433100 bytes (908 MB) SizeByes: 118928447300 bytes (1.2 GB) wastedBytes: 381696817 bytes (382 MB) userWastedPercent: 42.0485 % Inefficient Files: Count Wasted Space File Path 2 119 MB /usr/lib/locale/locale-archive … 2 0 B /usr/lib/systemd/system/runlevel0.target Results: FAIL: highestUserWastedPercent: too many bytes wasted, relative to the user bytes added (%-user-wasted-bytes=0.4204847104166557 > threshold=0.1) SKIP: highestWastedBytes: rule disabled FAIL: lowestEfficiency: image efficiency is too low (efficiency=0.7859187397531037 < threshold=0.9) Result:FAIL [Total:3] [Passed:0] [Failed:2] [Warn:0] [Skipped:1

命令格式

docker-squash -f <层数，依据docker history> -t 新镜像名称 原镜像名称

如下：

docker-squash -f 11 -t java-squash-11:1.0 java:3.0 2023-01-20 13:41:43,978 root INFO docker-squash version 1.0.10, Docker 5138787, API 1.39… 2023-01-20 13:41:43,978 root INFO Using v2 image format 2023-01-20 13:41:43,982 root INFO Old image has 25 layers 2023-01-20 13:41:43,982 root INFO Checking if squashing is necessary… 2023-01-20 13:41:43,982 root INFO Attempting to squash last 11 layers… 2023-01-20 13:41:43,982 root INFO Saving image sha256:63a1b1edb376fb12838dbbef7749effc1e8e6a973d33a47aee194eff6a173edd to /tmp/docker-squash-OlTnlF/old directory… 2023-01-20 13:42:03,134 root INFO Image saved! 2023-01-20 13:42:03,135 root INFO Squashing image java:3.0… 2023-01-20 13:42:03,136 root INFO Starting squashing… 2023-01-20 13:42:03,870 root INFO Squashing file /tmp/docker-squash-OlTnlF/old/ddf4a6e4ea15a7ca3825c70e116e40f3a01f25b7df008d4517c826ef78c03dc8/layer.tar… 2023-01-20 13:42:06,729 root INFO Squashing file /tmp/docker-squash-OlTnlF/old/7be6dcb4742458db7e2c77fb1ad9fd66555e6f08d28c74592a9b973740356919/layer.tar… 2023-01-20 13:42:19,510 root INFO Squashing finished! 2023-01-20 13:42:22,736 root INFO Original image size: 1166.43 MB 2023-01-20 13:42:22,736 root INFO Squashed image size: 1153.64 MB 2023-01-20 13:42:22,736 root INFO Image size decreased by 1.10 % 2023-01-20 13:42:22,736 root INFO New squashed image ID is 794121cd589230285a51d969aac97c5e936d732b0feefe5c05e78b027e850312 2023-01-20 13:42:30,244 root INFO Image registered in Docker daemon as java-squash-11:1.0 2023-01-20 13:42:30,369 root INFO Done

压缩后体积如下：

docker images | grep java java-squash-11 1.0 794121cd5892 About a minute ago 1.17GB java 3.0 63a1b1edb376 4 weeks ago 1.19GB

再次查看history

docker history java-squash-11:1.0 IMAGE CREATED CREATED BY SIZE COMMENT 794121cd5892 5 minutes ago 703MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY file:cbad11e2b6b88c38… 19.9MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY file:fd3bca1e9167d639… 21MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:2761dc242787e3ab8… 39.2MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:8a9faa49606db326b… 8.49MB <missing> 4 weeks ago /bin/sh -c #(nop) LABEL ImageName=reg.docke… 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL BaseRootfsLayerCoun… 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL BaseLayerCount=10 0B <missing> 4 weeks ago /bin/sh -c #(nop) LABEL baseOS=reg.docker.a… 0B <missing> 4 weeks ago /bin/sh -c #(nop) ENTRYPOINT [“”] 0B <missing> 4 weeks ago /bin/sh -c #(nop) CMD [“/sbin/init”] 0B <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:61a850de6501dac7b… 3.03kB <missing> 4 weeks ago /bin/sh -c sed -i -e s/root:[^:]*:/root:*:/… 94.6MB <missing> 4 weeks ago /bin/sh -c #(nop) COPY dir:ca4e72cff3459ccab… 395B <missing> 16 months ago 282MB Imported from –

再次dive 分析,镜像效率参数有些许改观

CI=true dive java-squash-11:1.0 Using default CI config Image Source: docker://java-squash-11:1.0 Fetching image… (this can take a while for large images) Analyzing image… efficiency: 79.9717 % UserSizeByes: 88655803400 bytes (887 MB) SizeByes: 116808817600 bytes (1.2 GB) wastedBytes: 360500520 bytes (360 MB) userWastedPercent: 40.6629 % Inefficient Files: Count Wasted Space File Path 2 119 MB /usr/lib/locale/locale-archive … 2 0 B /usr/lib/systemd/system/sysinit.target.wants/cryptsetup.target Results: FAIL: highestUserWastedPercent: too many bytes wasted, relative to the user bytes added (%-user-wasted-bytes=0.40662935326803434 > threshold=0.1) SKIP: highestWastedBytes: rule disabled FAIL: lowestEfficiency: image efficiency is too low (efficiency=0.7997167019774828 < threshold=0.9) Result:FAIL [Total:3] [Passed:0] [Failed:2] [Warn:0] [Skipped:1]

2.2、完全压缩

即压缩成一层

docker-squash -f 25 -t java-squash-all:1.0 java:3.0 2023-01-20 13:53:47,397 root INFO docker-squash version 1.0.10, Docker 5138787, API 1.39… 2023-01-20 13:53:47,397 root INFO Using v2 image format 2023-01-20 13:53:47,401 root INFO Old image has 25 layers 2023-01-20 13:53:47,401 root INFO Checking if squashing is necessary… 2023-01-20 13:53:47,401 root INFO Attempting to squash last 25 layers… 2023-01-20 13:53:47,401 root INFO Saving image sha256:63a1b1edb376fb12838dbbef7749effc1e8e6a973d33a47aee194eff6a173edd to /tmp/docker-squash-23fgk8/old directory… 2023-01-20 13:54:06,767 root INFO Image saved! 2023-01-20 13:54:06,767 root INFO Squashing image java:3.0… 2023-01-20 13:54:06,768 root INFO Starting squashing… 2023-01-20 13:54:06,768 root INFO Squashing file /tmp/docker-squash-23fgk8/old/ddf4a6e4ea15a7ca3825c70e116e40f3a01f25b7df008d4517c826ef78c03dc8/layer.tar… 2023-01-20 13:54:09,616 root INFO Squashing file /tmp/docker-squash-23fgk8/old/7be6dcb4742458db7e2c77fb1ad9fd66555e6f08d28c74592a9b973740356919/layer.tar… 2023-01-20 13:54:20,751 root INFO Squashing file /tmp/docker-squash-23fgk8/old/57be802a5ecce1545d1fef09e3f0b04cef12386e3995b8f98b827c1fa690f2dc/layer.tar… 2023-01-20 13:54:20,778 root INFO Squashing file /tmp/docker-squash-23fgk8/old/e113fadd42d1969d71a6d30cf402938436e9aba0de9af3ed2cdacec9e1842103/layer.tar… 2023-01-20 13:54:20,799 root INFO Squashing file /tmp/docker-squash-23fgk8/old/eca237f8359a3c25227be27fd27ebff05b510f06b9768639e3484741d9995073/layer.tar… 2023-01-20 13:54:20,809 root INFO Squashing file /tmp/docker-squash-23fgk8/old/2a47dbd3a252894e057a55d676f92625d24bfc8ad10a28b008e8b87638480ee6/layer.tar… 2023-01-20 13:54:20,871 root INFO Squashing file /tmp/docker-squash-23fgk8/old/bb207541a62d1cf5aa5dc13e971301949a22d2c3016e317d2190f1b51bde0167/layer.tar… 2023-01-20 13:54:20,876 root INFO Squashing file /tmp/docker-squash-23fgk8/old/354eb0981c6f10dcf5744386501d61c670f5bfbf77063c778b26e5c0450543f3/layer.tar… 2023-01-20 13:54:22,489 root INFO Squashing file /tmp/docker-squash-23fgk8/old/e31074992543d56a24b7da3457af34c7df7439ced6d9c0bf72ce4122fc380d81/layer.tar… 2023-01-20 13:54:22,492 root INFO Squashing file /tmp/docker-squash-23fgk8/old/1641651324846664fa3c233e4459767551e4ab9f081a718085216d30687e0877/layer.tar… 2023-01-20 13:54:52,084 root INFO Squashing finished! 2023-01-20 13:54:54,922 root INFO Original image size: 1166.43 MB 2023-01-20 13:54:54,922 root INFO Squashed image size: 981.28 MB 2023-01-20 13:54:54,922 root INFO Image size decreased by 15.87 % 2023-01-20 13:54:54,923 root INFO New squashed image ID is f508f38322ae8950e49bd8cbcf140356dd236654ecd6bf303440d93081f6f7c9 2023-01-20 13:55:04,532 root INFO Image registered in Docker daemon as java-squash-all:1.0 2023-01-20 13:55:04,624 root INFO Done

docker history 查看

docker history java-squash-all:1.0 IMAGE CREATED CREATED BY SIZE COMMENT f508f38322ae About a minute ago 989MB

docker images 比较一下大小

docker images | grep java java-squash-all 1.0 f508f38322ae 30 minutes ago 989MB java-squash-11 1.0 794121cd5892 42 minutes ago 1.17GB java 3.0 63a1b1edb376 4 weeks ago 1.19GB

dive 分析

CI=true dive java-squash-all:1.0 Using default CI config Image Source: docker://java-squash-all:1.0 Fetching image… (this can take a while for large images) Analyzing image… efficiency: 100.0000 % UserSizeByes: 0 bytes (0 B) SizeByes: 98904607400 bytes (989 MB) wastedBytes: 0 bytes (0 B) userWastedPercent: NaN % Inefficient Files: Count Wasted Space File Path None Results: PASS: highestUserWastedPercent SKIP: highestWastedBytes: rule disabled PASS: lowestEfficiency Result:PASS [Total:3] [Passed:2] [Failed:0] [Warn:0] [Skipped:1]

三、总结

1、docker-squash 根据docker history 层数对镜像进行压缩

2、-f 指定的层数越大，压缩效率越好

3、结合dive 工具，更直观展示压缩效率

4、压缩后的图像可以加载回Docker守护进程，即可以通过docker images 查看

5、使用docker-squash 也存在缺点，压缩为一层时无法根据docker history查看镜像构建历史。

参考文档

1、https://lequ7.com/guan-yu-docker-chuang-jian-zui-xiao-hua-de-rong-qi-jing-xiang-er.html2、https://mp.weixin.qq.com/s/S1Ib08SpQbf1SCbCutUoqQ3、https://blog.csdn.net/j_starry/article/details/1251855064、https://github.com/jwilder/docker-squash5、https://bbs.huaweicloud.com/blogs/3366846、https://github.com/dive/7、https://www.cnpython.com/pypi/docker-squash8、https://cloud.tencent.com/developer/article/1444954