gzip: use common bbunzip infrastructure - ~700 bytes code less

author: Denis Vlasenko <vda.linux@googlemail.com> 2007-03-14 00:07:51 +0000
committer: Denis Vlasenko <vda.linux@googlemail.com> 2007-03-14 00:07:51 +0000
commit: 75605788ff6be5a766a7e41da583d5e8f47d9ac4 (patch)
tree: 36853bddc7322d7a30c3dab2f064b8da7efe481e /docs
parent: 07766bb0e7adcefa5dd5a373986176a5cd42ed23 (diff)
download: busybox-w32-75605788ff6be5a766a7e41da583d5e8f47d9ac4.tar.gz
busybox-w32-75605788ff6be5a766a7e41da583d5e8f47d9ac4.tar.bz2
busybox-w32-75605788ff6be5a766a7e41da583d5e8f47d9ac4.zip
1 files changed, 88 insertions, 0 deletions
diff --git a/docs/keep_data_small.txt b/docs/keep_data_small.txt
new file mode 100644
index 000000000..88cc2bc66
--- /dev/null
+++ b/docs/keep_data_small.txt
@@ -0,0 +1,88 @@
+        Keeping data small
+When many applets are compiled into busybox, all rw data and
+bss for each applet are concatenated. Including those from libc,
+if static bbox is built. When bbox is started, _all_ this data
+is allocated, not just that one part for selected applet.
+What "allocated" exactly means, depends on arch.
+On nommu it's probably bites the most, actually using real
+RAM for rwdata and bss. On i386, bss is lazily allocated
+by COWed zero pages. Not sure about rwdata - also COW?
+Small experiment measures "parasitic" bbox memory consumption.
+Here we start 1000 "busybox sleep 10" in parallel.
+bbox binary is practically allyesconfig static one,
+built against uclibc:
+bash-3.2# nmeter '%t %c %b %m %p %[pn]'
+23:17:28 ..........  0  0 168M  0 147
+23:17:29 ..........  0  0 168M  0 147
+23:17:30 U.........  0  0 168M  1 147
+23:17:31 SU........  0 188k 181M 244 391
+23:17:32 SSSSUUU...  0  0 223M 757 1147
+23:17:33 UUU.......  0  0 223M  0 1147
+23:17:34 U.........  0  0 223M  1 1147
+23:17:35 ..........  0  0 223M  0 1147
+23:17:36 ..........  0  0 223M  0 1147
+23:17:37 S.........  0  0 223M  0 1147
+23:17:38 ..........  0  0 223M  1 1147
+23:17:39 ..........  0  0 223M  0 1147
+23:17:40 ..........  0  0 223M  0 1147
+23:17:41 ..........  0  0 210M  0 906
+23:17:42 ..........  0  0 168M  1 147
+23:17:43 ..........  0  0 168M  0 147
+This requires 55M of memory. Thus 1 trivial busybox applet
+takes 55k of userspace memory (nmeter doesn't account for kernel-side
+allocations). Definitely can be improved.
+Thus we should avoid large global data in our applets,
+and should minimize usage of libc functions which implicitly use
+such structures in libc.
+        Example 1
+One example how to reduce global data usage is in
+archival/libunarchive/decompress_unzip.c:
+/* This is somewhat complex-looking arrangement, but it allows
+ * to place decompressor state either in bss or in
+ * malloc'ed space simply by changing #defines below.
+ * Sizes on i386:
+ * text    data     bss     dec     hex
+ * 5256       0     108    5364    14f4 - bss
+ * 4915       0       0    4915    1333 - malloc
+ */
+#define STATE_IN_BSS 0
+#define STATE_IN_MALLOC 1
+This example completely eliminates globals in that module.
+Required memory is allocated in inflate_gunzip() [its main module]
+and then passed down to all subroutines which need to access globals
+as a parameter.
+        Example 2
+In case you don't want to pass this additional parameter everywhere,
+take a look at archival/gzip.c. Here all global data is replaced by
+singe global pointer (ptr_to_globals) to allocated storage.
+In order to not duplicate ptr_to_globals in every applet, you can
+reuse single common one. It is defined in libbb/messages.c
+as void *ptr_to_globals, but is NOT declared in libbb.h.
+You first define a struct:
+struct my_globals { int a; char buf[1000]; };
+and then declare that ptr_to_globals is a pointer to it:
+extern struct my_globals *ptr_to_globals;
+#define G (*ptr_to_globals)
+Linker magic enures that these two merge into single pointer object.
+Now initialize it in <applet>_main():
+        ptr_to_globals = xzalloc(sizeof(G));
+and you can reference "globals" by G.a, G.buf and so on, in any function.
author	Denis Vlasenko <vda.linux@googlemail.com>	2007-03-14 00:07:51 +0000
committer	Denis Vlasenko <vda.linux@googlemail.com>	2007-03-14 00:07:51 +0000
commit	75605788ff6be5a766a7e41da583d5e8f47d9ac4 (patch)
tree	36853bddc7322d7a30c3dab2f064b8da7efe481e /docs
parent	07766bb0e7adcefa5dd5a373986176a5cd42ed23 (diff)
download	busybox-w32-75605788ff6be5a766a7e41da583d5e8f47d9ac4.tar.gz busybox-w32-75605788ff6be5a766a7e41da583d5e8f47d9ac4.tar.bz2 busybox-w32-75605788ff6be5a766a7e41da583d5e8f47d9ac4.zip

diff --git a/docs/keep_data_small.txt b/docs/keep_data_small.txt new file mode 100644 index 000000000..88cc2bc66 --- /dev/null +++ b/docs/keep_data_small.txt
@@ -0,0 +1,88 @@
	1	Keeping data small
	2
	3	When many applets are compiled into busybox, all rw data and
	4	bss for each applet are concatenated. Including those from libc,
	5	if static bbox is built. When bbox is started, _all_ this data
	6	is allocated, not just that one part for selected applet.
	7
	8	What "allocated" exactly means, depends on arch.
	9	On nommu it's probably bites the most, actually using real
	10	RAM for rwdata and bss. On i386, bss is lazily allocated
	11	by COWed zero pages. Not sure about rwdata - also COW?
	12
	13	Small experiment measures "parasitic" bbox memory consumption.
	14	Here we start 1000 "busybox sleep 10" in parallel.
	15	bbox binary is practically allyesconfig static one,
	16	built against uclibc:
	17
	18	bash-3.2# nmeter '%t %c %b %m %p %[pn]'
	19	23:17:28 .......... 0 0 168M 0 147
	20	23:17:29 .......... 0 0 168M 0 147
	21	23:17:30 U......... 0 0 168M 1 147
	22	23:17:31 SU........ 0 188k 181M 244 391
	23	23:17:32 SSSSUUU... 0 0 223M 757 1147
	24	23:17:33 UUU....... 0 0 223M 0 1147
	25	23:17:34 U......... 0 0 223M 1 1147
	26	23:17:35 .......... 0 0 223M 0 1147
	27	23:17:36 .......... 0 0 223M 0 1147
	28	23:17:37 S......... 0 0 223M 0 1147
	29	23:17:38 .......... 0 0 223M 1 1147
	30	23:17:39 .......... 0 0 223M 0 1147
	31	23:17:40 .......... 0 0 223M 0 1147
	32	23:17:41 .......... 0 0 210M 0 906
	33	23:17:42 .......... 0 0 168M 1 147
	34	23:17:43 .......... 0 0 168M 0 147
	35
	36	This requires 55M of memory. Thus 1 trivial busybox applet
	37	takes 55k of userspace memory (nmeter doesn't account for kernel-side
	38	allocations). Definitely can be improved.
	39
	40	Thus we should avoid large global data in our applets,
	41	and should minimize usage of libc functions which implicitly use
	42	such structures in libc.
	43
	44	Example 1
	45
	46	One example how to reduce global data usage is in
	47	archival/libunarchive/decompress_unzip.c:
	48
	49	/* This is somewhat complex-looking arrangement, but it allows
	50	* to place decompressor state either in bss or in
	51	* malloc'ed space simply by changing #defines below.
	52	* Sizes on i386:
	53	* text data bss dec hex
	54	* 5256 0 108 5364 14f4 - bss
	55	* 4915 0 0 4915 1333 - malloc
	56	*/
	57	#define STATE_IN_BSS 0
	58	#define STATE_IN_MALLOC 1
	59
	60	This example completely eliminates globals in that module.
	61	Required memory is allocated in inflate_gunzip() [its main module]
	62	and then passed down to all subroutines which need to access globals
	63	as a parameter.
	64
	65	Example 2
	66
	67	In case you don't want to pass this additional parameter everywhere,
	68	take a look at archival/gzip.c. Here all global data is replaced by
	69	singe global pointer (ptr_to_globals) to allocated storage.
	70
	71	In order to not duplicate ptr_to_globals in every applet, you can
	72	reuse single common one. It is defined in libbb/messages.c
	73	as void *ptr_to_globals, but is NOT declared in libbb.h.
	74	You first define a struct:
	75
	76	struct my_globals { int a; char buf[1000]; };
	77
	78	and then declare that ptr_to_globals is a pointer to it:
	79
	80	extern struct my_globals *ptr_to_globals;
	81	#define G (*ptr_to_globals)
	82
	83	Linker magic enures that these two merge into single pointer object.
	84	Now initialize it in <applet>_main():
	85
	86	ptr_to_globals = xzalloc(sizeof(G));
	87
	88	and you can reference "globals" by G.a, G.buf and so on, in any function.