The libioil
library can be used to improve the performance of read and write
operations to DFuse from applications which use libc. The library bypasses the
kernel by intercepting POSIX read and write calls from the application and
servicing them directly in user-space.
Using libioil
To use the interception library, set LD_PRELOAD
to point to the shared
library in the DAOS install directory:
LD_PRELOAD
=
/usr/lib64/libioil.so
For example:
$
dd
if
=
/dev/zero
of
=
./foo
bs
=
1G
count
=
20
20
+0
records
in
20
+0
records
out 21474836480
bytes
(
21
GB,
20
GiB )
copied,
45
.6925
s,
470
MB/s
$
LD_PRELOAD
=
/usr/lib64/libioil.so
dd
if
=
/dev/zero
of
=
./bar
bs
=
1G
count
=
20
20
+0
records
in
20
+0
records
out 21474836480
bytes
(
21
GB,
20
GiB )
copied,
15
.735
s,
1
.4
GB/s
It is also possible to link applications with the interception library at
compile time with the -lioil
flag.
For evaluating performance using the interception library, see Performance considerations and Test performance .
Monitor activity
To enable activity reports for the interception library, set the D_IL_REPORT
environment variable. This returns a short summary of activity in the shared
library destructor, typically as a program exits.
The value assigned to D_IL_REPORT
defines the number of read and write calls
to log to stderr. To log all calls, set the value to -1
. To print the summary
at program exit only, set the value to 0
.
For example:
$
D_IL_REPORT
=
1
LD_PRELOAD
=
/usr/lib64/libioil.so
dd
if
=
/dev/zero
of
=
./bar
bs
=
1G
count
=
5
[
libioil ]
Intercepting
write
of
size
1073741824
5
+0
records
in
5
+0
records
out 5368709120
bytes
(
5
.4
GB,
5
.0
GiB )
copied,
4
.13708
s,
1
.3
GB/s
$
D_IL_REPORT
=
3
LD_PRELOAD
=
/usr/lib64/libioil.so
dd
if
=
/dev/zero
of
=
./bar
bs
=
1G
count
=
5
[
libioil ]
Intercepting
write
of
size
1073741824
[
libioil ]
Intercepting
write
of
size
1073741824
[
libioil ]
Intercepting
write
of
size
1073741824
5
+0
records
in
5
+0
records
out 5368709120
bytes
(
5
.4
GB,
5
.0
GiB )
copied,
4
.18626
s,
1
.3
GB/s
$
D_IL_REPORT
=
-1
LD_PRELOAD
=
/usr/lib64/libioil.so
dd
if
=
/dev/zero
of
=
./bar
bs
=
1G
count
=
5
[
libioil ]
Intercepting
write
of
size
1073741824
[
libioil ]
Intercepting
write
of
size
1073741824
[
libioil ]
Intercepting
write
of
size
1073741824
[
libioil ]
Intercepting
write
of
size
1073741824
[
libioil ]
Intercepting
write
of
size
1073741824
5
+0
records
in
5
+0
records
out 5368709120
bytes
(
5
.4
GB,
5
.0
GiB )
copied,
4
.12114
s,
1
.3
GB/s
Some programs, including most GNU utilities from the coreutils
package, have
a destructor function to close stderr on exit. This means that, while the
interception library works for many basic commands such as cp and cat, it's not
possible to see the summary generated by the interception library.
