Linux動態鏈接庫加載失敗一例

安裝pacemakerrpm包後，遇到啟動失敗的情況，原因和動態鏈接庫的加載有關，以下是詳細。

問題

編譯生成pacemaker 1.1.15的rpm包，然後在其它機器上安裝pacemaker rpm包後，啟動失敗。

[root@srdsdevapp73 ~]# service pacemaker startStarting Pacemaker Cluster Manager                         [FAILED] 

環境

• CentOS 6.3 64bit

原因

通過strace發現pacemaker啟動失敗由於加載庫libcoroipcc.so.4失敗

[root@srdsdevapp73 ~]# strace -f service pacemaker start...[pid 19960] writev(2, [{"pacemakerd", 10}, {": ", 2}, {"error while loading shared libra"..., 36}, {": ", 2}, {"libcoroipcc.so.4", 16}, {": ", 2}, {"cannot open shared object file", 30}, {": ... 

再用ldd檢查pacemakerd，發現總共有3個庫找不到

[root@srdsdevapp73 ~]# ldd /usr/sbin/pacemakerd    linux-vdso.so.1 =>  (0x00007fffc4c9f000)    libcrmcluster.so.4 => /usr/lib/libcrmcluster.so.4 (0x0000003cbac00000)    libstonithd.so.2 => /usr/lib/libstonithd.so.2 (0x0000003cba400000)    libcrmcommon.so.3 => /usr/lib/libcrmcommon.so.3 (0x0000003cb4c00000)    libm.so.6 => /lib64/libm.so.6 (0x0000003cb3c00000)    libcpg.so.4 => /usr/lib64/libcpg.so.4 (0x00007f3f72199000)    libcfg.so.6 => /usr/lib64/libcfg.so.6 (0x00007f3f71f95000)    libcmap.so.4 => /usr/lib64/libcmap.so.4 (0x00007f3f71d8f000)    libquorum.so.5 => /usr/lib64/libquorum.so.5 (0x00007f3f71b8b000)    libgnutls.so.26 => /usr/lib64/libgnutls.so.26 (0x0000003cb8800000)    libcorosync_common.so.4 => /usr/lib64/libcorosync_common.so.4 (0x00007f3f71988000)    libplumb.so.2 => /usr/lib64/libplumb.so.2 (0x00007f3f71754000)    libpils.so.2 => /usr/lib64/libpils.so.2 (0x00007f3f7154b000)    libqb.so.0 => /usr/lib64/libqb.so.0 (0x00007f3f712e6000)    libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003cb2c00000)    libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003cb7000000)    libxslt.so.1 => /usr/lib64/libxslt.so.1 (0x0000003cb4800000)    libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x0000003cb6000000)    libc.so.6 => /lib64/libc.so.6 (0x0000003cb2400000)    libuuid.so.1 => /lib64/libuuid.so.1 (0x0000003cb5000000)    libpam.so.0 => /lib64/libpam.so.0 (0x0000003cb6c00000)    librt.so.1 => /lib64/librt.so.1 (0x0000003cb3000000)    libdl.so.2 => /lib64/libdl.so.2 (0x0000003cb2800000)    libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x0000003cb3800000)    libltdl.so.7 => /usr/lib64/libltdl.so.7 (0x0000003cb8400000)    libcoroipcc.so.4 => not found    libcfg.so.4 => not found    libconfdb.so.4 => not found    libtasn1.so.3 => /usr/lib64/libtasn1.so.3 (0x0000003cb7800000)    libz.so.1 => /lib64/libz.so.1 (0x0000003cb3400000)    libgcrypt.so.11 => /lib64/libgcrypt.so.11 (0x0000003cb7400000)    /lib64/ld-linux-x86-64.so.2 (0x0000003cb2000000)    libaudit.so.1 => /lib64/libaudit.so.1 (0x0000003cb6400000)    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x0000003cb5c00000)    libgpg-error.so.0 => /lib64/libgpg-error.so.0 (0x0000003cb6800000)    libfreebl3.so => /lib64/libfreebl3.so (0x0000003cb5800000) 

上面有一段"/usr/lib/libcrmcluster.so.4"比較奇怪，確認後發現文件不對，是以前安裝的版本(不清楚當初怎麼安裝的了)。 正確的庫位置應該是"/usr/lib64/libcrmcluster.so.4"。 將老版本的pacemaker刪除後，一切正常

[root@srdsdevapp73 ~]# rm -f /usr/lib/libcrm*[root@srdsdevapp73 ~]# rm -f /usr/lib/libstonithd.*[root@srdsdevapp73 ~]# ldd /usr/sbin/pacemakerd    linux-vdso.so.1 =>  (0x00007fff9a3ff000)    libcrmcluster.so.4 => /usr/lib64/libcrmcluster.so.4 (0x00007f849a1fc000)    libstonithd.so.2 => /usr/lib64/libstonithd.so.2 (0x00007f8499fea000)    libcrmcommon.so.3 => /usr/lib64/libcrmcommon.so.3 (0x00007f8499d93000)    libm.so.6 => /lib64/libm.so.6 (0x0000003cb3c00000)    libcpg.so.4 => /usr/lib64/libcpg.so.4 (0x00007f8499b8c000)    libcfg.so.6 => /usr/lib64/libcfg.so.6 (0x00007f8499988000)    libcmap.so.4 => /usr/lib64/libcmap.so.4 (0x00007f8499782000)    libquorum.so.5 => /usr/lib64/libquorum.so.5 (0x00007f849957e000)    libgnutls.so.26 => /usr/lib64/libgnutls.so.26 (0x0000003cb8800000)    libcorosync_common.so.4 => /usr/lib64/libcorosync_common.so.4 (0x00007f849937b000)    libplumb.so.2 => /usr/lib64/libplumb.so.2 (0x00007f8499147000)    libpils.so.2 => /usr/lib64/libpils.so.2 (0x00007f8498f3e000)    libqb.so.0 => /usr/lib64/libqb.so.0 (0x00007f8498cd9000)    libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003cb2c00000)    libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003cb7000000)    libxslt.so.1 => /usr/lib64/libxslt.so.1 (0x0000003cb4800000)    libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x0000003cb6000000)    libc.so.6 => /lib64/libc.so.6 (0x0000003cb2400000)    libuuid.so.1 => /lib64/libuuid.so.1 (0x0000003cb5000000)    libpam.so.0 => /lib64/libpam.so.0 (0x0000003cb6c00000)    librt.so.1 => /lib64/librt.so.1 (0x0000003cb3000000)    libdl.so.2 => /lib64/libdl.so.2 (0x0000003cb2800000)    libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x0000003cb3800000)    libltdl.so.7 => /usr/lib64/libltdl.so.7 (0x0000003cb8400000)    libtasn1.so.3 => /usr/lib64/libtasn1.so.3 (0x0000003cb7800000)    libz.so.1 => /lib64/libz.so.1 (0x0000003cb3400000)    libgcrypt.so.11 => /lib64/libgcrypt.so.11 (0x0000003cb7400000)    /lib64/ld-linux-x86-64.so.2 (0x0000003cb2000000)    libaudit.so.1 => /lib64/libaudit.so.1 (0x0000003cb6400000)    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x0000003cb5c00000)    libgpg-error.so.0 => /lib64/libgpg-error.so.0 (0x0000003cb6800000)    libfreebl3.so => /lib64/libfreebl3.so (0x0000003cb5800000)[root@srdsdevapp73 ~]# service pacemaker startStarting Pacemaker Cluster Manager                         [  OK  ] 

總結

Linux下查找動態鏈接庫的默認路徑(未在/etc/ld.so.conf中設置，動態鏈接庫加載時會優先查找/etc/ld.so.cache中庫)的順序如下，如果有同名的庫文件擋在前面，可能導致動態鏈接庫加載失敗。

1. /lib
2. /usr/lib
3. /lib64
4. /usr/lib64