centos7下安装caffe以及pycaffe流程及问题解决
一、环境准备
安装依赖包
python、tesseract依赖：
yum install openssl openssl-devel gcc gcc-c++ autoconf automake libtool libjpeg libpng libtiff zlib libjpeg-devel libpng-devel libtiff-devel zlib-devel python-devel ImageMagick ImageMagick-devel ghostscript python-imaging vim (libyaml)
caffe依赖：
yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel openblas-devel
源码位置
下载部署包
二、安装Python3
- （1）创建文件夹：
mkdir /usr/local/python3
- （2）解压下载的文件：
tar xvJf Python-3.6.6.tar.xz
- （3）进入解压后目录，执行编译命令
./configure --prefix=/usr/local/python3
make && make install
- （4）创建软连接命令：
ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3
三、安装caffe
- （1）下载额外的rpm包（可能yum源没有），然后安装。
gflags-2.1.1-6.el7.x86_64.rpm glog-devel-0.3.3-8.el7.x86_64.rpm leveldb-1.12.0-11.el7.x86_64.rpm libaec-devel-1.0.4-1.el7.x86_64.rpm lmdb-libs-0.9.22-2.el7.x86_64.rpm
gflags-devel-2.1.1-6.el7.x86_64.rpm hdf5-1.8.12-10.el7.x86_64.rpm leveldb-devel-1.12.0-11.el7.x86_64.rpm lmdb-0.9.22-2.el7.x86_64.rpm
glog-0.3.3-8.el7.x86_64.rpm hdf5-devel-1.8.12-10.el7.x86_64.rpm libaec-1.0.4-1.el7.x86_64.rpm lmdb-devel-0.9.22-2.el7.x86_64.rpm
- （2）yum安装依赖包。
yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel
-（3）解压caffe源码包，然后执行：
cp Makefile.config.example Makefile.config
-（4）修改文件“Makefile.conf”，内容如下：
CPU_ONLY := 1
USE_OPENCV := 0
WITH_PYTHON_LAYER := 1
……
PYTHON_INCLUDE := /usr/local/python3/include/python3.6m \
/usr/local/python3/lib/python3.6/site-packages/numpy/core/include \
/usr/local/python3/
- （5）开始执行编译：
make all -j8
make test -j16
make runtest
注意：这里“-j8”等参数，表示使用多少个线程进行编译。
- （6） 编译pycaffe。
make pycaffe -j8
然后将编译完成之后的目录“/opt/soft/caffe/caffe-master/python/caffe”拷贝到python的site-packages目录中。
- 异常1: 遇到如下错误：
[root@aidm-1 caffe-master]# vim Makefile.config
[root@aidm-1 caffe-master]# make all -j8
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/ld: 找不到 -lcblas
/usr/bin/ld: 找不到 -latlas
collect2: 错误：ld 返回 1
make: *** [.build_release/lib/libcaffe.so.1.0.0] 错误 1
则进入 “usr/lib63/atlas”，创建两个软连接：
sudo ln -sv libsatlas.so.3.10 libcblas.so
sudo ln -sv libsatlas.so.3.10 libatlas.so
然后修改“Makefile.config”文件，在“LIBRARY_DIRS”后面追加 “/usr/lib64/atlas”。
- 异常2: 安装完成后，通过python调用出现如下错误：
[root@aidm-1 ~]# python3
Python 3.6.6 (default, Jun 26 2019, 17:33:11)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/python3/lib/python3.6/site-packages/caffe/__init__.py", line 1, in <module>
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File "/usr/local/python3/lib/python3.6/site-packages/caffe/pycaffe.py", line 13, in <module>
from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: /usr/local/python3/lib/python3.6/site-packages/caffe/_caffe.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE
要在“Makefile.config”文件中去掉这行注释：
WITH_PYTHON_LAYER := 1
然后，在/usr/lib64中添加如下两个文件：
libpython3.4m.so.1.0
libboost_python34.so.1.69.0
然后，重新编译。
- 异常3: python调用出现如下错误：
Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/abc/workplace/caffe/python/caffe/__init__.py", line 4, in <module>
from .proto.caffe_pb2 import TRAIN, TEST
File "/home/abc/workplace/caffe/python/caffe/proto/caffe_pb2.py", line 17, in <module>
serialized_pb='\n\x0b\x63\x61\x66\x66\x65.proto\x12\x05\x63\x61\x66\x66\x65\"\x1c\n\tBlobShape\x12\x0f\n\x03\x64i...'
File "/home/abc/anaconda3/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 824, in __new__
return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: expected bytes, str found
原因是之前编译的python包“caffe”中的proto/caffe_pb2.py文件有问题，找一个正确的替换下即可。
- 有用的linux命令
ls -l|grep "7月 2" | awk -F ' ' '{print $9;}' | xargs -i cp {} /home/aidm-algorithm/libs/ext-lib-all
问题
安装Python完成之后，执行pip3 list 命令之后报如下错误：
[root@aidm-2 ~]# pip3 list
Package Version
---------- -------
pip 10.0.1
setuptools 39.0.1
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping
- 原因：新版本pip默认使用https，而Python里面没有找到ssl模块（由于编译python的时候没有指定）。
- 解决方式1: 配置一个http的pip源，然后将这个源的域名设置为受信任的，参考操作如下：
mkdir ~/.pip
cd ~/.pip
touch pip.conf
然后添加如下内容：
[global]
index-url = http://pypi.douban.com/simple
[install]
trusted-host = pypi.douban.com
- 解决方式2: 首先检查系统中是否包含openssl和opeenssl-devel两个包，如果没有，则首先进行安装。然后重新编译python：
./configure --prefix=/usr/local/python3 --with-ssl
make && make install
