FoundationPose复现

见贤思齐

目录intro1电脑配置2环境配置2.1资源下载2.2conda环境3测试3.1初始demo4补充4.1cuda版本问题4.2Buildextensions报错4.3run_demo.py报错intro论文我就不多说了code:https://github.com/NVlabs/FoundationPosepaper:https://arxiv.org/abs/2312.083441电脑配置Linux环境：ubuntu22.04cuda版本：11.8显卡：GeForceRTX4080IDE：pycharm2环境配置2.1资源下载库gitclonehttps://github.com/NVlabs/FoundationPose.git1必须下载权重主目录下新建./weights/，放在该路径下测试数据主目录下新建./demo_data/，放在该路径下可选下载数据集太大了，我没下载model-freefew-shotversion2.2conda环境这里参考blogEigen33.4.0安装cd$HOME&wget-qhttps://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz&\tar-xzfeigen-3.4.0.tar.gz&\cdeigen-3.4.0&mkdirbuild&cdbuildcmake..-Wno-dev-DCMAKE_BUILD_TYPE=Release-DCMAKE_CXX_FLAGS=-std=c++14..sudomakeinstallcd$HOME&rm-rfeigen-3.4.0eigen-3.4.0.tar.gz123456建议不要使用官方github上的，我用下面的路径一直配置不对#InstallEigen33.4.0undercondaenvironmentcondainstallconda-forge::eigen=3.4.0exportCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH:/eigen/path/under/conda"123创建虚拟环境我这里都是在pycharm的终端中进行（个人习惯）#createcondaenvironmentcondacreate-nfoundationposepython=3.9#activatecondaenvironmentcondaactivatefoundationpose1234python库#installdependenciespython-mpipinstall-rrequirements.txt12下面的貌似自动给安装好了#InstallNVDiffRastpython-mpipinstall--quiet--no-cache-dirgit+https://github.com/NVlabs/nvdiffrast.git#Kaolin(Optional,neededifrunningmodel-freesetup)python-mpipinstall--quiet--no-cache-dirkaolin==0.15.0-fhttps://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.0.0_cu118.html#PyTorch3Dpython-mpipinstall--quiet--no-index--no-cache-dirpytorch3d-fhttps://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py39_cu118_pyt200/download.html123456这里可以检查下自己的torch、pytorch3d版本，终端输入pythonPython3.9.19(main,May62024,19:43:03)[GCC11.2.0]::Anaconda,Inc.onlinuxType"help","copyright","credits"or"license"formoreinformation.>>>importtorch,torchvision,pytorch3d>>>torch.__version__'2.0.0+cu118'>>>torchvision.__version__'0.15.1+cu118'>>>pytorch3d.__version__'0.7.3'>>>exit()1234567891011-构建C++扩展#BuildextensionsCMAKE_PREFIX_PATH=$CONDA_PREFIX/lib/python3.9/site-packages/pybind11/share/cmake/pybind11bashbuild_all_conda.sh12这里会花费比较长时间，并且可能会有各种Warning，只要最后能到100%，无视最后成功的输出[100%]LinkingCXXsharedmodulemycpp.cpython-39-x86_64-linux-gnu.solto-wrapper:warning:usingserialcompilationof2LTRANSjobs[100%]BuilttargetmycppObtainingfile:///home/user/code/embodied_AI/FoundationPose/bundlesdf/mycudaPreparingmetadata(setup.py)...doneInstallingcollectedpackages:commonRunningsetup.pydevelopforcommonSuccessfullyinstalledcommon-0.0.0123456783测试3.1初始demopythonrun_demo.py1成功弹出4补充4.1cuda版本问题我之前的所有代码都是在cuda11.6基础上的，所以这里需要进行一个cuda版本切换，否则会很麻烦（连环报错）按照blog一步步来官方下载11.8wgethttps://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.runsudoshcuda_11.8.0_520.61.05_linux.run12记得再下载cudnn更换~/.bashrc里的cuda路径并且source在虚拟环境终端下nvcc-Vnvcc:NVIDIA(R)CudacompilerdriverCopyright(c)2005-2022NVIDIACorporationBuiltonWed_Sep_21_10:33:58_PDT_2022Cudacompilationtools,release11.8,V11.8.89Buildcuda_11.8.r11.8/compiler.31833905_12345这里我还在conda中进行了下面操作condainstallcudatoolkit=11.81成功4.2Buildextensions报错报错bash:build_all_conda.sh:没有那个文件或目录1原因：路径没有在FoundationPose下遇到找不到Eigen3路径的在终端使用下面代码重新安装cd$HOME&wget-qhttps://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz&\tar-xzfeigen-3.4.0.tar.gz&\cdeigen-3.4.0&mkdirbuild&cdbuildcmake..-Wno-dev-DCMAKE_BUILD_TYPE=Release-DCMAKE_CXX_FLAGS=-std=c++14..sudomakeinstallcd$HOME&rm-rfeigen-3.4.0eigen-3.4.0.tar.gz123456torch版本对应如果cuda与torch会pytorch3d的版本不对应，也会报错，所以强烈建议按照官方的版本来，为自己11.6就是遇到各种问题换成11.8直接都解决4.3run_demo.py报错packages/torch/nn/modules/module.py",line1130,in_call_implreturnforward_call(*input,**kwargs)File"/home/zqy/anaconda3/envs/foundationpose/lib/python3.9/site-packages/torch/nn/modules/transformer.py",line437,inforwardreturntorch._transformer_encoder_layer_fwd(RuntimeError:expectedscalartypeHalfbutfoundFloat12345原因：torch的版本太低，切换成2.0以上就不会遇到了