Repository navigation
Duplicate #136 code and meet make errors #293
Description
I copied #136 codes and revised as Floop suggested that put 3 lines right below
Scope root = Scope::NewRootScope();
And I have many make errors.
If anyone could help I will be much appreciated.
I am running on the following conditions:
Freshly built Ubuntu 20.04LTS
CUDA 10.7
CuDnn 8.4.0
Bazel 5.1.1
GCC 9.4.0
make 4.2.1
tensoflow_cc build on 20-05-27, using tensorflow 2.9.0
I am using a PC with an Intel CPU, an Intel GPU and an NVidia T600.
I compiled tensorflow_cc with default CMakeLists file except for putting REQUIRE_CUDA ON, and make is successful.
Then I copied 136 codes:
#include "tensorflow/cc/client/client_session.h"
#include "tensorflow/cc/ops/standard_ops.h"
#include "tensorflow/core/framework/tensor.h"
#include "tensorflow/core/graph/default_device.h"
int main() {
using namespace tensorflow;
using namespace tensorflow::ops;
Scope root = Scope::NewRootScope();
GraphDef def;
TF_CHECK_OK(root.ToGraphDef(&def));
graph::SetDefaultDevice(false ? "/device:GPU:0" : "/cpu:0", &def);
// Matrix A = [3 2; -1 0]
auto A = Const(root, { {3.f, 2.f}, {-1.f, 0.f} });
// Vector b = [3 5]
auto b = Const(root, { {3.f, 5.f} });
// v = Ab^T
auto v = MatMul(root.WithOpName("v"), A, b, MatMul::TransposeB(true));
for (auto &node: *def.mutable_node()) {
node.set_device("/cpu:0");
std::cout << node.name() << " = '" << node.device() <<"'"<< std::endl;
}
std::cout << "=======================\n";
std::vector outputs;
ClientSession session(root);
// Run and fetch v
TF_CHECK_OK(session.Run({v}, &outputs));
// Expect outputs[0] == [19; -3]
LOG(INFO) << outputs[0].matrix();
return 0;
}
I have following make errors:
[ 50%] Building CXX object CMakeFiles/example.dir/example2.cpp.o
[100%] Linking CXX executable example
/usr/bin/ld: CMakeFiles/example.dir/example2.cpp.o: in function google::protobuf::Arena::AllocHook(std::type_info const*, unsigned long) const': example2.cpp:(.text._ZNK6google8protobuf5Arena9AllocHookEPKSt9type_infom[_ZNK6google8protobuf5Arena9AllocHookEPKSt9type_infom]+0x41): undefined reference to google::protobuf::Arena::OnArenaAllocation(std::type_info const*, unsigned long) const'
/usr/bin/ld: CMakeFiles/example.dir/example2.cpp.o: in function google::protobuf::internal::ArenaStringPtr::CreateInstance(google::protobuf::Arena*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*)': example2.cpp:(.text._ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x60): undefined reference to google::protobuf::internal::LogMessage::LogMessage(google::protobuf::LogLevel, char const*, int)'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x78): undefined reference to google::protobuf::internal::LogMessage::operator<<(char const*)' /usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x8d): undefined reference to google::protobuf::internal::LogFinisher::operator=(google::protobuf::internal::LogMessage&)'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x9d): undefined reference to google::protobuf::internal::LogMessage::~LogMessage()' /usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x221): undefined reference to google::protobuf::internal::ArenaImpl::AllocateAligned(unsigned long)'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x240): undefined reference to google::protobuf::internal::ArenaImpl::AllocateAlignedAndAddCleanup(unsigned long, void (*)(void*))' /usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN6google8protobuf8internal14ArenaStringPtr14CreateInstanceEPNS0_5ArenaEPKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x2d4): undefined reference to google::protobuf::internal::LogMessage::~LogMessage()'
/usr/bin/ld: CMakeFiles/example.dir/example2.cpp.o: in function google::protobuf::internal::GetEmptyStringAlreadyInited[abi:cxx11]()': example2.cpp:(.text._ZN6google8protobuf8internal27GetEmptyStringAlreadyInitedB5cxx11Ev[_ZN6google8protobuf8internal27GetEmptyStringAlreadyInitedB5cxx11Ev]+0xb): undefined reference to google::protobuf::internal::fixed_address_empty_string[abi:cxx11]'
/usr/bin/ld: CMakeFiles/example.dir/example2.cpp.o: in function tensorflow::NodeDef::set_device(char const*)': example2.cpp:(.text._ZN10tensorflow7NodeDef10set_deviceEPKc[_ZN10tensorflow7NodeDef10set_deviceEPKc]+0x4b): undefined reference to google::protobuf::internal::LogMessage::LogMessage(google::protobuf::LogLevel, char const*, int)'
/usr/bin/ld: example2.cpp:(.text._ZN10tensorflow7NodeDef10set_deviceEPKc[_ZN10tensorflow7NodeDef10set_deviceEPKc]+0x63): undefined reference to google::protobuf::internal::LogMessage::operator<<(char const*)' /usr/bin/ld: example2.cpp:(.text._ZN10tensorflow7NodeDef10set_deviceEPKc[_ZN10tensorflow7NodeDef10set_deviceEPKc]+0x75): undefined reference to google::protobuf::internal::LogFinisher::operator=(google::protobuf::internal::LogMessage&)'
/usr/bin/ld: example2.cpp:(.text._ZN10tensorflow7NodeDef10set_deviceEPKc[_ZN10tensorflow7NodeDef10set_deviceEPKc]+0x85): undefined reference to google::protobuf::internal::LogMessage::~LogMessage()' /usr/bin/ld: example2.cpp:(.text._ZN10tensorflow7NodeDef10set_deviceEPKc[_ZN10tensorflow7NodeDef10set_deviceEPKc]+0x11d): undefined reference to google::protobuf::internal::LogMessage::~LogMessage()'
/usr/bin/ld: CMakeFiles/example.dir/example2.cpp.o: in function google::protobuf::RepeatedPtrField<tensorflow::NodeDef>::TypeHandler::Type* google::protobuf::internal::RepeatedPtrFieldBase::Mutable<google::protobuf::RepeatedPtrField<tensorflow::NodeDef>::TypeHandler>(int)': example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0x49): undefined reference to google::protobuf::internal::LogMessage::LogMessage(google::protobuf::LogLevel, char const*, int)'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0x61): undefined reference to google::protobuf::internal::LogMessage::operator<<(char const*)' /usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0x73): undefined reference to google::protobuf::internal::LogFinisher::operator=(google::protobuf::internal::LogMessage&)'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0x83): undefined reference to google::protobuf::internal::LogMessage::~LogMessage()' /usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0xb1): undefined reference to google::protobuf::internal::LogMessage::LogMessage(google::protobuf::LogLevel, char const*, int)'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0xc9): undefined reference to google::protobuf::internal::LogMessage::operator<<(char const*)' /usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0xdb): undefined reference to google::protobuf::internal::LogFinisher::operator=(google::protobuf::internal::LogMessage&)'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0xeb): undefined reference to google::protobuf::internal::LogMessage::~LogMessage()' /usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0x12e): undefined reference to google::protobuf::internal::LogMessage::~LogMessage()'
/usr/bin/ld: example2.cpp:(.text._ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi[_ZN6google8protobuf8internal20RepeatedPtrFieldBase7MutableINS0_16RepeatedPtrFieldIN10tensorflow7NodeDefEE11TypeHandlerEEEPNT_4TypeEi]+0x150): undefined reference to `google::protobuf::internal::LogMessage::~LogMessage()'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/example.dir/build.make:87: example] Error 1
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/example.dir/all] Error 2
Activity
FloopCZ commentedon Jun 2, 2022
Can you try to link Protobuf to your program?
lancelot-ch commentedon Jun 3, 2022
Thanks for the reply. Sorry that I don't know exactly how to link Protobuf.
I installed Protobuf from source and the folder is at ~/protobuf.3.9.0
The project CMakeLists.txt file is as below:
CMakeLists.txt
cmake_minimum_required(VERSION 3.16)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)
find_package(TensorflowCC REQUIRED)
add_executable(test test.cpp)
target_include_directories(test PUBLIC
"${PROJECT_BINARY_DIR}"
)
add_subdirectory(abseil-cpp)
target_link_libraries(test TensorflowCC::TensorflowCC absl::strings)
FloopCZ commentedon Jun 3, 2022
See e.g., https://cmake.org/cmake/help/latest/module/FindProtobuf.html
lancelot-ch commentedon Jun 3, 2022
Thanks.
I tried include these 3 lines. And I get the following error:
$ cmake ..
CMake Error at CMakeLists.txt:20 (target_include_directories):
target_include_directories called with invalid arguments
And line 20 is :
target_include_directories(test ${Protobuf_INCLUDE_DIRS})
I am not sure what is wrong.
FloopCZ commentedon Jun 3, 2022
Please print the
${Protobuf_INCLUDE_DIRS}variable usingmessage()call. Maybe there are some spaces and you have to wrap it in".lancelot-ch commentedon Jun 3, 2022
Thanks.
I do see something weird.
cmake ..
/home/user/anaconda3/lib/libprotobuf.so-lpthread
/home/user/anaconda3/include
CMake Error at CMakeLists.txt:22 (target_include_directories):
target_include_directories called with invalid arguments
-- Configuring incomplete, errors occurred!
I recently installed Anaconda and I am wondering if that changed the path. Shall I just uninstall Anaconda and try?
FloopCZ commentedon Jun 3, 2022
You can try or you can try to change the PATH, but protobuf from anaconda is probably not the one you want. Anyway, the error you are seeing is most likely caused by something else, just add
${Protobuf_INCLUDE_DIRS}and${Protobuf_LIBRARIES}to thetarget_include_directoriesandtarget_link_librariesdirectives that you already had.lancelot-ch commentedon Jun 4, 2022
Thanks. Could I verify that I should install protobuf by myself(hopefully 3.9.0), and compile tensorflow_cc with
option(INSTALL_PROTOBUF "Install protobuf compatible with tensorflow version." OFF)
I am not sure if it is correct.
FloopCZ commentedon Jun 6, 2022
lancelot-ch commentedon Jun 7, 2022
I tried but I failed.
I do with this way:
I revise tensorflow_cc/tensorflow_cc/CMakeLists.txt
option(INSTALL_PROTOBUF "Install protobuf compatible with tensorflow version." ON) #previously it is OFF
I run cmake:
~/tensorflow_cc/tensorflow_cc/build$ cmake -DLOCAL_RAM_RESOURCES=2048 -DLOCAL_CPU_RESOURCES=4 ..
I see below error messages:
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Will build Protobuf from ''.
CMake Error at /usr/share/cmake-3.16/Modules/ExternalProject.cmake:2630 (message):
No download info given for 'protobuf-external' and its source directory:
/home/user/tensorflow_cc/tensorflow_cc/build/protobuf/src/protobuf-external
is not an existing non-empty directory. Please specify one of:
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/ExternalProject.cmake:3236 (_ep_add_download_command)
cmake/ProtobufExternal.cmake:17 (ExternalProject_Add)
CMakeLists.txt:36 (include)
Seems that it suggests that I get an external Protobuf?
Many thanks.
FloopCZ commentedon Jun 8, 2022
lancelot-ch commentedon Jun 8, 2022
I removed build folder and tried again. I got the same error.
Will build Protobuf from ''.
CMake Error at /usr/share/cmake-3.16/Modules/ExternalProject.cmake:2630 (message):
No download info given for 'protobuf-external' and its source directory:
/home/user/tensorflow_cc/tensorflow_cc/build/protobuf/src/protobuf-external
is not an existing non-empty directory. Please specify one of:
Call Stack (most recent call first):
/usr/share/cmake-3.16/Modules/ExternalProject.cmake:3236 (_ep_add_download_command)
cmake/ProtobufExternal.cmake:17 (ExternalProject_Add)
CMakeLists.txt:36 (include)
-- Configuring incomplete, errors occurred!
See also "/home/user/tensorflow_cc/tensorflow_cc/build/CMakeFiles/CMakeOutput.log".
lancelot-ch commentedon Jun 9, 2022
Many thanks to FloopCZ that with Docker I can make the program. The results is as following:
root@9c79a0ff0c79:/home/test/build# ./test
2022-06-09 01:18:04.209457: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable
TF_ENABLE_ONEDNN_OPTS=0.2022-06-09 01:18:04.218837: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX512F AVX512_VNNI
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-06-09 01:18:05.116681: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-09 01:18:05.139072: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-09 01:18:05.139187: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-09 01:18:05.455365: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-09 01:18:05.455501: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-09 01:18:05.455588: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-09 01:18:05.455691: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 2636 MB memory: -> device: 0, name: NVIDIA T600 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 7.5
2022-06-09 01:18:05.456816: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
However, I realized that even if I change the device from GPU to CPU (graph::SetDefaultDevice("/cpu:0", &def);
), the result is always using GPU device. I do not quite understand.
And also I hope to get the source code installing done if possible. Thanks a lot.
FloopCZ commentedon Jun 25, 2022
Hi, try to run
before running your program to disable the GPU
lancelot-ch commentedon Jul 1, 2022
Actually I am meant to use GPU to accelerate program running by using GPU. I have been successful to run program using tensorflow_cc on CPU. Anyway, I will keep trying.
FloopCZ commentedon Jul 16, 2022
ok, I think I misunderstood your previous message:
Is there anything else I can help with or should we close the issue?