绑定完请刷新页面
取消
刷新

分享好友

×
取消 复制
scylladb源码安装过程
2022-05-25 15:39:45

1.安装ubuntu
http://mirrors.163.com/ubuntu-releases/18.04.4/

使用以上的国内地址下载镜像速度更快,使用的ubuntu版本是ubuntu-18.04.5-desktop-amd64.iso。

虚拟机使用vmware,安装过程略过,可能遇到卡在安装vmtools,参考https://blog.csdn.net/qq_34415586/article/details/80347813解决。

安装完成后配置为国内源,参考https://www.cnblogs.com/bug132294/p/12749192.html。



2.下载scylladb源码
安装git:sudo apt-get install git

拉取scylla源码,使用了国内的git源:

git clone https://github.com.cnpmjs.org/scylladb/scylla.git

切换到tag:

git checkout scylla-4.1.0 -b scylla4.1.0

拉取子项目代码:

git submodule update --init --recursive



3.安装依赖
增加scylladb依赖库的源:

sudo add-apt-repository -y ppa:scylladb/ppa

安装依赖包:

sudo ./install-dependencies.sh



4.configure
install-dependencies.sh运行完毕后会提示configure的语法:

./configure.py --enable-dpdk --mode=release --static-thrift --static-boost --static-yaml-cpp --compiler=/opt/scylladb/bin/g++-7 --cflags="-I/opt/scylladb/include -L/opt/scylladb/lib/x86-linux-gnu/" --ldflags="-Wl,-rpath=/opt/scylladb/lib"



4.1版本应该需要g++8,需要安装一下。更新的版本可能需要g++10,目前需要使用源码安装。

安装g++8:

sudo apt-get install g++-8

运行configure.py,修改了g++的路径:

./configure.py --enable-dpdk --mode=release --static-thrift --static-boost --static-yaml-cpp --compiler=/usr/bin/g++-8 --cflags="-I/opt/scylladb/include -L/opt/scylladb/lib/x86-linux-gnu/" --ldflags="-Wl,-rpath=/opt/scylladb/lib"



报错:

CMake Error at cmake/SeastarDependencies.cmake:107 (find_package):
Could not find a configuration file for package "fmt" that is compatible
with requested version "5.0.0".
需要安装更新版的fmt,先删除原来的:sudo apt remove libfmt-dev,再安装:

wget -c http://archive.ubuntu.com/ubuntu/pool/universe/f/fmtlib/libfmt-dev_5.2.1+ds-2_amd64.deb

sudo apt install ./libfmt-dev_5.2.1+ds-2_amd64.deb



重新config,又报错:

No package 'jsoncpp' found

安装:

sudo apt-get install libjsoncpp-dev



重新config,又报错:

No package 'libxxhash' found

安装:

wget http://ftp.cn.debian.org/debian/pool/main/x/xxhash/libxxhash0_0.8.0-1_amd64.deb

sudo apt install ./libxxhash0_0.8.0-1_amd64.deb

wget http://mirrors.ustc.edu.cn/debian/pool/main/x/xxhash/libxxhash-dev_0.8.0-1_amd64.deb

sudo apt install ./libxxhash-dev_0.8.0-1_amd64.deb\



重新config,又报错:



Traceback (most recent call last):
File "./configure.py", line 1308, in <module>
proc_res = subprocess.run(["thrift", "-version"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'thrift': 'thrift'
安装thrift:

sudo apt install thrift-compiler



重新config,又报错:

No package 'lua' found

查看configure.py脚本,似乎需要5.3版本,因此安装:

sudo apt-get install liblua5.3-dev



重新config成功



5.编译
命令:

ninja release -j 6



执行报错:

File "./idl-compiler.py", line 25, in <module>
import pyparsing as pp
ModuleNotFoundError: No module named 'pyparsing'

这个是python的包,因此安装:

sudo apt-get install python-pip

使用-i清华源安装,也可以先配置到配置文件:

pip install pyparsing -i https://pypi.tuna.tsinghua.edu.cn/simple



再执行,下一个报错:

#include <thrift/Thrift.h>

安装:

sudo apt-get install scylla-libthrift010-dev



报错:

build/release/gen/alternator/expressionsLexer.hpp:55:10: fatal error: antlr3.hpp: No such file or directory
#include <antlr3.hpp>

安装:

sudo apt-get install scylla-antlr35-c++-dev

可能版本不大对,依赖没安装全,问题比较多



报错:

/usr/include/boost/icl/type_traits/type_to_string.hpp:56:12: error: partial specialization of ‘struct boost::icl::type_to_string<Unary<Type> >’ after instantiation of ‘struct boost::icl::type_to_string<std::__cxx11::basic_string<char> >’ [-fpermissive]

修改/usr/include/boost/icl/type_traits/type_to_string.hpp,调整了代码顺序:

namespace boost{ namespace icl
{
//--------------------------------------------------------------------------
template<class Type>
struct type_to_string
{
/** Convert the type to it's *tring */
static std::string apply();
};


//-------------------------------------------------------------------------
template<template<class> class Templ>
struct unary_template_to_string
{
static std::string apply();
};

template <template<class>class Unary, class Type>
struct type_to_string<Unary<Type> >
{
static std::string to_string()
{
return unary_template_to_string<Unary>::apply()+"<"+type_to_string<Type>::apply()+">";
}
};

// ---------------------------------------------------------------------------
template<template<class,class>class Templ>
struct binary_template_to_string
{
static std::string apply();
};

template <template<class Type1, class Type2>class Binary, class Type1, class Type2>
struct type_to_string<Binary<Type1, Type2> >
{
static std::string apply()
{
return binary_template_to_string<Binary>::apply()+
"<"+type_to_string<Type1>::apply()+","+type_to_string<Type2>::apply()+">";
}
};

//--------------------------------------------------------------------------
template<>inline std::string type_to_string<bool>::apply() { return "bool"; }
template<>inline std::string type_to_string<char>::apply() { return "char"; }
template<>inline std::string type_to_string<short>::apply(){ return "short"; }
template<>inline std::string type_to_string<int>::apply() { return "int"; }
template<>inline std::string type_to_string<long>::apply() { return "long"; }
template<>inline std::string type_to_string<long long>::apply(){ return "Long"; }

template<>inline std::string type_to_string<unsigned char>::apply(){ return "char+"; }
template<>inline std::string type_to_string<unsigned short>::apply(){ return "short+"; }
template<>inline std::string type_to_string<unsigned int>::apply() { return "int+"; }
template<>inline std::string type_to_string<unsigned long>::apply() { return "long+"; }
template<>inline std::string type_to_string<unsigned long long>::apply(){ return "Long+"; }

template<>inline std::string type_to_string<float>::apply() { return "flt"; }
template<>inline std::string type_to_string<double>::apply() { return "dbl"; }
template<>inline std::string type_to_string<std::string>::apply() { return "string"; }

}} // namespace boost icl


报错:

compress.cc:24:10: fatal error: snappy-c.h: No such file or directory
#include <snappy-c.h>

安装:

sudo apt-get install libsnappy-dev



报错:

g++-8: fatal error: Killed signal terminated program cc1plus

原因应该是虚拟机内存配了8G不大够用,减少线程数:ninja release -j 2



报错:

./alternator/rjson.hh:65:10: fatal error: rapidjson/document.h: No such file or directory
#include <rapidjson/document.h>

安装:

sudo apt-get install rapidjson-dev



报错一堆gnutls相关未定义:

In file included from alternator/auth.cc:26:
/usr/include/gnutls/crypto.h:35:10: error: ‘gnutls_cipher_algorithm_t’ has not been declared
gnutls_cipher_algorithm_t cipher,
^~~~~~~~~~~~~~~~~~~~~~~~~

修改alternator/auth.cc:增加#include <gnutls/gnutls.h>



报错:

g++-8: error: /usr/lib/x86_64-linux-gnu/libunistring.so: No such file or directory

安装:

sudo apt-get install libunistring-dev



报错:

build/release/gen/Cassandra.cpp:16366: error: undefined reference to 'apache::thrift::TApplicationException::write(apache::thrift::protocol::TProtocol*) const'
collect2: error: ld returned 1 exit status

修改:

sudo cp /opt/scylladb/lib/libthrift* /usr/lib



报错:

/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libboost_regex.a(icu.o):function boost::basic_regex<int, boost::icu_regex_traits>::imbue(icu_60::Locale): error: undefined reference to 'icu_60::Collator::createInstance(icu_60::Locale const&, UErrorCode&)'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libicuuc.a(udata.ao):function openCommonData(char const*, int, UErrorCode*): error: undefined reference to 'icudt60_dat'
/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libicuuc.a(udata.ao):function openCommonData(char const*, int, UErrorCode*): error: undefined reference to 'icudt60_dat'

搜这两个函数搜到两个库

./x86_64-linux-gnu/libicuuc.so.60

./x86_64-linux-gnu/libicui18n.so.60.2

编辑build.ninja,修改以下内容:

libs = -Wl,-Bstatic -lyaml-cpp -Wl,-Bdynamic -latomic -llz4 -lz -lsnappy -ljsoncpp -lstdc++fs -lcrypt -lcryptopp -lpthread -Wl,-Bstatic -lboost_date_time -lboost_regex -licuuc -Wl,-Bdynamic -lxxhash -llua5.3 -lsystemd

修改为:

libs = -licuuc -licui18n -Wl,-Bstatic -lyaml-cpp -Wl,-Bdynamic -latomic -llz4 -lz -lsnappy -ljsoncpp -lstdc++fs -lcrypt -lcryptopp -lpthread -Wl,-Bstatic -lboost_date_time -lboost_regex -licuuc -Wl,-Bdynamic -lxxhash -llua5.3 -lsystemd



报错:

/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/Scrt1.o:function _start: error: undefined reference to 'main'

此过程出错:

[18/149] LINK (stripped) build/release/test/boost/log_heap_test
FAILED: build/release/test/boost/log_heap_test

不知道为啥没有main函数,因为是test程序,应该不影响功能,在ninja.build中注释到这个编译。还有挺多这种的,同样处理。



修改后编译通过。



6.安装
sudo apt-get install patchelf

sudo apt-get install gawk

sudo ln -s /bin/gzip /usr/bin/gzip

sudo ln -s /sbin/ifconfig /usr/sbin/ifconfig

sudo apt-get install ethtool;sudo ln -s /sbin/ethtool /usr/sbin/ethtool

sudo ln -s /bin/netstat /usr/bin/netstat

sudo apt-get install hwloc

sudo apt-get install pigz

sudo mkdir /usr/lib/lib64;sudo ln -s /usr/lib/x86_64-linux-gnu/libthread_db.so /usr/lib/lib64/libthread_db.so

sudo ln -s /lib/x86_64-linux-gnu/libthread_db-1.0.so /lib64/libthread_db-1.0.so

#这个不知道是什么配置文件,touch了一个,可能有问题

sudo mkdir /etc/crypto-policies;sudo mkdir /etc/crypto-policies/back-ends;sudo touch /etc/crypto-policies/back-ends/gnutls.config



python3 -m compileall ./dist/common/scripts/ ./seastar/scripts/perftune.py ./tools/scyllatop

ninja -j 3 build/release/scylla-package.tar.gz



生产文件:./build/release/scylla-package.tar.gz

考到其他目录解压缩后,修改一下install.sh后运行:sudo ./install.sh

adjust_bin() {
local bin="$1"
# We could add --set-rpath too, but then debugedit (called by rpmbuild) barfs
# on the result. So use LD_LIBRARY_PATH in the thunk, below.
strip $root/$prefix/libexec/$bin
patchelf \
--set-interpreter "$prefix/libreloc/ld.so" \
"$root/$prefix/libexec/$bin"
cat > "$root/$prefix/bin/$bin" <<EOF
#!/bin/bash -e
export GNUTLS_SYSTEM_PRIORITY_FILE="\${GNUTLS_SYSTEM_PRIORITY_FILE-$prefix/libreloc/gnutls.config}"
export LD_LIBRARY_PATH="$prefix/libreloc"
exec -a "\$0" "$prefix/libexec/$bin" "\$@"
EOF
chmod +x "$root/$prefix/bin/$bin"
}



可以运行了:

~/software/scylla-package$ scylla --version
4.1.0-0.20201128.b443b2574
————————————————
版权声明:本文为CSDN博主「cocoti」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:https://blog.csdn.net/xuhaitao23/article/details/110259728

分享好友

分享这个小栈给你的朋友们,一起进步吧。

ScyllaDB
创建时间:2022-04-19 10:18:17
ScyllaDB 是用 C++ 重写的 Cassandra,每节点每秒处理 100 万 TPS。ScyllaDB 完全兼容 Apache Cassandra,拥有比 Cassandra 多 10x 倍的吞吐量,降低了延迟。 ScyllaDB 是性能优异的 NoSQL 列存储数据库。 ScyllaDB 在垃圾收集或者 Compaction 的时候不需要暂停;在常规生产负载的时候可以添加和删除节点。
展开
订阅须知

• 所有用户可根据关注领域订阅专区或所有专区

• 付费订阅:虚拟交易,一经交易不退款;若特殊情况,可3日内客服咨询

• 专区发布评论属默认订阅所评论专区(除付费小栈外)

技术专家

查看更多
  • 飘絮絮絮丶
    专家
戳我,来吐槽~