[Maxscale Filter] Community Bonding

During Community bonding with MariaDB as part of Google Sumer of Code 2016. I will be working on  setting up a working environment for the MaxScale filter project. The first step before starting work was to install and configure MaxScale. This post will give a general idea about the MaxScale configuration. Currently, Maxscale is not supported on OS X but most of the latest Linux distributions is just fine. I will be setting up Maxscale on CentOS7.

MariaDB MaxScale works with almost all versions of MariaDB and MYSQL.  I will be basing my work on MariaDB 10.1.

Installing MariaDB on CentOS7

Installing MariaDB on CentOS and most other rpm distributions is easy and straight forward,

     yum install mariadb-server mariadb-client

The above command will install both the client and server for you.

After Installation

After the installation completes, start MariaDB with:

sudo systemctl start mariadb

Then you can setup security to harden your installation by running the script

           mysql_secure_installation

This will assist in setting up a password for your installation.

Now we have MariaDB installed and server started. Lets go ahead and install Maxscale

Maxscale Installation from Source

In this post, i will describe how i build Maxscale from source on a fresh installation of Cent OS 7.

 

Now clone the GitHub project to your machine either via the web interface, your favorite graphical interface or the git command line

$ git clone https://github.com/mariadb-corporation/MaxScale
Cloning into 'MaxScale'...

This will create a Maxscale folder with the source code, now we are set to start building.

In order to build Maxscale , we will need a C compiler installed as well as some build tools and libraries. The following command should get all of them installed.

               sudo yum install mariadb-devel mariadb-embedded-devel libedit-                        devel gcc gcc-c++ ncurses-devel bison flex glibc-devel cmake libgcc                     perl make libtool openssl-devel libaio libaio-devel librabbitmq-devel                     libcurl-devel pcre-devel rpm-build

output:
Install 1 Package
Upgrade 2 Packages (+4 Dependent packages)

Total download size: 18 M
Is this ok [y/d/N]: y
Downloading packages:
updates/7/x86_64/prestodelta | 500 kB 00:00:06
Delta RPMs reduced 899 k of updates to 101 k (88% saved)
(1/7): pcre-8.32-15.el7_8.32-15.el7_2.1.x86_64.drpm | 62 kB 00:00:01
(2/7): pcre-devel-8.32-15.el7_8.32-15.el7_2.1.x86_64.drpm | 39 kB 00:00:03
(3/7): glibc-devel-2.17-106.el7_2.6.x86_64.rpm | 1.0 MB 00:00:31
(4/7): libedit-devel-3.0-12.20121213cvs.el7.x86_64.rpm | 32 kB 00:00:18
(5/7): glibc-headers-2.17-106.el7_2.6.x86_64.rpm | 662 kB 00:00:20
(6/7): glibc-2.17-106.el7_2.6.x86_64.rpm | 3.6 MB 00:01:03
(7/7): glibc-common-2.17-106.el7_2.6.x86_64.rpm | 11 MB 00:05:05
——————————————————————————————————————————————————
Total 57 kB/s | 17 MB 00:05:05
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : glibc-2.17-106.el7_2.6.x86_64 1/13
Updating : glibc-common-2.17-106.el7_2.6.x86_64 2/13
Updating : glibc-headers-2.17-106.el7_2.6.x86_64 3/13
Updating : pcre-8.32-15.el7_2.1.x86_64 4/13

Verifying : glibc-devel-2.17-106.el7_2.4.x86_64 12/13
Verifying : glibc-2.17-106.el7_2.4.x86_64 13/13

Installed:
libedit-devel.x86_64 0:3.0-12.20121213cvs.el7

Updated:
glibc-devel.x86_64 0:2.17-106.el7_2.6 pcre-devel.x86_64 0:8.32-15.el7_2.1

Dependency Updated:
glibc.x86_64 0:2.17-106.el7_2.6 glibc-common.x86_64 0:2.17-106.el7_2.6 glibc-headers.x86_64 0:2.17-106.el7_2.6 pcre.x86_64 0:8.32-15.el7_2.1

Complete!

 

The command should install the basic tools needed and in my case as seen on the output, I  have them installed so lets go ahead and try to build Maxscale.

cd Maxscale
   mkdir build
   cd build

Maxscale uses out of source build so

cmake ../

Output:

[maxscale@localhost build]$ cmake ../
— CMake version: 2.8.11
— The C compiler identification is GNU 4.8.5
— The CXX compiler identification is GNU 4.8.5
— Check for working C compiler: /usr/bin/cc
— Check for working C compiler: /usr/bin/cc — works
— Detecting C compiler ABI info
— Detecting C compiler ABI info – done
— Check for working CXX compiler: /usr/bin/c++
— Check for working CXX compiler: /usr/bin/c++ — works
— Detecting CXX compiler ABI info
— Detecting CXX compiler ABI info – done
— Looking for include file arpa/inet.h
— Looking for include file arpa/inet.h – found

……………………………………………………………………………
……………………………………………………………………………..

— MySQL provider: MariaDB
— Looking for pcre_stack_guard in MYSQL_EMBEDDED_LIBRARIES_STATIC-NOTFOUND
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
MYSQL_EMBEDDED_LIBRARIES_STATIC
linked by target “cmTryCompileExec1784562683” in directory /home/maxscale/Desktop/MaxScale/build/CMakeFiles/CMakeTmp

CMake Error: Internal CMake error, TryCompile configure of cmake failed
— Looking for pcre_stack_guard in MYSQL_EMBEDDED_LIBRARIES_STATIC-NOTFOUND – not found
— PCRE libs: /usr/lib64/libpcre.so
— PCRE include directory: /usr/include
— Embedded mysqld does not have pcre_stack_guard, linking with system pcre.
CMake Error at cmake/FindMySQL.cmake:115 (message):
Library not found: libmysqld. If your install of MySQL is in a non-default
location, please provide the location with -DMYSQL_EMBEDDED_LIBRARIES=<path
to library>
Call Stack (most recent call first):
CMakeLists.txt:37 (find_package)
— Configuring incomplete, errors occurred!

 

As seen in the output above the build was unsuccessful because we could not link against libmysqld. So how do we fix this?

We will build the library from source and retry building Maxscale source and linking against our non default installation of library.

Lets  download the Library and extract it as follows:

1  wget –content-disposition https://downloads.mariadb.org/f/mariadb-                 10.0.22/bintar-linux-glibc_214-x86_64/mariadb-10.0.22-linux-glibc_214-x86_64.tar.gz/from/http%3A/mirror.netinch.com/pub/mariadb/?serve

2  tar -axf mariadb-10.0.22-linux-glibc_214-x86_64.tar.gz

Now with our library downloaded and extracted to our system, lets locate the library location and copy the path to it.

from the terminal, type

locate libmysqld

Output:

[maxscale@localhost mariadb-10.0.22-linux-x86_64]$ locate libmysqld
/home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.a
/home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.so
/home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.so.18
/usr/lib64/mysql/libmysqld.so
/usr/lib64/mysql/libmysqld.so.18
[maxscale@localhost mariadb-10.0.22-linux-x86_64]$

As seen on the output, our library is located at  /home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.a

This is the path we need to pass to our build rule so lets retry and build Maxscale.

First, lets clean the old build by

rm CMakeCache.txt   or by deleting the build directory and recreating it.

So
cd build,

cmake ../DMYSQL_EMBEDDED_LIBRARIES=/home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.a

Output:
— CMake version: 2.8.11
— The C compiler identification is GNU 4.8.5
— The CXX compiler identification is GNU 4.8.5
— Check for working C compiler: /usr/bin/cc
— Check for working C compiler: /usr/bin/cc — works
— Detecting C compiler ABI info
— Detecting C compiler ABI info – done
— Check for working CXX compiler: /usr/bin/c++
— Check for working CXX compiler: /usr/bin/c++ — works
— Detecting CXX compiler ABI info
— Detecting CXX compiler ABI info – done
— Looking for include file arpa/inet.h

— Looking for include file sys/time.h – found
— Looking for include file sys/types.h
— Looking for include file sys/types.h – found
— Looking for include file sys/un.h
— Looking for include file sys/un.h – found
— Looking for include file time.h
— Looking for include file time.h – found
— Looking for include file unistd.h
— Looking for include file unistd.h – found
— RabbitMQ library not found.
— Found OpenSSL: /usr/lib64/libssl.so;/usr/lib64/libcrypto.so (found version “1.0.1e”)
— Valgrind found: /usr/bin/valgrind
— Found dynamic MySQL client library: /usr/lib64/mysql/libmysqlclient.so
— Static MySQL client library not found.
— Looking for LIBMARIADB
— Looking for LIBMARIADB – not found
— Found mysql_version.h: /usr/include/mysql/mysql_version.h
— MySQL version: 5.5.47
— MySQL provider: MariaDB
— Looking for pcre_stack_guard in /home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.a
— Looking for pcre_stack_guard in /home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.a – found
— Using embedded library: /home/maxscale/Desktop/mariadb-10.0.22-linux-x86_64/lib/libmysqld.a
— Using MySQL headers found at: /usr/include/mysql
— Using errmsg.sys found at: /usr/share/mysql/english/errmsg.sys
— Pandoc not found.
— Found libtcmalloc: /usr/lib64/libtcmalloc.so.4
— Could not find libjemalloc, using system default malloc instead.
— Found Git: /usr/bin/git (found version “1.8.3.1”)
— Found CURL: /usr/lib64/libcurl.so (found version “7.29.0”)
— RabbitMQ library not found.
— PCRE2 libs: /usr/local/lib/libpcre2-8.so
— PCRE2 include directory: /usr/local/include
— Building MariaDB Connector-C from source.
— Found git 1.8.3.1
— Commit ID: 4c025ccba9d90ec3f06e5fe3d0bd5ee685ec2cf5
— C Compiler supports: -Werror=format-security
— C Compiler supports: -Wno-unused-but-set-variable
— Using C99 standard
CMake Warning at server/modules/filter/CMakeLists.txt:9 (message):

— Found FLEX: /usr/bin/flex (found version “2.5.37”)
— Found BISON: /usr/bin/bison (found version “2.7”)
— Building MaxAdmin with editline: /usr/lib64/libedit.so
— Installing maxscale.conf to: /etc/ld.so.conf.d
— Installing startup scripts to: /etc/init.d
— Installing systemd service files to: /usr/lib/systemd/system
— Found Doxygen: /usr/bin/doxygen (found version “1.8.5”)
— Configuring done
— Generating done
— Build files have been written to: /home/maxscale/Desktop/MaxScale/build

 

Now our configure is successful and Maxscale is ready to be compiled.

Now run the following command to compile Maxscale.
make

Output

[ 1%] Performing download step (git clone) for ‘connector-c’
Cloning into ‘connector-c’…
remote: Counting objects: 5195, done.
remote: Total 5195 (delta 0), reused 0 (delta 0), pack-reused 5195
Receiving objects: 100% (5195/5195), 3.84 MiB | 88.00 KiB/s, done.
Resolving deltas: 100% (3797/3797), done.
Note: checking out ‘7fd72dfe3e5b889b974453b69f99c2e6fd4217c6’.

You are in ‘detached HEAD’ state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b new_branch_name

HEAD is now at 7fd72df… Make sure that on windows we include iconv.h from win-iconv, not a system one
[ 1%] No patch step for ‘connector-c’
[ 2%] No update step for ‘connector-c’
[ 3%] Performing configure step for ‘connector-c’
— The C compiler identification is GNU 4.8.5
— Check for working C compiler: /usr/bin/cc
— Check for working C compiler: /usr/bin/cc — works

………………………………………………………………………………….
………………………………………………………………………………….
[ 93%] Built target topfilter
Scanning dependencies of target hintfilter
[ 94%] Building C object server/modules/filter/hint/CMakeFiles/hintfilter.dir/hintfilter.c.o
[ 95%] Building C object server/modules/filter/hint/CMakeFiles/hintfilter.dir/hintparser.c.o
Linking C shared library libhintfilter.so
[ 95%] Built target hintfilter
[ 96%] [FLEX][token] Building scanner with flex 2.5.37
[ 96%] [BISON][ruleparser] Building parser with bison 2.7
Scanning dependencies of target dbfwfilter
[ 97%] Building C object server/modules/filter/dbfwfilter/CMakeFiles/dbfwfilter.dir/dbfwfilter.c.o
[ 97%] [BISON][ruleparser] Building parser with bison 2.7
[ 98%] Building C object server/modules/filter/dbfwfilter/CMakeFiles/dbfwfilter.dir/ruleparser.c.o
[ 98%] Building C object server/modules/filter/dbfwfilter/CMakeFiles/dbfwfilter.dir/token.c.o
Linking C shared library libdbfwfilter.so
[ 98%] Built target dbfwfilter
Scanning dependencies of target maxadmin
[100%] Building C object client/CMakeFiles/maxadmin.dir/maxadmin.c.o
Linking C executable maxadmin
[100%] Built target maxadmin

Now our Maxscale build is successful yay!!!

Lets go ahead and install it,

make install

 

 

Advertisements

2 thoughts on “[Maxscale Filter] Community Bonding

  1. Great to see this MaxScale related blog, congratulations! Please keep an eye on the latest MaxScale documentation, we are constantly trying to improve it, including the information on how to build from source. Also, we have the intention of creating a MaxScale development package, so that it would be possible to build a plugin such as a filter without needing to build the whole of MaxScale. This is obviously intended to make it easier to build modules for MaxScale. Unfortunately, I can’t say when this will be ready – it’s tricky fitting it in to our total workload 🙂

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s