Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Tools for developing MariaDB
Jira is the tool that is used for bug reporting, project planning and tracking in MariaDB development. It replaced the previous tool called WorkLog. Jira is also where you can find the MariaDB roadmap.
The MariaDB Jira is located at jira.mariadb.org
Everyone is welcome to sign up for a free account and submit issues, post new tasks or ideas, or to comment on existing issues. The sign up can be found next to the log in boxes on the front page or directly through this link.
After filling out the sign up form you can log in.
You can easily create links to Jira tasks in the documentation using their "MDEV" number. For example, the text 'mdev-191', when entered into the documentation becomes: .
This page is licensed: CC BY-SA / Gnu FDL
Here is the MariaDB Server (MDEV) Jira project workflow.
OPEN
Status assigned to newly created Jira issues
Indicates that the issue has been reported and awaits further action
Transitions
Needs feedback → NEEDS FEEDBACK
Start progress → IN PROGRESS
Close Issue → CLOSED
CONFIRMED
Status applied to validate the existence of a bug type of issue
Implies that the bug can be consistently reproduced and is scheduled for development
Transitions
Stop Progress → STALLED
NEEDS FEEDBACK
Status utilised when progress is contingent on receiving essential feedback
Feedback requests should be clearly communicated to the designated individual
Issue can be closed after staying in a month in NEED FEEDBACK with no activity
Transitions
IN PROGRESS
Status employed when the assignee is actively working on the issue
Denotes that the issue is being actively addressed and moving forward
Transitions
Stop Progress → STALLED
IN REVIEW
Status used to request a peer review of an issue, typically involving code evaluation
Seeking input from another assignee to ensure quality and adherence to standards
Transitions
Review done → STALLED
IN TESTING
Status employed to initiate testing of completed work, often involving a quality engineer
Ensuring that the implemented solution meets functional and performance requirements
Transitions
Stop Testing → STALLED
STALLED
Status indicating that progress on the issue has been temporarily halted
Distinguished from the Open status by the presence of prior work on the stalled issue
Transitions
Unconfirm issue → OPEN
CLOSED
Status assigned when work on the issue has been finalized
Signifies that the issue has been successfully resolved and no further action is expected
In the event that the issue requires reopening, it will revert to the Stalled status
The resolution is another field that will explain how it has been resolved.
This page is licensed: CC BY-SA / Gnu FDL
Request testing → IN TESTING
Needs feedback → NEEDS FEEDBACK
Close Issue → CLOSED
Request Review → IN REVIEW
Provide feedback → OPEN
No Feedback → CLOSED
From the end of 2023, we are moving the issues “IN PROGRESS”, which didn’t get any time to get meaningful updates from the assignee for 6 weeks in this status, to STALLED.
Request testing → IN TESTING
Needs feedback → NEEDS FEEDBACK
Close Issue → CLOSED
Request Review → IN REVIEW
Request testing → IN TESTING
Close Issue → CLOSED
Close Issue → CLOSED
Request testing → IN TESTING
Needs Feedback → NEEDS FEEDBACK
Start Progress → IN PROGRESS
Close Issue → CLOSED
Request Review → IN REVIEW
Transitions
Reopen Issue → STALLED

This page is licensed: CC BY-SA / Gnu FDL
This category contains articles documenting how the MariaDB Red Hat-based buildbot build and test VMs were created
This section documents some additions that have been made after the initial setup
Tricks and tips on how to use Git, the source control system MariaDB uses
Debian 4 has become so old that the apt repository has been moved out of the main Debian 4 mirror servers, and into the archive of old versions. This needs to be fixed by pointing the Debian 4 images to a different mirror:
64-bit:
kvm -m 512 -hda vm-debian4-amd64-install.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=e1000 -net user -nographic
sudo vi /etc/apt/sources.list
# replace http://ftp.dk.debian.org/debian/ with http://ftp.de.debian.org/archive/debian/32-bit:
kvm -m 512 -hda vm-debian4-i386-install.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=e1000 -net user -nographic
sudo vi /etc/apt/sources.list
# replace http://ftp.dk.debian.org/debian/ with http://ftp.de.debian.org/archive/debian/After that, it is necessary to re-do from scratch the -update and -update2 debian4 images (as these are built on top of the -install images).
This page is licensed: CC BY-SA / Gnu FDL
Requires cmake. Install cmake on all -build VMs (and other Unix-like machines) with:
wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz
tar -zxvf cmake-2.8.8.tar.gz
cd cmake-2.8.8;./configure
make
sudo make installThis page is licensed: CC BY-SA / Gnu FDL
Worklog has been replaced, please refer to for further information.
is the tool used to track all development of .
The MariaDB Worklog is open to everyone at (a free account on is required to suggest new tasks and add comments, votes, donations etc.). The account signup page for the wiki is .
If you find something in the worklog that you really would like to have done, you can commit to donate some money to the developer when this is done. (Search after "Make offer" on the worklog item you would like to sponsor).
If there is something in worklog which you would like to develop, you can contact us at 'maria-developers (at) lists.launchpad.com' or 'community (at) askmonty.org'. If you deliver a working solutions that is , you will get 60 % of the so far committed money. The rest of the money is kept by MariaDB Corporation Ab for help managing the project, code reviews, bug fixes, testing, maintenance, updates and merges to future MariaDB versions.
You can also add a link in the MariaDB worklog to tasks to the . Just refer to the MySQL task as #WL
These build instructions should in general be platform agnostic. It is based on a post by Adam M. Dutko on maria-discuss. See the .
To setup a build slave, the basic outline is:
Install Python (2.4 or later)
Install Twisted (twistedmatrix.com 8.0.x or later) - need Core, Mail, Web and Words (possibly Conch too)
Install ZopeInterface ()
See https://mariadb.org/the-onion-framework-a-new-approach-to-buildbot-configuration/
This page is licensed: CC BY-SA / Gnu FDL
General setup instructions are available in the , specifically in the section on .
In addition to installing BuildBot on the slave host, it is also necessary to install all the tools needed to branch MariaDB from Launchpad and compile it. It is a good idea to first manually branch the code from Launchpad and successfully build it, as otherwise a lot of time may be needed fixing things one at a time as new builds are started and fail in one way or the other.
Unfortunately, bzr is memory hungry, so at least 1 Gigabyte of memory is recommended (you may be able to squeeze through with less, but bzr is a real memory hog). A few Gigabytes of disk space are also needed to hold the build directory.
Here are some detailed instructions for various systems:
The source code for the Worklog application is hosted on Launchpad. The license is GPL.
This page is licensed: CC BY-SA / Gnu FDL
Install all the necessary compiler/build components.
Install Buildbot
Install Git and make sure you can clone from the main project
Get a username/login and password from the buildmaster (dbart on Libera.Chat)
Verify your buildbot can talk to the buildbot master instance and can run builds.
The following links will also help:
This page is licensed: CC BY-SA / Gnu FDL
See the Buildbot TODO for plans and ideas on improving Buildbot.
This page is licensed: CC BY-SA / Gnu FDL
Some additional/alternative libraries needs to be installed to handle readline and PAM correctly.
The newer libreadline is GPLv3 and so not compatible with the MariaDB/MySQL GPLv2 license. The PAM libraries are needed for the PAM plugin.
On the Centos and RHEL -build VMs, install the pam-devel package:
sudo yum install pam-develOn all the Debian/Ubuntu -build virtual machines, install libpam0g-dev:
sudo apt-get install libpam0g-devOn debian6/maverick/natty, install libreadline5-dev (replacing any libreadline6-dev already there):
sudo apt-get install libreadline5-devOn oneiric (and any newer, eg. Debian 7 or Ubuntu 12.04), the package is called libreadline-gplv2-dev:
sudo apt-get install libreadline-gplv2-devThis page is licensed: CC BY-SA / Gnu FDL
cd /kvm/vms
qemu-img create -f qcow2 vm-debian6-i386-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-serial.qcow2 -cdrom /kvm/debian-6a1-i386-netinst.iso -redir 'tcp:2245::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net userFrom base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:
Editing /etc/default/grub:
Add user buildbot, with disabled password. Add as sudo, and add ssh key.
See above for how to obtain my.seed
This page is licensed: CC BY-SA / Gnu FDL
The OQGraph storage engine needs a newer version of Boost that what is available on (most) distributions. The version installed is 1.42.0, available from.
The boost library is installed in all the builder virtual machine images with the following single command:
for i in "vm-hardy-amd64-build qemu64" "vm-hardy-i386-build qemu32,-nx" \
"vm-intrepid-amd64-build qemu64" "vm-intrepid-i386-build qemu32,-nx" \
"vm-karmic-amd64-build qemu64" "vm-karmic-i386-build qemu32,-nx" \
"vm-jaunty-amd64-build qemu64" "vm-jaunty-i386-deb-build qemu32,-nx" \
"vm-lucid-amd64-build qemu64" "vm-lucid-i386-build qemu32,-nx" \
"vm-maverick-amd64-build qemu64" "vm-maverick-i386-build qemu32,-nx" \
"vm-natty-amd64-build qemu64" "vm-natty-i386-build qemu64" \
"vm-oneiric-amd64-build qemu64" "vm-oneiric-i386-build qemu64" \
"vm-debian5-amd64-build qemu64" "vm-debian5-i386-build qemu32,-nx" \
"vm-debian4-amd64-build qemu64 --netdev=e1000" "vm-debian4-i386-build qemu32,-nx --netdev=e1000" \
"vm-centos5-i386-build qemu32,-nx" "vm-centos5-amd64-build qemu64" \
"vm-hardy-amd64-build qemu64" "vm-hardy-i386-build qemu32,-nx" \
"vm-jaunty-i386-deb-tarbake qemu32,-nx" ; do \
set $i; \
runvm -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 $3 $1.qcow2 \
"= scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_42_0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo mkdir -p /usr/local/src /usr/local/include" \
"cd /usr/local/src && sudo tar zxf /dev/shm/boost_1_42_0.tar.gz" \
"cd /usr/local/include && sudo ln -s ../src/boost_1_42_0/boost ." ; \
doneTo upgrade boost on the VMs to 1.49 I performed the following steps:
I initially tried to upgrade the VMs using a script like the one at the top of the page which was used to install boost 1.42, but I ran into issues with getting it to work on all of the VMs (it worked on some, but not on others). So I ended up using the steps below.
Copy the VM (keeping the original as a backup, in case something goes wrong):
Start the VM:
Copy the boost tar.gz file over to the VM:
ssh into the VM:
once inside the VM, perform the following steps:
This page is licensed: CC BY-SA / Gnu FDL
Create the VM:
cd /kvm/vms
qemu-img create -f qcow2 vm-debian4-i386-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-serial.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net userFrom base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:
Add to /boot/grub/menu.lst:
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consolealso add in menu.lst to kernel line:
Do these steps:
See the
article for how to make the 'my.seed' file.
This page is licensed: CC BY-SA / Gnu FDL
cd /kvm
wget http://ftp.klid.dk/ftp/centos/5.3/isos/x86_64/CentOS-5.3-x86_64-bin-DVD.iso
qemu-img create -f qcow2 vms/vm-centos5-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-serial.qcow2 -cdrom CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net userkvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-serial.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographicAdd to /boot/grub/menu.lst:
also add in menu.lst to kernel line (after removing `quiet splash'):
Run these commands:
Install compilers etc:
Download 5.0 rpm for shared-compat:
Install extra dependencies:
This page is licensed: CC BY-SA / Gnu FDL
cd /kvm/vms
qemu-img create -f qcow2 vm-debian4-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-serial.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot d -smp 2 -cpu qemu64 -net nic,model=e1000 -net userFrom base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:
Add to /boot/grub/menu.lst:
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consolealso add in menu.lst to kernel line:
See the
article for how to make the 'my.seed' file.
This page is licensed: CC BY-SA / Gnu FDL
Here are the steps I did when installing and configuring a buildbot slave on a PC-BSD 9 box.
Add buildbot user:
Python was already installed.
Bazaar was already installed.
NTP was already installed.
Install Zope3
Install Twisted
Install ccache
Run a test compile of MariaDB
Install buildbot
Create the buildbot slave
On the build master, add new entry to /etc/buildbot/maria-master-private.cfg
Remember the ${slave-name} and ${password} configured above, they're used in the next step.
Download netinst CD image debian-503-amd64-netinst.iso
From base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:
Add to /boot/grub/menu.lst:
also add in menu.lst to kernel line (after removing `quiet splash'):
Add user buildbot, with disabled password. Add as sudo, and add ssh key.
Commit emails for MariaDB are sent to . You can find the archive .
To allow others to see what you are working on in your MariaDB tree, you should:
to the email list
configure git to send your commits to .
Download the script. Configure as
Also you might want to check the for the post-commit trigger.
The post-commit git trigger uses
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-serial.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
su
apt-get install sudo openssh-server
visudo
# Add: %sudo ALL=NOPASSWD: ALL
# add user account to group sudo `addgroup <USER> sudo`.
# Copy in public ssh key.
# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-serial.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user
su
apt-get install sudo openssh-server
VISUAL=vi visudo
# Add at the end:t %sudo ALL=NOPASSWD: ALL
# Add account <USER> to group sudo
# Copy in public ssh key.
# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100console=tty0 console=ttyS0,115200n8# Add user buildbot, with disabled password. Add as sudo, and add ssh key.
sudo /usr/sbin/adduser --disabled-password buildbot
sudo /usr/sbin/adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .sshserial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consolekvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-serial.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user
su
apt-get install sudo openssh-server
VISUAL=vi visudo
# Add at the end:t %sudo ALL=NOPASSWD: ALL
# Add account <USER> to group sudo
# Copy in public ssh key.
# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100console=tty0 console=ttyS0,115200n8
# Add user buildbot, with disabled password. Add as sudo, and add ssh key.
sudo /usr/sbin/adduser --disabled-password buildbot
sudo /usr/sbin/adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .sshsudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-gruboldvm="vm-debian6-i386-build.qcow2"
newvm="vm-debian6-i386-build.upd.qcow2"
cp -avi ${oldvm} ${newvm}kvm -m 1024 -hda /kvm/vms/${newvm} -redir 'tcp:22775::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographicqemu-img create -b vm-debian4-i386-serial.qcow2 -f qcow2 vm-debian4-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-build.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user -nographic
sudo apt-get build-dep mysql-server-5.0
# Some latex packages fail to install because they complain that the
# source is more than 5 years old! I solved by setting back the clock a
# couple of years temporarily ...
sudo apt-get install devscripts doxygen texlive-latex-base gs lsb-release fakeroot libevent-dev libssl-dev zlib1g-dev libreadline5-devconsole=tty0 console=ttyS0,115200n8cat >>/etc/inittab <<END
# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
ENDqemu-img create -b vm-debian4-amd64-serial.qcow2 -f qcow2 vm-debian4-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-build.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic
sudo apt-get build-dep mysql-server-5.0
# Some latex packages fail to install because they complain that the
# source is more than 5 years old! I solved by setting back the clock a
# couple of years temporarily ...
sudo apt-get install devscripts doxygen texlive-latex-base gs lsb-release fakeroot libevent-dev libssl-dev zlib1g-dev libreadline5-devAlso, the post-commit trigger is just one approach. You can also use git-email on at least Debian and Fedora to send commit emails to the MariaDB commits email list.
This page is licensed: CC BY-SA / Gnu FDL
git config --global hooks.postcommitrecipients "commits@mariadb.org"
git config --global hooks.postcommitbranches "*"Add login prompt on serial console:
Create account.
Install compilers etc:
Download 5.0 rpm for shared-compat:
Install extra dependencies:
This page is licensed: CC BY-SA / Gnu FDL
cd /kvm
qemu-img create -f qcow2 vms/vm-centos5-i386-base.qcow2 8G
# ISO (dvd) install:
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-base.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net userconsole=tty0 console=ttyS0,115200n8Back on bsd9
Start the buildslave
Make the archive dir
Install Apache
Configure apache:
Install md5sum
This page is licensed: CC BY-SA / Gnu FDL
sudo adduser
buildbot
/bin/shcd /usr/ports/www/zope3
sudo make install clean
# accepted default options
cd /usr/ports/devel/py-zope.interface
sudo make install cleancd /usr/ports/devel/py-twisted
sudo make install clean
# accepted default optionsSee the General Principles
article for how to make the 'my.seed' file.
This page is licensed: CC BY-SA / Gnu FDL
cd /kvm/vms
qemu-img create -f qcow2 vm-debian6-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-serial.qcow2 -cdrom /kvm/debian-6a1-amd64-netinst.iso -redir 'tcp:2244::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net userSee the General
Principles article for how to make the 'my.seed' file.
This page is licensed: CC BY-SA / Gnu FDL
cd /kvm/vms
qemu-img create -f qcow2 vm-debian5-amd64-serial.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-serial.qcow2 -cdrom /kvm/debian-503-amd64-netinst.iso -redir 'tcp:2234::22' -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net userCreate /etc/event.d/ttyS0:
Add account and allow passwordless sudo:
This page is licensed: CC BY-SA / Gnu FDL
qemu-img create -f qcow2 vm-jaunty-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-jaunty-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net userkvm -m 1024 -hda vm-jaunty-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
sudo apt-get install emacs22-noxsudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .sshqemu-img create -b vm-debian6-i386-serial.qcow2 -f qcow2 vm-debian6-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-build.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo apt-get build-dep mysql-server-5.1
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline-devqemu-img create -b vm-debian6-i386-serial.qcow2 -f qcow2 vm-debian6-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-install.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic# No packages mostly!
sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
sudo debconf-set-selections /tmp/my.seedqemu-img create -b vm-debian6-i386-install.qcow2 -f qcow2 vm-debian6-i386-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-i386-upgrade.qcow2 -redir 'tcp:2245::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo apt-get install mysql-server-5.1
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"scp -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -P 22775 /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/ssh -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 22775 buildbot@localhostcd /usr/local/src
sudo tar zxf /dev/shm/boost_1_49_0.tar.gz
cd /usr/local/include/
sudo rm boost
sudo ln -s ../src/boost_1_49_0/boost .
sudo /sbin/shutdown -h nowqemu-img create -b vm-debian4-i386-serial.qcow2 -f qcow2 vm-debian4-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-install.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=e1000 -net user -nographic# No packages mostly!
sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
sudo debconf-set-selections /tmp/my.seedqemu-img create -b vm-debian4-i386-install.qcow2 -f qcow2 vm-debian4-i386-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-i386-upgrade.qcow2 -cdrom /kvm/debian-40r8-i386-netinst.iso -redir 'tcp:2241::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic
sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"useradd buildbot
# Password is disabled by default in Centos5.
usermod -a -G wheel buildbot
visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requiretty
# Put in public ssh key for own account and host buildbot account.
# Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.
su - buildbot
mkdir .ssh
chmod go-rwx .ssh
cat >.ssh/authorized_keys
chmod go-rwx .ssh/authorized_keysqemu-img create -b vm-centos5-amd64-serial.qcow2 -f qcow2 vm-centos5-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-build.qcow2 -cdrom /kvm/CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographicsudo yum groupinstall "Development Tools"
sudo yum install gperf readline-devel ncurses-devel libaio-devel openssl-devel zlib-devel perl perl\(DBI\)sudo mkdir -p /srv/shared/yum/CentOS/5/x86_64/RPMS/
cd /srv/shared/yum/CentOS/5/x86_64/RPMS/
sudo wget http://mirror.ourdelta.org/yum/CentOS/5/x86_64/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.x86_64.rpmqemu-img create -b vm-centos5-amd64-serial.qcow2 -f qcow2 vm-centos5-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-amd64-install.qcow2 -cdrom /kvm/CentOS-5.3-x86_64-bin-DVD.iso -redir 'tcp:2237::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographicsudo yum install perl perl\(DBI\)qemu-img create -b vm-debian4-amd64-serial.qcow2 -f qcow2 vm-debian4-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-install.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic# No packages mostly!
sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
sudo debconf-set-selections /tmp/my.seedqemu-img create -b vm-debian4-amd64-install.qcow2 -f qcow2 vm-debian4-amd64-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian4-amd64-upgrade.qcow2 -cdrom /kvm/debian-40r8-amd64-netinst.iso -redir 'tcp:2240::22' -boot c -smp 2 -cpu qemu64 -net nic,model=e1000 -net user -nographic
sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"(cd vms && qemu-img create -b vm-centos5-i386-base.qcow2 -f qcow2 vm-centos5-i386-serial.qcow2)
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-serial.qcow2 -cdrom CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographicserial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consolecat >>/etc/inittab <<END
# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
ENDuseradd buildbot
# Password is disabled by default in Centos5.
usermod -a -G wheel buildbot
visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requiretty
# Put in public ssh key for own account and host buildbot account.
# Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.
su - buildbot
mkdir .ssh
chmod go-rwx .ssh
cat >.ssh/authorized_keys
# Paste the id_dsa.pub key, see above.
chmod go-rwx .ssh/authorized_keysqemu-img create -b vm-centos5-i386-serial.qcow2 -f qcow2 vm-centos5-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-build.qcow2 -cdrom /kvm/CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographicsudo yum groupinstall "Development Tools"
sudo yum install gperf readline-devel ncurses-devel zlib-devel libaio-devel openssl-devel perl perl\(DBI\)sudo mkdir -p /srv/shared/yum/CentOS/5/i386/RPMS/
cd /srv/shared/yum/CentOS/5/i386/RPMS/
sudo wget http://mirror.ourdelta.org/yum/CentOS/5/i386/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.i386.rpmqemu-img create -b vm-centos5-i386-serial.qcow2 -f qcow2 vm-centos5-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-centos5-i386-install.qcow2 -cdrom /kvm/CentOS-5.3-i386-bin-DVD.iso -redir 'tcp:2225::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographicsudo yum install perl perl\(DBI\)cd /usr/ports/devel/ccache
sudo make install clean
# accepted default optionscd
cd src/maria/build
BUILD/compile-pentium64-max
# test compile appeared to workcd /usr/ports/devel/buildbot
sudo make install clean
# accepted default optionsslave-name=bsd9sudo su - buildbot
buildslave create-slave --usepty=0 /home/buildbot/maria-slave \
hasky.askmonty.org:9989 ${slave-name} ${password}
echo '${contact-email-address}' > /home/buildbot/maria-slave/info/admin
echo 'A host running PC-BSD 9.' > /home/buildbot/maria-slave/info/host
bzr init-repo maria-slave/bsd9
exitsudo su - buildbot
buildslave start maria-slavesudo su - buildbot
mkdir archive
exit
sudo ln -s /home/buildbot/archive /archivecd /usr/ports/www/apache22
sudo make install clean
# accepted default optionssudo su -s
echo 'apache22_enable="YES"' >> /etc/rc.conf
echo 'alias /archive "/archive"\
<Directory "/archive">\
Options All Multiviews\
AllowOverride All\
Order allow,deny\
Allow from all\
</Directory>' >> /usr/local/etc/apache22/httpd.conf
sudo /usr/local/etc/rc.d/apache22 startmd5sum already installed at /compat/linux/usr/bin/md5sum
edited /home/buildbot/.profile and added that dir to the path
# That didn't work, so did the following:
cd /usr/local/bin/
sudo ln -s /compat/linux/usr/bin/md5sum md5sumkvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-serial.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
su
apt-get install sudo openssh-server
visudo
# Add: %sudo ALL=NOPASSWD: ALL
# add user account to group sudo `addgroup <USER> sudo`.
# Copy in public ssh key.
# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grubsudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .sshqemu-img create -b vm-debian6-amd64-serial.qcow2 -f qcow2 vm-debian6-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-build.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo apt-get build-dep mysql-server-5.1
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline-devqemu-img create -b vm-debian6-amd64-serial.qcow2 -f qcow2 vm-debian6-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-install.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic# No packages mostly!
sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
sudo debconf-set-selections /tmp/my.seedqemu-img create -b vm-debian6-amd64-install.qcow2 -f qcow2 vm-debian6-amd64-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian6-amd64-upgrade.qcow2 -redir 'tcp:2244::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo apt-get install mysql-server-5.1
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-serial.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
su
apt-get install sudo openssh-server
visudo
# uncomment %sudo ALL=NOPASSWD: ALL
# add user account to group sudo `addgroup <USER> sudo`.
# Copy in public ssh key.
# Add in /etc/inittab:
S0:2345:respawn:/sbin/getty -L ttyS0 19200 vt100serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consoleconsole=tty0 console=ttyS0,115200n8sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Add all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .sshqemu-img create -b vm-debian5-amd64-serial.qcow2 -f qcow2 vm-debian5-amd64-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-build.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo apt-get build-dep mysql-server-5.0
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-devqemu-img create -b vm-debian5-amd64-serial.qcow2 -f qcow2 vm-debian5-amd64-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-install.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
# No packages mostly!
sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
sudo debconf-set-selections /tmp/my.seedqemu-img create -b vm-debian5-amd64-install.qcow2 -f qcow2 vm-debian5-amd64-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-amd64-upgrade.qcow2 -redir 'tcp:2234::22' -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consoleconsole=tty0 console=ttyS0,115200n8# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.
start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5
stop on runlevel 0
stop on runlevel 1
stop on runlevel 6
respawn
exec /sbin/getty 115200 ttyS0sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo EDITOR=emacs visudo
# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.
sudo su -s /bin/bash - buildbot
mkdir .ssh
cat >.ssh/authorized_keys
# Paste public key for buildbot user on host system.
chmod -R go-rwx .ssh
# Do manual login from host to guest once, to make sure .ssh/known_hosts is updated.qemu-img create -b vm-jaunty-amd64-serial.qcow2 -f qcow2 vm-jaunty-amd64-build.qcow2
kvm -m 1024 -hda vm-jaunty-amd64-build.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo apt-get build-dep mysql-server-5.1qemu-img create -b vm-jaunty-amd64-serial.qcow2 -f qcow2 vm-jaunty-amd64-install.qcow2
kvm -m 1024 -hda vm-jaunty-amd64-install.qcow2 -cdrom /kvm/ubuntu-9.04-server-amd64.iso -redir tcp:2227::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo adduser --system --group mysqlFor Ubuntu and Debian, a quick way to install much of what you need is:
If you're running a version of Debian or Ubuntu that doesn't have MariaDB, then do the following:
After running one (or both) of the above, run the following to catch things that they may have missed:
After setting up the build environment do a test build to confirm that things are working. First get the source code using the git instructions on the page, then follow the steps on the Generic Build Instructions page for building MariaDB using cmake. If your build succeeds, you're ready to move on to the next step of installing and configuring buildbot.
Do not hesitate to ask for help on the maria-developers mailing list or on IRC.
The easiest way to install buildbot on Ubuntu and Debian is to install the buildbot-slave package, like so:
Another way to install buildbot is using the Python pip package manager. Pip can be installed with:
Next install twisted and the buildbot-slave package using pip:
After the buildbot-slave package is installed (either via apt or pip), you need to create the builder using the buildslave create-slave command. As part of this command you will need to specify a name for your buildslave and a password. Both need to be given to the MariaDB Buildbot maintainers so that they can add your builder to the build pool. Ask on the maria-developers mailing list or on IRC for who these people are.
An example command for creating the slave is:
If you installed buildbot using pip, the convention is to create a buildbot user and then, as that user, create the buildslave in the home directory like so:
Put some appropriate info in info/admin and info/host files that are created, this will display on the information screen about your builder. See here for example: bb01
Submit your builder information to the MariaDB Buildbot admins. Also let them know if your machine can run multiple builds at the same time (and how many). After adding your builder's information to the main buildbot configuration, all that's left is for you to do is to start your builder.
If you installed your builder using apt, then you can start and stop it with:
If you installed your buildslave using pip, then do the following as the buildbot user in their home directory:
This page is licensed: CC BY-SA / Gnu FDL
Some KVM images end up not having enough space on them. In such cases, it is preferable to increase the size of the VM rather than to just delete an image and rebuild it from scratch. The steps outlined below document what was done to increase the size of the Red Hat 5 x86 VM and should be able to be easily adapted to other VMs, should they need the same treatment in the future.
Make a copy of the VM to work on (we don't want to change the original):
Using the qemu-img command, resize the image:
Not all versions of qemu-img can resize VMs.
Boot the newly resized image with gparted:
Connect to the VM using VNC from your local machine:
Midway through booting you'll have to reconnect
Use gparted to either expand the existing primary partition or, especially on VMs with LVM, add a new partition (since GParted can't change LVM partitions). Exit when finished and shutdown the VM.
Boot the VM again, this time without a VNC server:
login to the VM:
if expanded an existing partition: verify the new size
else if you created a new partition:
mount the new partition to tmp location
rsync contents of /home/ to the new partition
edit fstab to mount new partition to /home
mv /home to /home-old, create /home dir, mount /home
(optional) unmount /mnt
reboot and verify that things look good
if things do look good (new drive mounted OK, accounts work, etc...), delete /home-old
Move the old VM to -old and the -new VM to what the old VM used to be named
on other VM hosts, make a copy of the old file then rsync over the updated files (the copy helps speed up the rsync):
Test the new VM with a build to make sure it works
This page is licensed: CC BY-SA / Gnu FDL
qemu-img create -f qcow2 vm-lucid-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-lucid-amd64-serial.qcow2 -cdrom /kvm/lucid-server-amd64.iso -redir tcp:2238::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-lucid-amd64-serial.qcow2 -cdrom /kvm/lucid-server-amd64.iso -redir tcp:2238::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2238 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2238 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2238 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2238 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2238 ttyS0.conf buildbot@localhost:
ssh -p 2238 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'Enabling passwordless sudo:
Editing /boot/grub/menu.lst:
See above for how to obtain my.seed and sources.append.
This page is licensed: CC BY-SA / Gnu FDL
Download netinst CD image debian-503-i386-netinst.iso and install:
cd /kvm
qemu-img create -f qcow2 vms/vm-debian5-i386-base.qcow2 8G
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-base.qcow2 -cdrom /kvm/debian-503-i386-netinst.iso -redir 'tcp:2226::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net userFrom base install, setup for serial port, and setup accounts for passwordless ssh login and sudo:
Add to /boot/grub/menu.lst:
also add in menu.lst to kernel line (after removing `quiet splash'):
Add user buildbot, with disabled password. Add as sudo, and add ssh key.
See the article how to obtain the my.seed file.
This page is licensed: CC BY-SA / Gnu FDL
This vm is used to build source tarball and for Ubuntu 9.04 32-bit .deb.
First create and install image:
qemu-img create -f qcow2 vm-jaunty-i386-base.qcow2 8G
kvm -m 2047 -hda vm-jaunty-i386-base.qcow2 -cdrom ubuntu-9.04-server-i386.iso -redir tcp:2222::22 -boot d -cpu qemu32,-nx
# InstallCreate a vm, based on the first one, which is configured for serial port.
qemu-img create -b vm-jaunty-i386-base.qcow2 -f qcow2 vm-jaunty-i386-serial.qcow2
kvm -m 2047 -hda vm-jaunty-i386-base.qcow2 -redir tcp:2222::22 -boot c -cpu qemu32,-nxTo configure kernel and grub for serial console, add the following to /boot/grub/menu.lst:
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consolealso add in menu.lst to kernel line (after removing `quiet splash'):
console=tty0 console=ttyS0,115200n8Create /etc/event.d/ttyS0:
Add an account:
Copy public ssh key into /.ssh/authorized_keys
Enable passwordless sudo:
Create a new vm for building source tarballs:
Install compilers:
Copy in an existing bzr shared repository to buildbot/.bzr (or run bzr init-repo and bzr branch --no-tree lp:maria).
Create a new vm for building .debs:
Install compilers:
Create a new VM for testing .deb installs:
Install tools and local apt repository.
Setup default package config for debconf to enable unattended install. Copy in my.seed (see above) to vm.
Create a new VM for testing upgrades:
Prepare initial MySQL install with some test data.
This page is licensed: CC BY-SA / Gnu FDL
Ability on the web pages to mark certain builds of high priority, so they will be processed before others as build slaves become available. It must be possible to mark pending builds so not only individually, but also by branch, as well as by (branch,revision). The feature should probably interact in a meaningful way with the existing "nextBuild" ability to prioritise builds (). Eg. it could set appropriate properties somewhere that would be available to the nextBuild function.
Install buildbot-slave using or buildbot from (if you use old version of buildbot/buildbot-slave you should use buildbot command instead of buildslave-2.6 in following instructions).
Add user buildbot:
Make sure that you do not have a buildbot user and group on your system
If you do not have the group and user buildbot, then create the group and the user buildbot
To hide the user use:
Prepare environment:
The following steps were used to create a Red Hat 6 x86 buildslave.
When the VM boots. Go through the prompts.
Re-initialize the drive, when prompted.
Set the Hostname to "rhel6-x86".
Configure Network, set eth0 to "Connect Automatically"
Set the root password.
Set partitioning type to: Use All Space
sudo apt-get build-dep mariadb-serversudo apt-get build-dep mysql-serversudo apt-get install devscripts fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.11 dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev bzr scons check libboost-all-dev openssl epm libjudy-dev libjemalloc-dev libcrack2-dev git libkrb5-dev libcurl4-openssl-dev thrift-compiler libsystemd-dev dh-systemd libssl1.0.2 openjdk-8-jdk uuid-dev libnuma-dev gdb libarchive-dev libasio-dev dh-execsudo apt-get install buildbot-slavesudo apt-get install python-pipsudo pip install twisted==11.0.0
sudo pip install buildbot-slave==0.8.9sudo buildslave create-slave /var/lib/buildbot/slaves/maria buildbot.askmonty.org slavename passwordsudo buildslave create-slave ~/maria-slave buildbot.askmonty.org slavename passwordsudo /etc/init.d/buildslave start
sudo /etc/init.d/buildslave stopbuildslave start maria-slave
buildslave stop maria-slavecp -avi vm-rhel5-x86-build.qcow2 vm-rhel5-x86-build-new.qcow2qemu-img info vm-rhel5-x86-build-new.qcow2
qemu-img resize vm-rhel5-x86-build-new.qcow2 +10G
qemu-img info vm-rhel5-x86-build-new.qcow2
rsync -avP vm-rhel5-x86-build-new.qcow2 terrier:/kvm/vms/qemu-img create -f qcow2 vm-lucid-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-lucid-i386-serial.qcow2 -cdrom /kvm/lucid-server-i386.iso -redir tcp:2239::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-lucid-i386-serial.qcow2 -cdrom /kvm/lucid-server-i386.iso -redir tcp:2239::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2239 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2239 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2239 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2239 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2239 ttyS0.conf buildbot@localhost:
ssh -p 2239 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'sudo VISUAL=vi visudo
# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grubfor i in 'vm-lucid-amd64-serial.qcow2 2238 qemu64' 'vm-lucid-i386-serial.qcow2 2239 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
done# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.
start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5
stop on runlevel 0
stop on runlevel 1
stop on runlevel 6
respawn
exec /sbin/getty 115200 ttyS0sudo adduser --disabled-password buildbotsudo adduser buildbot sudo
# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.qemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-tarbake.qcow2sudo apt-get build-dep mysql-5.1-server
sudo apt-get install devscripts hardening-wrapper bzrcat >sources.append <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
for i in 'vm-lucid-amd64-serial.qcow2 2238 qemu64' 'vm-lucid-i386-serial.qcow2 2239 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
"= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
doneserial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consoleqemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-build.qcow2sudo apt-get build-dep mysql-5.1-server
sudo apt-get install devscripts hardening-wrapperqemu-img create -b vm-jaunty-i386-serial.qcow2 -f qcow2 vm-jaunty-i386-deb-install.qcow2sudo debconf-set-selections /tmp/my.seedBuildbot can be started/stopped manually with these commands (it's a good idea to start and stop it to see if it is set up correctly):
If you installed buildbot from fink, please make sure that the buildbot user is using the environment settings. Your .profile should contain follwing line:
In order to make buildbot start on system boot, you'll need to create /Library/LaunchDaemons/net.sourceforge.buildbot.plist file with the following contents (modified example from buildbot wiki):
If you installed buildbot from fink, then you can edit and copy the plist file
Your plist file should similar to this one after editing:
Note: you have to start your buildslave via launchd, otherwise you will run into several problems. For further details, please refer to Using Launchd
This page is licensed: CC BY-SA / Gnu FDL
# Check for group with id 101
id -g -nr 101
# Check for user with id 101
id -u -nr 101Select "Write Changes to Disk" on the popup that appears.
Set the software set to "Virtual Host".
After clicking next on the install-type page, the installation will finally start.
After installation completes, click reboot. Then shutdown the VM.
Login as root.
Add to /boot/grub/menu.lst:
also add in menu.lst to kernel line (after removing 'quiet'):
Add login prompt on serial console:
With the network up and running, it's time to add a user so that we don't have to login as root all the time.
Now logout and then ssh to the VM as the buildbot user. On my local box I added the following to my /.ssh/config file to make logging in easier:
With the above in place I can simply type:
...to connect to the vm.
Register the system with RHN:
Choose defaults when registering. After the process is complete:
The first time you update you'll be prompted to import some GPG keys from Red Hat. The updating process may take a while, depending on the number of updates and the speed of your Internet connection.
After updating shutdown so we can make more copies.
Wait for the system to boot.
Install compilers etc:
If the "Development Tools" group is not available, the following lines will install the packages from it:
Other packages to install:
Create rpm directories and download 5.0 rpm for shared-compat:
Install extra dependencies:
This page is licensed: CC BY-SA / Gnu FDL
vm=vm-rhel5-x86-build-new.qcow2
kvm -m 2048 -hda /kvm/vms/${vm} -cdrom /kvm/iso/gparted-live-0.14.1-6-i486.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:22666-:22vncviewer -via <vmhost> localhostkvm -m 2048 -hda /kvm/vms/${vm} -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:22666-:22 -nographicssh -t -p 22666 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa buildbot@localhostsudo mount /dev/hda3 /mntsudo rsync -avP /home/ /mnt/sudo vi /etc/fstabsudo mv -vi /home /home-old;sudo mkdir -v /home;sudo mount /homesudo umount /mntsudo /sbin/shutdown -h nowsudo mv -vi /kvm/vms/vm-rhel5-x86-build.qcow2 vm-rhel5-x86-build-old.qcow2; sudo mv -vi /kvm/vms/vm-rhel5-x86-build-new.qcow2 /kvm/vms/vm-rhel5-x86-build.qcow2sudo cp -avi /kvm/vms/vm-rhel5-x86-build.qcow2 /kvm/vms/vm-rhel5-x86-build-old.qcow2
sudo rsync -avP terrier.askmonty.org::kvm/vms/vm-rhel5-x86-build* /kvm/vms/for i in 'vm-lucid-amd64-install.qcow2 2238 qemu64' 'vm-lucid-i386-install.qcow2 2239 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
doneqemu-img create -b vm-debian5-i386-base.qcow2 -f qcow2 vm-debian5-i386-serial.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-serial.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
su
apt-get install sudo openssh-server emacs22-nox
visudo
# uncomment %sudo ALL=NOPASSWD: ALL
# add user account to group sudo.
# Copy in public ssh key.
# Add in /etc/inittab:
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100console=tty0 console=ttyS0,115200n8sudo adduser --disabled-password buildbot
sudo adduser buildbot sudo
sudo su - buildbot
mkdir .ssh
# Paste all necessary keys.
cat >.ssh/authorized_keys
chmod -R go-rwx .sshqemu-img create -b vm-debian5-i386-serial.qcow2 -f qcow2 vm-debian5-i386-build.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-build.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
sudo apt-get build-dep mysql-server-5.0
sudo apt-get install devscripts hardening-wrapper doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-devqemu-img create -b vm-debian5-i386-serial.qcow2 -f qcow2 vm-debian5-i386-install.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-install.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
# No packages mostly!
sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
END
sudo debconf-set-selections /tmp/my.seedqemu-img create -b vm-debian5-i386-install.qcow2 -f qcow2 vm-debian5-i386-upgrade.qcow2
kvm -m 2047 -hda /kvm/vms/vm-debian5-i386-upgrade.qcow2 -redir 'tcp:2226::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
sudo apt-get install mysql-server-5.0
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"sudo apt-get install debconf-utils
cat >>/etc/apt/sources.list <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
ENDqemu-img create -b vm-jaunty-i386-deb-install.qcow2 -f qcow2 vm-jaunty-i386-deb-upgrade.qcow2sudo apt-get install mysql-server-5.1
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"# Create group buildbot with group id 101
GROUP="buildbot"
dscl . create /groups/$GROUP
dscl . create /groups/$GROUP name $GROUP
dscl . create /groups/$GROUP passwd "*"
dscl . create /groups/$GROUP gid 101
# Create user buildbot with id 101
BUILDSLAVE_HOME=/var/lib/buildslave
USER=buildbot
mkdir -p $BUILDSLAVE_HOME
dscl . -create /Users/$USER
dscl . -create /Users/$USER RealName "Buildbot slave"
dscl . -create /Users/$USER NFSHomeDirectory $BUILDSLAVE_HOME
dscl . -create /Users/$USER UserShell /bin/bash
dscl . -create /Users/$USER UniqueID 101
dscl . -create /Users/$USER PrimaryGroupID 101
chown 101:101 $BUILDSLAVE_HOMEdefaults write /Library/Preferences/com.apple.loginwindow HiddenUsersList -array-add buildbotsudo -i -u buildbot
buildslave-2.6 create-slave --usepty=0 maria-slave hasky.askmonty.org:9989 <slavename> <passwd>
bzr init-repo maria-slave/<slavedirectory>
$EDITOR maria-slave/info/admin
$EDITOR maria-slave/info/host
logoutsudo - buildbot
more .profile
test -r /sw/bin/init.sh && . /sw/bin/init.shsudo -i -u buildbot
buildslave-2.6 start maria-slave
buildslave-2.6 stop maria-slave
logout<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>StandardOutPath</key>
<string>twistd.log</string>
<key>StandardErrorPath</key>
<string>twistd-err.log</string>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>/opt/local/bin:/sbin:/usr/sbin:/bin:/usr/bin</string>
<key>PYTHONPATH</key>
<string>/opt/local/lib/python2.5/site-packages</string>
</dict>
<key>GroupName</key>
<string>daemon</string>
<key>KeepAlive</key>
<dict>
<key>SuccessfulExit</key>
<false/>
</dict>
<key>Label</key>
<string>net.sourceforge.buildbot.slave.test</string>
<key>ProgramArguments</key>
<array>
<string>/opt/local/bin/buildslave-2.6</string>
<string>start</string>
<string>--nodaemon</string>
<string>maria-slave</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>UserName</key>
<string>buildbot</string>
<key>WorkingDirectory</key>
<string>/var/lib/buildslave/</string>
</dict>
</plist>$EDITOR /sw/share/doc/buildbot-py26/contrib/os-x/net.sourceforge.buildbot.slave.plist<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd
">
<plist version="1.0">
<dict>
<key>Label</key>
<string>net.sourceforge.buildbot.slave</string>
<!-- Change this to the user you want to run buildbot as -->
<key>UserName</key>
<string>buildbot</string>
<!-- Change this to your buildbot working directory -->
<key>WorkingDirectory</key>
<string>/Volumes/MiniHD2/ServiceData/buildslave/maria-slave</string>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>/sw/bin:/sw/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin</string>
</dict>
<key>ProgramArguments</key>
<array>
<string>/sw/bin/twistd</string>
<string>--nodaemon</string>
<string>--python=buildbot.tac</string>
<string>--logfile=buildbot.log</string>
<string>--prefix=slave</string>
</array>
<key>KeepAlive</key>
<dict>
<key>SuccessfulExit</key>
<false/>
</dict>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>sudo cp /sw/share/doc/buildbot-py26/contrib/os-x/net.sourceforge.buildbot.slave.plist /Library/LaunchDaemons/cd vms
qemu-img create -f qcow2 vm-rhel6-x86-base.qcow2 8G
kvm -m 1024 -hda vm-rhel6-x86-base.qcow2 -cdrom ../iso/red-hat/rhel-server-6.0-i386-dvd.iso -redir 'tcp:22275::22' -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net usercd vms
qemu-img create -b vm-rhel6-x86-base.qcow2 -f qcow2 vm-rhel6-x86-serial.qcow2
kvm -m 1024 -hda vm-rhel6-x86-serial.qcow2 -redir 'tcp:22275::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net userserial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consoleconsole=tty0 console=ttyS0,115200n8cat >>/etc/inittab <<END
# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
ENDuseradd buildbot
# Password is disabled by default in Centos5.
usermod -a -G wheel buildbot
visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requiretty
# Put in public ssh key for own account and host buildbot account.
# Note that Centos5 seems to require .ssh/authorized_keys chmod go-rwx.
su - buildbot
mkdir .ssh
chmod go-rwx .ssh
# scp the key from the vm host over to .ssh/authorized_keys
chmod go-rwx .ssh/authorized_keysHost rhel6-x86
HostName localhost
User buildbot
IdentityFile ~/.ssh/id_rsa
Port 2225ssh rhel6-x86sudo rhn_registersudo yum updatesudo shutdown -h nowqemu-img create -b vm-rhel6-x86-serial.qcow2 -f qcow2 vm-rhel6-x86-build.qcow2
kvm -m 1024 -hda vm-rhel6-x86-build.qcow2 -redir 'tcp:22275::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographicsudo yum groupinstall "Development Tools"
sudo yum install libaio-devel openssl-devel# Mandatory Packages:
sudo yum install autoconf automake binutils bison flex gcc gcc-c++ gettext
sudo yum install libtool make patch pkgconfig redhat-rpm-config rpm-build
# Default Packages:
sudo yum install byacc cscope ctags cvs diffstat doxygen elfutils gcc-gfortran
sudo yum install git indent intltool patchutils rcs subversion swig systemtap
# Optional Packages:
sudo yum install ElectricFence ant babel bzr chrpath cmake compat-gcc-34
sudo yum install compat-gcc-34-c++ compat-gcc-34-g77 dejagnu expect gcc-gnat
sudo yum install gcc-java gcc-objc gcc-objc++ imake jpackage-utils libstdc++-docs
sudo yum install mercurial mod_dav_svn nasm perltidy python-docs rpmdevtools
sudo yum install rpmlint systemtap-sdt-devel systemtap-serversudo yum install gperf readline-devel ncurses-devel zlib-devel perl perl\(DBI\)sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS
sudo mkdir -vp /srv/shared/yum/CentOS/5/i386/RPMS/
cd /srv/shared/yum/CentOS/5/i386/RPMS/
sudo wget http://mirror.ourdelta.org/yum/CentOS/5/i386/RPMS/MySQL-OurDelta-shared-5.0.87.d10-65.el5.i386.rpmqemu-img create -b vm-rhel6-x86-serial.qcow2 -f qcow2 vm-rhel6-x86-install.qcow2
kvm -m 1024 -hda vm-rhel6-x86-install.qcow2 -redir 'tcp:22275::22' -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographicsudo yum install perl perl\(DBI\)We need a way to kill all running builds, and remove all pending builds, for a given combination of branch and revision. It is possible to manually cancel a pending or running build, but this becomes extremely inconvenient when there are many builders.
Comments from buildbot mailing list: "There is already a button (hidden on the change page) for stopping all builds associated to a change. Unfortunately, the only way to get to this page seems to be through the waterfall." "Note that currently, the button on the change page only cancels still-pending builds, it does not stop builds in progress."
Fix problem when pushing multiple bzr revisions at one time:
Buildbot should wait a few moments before starting a build, and then start building the latest (currently it seems to build the first revision immedeately, and even often seems to not build the latest revision at all, getting confused about the order)
When pushing a change that changes revision numbers, the grid display can get really confused, would be nice to fix this somehow.
Any failure other than compile/test failure should mean a re-run. Failures like buildslave reboot or loss of connection or timeout of bazaar operation should be interpreted as if the build has never happened, and the system should re-try building.
In the bzr source step, retry the checkout a couple of times if it fails (to be more robust against temporary Launchpad problems, which we see a number of). Could maybe be done as a general Buildbot optional feature for all revision control systems supported, or maybe specific for the bzr depending on what can work.
Get PBXT test suite fixed in --valgrind; once that is done
include it in the list of main suites so that it is tested on all platforms
(and remove the special extra pbxt test steps in some Buildbot builds, as
they will no longer be needed).
Load statistics. We need it to be able to see if adding another tree will starve the buildbot:
Percentage of busy time for available slaves
Median/Max response time (How much time do I have to wait after I've pushed for test run to complete)
Fix that mysql-test-run uses --skip-ssl by default.
Fix that mysql-test-run.pl --mem doesn't properly clean
up /dev/shm/var* (so that we can use --mem
and --parallel option to speed up the more powerful
machines).
For triggered builds (ie. one builder makes a source tarball which another builder consumes), make it so that the pushed changesets are also transfered, so the blamelist on the build page works (currently the blame list is empty).
Upgrade to Buildbot 0.8 on the master. Then use the facility to use a MariaDB server as backend for both speedup (hopefully) and more flexible querying of buildbot history.
It should be easier to setup for a new user. Ideally, there should be one tarball that one can download, unpack and run. Within the tarball there should be a script and/or config files which allow to add the buildslave into automatic startup. The following should not be there:
Dependence of buildslave on a load of python libraries. Ship them.
Same for bzr
Same for mysqld itself. Dl/make/make-install everything we need in the buildbot directory.
If the above is not done, we'll never get much participation from old or unusual machines. The owners of such machines do not have the latest libraries, installing them requires some manual effort, which simply sets the bar too high.
Check archivist observation/bug that build slave uname info disappears when
slave is disconnected.
Self refresh at a time interval (ajax maybe to reduce transfer) so users can know when to fire up a slave.
In the waterfall page, second line from top is "current activity". When displaying a particular branch ("?branch=5.1"), the "current activity" for hosts that are currently building another branch could also display the name of the branch (as a link to waterfall page for that branch) to avoid confusion that it is active on the current branch.
Run with --valgrind-mysqld instead
of --valgrind. Little point in Valgrinding mysqltest when we
in any case ignore any errors in that program.
Show pending builds for a slave on the 'builders' page, eg.
Show dates in local time of the client. One way is with Javascript: (would be nice if some dates were still shown when javascript is not available).
Check if InnoDB is fixed sufficiently to have no Valgrind leaks
with innodb_use_sys_malloc; if so remove hack
in mysql-test-run.pl to disable this in valgrind case.
Fix problem that illegal regexp causes build exception (example:). The problem is that Buildbot grabs regexps from the suppression file in the MariaDB tree and tries to compile them; if this throws exception, then the exception should be caught (and possibly warned about), not crash the whole build.
Add apt-get source mariadb-server checking in buildbot (for package
testing).
Add testing of bintar package with real start of server and also test on different distro/version than what was built (eg hardy<->jaunty).
In package build, sign .debs with dummy key, to more accurately test the real build process.
Switch to build the bintar packages on centos 5, to work with older glibc versions? Or maybe even Debian 4 which is older I think? But do the install test in Buildbot also on other/newer distros (should probably test on several in each build).
Add a test step for .deb that tests upgrading from an earlier version of MariaDB (currently we only check upgrade from MySQL).
Add a 'follow log' feature to runvm (which does ssh guest tail -f log > log). Use this in the Buildbot packaging step that runs mysql-test-run.pl,
so that we can get mysqld.X.err.Y server logs available like for the non-kvm
builds.
For runvm: add the options "-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" to the ssh command used to login to the guest vm, so we do not get a login failure due to different host keys in different guests.
Idea is to set it up so that each developer/group has a stating tree. Any push to this tree will first get a full round of testing in Buildbot. If all results are green after this, it will automatically be pushed to the main tree. If another push gets in first, it will automatically merge the new stuff and re-try a full Buildbot test. If there is a problem (test failure or merge conflict), it will send mail with details.
Get something working, simple initially.
Staging tree per-captain
mysqltest --require not-staging to speed up Valgrind
This page is licensed: CC BY-SA / Gnu FDL
This page describes the database schema used by Buildbot to save results from test runs.
The idea is to be able to use this data from outside of Buildbot for things like additional web pages presenting test results, or search/data mining facilities for searching for test failures.
The plan is to make remote database connections available to community members. For this, we need to set up a slave host replicating the master Buildbot database (which would in any case be good to isolate the running Buildbot from possibly high load from queries).
However, for now the database access is only available locally on the machine (hasky) running the buildbot master.
The most current information about the schema used is available in the file buildbot/process/mtrlogobserver.py in the Buildbot sources. As the code evolves and more kinds of information is made available in the database, the schema might be extended, but the schema description in the source code should always be up-to-date.
test_run tableThis table has one row for every test run that Buildbot does. Thus, each row corresponds to one cell in the [ Waterfall display]. The format of the table is as follows:
id: Primary key, just an auto_increment id.
branch: This is the name of the bzr branch of the test run.
revision: The Bzr revision number tested.
platform: The name of the builder that ran the test.
test_failure tableThis table has one row for every test failure encountered:
test_run_id: This identifies the test run in which the test failure occured (eg. it is a foreign key to id in table test_run).
test_name: The name of the test that failed, eg. main.information_schema.
test_variant: Some tests are run multiple times in different variants. Ie. many replication tests are run under both statement-based, mixed-mode, and row-based replication. The variant will be 'stmt', 'mix', or 'row' accordingly. For tests that do not have multiple variants, the value will be the empty string (ie. not a NULL value).
test_warnings tableThis table holds information about test problems that were detected after a test case ran, during server restart (typically by finding an error or warning message in the server error log files). A typical example of this is a memory leak or a crash during server shutdown.
Such a failure can not be attributed to a specific test case, as it could be caused by any of the tests run against the server since last restart, or could even be a general problem not caused by any test case. Instead, for each occurence, this table provides a list of names of the tests that were run by the server prior to detecting the error or warning.
test_run_id: Identifies the corresponding row in table test_run.
list_id: This is a counter for occurences of warnings within each test run (ie. it starts over from 0 again for each different value of test_run_id).
list_idx: This is a counter for each test name (ie. it starts over from 0 again for each different value of test_run_id ''and'' list_id).
Show all platforms that failed for a particular revision of a particular branch:
Find failures similar to a given failure being investigated:
Check which branches a specific kind of failure has occured in:
Find all test runs where a given test was run against a server that later had warnings in the error log, and also count the number of occurences of this event in each run:
This page is licensed: CC BY-SA / Gnu FDL
CassandraSE is no longer actively being developed and has been removed in MariaDB 10.6. See MDEV-23024.
Here are the steps I took to install Cassandra on the Fulltest VMs.
backed up the fulltest VMs with:
rsync -avP /kvm/vms/*fulltest* host:/destination/path/boot the amd64 fulltest VM:
login to the VM:
in the VM, install Cassandra:
in the VM, launch the cassandra-cli program and test the Cassandra installation:
Output of the above:
in the VM, shut it down:
Do steps 2-6 for vm-precise-i386-fulltest.qcow2. The output of the testing step was:
on the other build hosts, rsync the files over:
This page is licensed: CC BY-SA / Gnu FDL
This part of the Buildbot setup uses KVM virtual machines to build packages on a wide range of Linux distributions and test the different packages in various ways.
The builds run on a single host, which has an Intel Core i7 860 @ 2.80GHz CPU, 8 GByte of RAM, and a 500GByte disk. The build host is installed with Ubuntu 9.04 Server 64-bit. The current box is capable of running up to 3 builds in parallel. The entire set of 19 builds take around 3.5 hours to complete.
The build host has no virtual machines that are kept running when idle. Instead each build boots up and shuts down virtual machines on the fly as needed, using the runvm tool.
The details of the setup (installation) of the virtual machines are here.
(Please check the Buildbot configuration for details, in case this information becomes out of date).
At the time of writing, the builds done are these:
Source tarball build (on a 32-bit Ubuntu 9.04 VM)
Binary tarballs, i386 and amd64, on Ubuntu 8.04 VMs.
Ubuntu, i386 and amd64: 8.04, 8.10, 9.04, 9.10, 10.04 (alpha).
Debian, i386 and amd64, Debian 4 and Debian 5.
The builds use the packaging scripts from .
In all builds (except for source tarball), the binary package is built, and subsequently attempted installed. The build is done on a virtual machine with compilers and other necessary build tools installed. The virtual machines are cloned from a reference image before each build (using the --base-image option to runvm). This way, the original images are not modified in any way, so there is no risk of "pollution" from previous builds, and no need to carefully clean up after a build or test.
The source tarball build is done in a virtual machine that is not reset after every build. This is to preserve the bzr shared repository setup inside the virtual machine. This way it is only necessary in each build to download new changes from Launchpad, which saves significant amounts of time (and Launchpad bandwidth). Since this particular build only works inside the source checkout directory, the risk of cross-build pollution is minimal.
The source tarball build is the only one that is triggered by bzr commits. After it has successfully built a source tarball, the tarball is uploaded to the Buildbot master, and all of the other builders are triggered to start building. Each will download the source tarball from the master and build from that rather than from a bzr checkout (this is the correct way, as we want to ensure that users building from the source tarball themselves get the same results as the packages built by us).
Since we use the OurDelta scripts, which use a "bakery" for the build scripts similar to a source tarball, there is a bakery tarball similarly generated from bzr and uploaded to the master.
Because of this triggering, it is not possible to manually force a Buildbot run on "latest bzr version" on the non-tarball builders. Instead to force a build, it is necessary to set the build properties for source tarball and bakery manually to the files one wants to tests. But it is often easier just to pick an existing build (with correct tarball) and use the "resubmit build" feature instead. Another option is to force build on the tarball build only; it will then in turn trigger all of the other builds.
On all builds (except source tarball), after the build a test is performed where the resulting package is installed in a virtual machine and basic testing is done.
The install test is done in a separate virtual machine from the one in which it was built, with no build tools and a very minimal install (to check that we have no hidden dependencies). The install test is currently very basic, basically just create a table and insert a row (it would be a nice ToDo to extend this testing to a more complete "smoke test").
Note that for .deb (Debian and Ubuntu), we set up a small fake local apt repository, so that we can properly test that ''apt-get install'' is able to pull in any extra dependencies without anything special needed on the part of the user.
On .deb builders (Debian and Ubuntu), we do three additional upgrade tests.
Two of them install the newly built MariaDB packages on top of an already installed MySQL package (the default MySQL package on that particular distro version), as well as on top of an earlier version of the MariaDB packages. For this, separate virtual machine images are used, based on the one used in the install test, but in addition with MySQL/MariaDB pre-installed.
The third test is performed on a clean image without any pre-installed MySQL or MariaDB packages. The test installs the latest available release of the same major version from MariaDB public repo, and then runs dist-upgrade using the local repo as a source of the new packages. The main goal of this test is to check that regular system upgrades performed via dist-upgrade work as expected in regard to MariaDB server.
In all tests, some very basic test data is also created in the installation, so we can test that replacing the old server installation with our new MariaDB package does not nuke the user's existing data! The test is also useful to check that the upgrade runs correctly with respect to dependencies etc, something that is quite complex to get right with the .deb packaging.
On .rpm builders (CentOS, Fedora, RHEL, SLES, openSUSE) we have numerous upgrade tests, although only a small portion of them is run for every branch.
All tests upgrade an already existing installation to the new MariaDB packages. The variables are:
what kind of existing installation is being upgraded: MariaDB server, MariaDB Galera, MySQL, Percona server, or mysql/mariadb installations provided by distributions;
which command is used for upgrade: it can either be performed via upgrade/update (depending on the package manager), or install, or dist-upgrade;
which packages are installed and upgraded: it can be a minimal set, server + client (plus whatever dependencies they bring), or a full set of packages.
In some cases, the tests can also set additional command-line options or environment variables for the package manager. It is normally done when the default installation does not work, and the package manager suggests using a different set of parameters.
Each test uses a clean VM image, installs an initial set of packages, and then uses one of the commands to upgrade the installation. It further checks that the server is upgraded, restarted and is reachable.
On the .deb builders (Debian and Ubuntu), we additionally do a full mysql-test-run.pl test run. This is done using the package mariadb-test.
Note that currently, due to , on newer Ubuntus the default apparmor profile prevents running mysql-test-run.pl. Until this is fixed, the tests need to uninstall apparmor before running the test suite.
This page is licensed: CC BY-SA / Gnu FDL
NOTE #1:It would probably make sense to create a seperate zone for the buildbot on your Solaris 10 system. That is left up to you!
NOTE #2:You might need to install the Zope Interface package for some of the Python pieces to work properly. Do this after installing Python 2.5. You can find information on the Zope Interface on their site.
Unless you want to spend time optimizing the underlying Solaris 10 (SPARC) installation, it is strongly suggested that you choose the "Entire" Software Group during the installation process. Once you have your Solaris 10 (SPARC) system on the network and able to access the Internet you can use the following suggestions to setup your buildbot:
By default Solaris 10 comes packages with Python 2.6.x. For compatibility reasons, you need to install Python 2.5.x. The author found it easiest to use Sunfreeware (ftp://ftp.sunfreeware.com/pub/freeware/sparc/5.10/):
If you are unfamiliar with howto install packages from Sunfreeware please read their .
Setup your environment:
Create a user:
It is imperative that you not use Bash as the buildbot user's shell. It could have been a dependency issue but I literally spent several days trying to solve why the buildbot wasn't checking out code with bzr only to discover that using the default Solaris shell fixed the problem. If someone comes up with a solution please let us know on the mailing list.
Adjust the global profile (you could be more granular here but since I'm setting up a dedicated system I wasn't):
Install the latest Twisted:
Install required packages from Sunfreeware:
Install the Bazaar DVCS:
You can find a package on .
Install the latest Bazaar for Solaris 10. At the time of this writing it was 1.14.1: .
Volunteer your buildbot to the Maria team. Follow the directions listed in the Volunteering to run a build slave section of the page.
Create the buildbot as the buildbot user:
Replace ${buildbotname} with the buildbot name you received from the MariaDB developers. Replace with the password they gave you. You can adjust these and other parameters anytime within the maria-slave/buildbot.tac file at anytime in the future.
Create a shared Bazaar repository in the buildbot build directory:
Replace ${buildbotname} like you did in Step #7.
Attach the buildbot to the master (this assumes you are not logged in as buildbot):
OPTIONAL: You can create a proper service by following (and modifying as necessary) these
Please ask on the #maria channel on irc.freenode.net if you have problems or questions setting up your buildbot. Or ask on the on the mailing list.
Check the status of your buildbot through the page.
This howto was contributed by Adam Dutko. The original version is .
This page is licensed: CC BY-SA / Gnu FDL
The installations are kept minimal, picking mostly default options. This helps ensure our stuff works on default installs, and also saves effort during the creation of the virtual machines.
Since most of the logic happens in the Buildbot slave on a host machine, and inside the Buildbot master, there is no need to install Buildbot or other similar complex stuff inside the VMs.
The virtual machines are configured with ssh access. An account called 'buildbot' is created, with passwordless login (using ssh public key authentication), and with passwordless sudo access (needed for automated scripting). The public key for user buildbot is installed in every VM, and the private key is added to the account running the Buildbot master on the host machine.
To generate the ssh keys:
Leave the passphrase empty. Run this as the user running the Buildbot slave on the KVM host machine. The resulting /.ssh/id_dsa.pub will be needed below for each virtual machine install.
The vms are configured to use the (emulated) serial port as the console. When running KVM on the host, the console then maps to the stdin/stdout. This is useful to get kernel log messages available to easier debug any problems. The bootloader Grub is similarly configured to use the serial port.
The vms are also set up to give a login prompt on the serial port (running getty). In retrospect this has proven not really needed, as if needing to manually log in and investigate things, it is often as easy to just run KVM in graphics mode. It may be useful in cases where the host machine is remote though (in this case graphics mode does still work but can be a bit slow).
To configure for serial console, the following two files are referenced below, and should be prepared in advance:
ttyS0ttyS0.confThe MariaDB (and MySQL) package on Debian and Ubuntu prompts the user about the root password to set. We want to test this important step, but we do not want the prompt of course. To do this we use the following configuration file for defaults for debconf. This file is needed in the steps below under the name "my.seed" (be careful to preserve it exactly as here, including trailing space and tab characters!)
For some background see . The file my.seed can be generated from an existing install using debconf-get-selections.
For Debian/Ubuntu, we add a local repository to the list of apt source to be
able to test apt-get install. For this, the following file "sources.append"
must be prepared:
The default network card emulated by KVM has poor performance. To solve this we instead use the "virtio" network device, using the KVM options "-net nic,model=virtio -net user". Except on Debian 4, which has an old kernel without support for virtio. Background info .
A detail is that some 32-bit vms crashed during boot with default options. This was fixed by using the kvm "-cpu qemu32,-nx" option. Some background information .
This page is licensed: CC BY-SA / Gnu FDL
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote_host} with the host the
vm is running on.
Note: When you activate the install, vncviewer will disconnect after a bit with a complaint about the rect being too large. This is fine. The installer has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
This step creates virtual machine images used to do an important additional upgrade test for .debs. Each virtual machine is pre-installed with an older version of MariaDB, and the test tries upgrading this version to the newly build one (to check that dependencies etc. work out correctly).
This step may not be easily possible to replicate exactly for new or updated images. For example, when we add a new platform/distro, we will not have the same old version of MariaDB available for installation on the new planform. Or if needing to re-create an image in the future, the original old .debs used before may no longer be available. However, this is not a big problem, as we can just use whatever version of MariaDB is available. In fact, while we cannot reasonably test every possible upgrade combination between MariaDB versions, it is still useful to test upgrades from different versions on different platforms, to increase coverage a bit.
The bulk of the images were installed with the following two loops, using packages from the OurDelta repository for , which was the one available from there at the time of installation.
The Ubuntu 10.04 "lucid" images were installed manually (as no packages for lucid were available from OurDelta at the time of installation):
Create and boot 64-bit lucid upgrade image:
Create directory inside the image:
Copy in packages to be installed from outside:
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The Fedora installer has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The Fedora installer has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
vm=vm-precise-amd64-fulltest.qcow2
kvm -m 2048 -hda /kvm/vms/${vm} -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:22666-:22 -nographicssh-keygen -t dsaCentos 5, i386 and amd64.
dt: Date when the buildbot run was started.
bbnum: The Buildbot '''build number''' which together with platform uniquely identifies a build within Buildbot.
typ: Concise abbreviation describing the kind of test. For example pr for --ps-protocol with row based replication, or nm for normal run with mixed-mode replication.
info: Short textual description of the kind of test run.
info_text: This is a short description that mysql-test-run.pl sometimes gives for some kinds of test failures (for example "timeout").
failure_text: This is the entire output from mysql-test-run.pl concerning this test failure. It usually contains the diff against the result file, a stacktrace for a crash, etc. This is useful to run LIKE queries against when searching for test failures similar to one being investigated.
test_name: The name of the test run by the server prior to seeing the warning.
ssh -t -p 22666 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /kvm/vms/ssh-keys/id_dsa dbart@localhostKeymap Selection: Do not set non-default key mapping
Hostname: to freebsd-92-amd64 or freebsd-92-i386
Distribution Select: deselect "games", leave "ports" selected
Partitioning: Guided and use the entire disk
Accept the default partitioning (Finish then Commit)
Archive Extraction: ...wait while installer installs the OS...
Network Configuration: choose vtnet0
configure ipv4 w/ DHCP
do not configure ipv6
System Configuration: leave sshd selected, don't select the others
Dumpdev Configuration: do not enable crash dumps
add a buildbot user
add user to 'wheel' group
Final Configuration: Apply configuration and exit
Manual Configuration: yes to opening a shell
echo 'console="comconsole"' >> /boot/loader.conf
Edit /etc/ttys and change off to on and dialup to vt100 for the ttyu0 entry.
shutdown -p now
Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:
Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.
This page is licensed: CC BY-SA / Gnu FDL
Under "Network Configuration" set the hostnames to fedora18-amd64 and fedora18-i386
Change "Software Selection" to "Minimal Install" (default is "Gnome Desktop")
For "Installation Destination" select the disk then click continue.
do not check the encryption checkbox
On "Installation Options" screen, expand the "Partition scheme configuration" box and select a "Partition type" of "Standard Partition"
While installing, set the root password
When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:
Until a non-root user is installed you must connect as root. SSH is preferred, so that's what we'll do first. Login as root.
After logging in as root, install proper ssh and then create a local user:
Enable password-less sudo and serial console:
Still logged in as root, add to /boot/grub/menu.lst:
Editing /boot/grub/menu.lst:
Logout as root, and then, from the VM host server:
Create a .ssh folder:
Copy over the authorized keys file:
Set permissions on the .ssh folder correctly:
Create the buildbot user:
su to the local buildbot user and ssh to the vm to put the key in known_hosts:
For i386:
For amd64:
Upload the ttyS0 file and put it where it goes:
Update the VM:
Once logged in:
After updating, shut down the VM:
This page is licensed: CC BY-SA / Gnu FDL
Set the hostname to ubuntu-quantal-amd64 or ubuntu-quantal-i386
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server
Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:
Enabling passwordless sudo:
Editing /boot/grub/menu.lst:
See Buildbot Setup for Virtual Machines - General Principles for how to obtain my.seed and sources.append.
The steps below are based on the Natty steps on Installing VM images for testing .deb upgrade between versions.
Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.
This page is licensed: CC BY-SA / Gnu FDL
Set the language to English(US)
Set the timezone to Etc/Greenwich Mean Time timezone
Change "Software Selection" to "Minimal Install" (default is "Gnome Desktop")
Under "Network Configuration" set the hostnames to fedora20-amd64 and fedora20-i386
For "Installation Destination" select the disk then click continue.
On "Installation Options" screen, select the "Partition scheme" drop-down menu and select "Standard Partition". We do not want LVM.
do not check the encryption checkbox
Select the "Begin installation" button to start the install
While installing, set the root password and an initial user.
Be sure the initial user is an administrator
When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM after the reboot fails and start it up again with:
Log in using the initial user created during the install:
After logging in, add the initial user to the "wheel" group.
Enable password-less sudo for the "wheel" group and serial console sudo:
Edit /boot/grub/menu.lst:
Logout and then, from the VM host server:
Create a .ssh folder:
Copy over the authorized keys file:
Set permissions on the .ssh folder correctly:
Create the buildbot user:
su to the local buildbot user and ssh to the vm to put the key in known_hosts:
For i386:
For amd64:
Upload the ttyS0 file and put it where it goes:
Update the VM:
Once logged in:
Change selinux policy to permissive:
After updating, exit, then shut down the VM:
This page is licensed: CC BY-SA / Gnu FDL
Set the hostname to ubuntu-precise
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server
Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:
Enabling passwordless sudo:
Editing /boot/grub/menu.lst:
Also:
See Buildbot Setup for Virtual Machines - General Principles for how to obtain my.seed and sources.append.
The steps below are based on the Natty steps on Installing VM images for testing .deb upgrade between versions.
64-bit Ubuntu precise:
32-bit Ubuntu precise:
This page is licensed: CC BY-SA / Gnu FDL
For 8.04/8.10/9.04, /boot/grub/menu.lst is edited as follows:
For 9.10, /boot/grub/menu.lst is edited as follows:
See the General Principles
article for how to make the 'my.seed' and 'sources.append' files.
This page is licensed: CC BY-SA / Gnu FDL
sudo VISUAL=vi visudo
# uncomment `%sudo ALL=NOPASSWD: ALL' line in `visudo`, and move to end.Perf records at a high frequency, so only a short recording is sufficient to answer the question. If this is too short adjust the frequency at which perf does its recordings.
The -g option here records the calling stack. Because seeing time in a mutex function isn't particularly interesting without knowing which mutex it is.
To view a recording, you need the debug symbols for your executable. See this page on getting the debug symbols available.
Changing the ownership of the recording means you can run perf report without sudo.
This shows where in the process MariaDB is spending most of its time at the top level. MariaDB uses threads per user connection so this will usually show a significant time in a handle_connection function. There are background threads that also run, so this can quickly show if its connection related time or a background thread.
To see which low level functions are consuming the most time, --no-children means that each function listed include only the time that is being spend it this function and excluding the other functions it calls.
Expanding out the function shows the complete call stack again. Multiple functions may have called the function you are looking at so there may be a different frequency breakdown.
A more complete example of performance analysis using perf is on this Percona community blog article.
One can add tracepoints at function entry/exit (and other locations too):
Something like:
Note: -a means system-wide.
There's also -p $PID option
This page is licensed: CC BY-SA / Gnu FDL
CREATE TABLE test_run(
id INT PRIMARY KEY AUTO_INCREMENT,
branch VARCHAR(100),
revision VARCHAR(32) NOT NULL,
platform VARCHAR(100) NOT NULL,
dt TIMESTAMP NOT NULL,
bbnum INT NOT NULL,
typ VARCHAR(32) NOT NULL,
info VARCHAR(255),
KEY (branch, revision),
KEY (dt),
KEY (platform, bbnum)
) ENGINE=innodbCREATE TABLE test_failure(
test_run_id INT NOT NULL,
test_name VARCHAR(100) NOT NULL,
test_variant VARCHAR(16) NOT NULL,
info_text VARCHAR(255),
failure_text TEXT,
PRIMARY KEY (test_run_id, test_name, test_variant)
) ENGINE=innodbCREATE TABLE test_warnings(
test_run_id INT NOT NULL,
list_id INT NOT NULL,
list_idx INT NOT NULL,
test_name VARCHAR(100) NOT NULL,
PRIMARY KEY (test_run_id, list_id, list_idx)
) ENGINE=innodbSELECT platform
FROM test_run r
WHERE branch = 'mysql-6.0-testing2'
AND revision = '2819'
AND (EXISTS (SELECT * FROM test_failure f WHERE f.test_run_id = r.id)
OR EXISTS (SELECT * FROM test_warnings w WHERE w.test_run_id = r.id));SELECT branch, revision, platform, test_name, test_variant, failure_text
FROM test_failure f
INNER JOIN test_run r ON (f.test_run_id = r.id)
WHERE failure_text LIKE "%--protocol=TCP' failed%";SELECT branch, COUNT(*)
FROM test_failure f
INNER JOIN test_run r ON (f.test_run_id = r.id)
WHERE failure_text LIKE "%--protocol=TCP' failed%"
GROUP BY branch;SELECT branch, revision, platform, COUNT(*)
FROM test_warnings w
INNER JOIN test_run r ON (w.test_run_id = r.id)
WHERE test_name = 'rpl.rpl_plugin_load'
GROUP BY r.id;sudo vi /etc/apt/sources.list.d/cassandra.list
# paste in the following two lines:
deb http://www.apache.org/dist/cassandra/debian 11x main
deb-src http://www.apache.org/dist/cassandra/debian 11x main
gpg --keyserver pgp.mit.edu --recv-keys F758CE318D77295D
gpg --export --armor F758CE318D77295D | sudo apt-key add -
gpg --keyserver pgp.mit.edu --recv-keys 2B5C1B00
gpg --export --armor 2B5C1B00 | sudo apt-key add -
sudo apt-get update
sudo apt-get install cassandraCREATE keyspace DEMO;
USE DEMO;
CREATE COLUMN family Users
WITH key_validation_class = 'UTF8Type'
AND comparator = 'UTF8Type'
AND default_validation_class = 'UTF8Type';
SET Users[1234][name] = scott;
SET Users[1234][password] = tiger;
GET Users[1234];
quit;dbart@ubuntu-precise-amd64:~$
cassandra-cli
Connected to: "Test Cluster" on 127.0.0.1/9160
Welcome to Cassandra CLI version 1.1.9
Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.
[default@unknown]
create keyspace DEMO;
622a672f-dd03-37bf-bf78-f3e99a8f18a6
Waiting for schema agreement...
... schemas agree across the cluster
[default@unknown]
use DEMO;
Authenticated to keyspace: DEMO
[default@DEMO]
create column family Users
...
with key_validation_class = 'UTF8Type'
...
and comparator = 'UTF8Type'
...
and default_validation_class = 'UTF8Type';
605eea14-d3e5-3d1d-ab1d-f4863c814538
Waiting for schema agreement...
... schemas agree across the cluster
[default@DEMO]
set Users[1234][name] = scott;
Value inserted.
Elapsed time: 46 msec(s).
[default@DEMO]
set Users[1234][password] = tiger;
Value inserted.
Elapsed time: 2.77 msec(s).
[default@DEMO]
get Users[1234];
=> (column=name, value=scott, timestamp=1361818884084000)
=> (column=password, value=tiger, timestamp=1361818887944000)
Returned 2 results.
Elapsed time: 53 msec(s).
[default@DEMO]
quit;sudo shutdown -h nowdbart@ubuntu-precise-i386:~$
cassandra-cli
Connected to: "Test Cluster" on 127.0.0.1/9160
Welcome to Cassandra CLI version 1.1.9
Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.
[default@unknown]
create keyspace DEMO;
5eafc25e-71b6-3585-9db1-891b3348790c
Waiting for schema agreement...
... schemas agree across the cluster
[default@unknown]
use DEMO;
Authenticated to keyspace: DEMO
[default@DEMO]
create column family Users
...
with key_validation_class = 'UTF8Type'
...
and comparator = 'UTF8Type'
...
and default_validation_class = 'UTF8Type';
9c2ad7bc-8dc0-35ce-8067-4dc4577319f1
Waiting for schema agreement...
... schemas agree across the cluster
[default@DEMO]
set Users[1234][name] = scott;
Value inserted.
Elapsed time: 51 msec(s).
[default@DEMO]
set Users[1234][password] = tiger;
Value inserted.
Elapsed time: 2.44 msec(s).
[default@DEMO]
get Users[1234];
=> (column=name, value=scott, timestamp=1361819341068000)
=> (column=password, value=tiger, timestamp=1361819345337000)
Returned 2 results.
Elapsed time: 57 msec(s).
[default@DEMO]
quit;rsync -avP host::kvm/vms/*fulltest* /kvm/vms/$ cd /tmp
$ ftp ftp.sunfreeware.com
Name: anonymous
Password: (your email address)
ftp> bin
ftp> cd pub/freeware/sparc/5.10/
ftp> get python-2.5.4-sol10-sparc-local.gz
ftp> quit
$ gunzip python-2.5.4-sol10-sparc-local.gz
$ pkgadd -d python-2.5.4-sol10-sparc-local$ useradd -d /export/home/buildbot -m buildbot$ vi /etc/profile
LD_LIBRARY_PATH=/opt/csw/lib:/usr/local/lib:/usr/sfw/lib:$LD_LIBRARY_PATH
# Add required libraries
PYTHONPATH=/usr/local/lib/python2.5/site-packages:$PYTHONPATH
# Makes Python 2.5 the default
PATH=/usr/local/bin:/usr/bin:/usr/sbin:/etc:/usr/sfw/bin:$PATH
# Puts "local" packages in your path
export LOGNAME PATH PYTHONPATH LD_LIBRARY_PATH$ wget tmrc.mit.edu/mirror/twisted/Twisted/9.0/Twisted-9.0.0.tar.bz2
$ bunzip2 Twisted-9.0.0.tar.bz2
$ tar -xf Twisted-9.0.0.tar
$ cd Twisted-9.0.0
$ python setup.py build installautomake, autoconf, gcc, m4, md5, openssl, libsigsegv, Tcl, Tk, perl,
libtool, sed, libgcc, gcc, libintl, libiconv, zlib, binutils, groff, cmakebuildbot create-slave --usepty=0 /export/home/buildbot/maria-slave
hasky.askmonty.org:9989 ${buildbotname} <passwd>HOME=/export/home/buildbot; bzr init-repo maria-slave/${buildbotname}sudo su - buildbot -c "/usr/bin/buildbot start /export/home/buildbot/maria-slave"# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.
start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5
stop on runlevel 0
stop on runlevel 1
stop on runlevel 6
respawn
exec /sbin/getty 115200 ttyS0# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.
start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]
respawn
exec /sbin/getty -L 115200 ttyS0 vt102mariadb-server-5.1 mysql-server/root_password_again password rootpass
mariadb-server-5.1 mysql-server/root_password password rootpass
mariadb-server-5.1 mysql-server/error_setting_password error
mariadb-server-5.1 mysql-server-5.1/nis_warning note
mariadb-server-5.1 mysql-server-5.1/really_downgrade boolean false
mariadb-server-5.1 mysql-server-5.1/start_on_boot boolean true
mariadb-server-5.1 mysql-server-5.1/postrm_remove_databases boolean false
mariadb-server-5.1 mysql-server/password_mismatch error
mariadb-server-5.1 mysql-server/no_upgrade_when_using_ndb errordeb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/qemu-img create -f qcow2 /kvm/vms/vm-freebsd92-amd64-serial.qcow2 15G
qemu-img create -f qcow2 /kvm/vms/vm-freebsd92-i386-serial.qcow2 15Gkvm -m 2048 -hda /kvm/vms/vm-freebsd92-amd64-serial.qcow2 -cdrom /kvm/iso/freebsd/9.2/FreeBSD-9.2-RELEASE-amd64-dvd1.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2283-:22
kvm -m 2048 -hda /kvm/vms/vm-freebsd92-i386-serial.qcow2 -cdrom /kvm/iso/freebsd/9.2/FreeBSD-9.2-RELEASE-i386-dvd1.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2284-:22vncviewer -via ${remote_host} localhostkvm -m 2048 -hda /kvm/vms/vm-freebsd92-amd64-serial.qcow2 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2283-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-freebsd92-i386-serial.qcow2 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2284-:22 -nographic
ssh -p 2283 localhost
ssh -p 2284 localhost
su -
cd /usr/ports/security/sudo; make install clean # accept defaults
visudo # uncomment the '# %wheel ALL=(ALL) NOPASSWD: ALL' line
exit
cd /usr/ports/ports-mgmt/portmaster; sudo make install clean # select both autocomplete options
ssh hasky.askmonty.org
exit
sudo su buildbot -
ssh hasky.askmonty.org
exit
exit
exit
scp -P 2283 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2283 /kvm/vms/authorized_keys buildbot@localhost:.ssh/
scp -P 2284 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2284 /kvm/vms/authorized_keys buildbot@localhost:.ssh/
ssh -p 2283 localhost
ssh -p 2284 localhost
sudo portsnap fetch extract
sudo portsnap fetch update
sudo portmaster -Da
sudo shutdown -p nowfor i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"cd /usr/ports/devel/libevent; sudo make install clean BATCH=yes" \
"cd /usr/ports/devel/boost-all; sudo make install clean BATCH=yes" \
"cd /usr/ports/devel/cmake; sudo make install clean BATCH=yes" \
"cd /usr/ports/devel/thrift/; sudo make install clean BATCH=yes" \
"cd /usr/ports/devel/bzr/; sudo make install clean BATCH=yes" \
"cd /usr/ports/devel/bison/; sudo make install clean BATCH=yes" \
"cd /usr/ports/databases/unixODBC; sudo make install clean BATCH=yes" \
"cd /usr/ports/devel/doxygen; sudo make install clean BATCH=yes"; \
doneqemu-img create -b /kvm/vms/vm-freebsd92-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-freebsd92-amd64-install.qcow2
qemu-img create -b /kvm/vms/vm-freebsd92-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-freebsd92-i386-install.qcow2
for i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo portsnap cron update"; \
donefor i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"cd /usr/ports/databases/mysql55-server; sudo make install clean BATCH=yes" \
"sudo /usr/local/etc/rc.d/mysql-server onestart" \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
donefor i in '/kvm/vms/vm-freebsd92-amd64-serial.qcow2 2283 qemu64' '/kvm/vms/vm-freebsd92-i386-serial.qcow2 2284 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
"cd /usr/ports/databases/mariadb55-server; sudo make install clean BATCH=yes" \
"sudo /usr/local/etc/rc.d/mysql-server onestart" \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"'; \
done# freebsd92-amd64
cp -avi /kvm/vms/vm-freebsd92-amd64-install.qcow2 /kvm/vms/vm-freebsd92-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-freebsd92-amd64-test.qcow2 -redir tcp:2283::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2283 buildbot@localhost sudo shutdown -p now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-freebsd92-amd64-test.qcow2
# freebsd92-i386
cp -avi /kvm/vms/vm-freebsd92-i386-install.qcow2 /kvm/vms/vm-freebsd92-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-freebsd92-i386-test.qcow2 -redir tcp:2284::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2284 buildbot@localhost sudo shutdown -p now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-freebsd92-i386-test.qcow2qemu-img create -f qcow2 /kvm/vms/vm-fedora18-i386-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-fedora18-amd64-serial.qcow2 10Gkvm -m 2048 -hda /kvm/vms/vm-fedora18-i386-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-18-i386-DVD.iso -redir tcp:2277::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora18-amd64-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-18-x86_64-DVD.iso -redir tcp:2278::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net uservncviewer -via ${remote-host} localhostkvm -m 2048 -hda /kvm/vms/vm-fedora18-i386-serial.qcow2 -redir tcp:2277::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora18-amd64-serial.qcow2 -redir tcp:2278::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net userssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhostadduser ${username}
usermod -a -G wheel ${username}
passwd ${username}visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requirettysudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
grub2-mkconfig -o /boot/grub2/grub.cfgssh -t -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"scp -P 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/ssh -t -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"ssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'sudo su - buildbot
ssh -p 2277 buildbot@localhost
# exit, then exit againsudo su - buildbot
ssh -p 2278 buildbot@localhost
# exit, then exit againscp -P 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
ssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'ssh -p 2277 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2278 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum updatesudo shutdown -h nowfor i in '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo yum -y groupinstall 'Development Tools'" \
"sudo yum -y install cmake tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel libxml2 libxml2-devel unixODBC-devel bzr perl perl\(DBI\)" \
"sudo yum -y remove systemtap-sdt-dev" \
"bzr co --lightweight lp:mariadb-native-client" \
"sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
"cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
"cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" \
"cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -vs ../src/boost_1_49_0/boost ." ; \
donefor i in '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo yum -y update" \
"sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI" ; \
donefor i in '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo yum -y update" \
"sudo yum -y install patch tar mysql-server libtool-ltdl unixODBC" \
"sudo systemctl enable mysqld.service" \
"sudo systemctl start mysqld.service" \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
donefor i in '/kvm/vms/vm-fedora18-amd64-serial.qcow2 2278 qemu64' '/kvm/vms/vm-fedora18-i386-serial.qcow2 2277 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB-${2}.repo buildbot@localhost:/tmp/MariaDB.repo" \
'sudo rpm --verbose --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB' \
'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
'sudo yum -y remove mysql-libs' \
'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test libtool-ltdl unixODBC' \
'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix patch tar' \
'sudo /etc/init.d/mysqld start' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
doneqemu-img create -f qcow2 /kvm/vms/vm-quantal-amd64-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-quantal-i386-serial.qcow2 10Gkvm -m 2048 -hda /kvm/vms/vm-quantal-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-amd64.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2275-:22
kvm -m 2048 -hda /kvm/vms/vm-quantal-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-i386.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2276-:22vncviewer -via ${remote-host} localhostkvm -m 2048 -hda /kvm/vms/vm-quantal-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-amd64.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2275-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-quantal-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.10-server-i386.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2276-:22 -nographic
ssh -p 2275 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -p 2276 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2275 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2276 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
scp -P 2275 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2276 /kvm/vms/authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2275 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2276 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
scp -P 2275 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2276 /kvm/vms/ttyS0.conf buildbot@localhost:
ssh -p 2275 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2276 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2275 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2276 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grub
# exit back to the host serverfor i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 defoma dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern psfontmgr texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev" \
"cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
"cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" \
"cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -vs ../src/boost_1_49_0/boost ." ; \
donefor i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
donefor i in '/kvm/vms/vm-quantal-amd64-serial.qcow2 2275 qemu64' '/kvm/vms/vm-quantal-i386-serial.qcow2 2276 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"= scp -P $2 /kvm/vms/mariadb-quantal.list buildbot@localhost:/tmp/tmp.list" \
"sudo debconf-set-selections /tmp/my55.seed" \
'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
'sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0xcbcb082a1bb943db' \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-server-5.5 mariadb-client mariadb-client-5.5 mariadb-test libmariadbclient-dev' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done# quantal-amd64
cp -avi /kvm/vms/vm-quantal-amd64-install.qcow2 /kvm/vms/vm-quantal-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-quantal-amd64-test.qcow2 -redir tcp:2275::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2275 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-quantal-amd64-test.qcow2
# quantal-i386
cp -avi /kvm/vms/vm-quantal-i386-install.qcow2 /kvm/vms/vm-quantal-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-quantal-i386-test.qcow2 -redir tcp:2276::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2276 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-quantal-i386-test.qcow2qemu-img create -f qcow2 /kvm/vms/vm-fedora20-i386-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-fedora20-amd64-serial.qcow2 20Gkvm -m 2048 -hda /kvm/vms/vm-fedora20-i386-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-20-i386-DVD.iso -redir tcp:2291::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora20-amd64-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-20-x86_64-DVD.iso -redir tcp:2292::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net uservncviewer -via ${remote-host} localhostkvm -m 2048 -hda /kvm/vms/vm-fedora20-i386-serial.qcow2 -redir tcp:2291::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora20-amd64-serial.qcow2 -redir tcp:2292::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net userssh -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost
ssh -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhostsudo usermod -a -G wheel ${username}sudo visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requirettysudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo grub2-mkconfig -o /boot/grub2/grub.cfgssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"scp -P 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -vR go-rwx .ssh"
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -vR go-rwx .ssh"ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -vR go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -vR go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'sudo su - buildbot
ssh -p 2291 buildbot@localhost
# exit, then exit againsudo su - buildbot
ssh -p 2292 buildbot@localhost
# exit, then exit againscp -P 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'ssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum -y updatesudo sed -i 's/SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/configexitssh -t -p 2291 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo shutdown -h now;exit'
ssh -t -p 2292 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo shutdown -h now;exit'for i in '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_build_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo yum -y groupinstall 'Development Tools'" \
"sudo yum -y install yum-utils" \
"sudo yum-builddep -y mariadb" \
"sudo yum -y install automake libtool flex pkgconfig gcc-c++ libevent-devel python-devel ruby-devel rpm-build Judy-devel" \
"sudo yum -y install cmake tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel" \
"sudo yum -y install libxml2 libxml2-devel bison bison-devel boost-devel unixODBC-devel bzr perl perl\(DBI\)" \
"sudo yum -y remove systemtap-sdt-dev" \
"bzr co --lightweight lp:mariadb-native-client" \
"sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
"cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
"cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
donefor i in '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_install_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo yum -y update" \
"sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI unixODBC Judy" ; \
donefor i in '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_upgrade_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo yum -y update" \
"sudo yum -y install patch tar mariadb-server libtool-ltdl unixODBC Judy" \
"sudo systemctl enable mariadb.service" \
"sudo systemctl start mariadb.service" \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
donefor i in '/kvm/vms/vm-fedora20-amd64-serial.qcow2 2292 qemu64' '/kvm/vms/vm-fedora20-i386-serial.qcow2 2291 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_upgrade2_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB-${2}.repo buildbot@localhost:/tmp/MariaDB.repo" \
'sudo rpm --verbose --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB' \
'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
'sudo yum -y remove mysql-libs' \
'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test libtool-ltdl unixODBC Judy' \
'sudo yum -y install cronie cronie-anacron crontabs.noarch patch tar' \
'sudo /etc/init.d/mysql start' \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
doneqemu-img create -f qcow2 /kvm/vms/vm-precise-amd64-serial.qcow2 8G
qemu-img create -f qcow2 /kvm/vms/vm-precise-i386-serial.qcow2 8Gkvm -m 1024 -hda /kvm/vms/vm-precise-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-amd64.iso -redir tcp:22255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-precise-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-i386.iso -redir tcp:22256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net uservncviewer -via ${remote-host} localhostkvm -m 1024 -hda /kvm/vms/vm-precise-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-amd64.iso -redir tcp:22255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-precise-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-12.04-server-i386.iso -redir tcp:22256::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 22255 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -p 22256 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 22255 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 22256 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
scp -P 22255 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 22256 /kvm/vms/authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 22255 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 22256 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
scp -P 22255 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 22256 /kvm/vms/ttyS0.conf buildbot@localhost:
ssh -p 22255 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 22256 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grubfor i in '/kvm/vms/vm-precise-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-precise-i386-serial.qcow2 22256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 defoma dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern psfontmgr texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils" ; \
donefor i in '/kvm/vms/vm-precise-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-precise-i386-serial.qcow2 22256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
"= scp -P $2 /kvm/vms/my.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in '/kvm/vms/vm-precise-amd64-install.qcow2 22255 qemu64' '/kvm/vms/vm-precise-i386-install.qcow2 22256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y libaio1 mysql-server-5.5' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
doneqemu-img create -b vm-precise-amd64-install.qcow2 -f qcow2 vm-precise-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-precise-amd64-upgrade2.qcow2 -redir 'tcp:22200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 22200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y libaio1
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h nowqemu-img create -b vm-precise-i386-install.qcow2 -f qcow2 vm-precise-i386-upgrade2.qcow2
kvm -m 512 -hda vm-precise-i386-upgrade2.qcow2 -redir 'tcp:22200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 22200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.5/ubuntu oneiric main
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y libaio1
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h nowqemu-img create -f qcow2 vm-hardy-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2228 localhost
# edit /boot/grub/menu.lst and visudo, see below
sudo /sbin/shutdown -h now
kvm -m 1024 -hda vm-hardy-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-amd64.iso -redir tcp:2228::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -t -p 2228 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2228 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2228 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2228 ttyS0 buildbot@localhost:
ssh -p 2228 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'qemu-img create -f qcow2 vm-hardy-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-hardy-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-i386.iso -redir tcp:2229::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-hardy-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.04.3-server-i386.iso -redir tcp:2229::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2229 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2229 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2229 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2229 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2229 ttyS0 buildbot@localhost:
ssh -p 2229 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'qemu-img create -f qcow2 vm-intrepid-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-intrepid-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-amd64.iso -redir tcp:2230::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-intrepid-amd64-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-amd64.iso -redir tcp:2230::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2230 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2230 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2230 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2230 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2230 ttyS0 buildbot@localhost:
ssh -p 2230 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'qemu-img create -f qcow2 vm-intrepid-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-intrepid-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-i386.iso -redir tcp:2231::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-intrepid-i386-serial.qcow2 -cdrom /kvm/ubuntu-8.10-server-i386.iso -redir tcp:2231::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2231 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2231 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2231 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2231 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2231 ttyS0 buildbot@localhost:
ssh -p 2231 buildbot@localhost 'sudo cp ttyS0 /etc/event.d; rm ttyS0; sudo shutdown -h now'qemu-img create -f qcow2 vm-karmic-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-karmic-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-amd64.iso -redir tcp:2232::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-karmic-amd64-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-amd64.iso -redir tcp:2232::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2232 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2232 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2232 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2232 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2232 ttyS0.conf buildbot@localhost:
ssh -p 2232 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'qemu-img create -f qcow2 vm-karmic-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-karmic-i386-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-i386.iso -redir tcp:2233::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-karmic-i386-serial.qcow2 -cdrom /kvm/ubuntu-9.10-server-i386.iso -redir tcp:2233::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2233 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2233 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2233 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2233 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2233 ttyS0.conf buildbot@localhost:
ssh -p 2233 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'sudo vi /boot/grub/menu.lst
# Add this:
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial console
# Also add in menu.lst to kernel line (after removing `quiet splash'):
console=tty0 console=ttyS0,115200n8sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grubfor i in 'vm-hardy-amd64-serial.qcow2 2228 qemu64' 'vm-hardy-i386-serial.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-serial.qcow2 2230 qemu64' 'vm-intrepid-i386-serial.qcow2 2231 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get build-dep -y mysql-server-5.0" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
done
for i in 'vm-karmic-amd64-serial.qcow2 2232 qemu64' 'vm-karmic-i386-serial.qcow2 2233 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline5-dev" ; \
donefor i in 'vm-hardy-amd64-serial.qcow2 2228 qemu64' 'vm-hardy-i386-serial.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-serial.qcow2 2230 qemu64' 'vm-intrepid-i386-serial.qcow2 2231 qemu32,-nx' 'vm-karmic-amd64-serial.qcow2 2232 qemu64' 'vm-karmic-i386-serial.qcow2 2233 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
"= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in 'vm-hardy-amd64-install.qcow2 2228 qemu64' 'vm-hardy-i386-install.qcow2 2229 qemu32,-nx' 'vm-intrepid-amd64-install.qcow2 2230 qemu64' 'vm-intrepid-i386-install.qcow2 2231 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.0'
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
done
for i in 'vm-karmic-amd64-install.qcow2 2232 qemu64' 'vm-karmic-i386-install.qcow2 2233 qemu32,-nx' ; do \
set $i; \
runvm --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
donesudo perf record -p ${pidof mysqld} -g -o sample.perf -- sleep 5sudo chown $USER: sample.perf
perf report -i sample.perf -gperf report -i sample.perf -g --no-childrensudo perf probe -x /path/to/ha_rocksdb.so --add rocksdb_prepare
sudo perf probe -x /path/to/ha_rocksdb.so --add rocksdb_prepare%returnsudo perf probe -lperf record -e 'probe_ha_rocksdb:*' -a -- sleep 60perf record -e 'probe_ha_rocksdb:*' -p $(pidof mysqld) -- sleep 60perf scriptNow the same, but for 32-bit lucid:
Create directory inside the image:
Copy in packages to be installed from outside:
Install the MariaDB packages, remove package dir, and upgrade to latest security fixes:
The Ubuntu 10.10 "maverick" images were likewise installed manually:
Create and boot 64-bit maverick upgrade image:
Same for 32-bit maverick:
For Ubuntu 11.04 "natty", mariadb packages were installed from the repository for the previous version.
64-bit Ubuntu natty:
32-bit Ubuntu natty:
For Ubuntu 11.10 "oneiric". The steps performed are listed at: Buildbot Setup for Virtual Machines - Ubuntu 11.10 "oneiric" (in the VMs for Upgrade Testing section)
This page is licensed: CC BY-SA / Gnu FDL
for i in "vm-hardy-amd64-install qemu64 hardy" "vm-hardy-i386-install qemu32,-nx hardy" \
"vm-intrepid-amd64-install qemu64 intrepid" "vm-intrepid-i386-install qemu32,-nx intrepid" \
"vm-karmic-amd64-install qemu64 karmic" "vm-karmic-i386-install qemu32,-nx karmic" \
"vm-jaunty-amd64-install qemu64 jaunty" "vm-jaunty-i386-deb-install qemu32,-nx jaunty" \
"vm-debian5-amd64-install qemu64 lenny" "vm-debian5-i386-install qemu32,-nx lenny" ; do \
set $i; \
runvm -b $1.qcow2 -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 $(echo $1 | sed -e 's/-install/-upgrade2/').qcow2 \
"wget -O- http://ourdelta.org/deb/ourdelta.gpg | sudo apt-key add -" \
"sudo sh -c \"echo 'deb http://mirror.ourdelta.org/deb $3 mariadb-ourdelta' > /etc/apt/sources.list.d/ourdelta.list\"" \
"sudo apt-get update || true" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-test libmariadbclient-dev || true" \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
"sudo rm /etc/apt/sources.list.d/ourdelta.list" \
"sudo apt-get update || true" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y || true" ; \
doneInstall pywin32. Make sure the version matches your Python version perfectly, and get the .exe file, not the zip file.
Note: As of June 2016, used pywin32-220.win-amd64-py2.7.exe, it seems to work.
Install Twisted
Note: As of June 2016, used Twisted 16.2.0 for Python 2.7 64 bits
Install buildbot: Get the zip file and unpack it. In an administrator shell, cd to the buildbot dir and run "python setup.py install". After that, the unpacked buildbot directory is no longer needed.
Note: As of June 2016, used buildbot 0.8.12.
When this has been done, you run these commands:
Adjust paths in the last command according to the location of your unpacked buildbot.
You get and from one of the MariaDB Corporation or MariaDB Foundation people handling the buildbot system. can be whatever you want - is a good choice.
Please edit the files \info\admin and \info\host with your information. The host information should include your Windows version, 32 or 64 bit, and your Visual Studio version.
You can test the buildbot slave with this command:
When buildbot starts, you should confgure it as a service instead of manually. See the instructions here. It's under the section "Windows Buildbot service setup". This document also has the generic Windows Buildbot installation documentation.
When buildbot is running in user session, and application that is started by buildbot crashes, you'll get a crash popup. There are popups that are generated by post-mortem debugging and there are popups that are generated by Windows Error reporting. It is hard (or impossible for those who did not try this before) to get rid of all popups. Thus, do run buildbot as service. There are no popups in services.
MTR attempts to call cdb via the Perl backticks operator, when mysqld.exe crashes. On the first run, cdb is downloading public Windows symbols from msdl.microsoft.com/download/symbols. The symbols are cached in C:\cdb_symbols, and following runs will be faster, however crashdump analysis on very first crash will typically take some time.
If you find this bothering, start the test with --mysqld=--gdb which will cause no crashdump files to be created and thus will prevent cdb from being called.
This is due to the fact that buildbot does not clean up any processes left over from a previous run, and those processes may hold locks on files that are needed for a new build to start. Current workaround is to use windows job objects that allow to terminate entire process trees. We use special "process launcher" utility called "dojob . This will also require changing buildbot configuration for the builder.
Download dojob.cpp, and compile it with
Then, put dojob.exe into a directory in the PATH environment variable. Then, change buildbot configuration to use "dojob" for every command, as a replacement for "cmd /c", e.g
(Seen very seldom?). This usually happens after retrying multiple failing tests multiple times. It appears that MTR's --retries= option is not safe to use on Windows. The solution is to run the test with no --retries.
This is a sympthom of intermittent network issues, which cause Buildbot to abort the current build altogether. The following workarounds are possible:
In your buildbot.tac file, specify a higher keepalive value, such as 60000.
Note: For the current versions (as of June 2016), the opposite solution worked: keepalive = 60. KeepAliveTime in the Windows registry has been set to 60000 as suggested below, but there is no proof it made any difference.
If you are running the Windows host inside VMWare, substitute the e1000 network adapter with vmxnet3
If your build host is behind a firewall, makes sure that the firewall does not time out any idle connections quickly. Configure at least a 24-hour timeout.
If your build host is behind a firewall, consider disabling the built-in Windows Firewall in order to avoid one potential point of failure.
Modify the Windows KeepAliveTime registry setting to a lower value, such as 60000 (equal to 60 seconds). For more information, see
Make sure the buildbot master does not experience prolonged bouts of 100% CPU activity, as this may prevent keepalives from working. If the buildbot master twisted.log says that data is frequently being loaded from on-disk pickles, increase the buildCacheSize in the master configuration file to be more than the number of builds per builder that the log file reports are being loaded.
In case the default Windows buildbot setup is not sufficiently reliable due to "connection lost in a non-clean fashion" errors, the following setup can be used instead. It runs the buildbot deamon on a linux host while doing the builds on Windows, thus working around Twisted issues on Windows.
Note that the procedure below significantly degrades the overall security of your Windows host. It is strongly recommended that a properly-firewalled, standalone virtual machine is used.
Set the Windows server to auto-login with the user the builds will run as. This will prevent any issues relating to running things as a service and the privilege considerations associated with services;
Install FreeSSHd on Windows. The OpenSSHd/Cygwin combo is not likely to work.
Do not run FreeSSHd it as a service, but rather as a startup console application for the user you are running the builds under. This allows FreeSSHd UI to properly operate;
Generate a private/public key pair using PuTTYGen and place the public part in a properly named file in the key directory specified in the FreeSSHd GUI
Using the FreeSSHd GUI, create a new user and set it to use key authentication;
Set the SFTP home path to the directory where builds will take place.
Disable UAC (User Access Control) (or set to lowest level) by using the Users Control Panel. Add your buildbot user to the Administrators group.
Export the private key from PuTTYGen and copy it to a Linux host.
Install buildbot on the Linux host
The buildbot configuration file will then look something like:
The following buildbot considerations apply under this setup:
buildbot will not perform a proper cleanup of the build directory on the Windows host before each build (it will only clean up things on the linux side). So, a directory cleanup via rmdir must be placed as an explicit first step in the build sequence;
Bzr() can not be used to check out BZR trees, as it expects that the bzr checkout command will run locally. Instead, an explicit call to bzr checkout via a SSH command must be used;
This page is licensed: CC BY-SA / Gnu FDL
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The CentOS installer has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
The Installer will throw up a "Storage Device Warning", choose "Yes, discard any data"
Set the hostname to centos6-amd64 (or centos6-i386)
Click the "Configure Network" button on the Hostname screen.
Edit System eth0 to "connect automatically"
Apply and then close the "Network Connections" window
Set Timezone to Europe/Helsinki (keep "System clock uses UTC" checked)
When partitioning disks, choose "Use All Space"
do not check the "Encrypt system" checkbox
do check the "Review and modify partitioning layout" checkbox
Delete the LVM stuff and leaving the sda1 partition alone, repartition the physical volume as follows
Minimal install
Customize Later
When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:
You may connect via VNC as before, but ssh is probably preferred. Login as root.
Now that the VM is installed, it's time to configure it.
After logging in as root, create a local user
Enable password-less sudo and serial console:
Still logged in as root, add to /boot/grub/menu.lst:
also add in menu.lst to kernel line (after removing 'quiet'):
Add login prompt on serial console:
Logout as root, and then, from the VM host server:
Install proper ssh:
Create a .ssh folder:
Copy over the authorized keys file:
Set permissions on the .ssh folder correctly:
Create the buildbot user:
Upload the ttyS0.conf file and put it where it goes:
Update the VM:
Once logged in:
After updating, shut down the VM:
Also:
MariaDB.local.repo points at a local directory on the VM. MariaDB.repo points at the real MariaDB YUM repository.
The MariaDB upgrade testing VMs were not built. There is currently an error with installing MariaDB from the YUM repo.
This page is licensed: CC BY-SA / Gnu FDL
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The Fedora installer has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
The Installer will throw up a "Storage Device Warning", choose "Yes, discard any data"
Set the hostname to fedora17-amd64 (or fedora17-i386)
Click the "Configure Network" button on the Hostname screen.
Edit System eth0 to "connect automatically"
Apply and then close the "Network Connections" window
When partitioning disks, choose "Use All Space"
uncheck the "Use LVM" checkbox
do not check the "Encrypt system" checkbox
Minimal install
Customize Later
When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:
Until the extra user is installed you must connect via VNC as before. SSH is preferred, so that's what we'll do first. Login as root.
After logging in as root, install proper ssh and then create a local user:
Enable password-less sudo and serial console:
Still logged in as root, add to /boot/grub/menu.lst:
Editing /boot/grub/menu.lst:
Logout as root, and then, from the VM host server:
Create a .ssh folder:
Copy over the authorized keys file:
Set permissions on the .ssh folder correctly:
Create the buildbot user:
su to the local buildbot user and ssh to the vm to put the key in known_hosts:
For i386:
For amd64:
Upload the ttyS0 file and put it where it goes:
Update the VM:
Once logged in:
After updating, shut down the VM:
On the VM host server:
Boot the VMs with:
Copy over the boost tar file:
Login to the VMs:
Once logged in:
On the VM host server:
Boot the VMs with:
Login to the VMs:
Once logged in:
On the VM host server:
Boot the VMs with:
Login to the VMs:
Once logged in:
The MariaDB upgrade testing VMs were not built. Once we have MariaDB Fedora 17 RPMs, then I will attempt building this VM. For now, the placeholder text below is copied from the page.
This page is licensed: CC BY-SA / Gnu FDL
This page documents the general setup process for a server that is acting as virtual machine host, like those documented in the Buildbot Setup for Virtual Machines section.
Provision hardware with most recent Ubuntu LTS release
Add host to DNS
Apply updates (replace <host> with hostname)
install some favorite packages (these aren't necessarily required, but I like them)
a buildbot admin needs to add the new host to the allowed list of rsync clients on the VM master (whichever host is the official host of VM files) The VM master changes periodically, so check to make sure you have the correct one.
make a /kvm/ dir and rsync it with the VM master above. The dir often
resides at /home/kvm/ (or wherever the storage drive is) and is then
linked to /kvm/. The VMs rely on the /kvm/vms/ path, so
the /kvm/ location is required.
detatch from screen session with Ctrl+a d
Configure vim.basic as the default editor (optional)
install buildbot-slave, bzr, and kvm
add a default user, and then add the user to the appropriate groups
logout then back in as the default user and change the password
set up the /.ssh/authorized_keys file so you can login that way
create other standard users and set up their ssh keys (optional)
turn off password login (WARNING: be sure to have your ssh key setup before doing this!) and disallow all root logins and password logins (it is safer to only allow logins using ssh keys with regular users):
checkout mariadb-tools
put runvm in the right place
add the buildbot user to the kvm and tty groups
A buildbot admin will need to add this builder
to the maria-master-private.cfg file on the ${buildmaster} and
also add it to the c['slaves'] array in maria-master.cfg then
create the buildslave using the hostname and whatever ${password} was
agreed upon by you and the buildbot admin:
add the following to /etc/default/buildslave (replace ${hostname}
with the name of the host)
edit the admin and host files and add contact information and details on the builder:
copy over the buildbot .ssh dir from terrier:
Edit /etc/passwd and change the buildbot user's shell from /bin/false
to /bin/bash
su to the buildbot user and copy in the /etc/skel files
change ownership of the buildbot/slaves dir to buildbot:buildbot
move the /var/lib/buildbot directory to /home (or whatever
location you want to use to store things) and then link it back
update /etc/default/locale and change it to: LANG=en_US.UTF-8
monitor the rsync, wait for it to finish
once the rsync is finished, test the runvm script
Remove the "testtest" VMs we created above
Start the buildslave
ssh to ${buildmaster} and add this new host to kvm_slaves in
the maria-master.cfg file
still on ${buildmaster}, test and then reload buildbot
This page is licensed: CC BY-SA / Gnu FDL
The goal of MariaDB Foundation Buildbot is to ensure that the MariaDB Server is being thoroughly tested on all supported platforms and environments. We are currently running 100 different configurations for the following platforms:
x64 and x86
aarch64
ppc64le
s390x
and operating systems:
Debian 9, 10, 11 and Sid
Ubuntu 16.04, 18.04, 20.04 and 21.04
Fedora 33 and 34
CentOS 7 and 8
Moreover, we run other ecosystem tests for:
PHP
DBdeployer
MySqlJS
PyMySQL
Packages built by buildbot can be downloaded from .
The MariaDB Foundation uses Buildbot, a continuous integration and testing framework to test and create MariaDB Server packages. It is hosted onand ensures that each push to the MariaDB Server GitHub repository is properly tested.
Buildbot should be used by each MariaDB developer to ensure that the new changes that are made are properly tested on all supported platforms and environments. In order to enforce this, Buildbot is used for . However, even though branch protection is enabled, only a selected few builders are part of it. So, it is the developer's responsibility to monitor all the builders and make sure that everything is fine before making the final push to the main MariaDB branch.
Changes/Repository - Any change that occurs in the source code (commit)
Build Master - The main process that runs on a dedicated machine. It checks for changes in the source code and is in charge of scheduling builds
Build - The actual configuration that is tested. It consists of a sequence of steps that define the actual test (e.g. get source code, compile, run tests, etc)
Buildbot Worker - The process which waits for commands from the Build Master in order to run a build
As it comes to the Buildbot Master, we use a multi-master configuration. This means that we have multiple running master processes. So, we have a dedicated master for the user interface and several other that deal with looking for changes and scheduling builds.
Each time a push is made to the MariaDB Server Repository, it is detected by the buildbot master which schedules all the builds. Each build defines a different test configuration. We mainly use Docker Latent Workers which means that for each build, the master starts a Docker container on a remote machine. The container is configured to run the buildbot-worker process on startup. This process can now receive instructions from the master. In this way, by using latent workers there isn’t a buildbot-worker process continuously running on the worker machine. Instead, for each build a separate container is started.
Below, you can find a detailed step by step overview of what happens after a push is made to the MariaDB Server repository:
Step 1: Detect a new change in the MariaDB Server repository
Trigger source tarball creation
Step 2: Tarball creation
Clone the repository and create a source tarball corresponding to the latest changes
The information below refers to the old Buildbot (), and not the new Buildbot (). The information is old, outdated, or otherwise currently incorrect.
The current state of the MariaDB trees with respect to build or test failures is always available from the page.
The BuildBot setup polls the Launchpad trees every 5 minutes for changes. Whenever a new push is found in one of our trees, the new code is compiled and run through the test suite.
If all platforms are green after this, everything is good. If not, it means there is a problem with the push, and someone needs to look into it ASAP. If it was your push, then the someone who needs to look at it is you!
BuildBot is a generic, GPL'ed program providing a continuous integration test framework. For more information on BuildBot, see the .
Many of our build hosts are run by members, and we are always looking for additional volunteers to help us cover additional platforms or build options in BuildBot.
If you are able to provide a spare machine for this purpose, your help is greatly appreciated! This is a good way to get involved without having to spend a lot of time on it. Get started by writing an email to 'developers (at) lists.mariadb.org' with an offer to run a BuildBot slave.
See .
Sometimes you need to work when your computer is busy running tests for buildbot. We've added a new feature to the mysql-test-run.pl script which allows you to stop it temporarily so you can use your computer and then restart the tests when you're ready.
To do this, define the environment variable "MTR_STOP_FILE". Whenever the file specified by this environment variable exists, the mysql-test-run.pl script will stop as soon as it is able to do so (i.e. it won't stop immediately). When the file is removed, the mysql-test-run.pl script will continue from where it left off.
If you plan on using this feature you should also set the "MTR_STOP_KEEP_ALIVE" environment variable with a value of 120. This will make the script print messages to buildbot every 2 minutes which will prevent a timeout.
Buildbot saves the results of test runs in a database, to be used for enhanced reporting on web pages without need to change the Buildbot code, and for data mining when investigating test failures.
The database schema is documented under . The schema is likely to evolve as we gradually add more kinds of information.
For now, the data is not externally available. But the plan is to set up a slave database to replicate the data, and provide access (eg. remote database accounts) to members of the community with interesting ideas about how to present or mine this data, or who are just curious to play with it. If anyone has an interest in this, or wants to volunteer a slave host for this purpose, please send a . The more people show interest in this, the faster it is likely to happen!
We are developing new reports fed off the test results in the database. These reports will be located . The first report is the report. This report allows all test failures to be searched.
Here is some information on how our Buildbot installation is set up and maintained:
The configuration file is included in the repository.
The building and testing of binary packages is documented on the page.
We developed a small tool, , which is used to do some of the builds inside a virtual machine, mostly to test builds of binary packages.
The page describes how we developed some of the enhancements to BuildBot that we use and have contributed upstream.
The page
This page is licensed: CC BY-SA / Gnu FDL
If you just want to get the latest source and you don't require the ability to push changes to the MariaDB repository, you can use the following command to check out the latest 10.5 branch:
Set your name with git
Go to and add your public SSH key ().
Clone the repository
Config repository pull and alias for fast forward:
In git commit messages are normally formatted as
That is, the first line is considered a subject, much like email subject. Many git commands and pages on github only show the commit subject, not the complete comment. After the subject goes an empty line, then the detailed description of the comment. Please, structure your commit comments this way, don't forget the empty line.
This is an important concept, and git branches do not have equivalents in bzr.
In Bazaar, we all used to have one shared repository, within which there were many branches. This seems to be impossible with git?
In Git, each repository has only one branch that is currently checked out.
To see which branches exists locally and remotely:
Note that if the output from git branch --all is remotes/origin/XXX you should just use XXX as branch name.
If you get the following error on pull:
Instead of removing your copy and then clone, you can do:
Note: branches whose names start with bb- are automatically put into the buildbot.
CAVEAT UTILITOR. Check the manual before running!
bzr status is git status
bzr diff is git diff
bzr log is git log
GUIs
bzr gcommit is git citool
bzr viz is gitk
bzr gannotate is git gui blame
In the MariaDB project, it is a good idea (and a long tradition since MySQL Ab) to have all your commits sent to a commits@mariadb.org mailing list. It allows others to follow the progress, comment, etc.
A script and instructions on how to setup commit triggers in Git are here:. Jira task for commit trigger was .
If you add code on behalf of someone else, please attribute the code to the original author:
Run git citool and move changed files to staged.
Don't commit, abort instead
run git commit --author="Original author name <email_address>"
The above is needed as git citool can't handle the --author option.
At the end of the pull request page there is a button "Merge pull request" and next to it a link "command line instructions". Click the link, you'll see something like
Step 1: From your project repository, check out a new branch and test the changes.
Step 2: Merge the changes and update on GitHub.
Note where to pull from — https://github.com/joeuser/mariadb-server cool-feature.
Now, checkout the branch you want to merge it to, say, bb-10.3-stage, and do the following
Now's the time to compile the code, test it, fix, if necessary. Then
If you want to do small changes to the pull request, do it in a separate commit, after git rebase @{-1} above. If you want to do big changes to the pull request, perhaps you shouldn't merge it in the first place, but ask the contributor to fix it?
You are working on a branch (NEW_FEATURE) and would like to have into that branch all changes from the main branch (10.1).
You've just fixed and committed a bug in the main 10.1 branch and want to merge it with the latest 10.1. Often a rebase is better in this case. Assuming your current checked out branch is 10.1:
This will work even if you have done multiple commits in your local 10.1 tree.
What to do when you have fixed a bug in the main tree but notices that someone has changed the tree since you pulled last time. This approach ensures that your patch is done in one block and not spread out over several change sets.
This page is licensed: CC BY-SA / Gnu FDL
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
Set the hostname to ubuntu-raring-amd64 or ubuntu-raring-i386
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server
Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:
Enabling passwordless sudo:
Editing /boot/grub/menu.lst:
See for how to obtain my.seed and sources.append.
The steps below are based on the Natty steps on .
Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.
This page is licensed: CC BY-SA / Gnu FDL
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
Set the hostname to ubuntu-saucy-amd64 or ubuntu-saucy-i386
do not encrypt the home directory
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server
Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:
Enabling passwordless sudo:
Editing /boot/grub/menu.lst:
See for how to obtain my.seed and sources.append.
Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.
This page is licensed: CC BY-SA / Gnu FDL
Enabling passwordless sudo:
sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'Editing /boot/grub/menu.lst:
Install cmake (required for ) with:
Also:
See above for how to obtain my.seed and sources.append.
The steps below are based on the Natty steps on .
64-bit Ubuntu oneiric:
32-bit Ubuntu oneiric:
This page is licensed: CC BY-SA / Gnu FDL
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. Ubuntu has just resized the vnc screen. Simply reconnect.
During the install, pick default options for the most part, with the following notes:
Set the hostname to ubuntu-trusty-amd64 or ubuntu-trusty-i386
(depending on which architecture we're installing)
do not encrypt the home directory
When prompted if the timezone is correct, answer "No" and when the list comes up, scroll to the bottom and choose "UTC"
When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
No automatic updates
Choose software to install: OpenSSH server
Reboot when prompted. It will fail, so just kill the kvm process with^Ctrl+C.
Now that the VM is installed, it's time to configure it. If you have the memory you can do the following simultaneously:
set vim as the preferred editor
Enabling passwordless sudo:
edit /boot/grub/menu.lst:
create the .ssh directory and the sudo group
copy the authorized_keys file up to the VMs
create the buildbot user on the amd64 VM
create the buildbot user on the x86 VM
copy the ttyS0 file to the VMs
Apply all OS updates
copy the ttyS0 file to its proper location and shut down the VM
See for how to obtain my.seed and sources.append.
Do the following on each kvm host server (terrier, terrier2, i7, etc...) to add the VMs to known_hosts.
This page is licensed: CC BY-SA / Gnu FDL
Start each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The Fedora installer has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
Pages about buildbot
As an example, you can see what's happening with 11.3 .
for i in "vm-debian4-amd64-install qemu64 etch /kvm/debian-40r8-amd64-netinst.iso" "vm-debian4-i386-install qemu32,-nx etch /kvm/debian-40r8-i386-netinst.iso" ; do \
set $i; \
runvm -b $1.qcow2 -m 512 --smp=1 --port=2200 --user=buildbot --cpu=$2 --netdev=e1000 --kvm=-cdrom --kvm=$4 $(echo $1 | sed -e 's/-install/-upgrade2/').qcow2 \
"wget -O- http://ourdelta.org/deb/ourdelta.gpg | sudo apt-key add -" \
"sudo sh -c \"echo 'deb http://mirror.ourdelta.org/deb $3 mariadb-ourdelta' > /etc/apt/sources.list.d/ourdelta.list\"" \
"sudo apt-get update || true" \
"sudo sh -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-test libmariadbclient-dev' || true" \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
"sudo rm /etc/apt/sources.list.d/ourdelta.list" \
"sudo apt-get update || true" \
"sudo sh -c 'DEBIAN_FRONTEND=noninteractive apt-get upgrade -y' || true" ; \
doneqemu-img create -b vm-lucid-amd64-install.qcow2 -f qcow2 vm-lucid-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-lucid-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographicmkdir buildbotscp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-277/kvm-deb-lucid-amd64/debs buildbot@localhost:buildbot/sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -yqemu-img create -b vm-lucid-i386-install.qcow2 -f qcow2 vm-lucid-i386-upgrade2.qcow2
kvm -m 512 -hda vm-lucid-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu32,-nx -net nic,model=virtio -net user -nographicmkdir buildbotscp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-277/kvm-deb-lucid-x86/debs buildbot@localhost:buildbot/sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -yqemu-img create -b vm-maverick-amd64-install.qcow2 -f qcow2 vm-maverick-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-maverick-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost mkdir buildbot
scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-619/kvm-deb-maverick-amd64/debs buildbot@localhost:buildbot/
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -yqemu-img create -b vm-maverick-i386-install.qcow2 -f qcow2 vm-maverick-i386-upgrade2.qcow2
kvm -m 512 -hda vm-maverick-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost mkdir buildbot
scp -P 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /archive/pack/mariadb-5.1-knielsen/build-619/kvm-deb-maverick-x86/debs buildbot@localhost:buildbot/
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
rm -Rf buildbot
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -yqemu-img create -b vm-natty-amd64-install.qcow2 -f qcow2 vm-natty-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-natty-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -yqemu-img create -b vm-natty-i386-install.qcow2 -f qcow2 vm-natty-i386-upgrade2.qcow2
kvm -m 512 -hda vm-natty-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.1/ubuntu maverick main
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -ycd <somewhere>
mkdir buildbot
cd buildbot
bzr init-repo .
C:\buildbot\buildbot-slave-0.8.3\build\scripts-2.7\buildslave.bat create-slave --usepty=0 <slavedir> hasky.askmonty.org:9989 <slavename> <passwd>C:\buildbot\buildbot-slave-0.8.3\build\scripts-2.7\buildslave.bat start <somewhere>\buildbot\<slavedir>cl dojob.cppfactory.addStep(Compile(
name = "cmake",
command=["dojob", WithProperties("cd c:\\buildbot\\%(buildername)s\\build && cmake .")]
));f_win2008r2_i386_release.addStep(ShellCommand(
command=["ssh", "-i", "/home/buildbot/keys/id_rsa", "buildbot@win2008r2-build", "cmd", "/C", "whoami"]
));qemu-img create -f qcow2 /kvm/vms/vm-centos6-amd64-serial.qcow2 8G
qemu-img create -f qcow2 /kvm/vms/vm-centos6-i386-serial.qcow2 8Gkvm -m 1024 -hda /kvm/vms/vm-centos6-amd64-serial.qcow2 -cdrom /kvm/iso/centos/CentOS-6.2-x86_64-bin-DVD1.iso -redir tcp:22255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-centos6-i386-serial.qcow2 -cdrom /kvm/iso/centos/CentOS-6.2-i386-bin-DVD1.iso -redir tcp:22256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net uservncviewer -via ${remote-host} localhostqemu-img create -f qcow2 /kvm/vms/vm-fedora17-i386-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-fedora17-amd64-serial.qcow2 10Gkvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-17-i386-DVD.iso -redir tcp:2265::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-17-x86_64-DVD.iso -redir tcp:2266::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net uservncviewer -via ${remote-host} localhostssh <host>.mariadb.net
sudo apt-get update
sudo apt-get dist-upgradegit clone -b 10.5 https://github.com/MariaDB/server.gitqemu-img create -f qcow2 /kvm/vms/vm-raring-amd64-serial.qcow2 15G
qemu-img create -f qcow2 /kvm/vms/vm-raring-i386-serial.qcow2 15Gkvm -m 2048 -hda /kvm/vms/vm-raring-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-amd64.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2279-:22
kvm -m 2048 -hda /kvm/vms/vm-raring-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-i386.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2280-:22vncviewer -via ${remote-host} localhostqemu-img create -f qcow2 /kvm/vms/vm-saucy-amd64-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-saucy-i386-serial.qcow2 20Gkvm -m 2048 -hda /kvm/vms/vm-saucy-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-amd64.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2287-:22
kvm -m 2048 -hda /kvm/vms/vm-saucy-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-i386.iso -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2288-:22vncviewer -via ${remote_host} localhostqemu-img create -f qcow2 /kvm/vms/vm-oneiric-amd64-serial.qcow2 8G
qemu-img create -f qcow2 /kvm/vms/vm-oneiric-i386-serial.qcow2 8G
# Start each VM booting from the server install iso one at a time and perform
# the following install steps:
kvm -m 1024 -hda /kvm/vms/vm-oneiric-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-amd64.iso -redir tcp:2255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-oneiric-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-i386.iso -redir tcp:2256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Once running you can connect to the VNC server from your local host with:
#
# vncviewer -via ${remote-host} localhost
#
# (replace ${remote-host} with the host the vm is running on)
# Install, picking default options mostly, with the following notes:
# * Set the hostname to ubuntu-oneiric
# * When partitioning disks, choose "Guided - use entire disk" (we do not want LVM)
# * No automatic updates
# * Choose software to install: OpenSSH server
# Now that the VM is installed, it's time to configure it.
# If you have the memory you can do the following simultaneously:
kvm -m 1024 -hda /kvm/vms/vm-oneiric-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-amd64.iso -redir tcp:2255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-oneiric-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.10-server-i386.iso -redir tcp:2256::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2255 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -p 2256 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2255 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2256 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
scp -P 2255 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2256 /kvm/vms/authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2255 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2256 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
scp -P 2255 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2256 /kvm/vms/ttyS0.conf buildbot@localhost:
ssh -p 2255 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2256 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grubfor i in '/kvm/vms/vm-oneiric-amd64-serial.qcow2 2255 qemu64' '/kvm/vms/vm-oneiric-i386-serial.qcow2 2256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
doneqemu-img create -f qcow2 /kvm/vms/vm-trusty-amd64-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-trusty-i386-serial.qcow2 20Gkvm -m 2048 -hda /kvm/vms/vm-trusty-amd64-serial.qcow2 \
-cdrom /kvm/iso/ubuntu/trusty-server-amd64.iso \
-boot d -smp 2 -cpu qemu64 \
-net nic,model=virtio \
-net user,hostfwd=tcp:127.0.0.1:2293-:22
kvm -m 2048 -hda /kvm/vms/vm-trusty-i386-serial.qcow2 \
-cdrom /kvm/iso/ubuntu/trusty-server-i386.iso \
-boot d -smp 2 -cpu qemu64 \
-net nic,model=virtio \
-net user,hostfwd=tcp:127.0.0.1:2294-:22vncviewer -via ${remote_host} localhostSLES 12 and 15
OpenSUSE 15 and 42
Windows
AIX-7.2
Trigger bintar builds
Step 3: Bintar builds
Fetch the source tarball previously created
Compile
Test (mysql-test-run)
Save bintar
Trigger package creation builds
Trigger ecosystem builds
Step 4.1: Package creation
Fetch source
Create packages
Save packages
Trigger installation builds
Step 4.2: Ecosystem tests
Step 5: Installation builds
Fetch packages
Test if they install successfully
Trigger upgrade builds
Step 6: Upgrade builds
Test if the previously MariaDB Server version can be successfully upgraded to the current one

This section documents the exact steps used to set up each virtual machine used in the MariaDB Buildbot for building packages and testing. There are lots of boring details, but the information here should be useful for reference, and as a help when installing new virtual images for additional distros and versions.
The articles here document how the virtual machine images were actually set up, not the most efficient way to recreate all of them. Thus, some historical artifacts remain, reflecting the order they were initially added and lessons learned along the way.
For example, the steps for the earliest VMs are somewhat overly enthusiastic
about using the qemu-img create -b option to make one vm a copy-on-write copy
on top of another image. This makes it harder to modify the images afterwards,
since if the base image is modified, any copy-on-write vm on top of it needs to
be re-created.
The descriptions do not go into painful detail about the installation procedure. Generally the .iso for the distro was downloaded, and the default, non-graphical installer started. Installing Linux these days is quite easy, and generally just defaults were picked. With respect to harddisk partitioning, generally an 8Gb image was configured, of which 1/2 Gb was reserved for swap and the rest for a single root partition "/".
For details on setting up a host server to run these VMs, see Buildbot Setup for VM host.
sda2
5672
/
ext4
yes
sda3
(max allowable)
(n/a)
swap
yes
bzr revert is git reset --hard
bzr revert filename is git checkout filename
bzr parent is git remote -v (but there are more detailed commands)
bzr parent to-default-mariadb-repo git remote set-url origin git@github.com:MariaDB/server.git
bzr push is git push REMOTENAME BRANCHNAME. REMOTENAME is typically "origin", for example: git push origin HEAD:10.3-new-feature. The HEAD: stands for "from current branch".
bzr clean-tree --ignored is git clean -Xdf (note the capital X!)
bzr root is git rev-parse --show-toplevel
bzr missing --mine-only is git cherry -v origin (kind-of).
Under "Network Configuration" set the hostnames to fedora19-amd64 and fedora19-i386
Change "Software Selection" to "Minimal Install" (default is "Gnome Desktop")
For "Installation Destination" select the disk then click continue.
do not check the encryption checkbox
On "Installation Options" screen, expand the "Partition scheme configuration" box and select a "Partition type" of "Standard Partition"
While installing, set the root password and an initial user
When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:
Log in using the initial user created during the install:
After logging in, add the initial user to the "wheel" group.
Enable password-less sudo for the "wheel" group and serial console sudo:
Edit /boot/grub/menu.lst:
Logout as root, and then, from the VM host server:
Create a .ssh folder:
Copy over the authorized keys file:
Set permissions on the .ssh folder correctly:
Create the buildbot user:
su to the local buildbot user and ssh to the vm to put the key in known_hosts:
For i386:
For amd64:
Upload the ttyS0 file and put it where it goes:
Update the VM:
Once logged in:
Change selinux policy to permissive:
After updating, shut down the VM:
This page is licensed: CC BY-SA / Gnu FDL
kvm -m 1024 -hda /kvm/vms/vm-centos6-amd64-serial.qcow2 -redir tcp:22255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-centos6-i386-serial.qcow2 -redir tcp:22256::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net userssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhostadduser ${username}
usermod -a -G wheel ${username}
passwd ${username}visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requirettyserial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=3 serial consoleconsole=tty0 console=ttyS0,115200n8cat >>/etc/inittab <<END
# Serial console.
S0:2345:respawn:/sbin/agetty -h -L ttyS0 19200 vt100
ENDssh -t -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "sudo yum install openssh-server openssh-clients"
ssh -t -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "sudo yum install openssh-server openssh-clients"ssh -t -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"scp -P 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/ssh -t -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"ssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'scp -P 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0.conf buildbot@localhost:
ssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf;'
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf;'ssh -p 22255 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 22256 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum updatesudo shutdown -h nowfor i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo yum -y groupinstall 'Development Tools'" \
"sudo yum -y install wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel libxml2 libxml2-devel unixODBC-devel bzr perl perl\(DBI\)" \
"sudo yum -y remove systemtap-sdt-dev" \
"bzr co --lightweight lp:mariadb-native-client" \
"sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
"wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz;tar -zxvf cmake-2.8.8.tar.gz;cd cmake-2.8.8;./configure;make;sudo make install"; \
donefor i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo yum -y update" \
"sudo yum -y install patch libaio perl perl-Time-HiRes perl-DBI libtool-ltdl unixODBC" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
"sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
donefor i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo yum -y update" \
'sudo yum -y install mysql-server libtool-ltdl unixODBC' \
'sudo /etc/init.d/mysqld start' \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
"sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
donefor i in '/kvm/vms/vm-centos6-amd64-serial.qcow2 22255 qemu64' '/kvm/vms/vm-centos6-i386-serial.qcow2 22256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.repo buildbot@localhost:/tmp/" \
'sudo rpm --verbose --import http://downloads.mariadb.org/repo/RPM-GPG-KEY-MariaDB' \
'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
'sudo yum -y remove mysql-libs' \
'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test' \
'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix' \
'sudo /etc/init.d/mysqld start' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
donekvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-serial.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-serial.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net userssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost/sbin/chkconfig --level 35 network on
ifup eth0
yum install openssh-server openssh-clients
adduser ${username}
usermod -a -G wheel ${username}
passwd ${username}visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requirettysudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
grub2-mkconfig -o /boot/grub2/grub.cfgssh -t -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"scp -P 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/ssh -t -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'sudo su - buildbot
ssh -p 2265 buildbot@localhost
# exit, then exit againsudo su - buildbot
ssh -p 2266 buildbot@localhost
# exit, then exit againscp -P 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum updatesudo shutdown -h nowqemu-img create -b /kvm/vms/vm-fedora17-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-i386-build.qcow2
qemu-img create -b /kvm/vms/vm-fedora17-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-amd64-build.qcow2kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-build.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-build.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographicscp -P 2265 -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/
scp -P 2266 -i /kvm/vms/ssh-keys/id_dsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/ssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum -y groupinstall 'Development Tools'
sudo yum -y install tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel perl perl\(DBI\)
sudo yum -y remove systemtap-sdt-dev
sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS
# cmake
wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz;tar -zxvf cmake-2.8.8.tar.gz;cd cmake-2.8.8;./configure;make;sudo make install
# boost
cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -s ../src/boost_1_49_0/boost .
# shutdown the VM
sudo shutdown -h nowqemu-img create -b /kvm/vms/vm-fedora17-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-i386-install.qcow2
qemu-img create -b /kvm/vms/vm-fedora17-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-amd64-install.qcow2kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-install.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-install.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographicssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum -y update
sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI
# shutdown the VM
sudo shutdown -h nowqemu-img create -b /kvm/vms/vm-fedora17-i386-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-i386-upgrade.qcow2
qemu-img create -b /kvm/vms/vm-fedora17-amd64-serial.qcow2 -f qcow2 /kvm/vms/vm-fedora17-amd64-upgrade.qcow2kvm -m 1024 -hda /kvm/vms/vm-fedora17-i386-upgrade.qcow2 -redir tcp:2265::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
kvm -m 1024 -hda /kvm/vms/vm-fedora17-amd64-upgrade.qcow2 -redir tcp:2266::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographicssh -p 2265 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2266 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum -y update
sudo yum -y install tar mysql-server
sudo systemctl enable mysqld.service
sudo systemctl start mysqld.service
mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
# shutdown the VM
sudo shutdown -h nowfor i in '/kvm/vms/vm-fedora17-amd64-serial.qcow2 2266 qemu64' '/kvm/vms/vm-fedora17-i386-serial.qcow2 2265 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.repo buildbot@localhost:/tmp/" \
'sudo rpm --verbose --import http://downloads.mariadb.org/repo/RPM-GPG-KEY-MariaDB' \
'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
'sudo yum -y remove mysql-libs' \
'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test' \
'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix' \
'sudo /etc/init.d/mysqld start' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
donesudo apt-get install tree renameutils vim-noxvi /etc/rsyncd.confvm_master="hostname" # ask for this from a buildbot admin
mkdir /home/kvm
ln -sv /home/kvm /kvm
rsync --dry-run --delete --exclude=deprecated \\
--exclude=iso --exclude=lost+found -avPL ${vm_master}::kvm/ /home/kvm/
# assuming the dry-run looks good, start the "real" rsync in a screen session
screen
rsync --delete --exclude=deprecated --exclude=iso --exclude=lost+found -avPL ${vm_master}::kvm/ /home/kvm/update-alternatives --config editorsudo apt-get install bzr git buildbot-slave qemu kvm
sudo apt-get install libsdl2-2.0-0username="mydefaultusername"
sudo adduser ${username}
for group in sudo tty kvm;do
sudo adduser ${username} ${group}
donesudo perl -i -pe "s/#PasswordAuthentication yes/PasswordAuthentication no/" /etc/ssh/sshd_config
sudo perl -i -pe "s/PermitRootLogin yes/PermitRootLogin no/" /etc/ssh/sshd_config
sudo /etc/init.d/ssh restartmkdir ~/src
cd ~/src/
bzr branch lp:mariadb-toolssudo cp -v ~/src/mariadb-tools/buildbot/runvm /usr/local/bin/
ls -l /usr/local/bin/sudo adduser buildbot kvm
sudo adduser buildbot ttysudo buildslave create-slave /var/lib/buildbot/slaves/maria buildbot.askmonty.org ${host} ${password}HOME=/var/lib/buildbot
SLAVE_ENABLED[1]=1
SLAVE_NAME[1]="${hostname} maria slave"
SLAVE_USER[1]="buildbot"
SLAVE_BASEDIR[1]="$HOME/slaves/maria"
SLAVE_OPTIONS[1]=""
SLAVE_PREFIXCMD[1]=""sudo vi /var/lib/buildbot/slaves/maria/info/*scp terrier.askmonty.org:buildbot-ssh.tar.gz .
cd /var/lib/buildbot
sudo tar -zxvf ~/buildbot-ssh.tar.gz
sudo chown -Rv buildbot: .ssh
sudo chmod -v 700 .ssh
sudo chmod -Rv go-r .sshsudo su - buildbot
cp -v /etc/skel/.bash* .
cp -v /etc/skel/.profile .
exitsudo chown -Rv buildbot:buildbot ~buildbot/slavessudo mv -vi /var/lib/buildbot /home/;cd /var/lib/;sudo ln -sv /home/buildbot ./sudo vi /etc/default/locale
sudo locale-gensudo su - buildbot
for i in '/kvm/vms/vm-xenial-amd64-serial.qcow2 6666 qemu64' '/kvm/vms/vm-xenial-i386-serial.qcow2 6666 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/testtest/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils unixodbc libxml2 libjudydebian1" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donerm -v /kvm/vms/*testtest*sudo /etc/init.d/buildslave start
tail -f ~buildbot/slaves/maria/twistd.logsudo vi /etc/buildbot/maria-master.cfgcd /etc/buildbot
sudo -u buildbot PYTHONPATH=/usr/local/buildbot/lib/python python -c 'exec open("maria-master.cfg", "r")'
sudo /etc/init.d/buildmaster reload
sudo tail -f /var/lib/buildbot/maria/twistd.loggit config --global user.name "Ivan Ivanov"
git config --global user.email "ivan@mariadb.com"git clone git@github.com:MariaDB/server.git
cd server
git checkout 10.5git config pull.ff only
git config --global alias.ff "merge --ff-only"subject
body
more body
...git branchgit branch --allgit checkout branch-namebranch clone old_directory new_directory
cd new_directory
git remote set-url origin git@github.com:MariaDB/server.git
git pullgit reset --hard HEAD^shell> git pull
X11 forwarding request failed on channel 0
fatal: Not possible to fast-forward, aborting.git reset --hard origin/##branch-name##git checkout -b mariadb-server-joeuser-cool-feature 10.3
git pull https://github.com/joeuser/mariadb-server cool-featuregit checkout 10.3
git merge --no-ff mariadb-server-joeuser-cool-feature
git push origin 10.3git fetch https://github.com/joeuser/mariadb-server cool-feature
git checkout FETCH_HEAD
git rebase @{-1}git checkout @{-1}
git ff @{-1}git showgit checkout 10.1
git pull
git checkout NEW_FEATURE
git rebase 10.1git fetch origin
git rebase origin/10.1git clone 10.1
cd 10.1
< fix a bug here>
git citool
git push
# ^ and the above fails, because someone has pushed to 10.1 in between
git branch tmp
# ^ copy our work to branch named 'tmp'
get checkout 10.1
git reset --hard HEAD^
# ^ remove our work from '10.1' local branch'
git pull
# ^ get changes from remote
git checkout tmp
git rebase 10.1
# ^ switch to 'tmp' and try to rebase 'tmp' branch on top of 10.1 branch.
# here you will be asked to merge if necessary
git checkout 10.1
git pull --ff . tmp
# ^ switch back to the '10.1' branch, and pull from 'tmp' branch.
git branch -D tmp
#^ this removes the tmp. branch
git pushsh> git tag --contain e65f667bb60244610512efd7491fc77eccceb9db
mariadb-10.0.30
mariadb-10.1.22
mariadb-10.1.23
mariadb-10.2.5
mariadb-10.3.0
mariadb-galera-10.0.30kvm -m 2048 -hda /kvm/vms/vm-raring-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-amd64.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2279-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-raring-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.04-server-i386.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2280-:22 -nographic
ssh -p 2279 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -p 2280 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2279 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2280 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
scp -P 2279 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2280 /kvm/vms/authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2279 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2280 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
scp -P 2279 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2280 /kvm/vms/ttyS0.conf buildbot@localhost:
ssh -p 2279 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2280 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2279 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2280 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_HIDDEN_TIMEOUT_QUIET=false
GRUB_TIMEOUT=0
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grub
# exit back to the host serverfor i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/boost_1_49_0.tar.gz buildbot@localhost:/dev/shm/" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev bzr" \
"bzr co --lightweight lp:mariadb-native-client" \
"cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
"cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" \
"cd /usr/local/src;sudo tar zxf /dev/shm/boost_1_49_0.tar.gz;cd /usr/local/include/;sudo ln -vs ../src/boost_1_49_0/boost ." ; \
donefor i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
donefor i in '/kvm/vms/vm-raring-amd64-serial.qcow2 2279 qemu64' '/kvm/vms/vm-raring-i386-serial.qcow2 2280 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"= scp -P $2 /kvm/vms/mariadb-raring.list buildbot@localhost:/tmp/tmp.list" \
"sudo debconf-set-selections /tmp/my55.seed" \
'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
'sudo apt-key adv --recv-keys --keyserver pgp.mit.edu 0xcbcb082a1bb943db' \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-server-5.5 mariadb-client mariadb-client-5.5 mariadb-test libmariadbclient-dev mysql-common=5.5.30-mariadb1~quantal libmysqlclient18=5.5.30-mariadb1~quantal' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done# raring-amd64
cp -avi /kvm/vms/vm-raring-amd64-install.qcow2 /kvm/vms/vm-raring-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-raring-amd64-test.qcow2 -redir tcp:2279::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2279 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-raring-amd64-test.qcow2
# raring-i386
cp -avi /kvm/vms/vm-raring-i386-install.qcow2 /kvm/vms/vm-raring-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-raring-i386-test.qcow2 -redir tcp:2280::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2280 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-raring-i386-test.qcow2kvm -m 2048 -hda /kvm/vms/vm-saucy-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-amd64.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2287-:22 -nographic
kvm -m 2048 -hda /kvm/vms/vm-saucy-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu/ubuntu-13.10-server-i386.iso -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:2288-:22 -nographic
ssh -p 2287 localhost
sudo update-alternatives --config editor
# edit /boot/grub/menu.lst and visudo, see below
ssh -p 2288 localhost
sudo update-alternatives --config editor
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2287 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2288 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
scp -P 2287 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2288 /kvm/vms/authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2287 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2288 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
scp -P 2287 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -P 2288 /kvm/vms/ttyS0.conf buildbot@localhost:
ssh -p 2287 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2288 buildbot@localhost 'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -p 2287 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -p 2288 buildbot@localhost 'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_HIDDEN_TIMEOUT_QUIET=false
GRUB_TIMEOUT=0
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grub
# exit back to the host serverfor i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev bzr scons check libboost-all-dev openssl epm" \
"bzr co --lightweight lp:mariadb-native-client" \
"cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
"cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
donefor i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils unixodbc libxml2" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
donefor i in '/kvm/vms/vm-saucy-amd64-serial.qcow2 2287 qemu64' '/kvm/vms/vm-saucy-i386-serial.qcow2 2288 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"= scp -P $2 /kvm/vms/mariadb-saucy.list buildbot@localhost:/tmp/tmp.list" \
"sudo debconf-set-selections /tmp/my55.seed" \
'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
'sudo apt-key adv --recv-keys --keyserver pgp.mit.edu 0xcbcb082a1bb943db' \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done# saucy-amd64
cp -avi /kvm/vms/vm-saucy-amd64-install.qcow2 /kvm/vms/vm-saucy-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-saucy-amd64-test.qcow2 -redir tcp:2287::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2287 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-saucy-amd64-test.qcow2
# saucy-i386
cp -avi /kvm/vms/vm-saucy-i386-install.qcow2 /kvm/vms/vm-saucy-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-saucy-i386-test.qcow2 -redir tcp:2288::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2288 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-saucy-i386-test.qcow2qemu-system-x86_64 -m 2048 -hda /kvm/vms/vm-trusty-amd64-serial.qcow2 \
-boot c -smp 2 -cpu qemu64 \
-net nic,model=virtio \
-net user,hostfwd=tcp:127.0.0.1:2293-:22 \
-nographic
qemu-system-x86_64 -m 2048 -hda /kvm/vms/vm-trusty-i386-serial.qcow2 \
-boot c -smp 2 -cpu qemu64 \
-net nic,model=virtio \
-net user,hostfwd=tcp:127.0.0.1:2294-:22 \
-nographicssh -t -p 2293 localhost sudo update-alternatives --config editor
ssh -t -p 2294 localhost sudo update-alternatives --config editorsudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_HIDDEN_TIMEOUT_QUIET=false
GRUB_TIMEOUT=0
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grub
# exit back to the host serverssh -t -p 2293 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"
ssh -t -p 2294 localhost "mkdir -v .ssh; sudo addgroup $USER sudo"scp -P 2293 /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2294 /kvm/vms/authorized_keys localhost:.ssh/echo $'Buildbot\n\n\n\n\ny' | ssh -p 2293 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'echo $'Buildbot\n\n\n\n\ny' | ssh -p 2294 localhost 'chmod -vR go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir -v ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'scp -i -P 2293 /kvm/vms/ttyS0.conf buildbot@localhost:
scp -i -P 2294 /kvm/vms/ttyS0.conf buildbot@localhost:ssh -t -i -p 2293 buildbot@localhost \
'sudo apt-get update && sudo apt-get -y dist-upgrade;'
ssh -t -i -p 2294 buildbot@localhost \
'sudo apt-get update && sudo apt-get -y dist-upgrade;'ssh -t -i -p 2293 buildbot@localhost \
'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'
ssh -t -i -p 2294 buildbot@localhost \
'sudo cp -vi ttyS0.conf /etc/init/; rm -v ttyS0.conf; sudo shutdown -h now'for i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.5" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libpam0g-dev libreadline-gplv2-dev autoconf automake automake1.9 dpatch ghostscript-x libfontenc1 libjpeg62 libltdl-dev libltdl7 libmail-sendmail-perl libxfont1 lmodern texlive-latex-base-doc ttf-dejavu ttf-dejavu-extra libaio-dev xfonts-encodings xfonts-utils libxml2-dev unixodbc-dev bzr scons check libboost-all-dev openssl epm libjudy-dev libjemalloc-dev" \
"bzr co --lightweight lp:mariadb-native-client" \
"cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
"cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
donefor i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils unixodbc libxml2 libjudydebian1 libjemalloc1" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my55.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.5 libjemalloc1' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
donefor i in '/kvm/vms/vm-trusty-amd64-serial.qcow2 2293 qemu64' '/kvm/vms/vm-trusty-i386-serial.qcow2 2294 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
"= scp -P $2 /kvm/vms/my55.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"= scp -P $2 /kvm/vms/mariadb-trusty.list buildbot@localhost:/tmp/tmp.list" \
"sudo debconf-set-selections /tmp/my55.seed" \
'sudo mv -vi /tmp/tmp.list /etc/apt/sources.list.d/' \
'sudo apt-key adv --recv-keys --keyserver pgp.mit.edu 0xcbcb082a1bb943db' \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server libjemalloc1' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/apt/sources.list.d/tmp.list' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get update' \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y patch libaio1 debconf-utils' \
'sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y'; \
done# trusty-amd64
cp -avi /kvm/vms/vm-trusty-amd64-install.qcow2 /kvm/vms/vm-trusty-amd64-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-trusty-amd64-test.qcow2 -redir tcp:2293::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2293 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-trusty-amd64-test.qcow2
# trusty-i386
cp -avi /kvm/vms/vm-trusty-i386-install.qcow2 /kvm/vms/vm-trusty-i386-test.qcow2
kvm -m 1024 -hda /kvm/vms/vm-trusty-i386-test.qcow2 -redir tcp:2294::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
sudo su - buildbot
ssh -p 2294 buildbot@localhost sudo shutdown -h now
# answer "yes" when prompted
exit # the buildbot user
rm -v /kvm/vms/vm-trusty-i386-test.qcow2qemu-img create -f qcow2 /kvm/vms/vm-fedora19-i386-serial.qcow2 20G
qemu-img create -f qcow2 /kvm/vms/vm-fedora19-amd64-serial.qcow2 20Gkvm -m 2048 -hda /kvm/vms/vm-fedora19-i386-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-19-i386-DVD.iso -redir tcp:2289::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora19-amd64-serial.qcow2 -cdrom /ds413/iso/fedora/Fedora-19-x86_64-DVD.iso -redir tcp:2290::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net uservncviewer -via ${remote-host} localhostkvm -m 2048 -hda /kvm/vms/vm-fedora19-i386-serial.qcow2 -redir tcp:2289::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user
kvm -m 2048 -hda /kvm/vms/vm-fedora19-amd64-serial.qcow2 -redir tcp:2290::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net userssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhostusermod -a -G wheel ${username}visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requirettysudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
grub2-mkconfig -o /boot/grub2/grub.cfgssh -t -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"scp -P 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/ssh -t -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"ssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'sudo su - buildbot
ssh -p 2289 buildbot@localhost
# exit, then exit againsudo su - buildbot
ssh -p 2290 buildbot@localhost
# exit, then exit againscp -P 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
ssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mkdir -v /etc/event.d;sudo mv -vi ttyS0 /etc/event.d/;'ssh -p 2289 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2290 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum updatesudo vi /etc/selinux/config
# change:
SELINUX=enforcing
# to:
SELINUX=permissivesudo shutdown -h nowfor i in '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/thrift-0.9.0.tar.gz buildbot@localhost:/dev/shm/" \
"sudo yum -y groupinstall 'Development Tools'" \
"sudo yum -y install yum-utils" \
"sudo yum-builddep -y mariadb" \
"sudo yum -y install automake libtool flex pkgconfig gcc-c++ libevent-devel python-devel ruby-devel rpm-build" \
"sudo yum -y install cmake tar wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel" \
"sudo yum -y install libxml2 libxml2-devel bison bison-devel boost-devel unixODBC-devel bzr perl perl\(DBI\)" \
"sudo yum -y remove systemtap-sdt-dev" \
"bzr co --lightweight lp:mariadb-native-client" \
"sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
"cd /usr/local/src;sudo tar zxf /dev/shm/thrift-0.9.0.tar.gz;pwd;ls" \
"cd /usr/local/src/thrift-0.9.0;echo;pwd;sudo ./configure --prefix=/usr --enable-shared=no --enable-static=yes CXXFLAGS=-fPIC CFLAGS=-fPIC && echo && echo 'now making' && echo && sleep 3 && sudo make && echo && echo 'now installing' && echo && sleep 3 && sudo make install" ; \
donefor i in '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo yum -y update" \
"sudo yum -y install patch tar libaio perl perl-Time-HiRes perl-DBI unixODBC" ; \
donefor i in '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo yum -y update" \
"sudo yum -y install patch tar mysql-server libtool-ltdl unixODBC" \
"sudo systemctl enable mysqld.service" \
"sudo systemctl start mysqld.service" \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ; \
donefor i in '/kvm/vms/vm-fedora19-amd64-serial.qcow2 2290 qemu64' '/kvm/vms/vm-fedora19-i386-serial.qcow2 2289 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB-${2}.repo buildbot@localhost:/tmp/MariaDB.repo" \
'sudo rpm --verbose --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB' \
'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
'sudo yum -y remove mysql-libs' \
'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test libtool-ltdl unixODBC' \
'sudo yum -y install cronie cronie-anacron crontabs.noarch patch tar' \
'sudo /etc/init.d/mysql start' \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
donekvm -m 1024 -hda ${vm} -redir tcp:22565::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -o IdentityFile=.ssh/buildbot.id_dsa -p 22565 buildbot@localhost
wget http://www.cmake.org/files/v2.8/cmake-2.8.6.tar.gz
tar -zxvf cmake-2.8.6.tar.gz
cd cmake-2.8.6;./configure
make
sudo make install
sudo /sbin/shutdown -h nowfor i in '/kvm/vms/vm-oneiric-amd64-serial.qcow2 2255 qemu64' '/kvm/vms/vm-oneiric-i386-serial.qcow2 2256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
"= scp -P $2 /kvm/vms/my.seed /kvm/vms/sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in '/kvm/vms/vm-oneiric-amd64-install.qcow2 2255 qemu64' '/kvm/vms/vm-oneiric-i386-install.qcow2 2256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
doneqemu-img create -b vm-oneiric-amd64-install.qcow2 -f qcow2 vm-oneiric-amd64-upgrade2.qcow2
kvm -m 512 -hda vm-oneiric-amd64-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo rm /etc/apparmor.d/usr.sbin.mysqld
sudo /etc/init.d/apparmor restart
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h nowqemu-img create -b vm-oneiric-i386-install.qcow2 -f qcow2 vm-oneiric-i386-upgrade2.qcow2
kvm -m 512 -hda vm-oneiric-i386-upgrade2.qcow2 -redir 'tcp:2200::22' -boot c -smp 1 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2200 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
sudo sh -c 'cat > /etc/apt/sources.list.d/tmp.list'
deb http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main
deb-src http://ftp.osuosl.org/pub/mariadb/repo/5.2/ubuntu natty main
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1BB943DB
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated mariadb-server mariadb-test libmariadbclient-dev
sudo rm /etc/apparmor.d/usr.sbin.mysqld
sudo /etc/init.d/apparmor restart
sudo apt-get -f install
mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"
sudo rm /etc/apt/sources.list.d/tmp.list
sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
sudo shutdown -h nowSee above for how to obtain my.seed and sources.append.
This page is licensed: CC BY-SA / Gnu FDL
qemu-img create -f qcow2 vm-natty-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-natty-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-amd64.iso -redir tcp:2255::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-natty-amd64-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-amd64.iso -redir tcp:2255::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2255 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2255 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2255 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2255 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2255 ttyS0.conf buildbot@localhost:
ssh -p 2255 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'See the General Principles
article for how to make the 'my.seed' and 'sources.append' files.
This page is licensed: CC BY-SA / Gnu FDL
qemu-img create -f qcow2 vm-maverick-amd64-serial.qcow2 8G
kvm -m 1024 -hda vm-maverick-amd64-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-amd64.iso -redir tcp:2246::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-maverick-amd64-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-amd64.iso -redir tcp:2246::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2246 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2246 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2246 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2246 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2246 ttyS0.conf buildbot@localhost:
ssh -p 2246 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'qemu-img create -f qcow2 vm-natty-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-natty-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-i386.iso -redir tcp:2256::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-natty-i386-serial.qcow2 -cdrom /kvm/iso/ubuntu-11.04-server-i386.iso -redir tcp:2256::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net user -nographic
ssh -p 2256 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2256 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2256 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2256 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2256 ttyS0.conf buildbot@localhost:
ssh -p 2256 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grubfor i in 'vm-natty-amd64-serial.qcow2 2255 qemu64' 'vm-natty-i386-serial.qcow2 2256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
donefor i in 'vm-natty-amd64-serial.qcow2 2255 qemu64' 'vm-natty-i386-serial.qcow2 2256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
"= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in 'vm-natty-amd64-install.qcow2 2255 qemu64' 'vm-natty-i386-install.qcow2 2256 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
doneqemu-img create -f qcow2 vm-maverick-i386-serial.qcow2 8G
kvm -m 1024 -hda vm-maverick-i386-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-i386.iso -redir tcp:2247::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
# Install, picking default options mostly, only adding openssh server.
kvm -m 1024 -hda vm-maverick-i386-serial.qcow2 -cdrom /kvm/ubuntu-10.10-server-i386.iso -redir tcp:2247::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user -nographic
ssh -p 2247 localhost
# edit /boot/grub/menu.lst and visudo, see below
ssh -t -p 2247 localhost "mkdir .ssh; sudo addgroup $USER sudo"
scp -P 2247 authorized_keys localhost:.ssh/
echo $'Buildbot\n\n\n\n\ny' | ssh -p 2247 localhost 'chmod -R go-rwx .ssh; sudo adduser --disabled-password buildbot; sudo addgroup buildbot sudo; sudo mkdir ~buildbot/.ssh; sudo cp .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -R buildbot:buildbot ~buildbot/.ssh; sudo chmod -R go-rwx ~buildbot/.ssh'
scp -P 2247 ttyS0.conf buildbot@localhost:
ssh -p 2247 buildbot@localhost 'sudo cp ttyS0.conf /etc/init/; rm ttyS0.conf; sudo shutdown -h now'sudo VISUAL=vi visudo
# Add line at end: `%sudo ALL=NOPASSWD: ALL'sudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
sudo update-grubfor i in 'vm-maverick-amd64-serial.qcow2 2246 qemu64' 'vm-maverick-i386-serial.qcow2 2247 qemu32,-nx' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get -y build-dep mysql-server-5.1" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y devscripts hardening-wrapper fakeroot doxygen texlive-latex-base ghostscript libevent-dev libssl-dev zlib1g-dev libreadline6-dev" ; \
donecat >sources.append <<END
deb file:///home/buildbot/buildbot/debs binary/
deb-src file:///home/buildbot/buildbot/debs source/
ENDfor i in 'vm-maverick-amd64-serial.qcow2 2246 qemu64' 'vm-maverick-i386-serial.qcow2 2247 qemu32,-nx' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get update" \
"sudo DEBIAN_FRONTEND=noninteractive apt-get install -y debconf-utils" \
"= scp -P $2 my.seed sources.append buildbot@localhost:/tmp/" \
"sudo debconf-set-selections /tmp/my.seed" \
"sudo sh -c 'cat /tmp/sources.append >> /etc/apt/sources.list'"; \
donefor i in 'vm-maverick-amd64-install.qcow2 2246 qemu64' 'vm-maverick-i386-install.qcow2 2247 qemu32,-nx' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/install/upgrade/')" \
'sudo DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server-5.1' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' ;\
doneMariaDB uses branch protection to ensure that pushes to the MariaDB Server git repository cannot be made without first passing a series of tests. This page aims to describe what a developer should do in order to ensure that their changes get pushed to the main repository.
The current protected branches are the main branches 10.X/11.X and release branches bb-10.X-release/bb-11.X-release*
git rebase 10.11Push to bb-* branch
Wait for all the checks to run
If all checks are successful you can push to the main branch to be the same as bb-* branch
If not (failed checks), check the build status, make changes accordingly and repeat this flow
The following are the only required builders at this point:
amd64-debian-11-debug-ps-embedded
amd64-debian-12
amd64-debian-12-debug-embedded
amd64-debian-11-msan
amd64-fedora-40
amd64-last-N-failed
amd64-ubuntu-2004-clang
amd64-ubuntu-2004-debug
amd64-ubuntu-2204-debug-ps
amd64-windows (?)
The protected branches builders run a subset of tests in order to avoid sporadic failures. The list is defined by:
All other checks are not required in order to make the push. However, please take a look at all the builders and look for failures.
The build status can be seen directly from GitHub near the commit message. Below, you can find an example (please note the yellow dot near the commit message):
If you click on the yellow dot, you can find the list of checks performed and their status, as shown below:
By clicking on Details you will be redirected to the buildbot page showing all the build details.
Alternatively, you can look for the builds on the buildbot Grid View. This allows filtering by branch, so to only see changes for one particular branch you can use grid?branch=10.5.
Yellow dot - specifies that one or more of checks are not yet completed.
Action item: Wait for all the checks to finish
Green tick - all checks were successful
Action item: No specific action is needed. The push can go through!
Red X - one or more checks have failed
Action item: Look at the status of failing builds and fix potential issues
Note: Only buildbot and not external CI tests (Travis, AppVeyor, etc) are currently in the protected branches criteria. Please take note of other failures, and if they are acceptable and explainable, then merge.
In some cases it might be useful to re-trigger one or more checks. This can easily be done from the buildbot.mariadb.org interface. The steps are:
Login to buildbot.mariadb.org using the GitHub credentials: Hit the "Anonymous" button from the upper right corner and then "Login with GitHub" as shown below:
Open the build details of the build you want to re-trigger
Hit the "Rebuild" button from the upper right corner
Note: If you want to re-trigger all checks, the easiest approach is to make a new push. Alternatively, you can follow the above steps for all the checks
This page is licensed: CC BY-SA / Gnu FDL
The following steps were used to create a Solaris 10 x86 BuildSlave.
I started with a default install of Solaris 10.
First I added a new user with:
groupadd sudo
useradd -d /export/home/${username} -m -s /bin/bash -g staff -G sudo ${username}
passwd ${username}I then logged in as the new user and setup an ssh key.
Now to install software
Prior to actually installing the software, I adjusted the global profile so that the /usr/local/ dirs were included in the various PATHs:
vi /etc/profile
# Add the following lines:
LD_LIBRARY_PATH=/opt/csw/lib:/usr/local/lib:/usr/sfw/lib:$LD_LIBRARY_PATH # Add required libraries
PYTHONPATH=/usr/local/lib/python2.5/site-packages:$PYTHONPATH
PATH=/usr/local/bin:/usr/bin:/usr/sbin:/etc:/usr/sfw/bin:$PATH # Puts "local" packages in your path
export LOGNAME PATH PYTHONPATH LD_LIBRARY_PATHThe extra software, I downloaded from sunfreeware:
With all of the software downloaded, I next setup and configured sudo and python:
With sudo now working, I logged out and then back in. I then installed the other packages:
With those packages installed it was time to install the pieces of software which don't have pre-built packages:
Install Zope Interface:
cd /tmp
wget http://www.zope.org/Products/ZopeInterface/3.3.0/zope.interface-3.3.0.tar.gz
gunzip -v zope.interface-3.3.0.tar.gz
gtar -xf zope.interface-3.3.0.tar
cd zope.interface-3.3.0/
python setup.py build
sudo python setup.py installInstall the latest Twisted framework:
Install Bazaar:
Install ccache:
Configure and start NTP:
Check out and make a test build of MariaDB:
Add a user for buildbot:
Install Buildbot:
Create the buildbot as the buildbot user:
On the build master, add new entry to /etc/buildbot/maria-master-private.cfg
Remember the ${slave-name} and ${password} configured above, they're used in the next step.
Back on the solaris machine:
Now start the slave:
That's the basic process.
This page is licensed: CC BY-SA / Gnu FDL
--suite=main,spider,spider/bg,spider/bugfix,spider/feature,spider/regression/e1121,spider/regression/e112122 --skip-test="^stack_crash$|^float$|^derived_split_innodb$|^mysql_client_test$|^kill$|^processlist_not_embedded$|^sp-big$cd /tmp
ftp ftp.sunfreeware.com
anonymous
none
bin
cd pub/freeware/intel/10/
mget python-2.5.1-sol10-x86-local.gz sudo-1.7.4p4-sol10-x86-local.gz libintl-3.4.0-sol10-x86-local.gz libgcc-3.4.6-sol10-x86-local.gz libiconv-1.13.1-sol10-x86-local.gz
mget automake-1.9-sol10-intel-local.gz autogen-5.9.8-sol10-x86-local.gz autoconf-2.68-sol10-x86-local.gz gcc-4.5.1-sol10-x86-local.gz
mget m4-1.4.15-sol10-x86-local.gz libtool-2.4-sol10-x86-local.gz gmp-4.2.1-sol10-x86-local.gz
mget md5-6142000-sol10-intel-local.gz openssl-1.0.0c-sol10-x86-local.gz libsigsegv-2.9-sol10-x86-local.gz tcl-8.5.9-sol10-x86-local.gz tk-8.5.9-sol10-x86-local.gz perl-5.12.2-sol10-x86-local.gz
mget libtool-2.4-sol10-x86-local.gz sed-4.2.1-sol10-x86-local.gz zlib-1.2.5-sol10-x86-local.gz binutils-2.21-sol10-x86-local.gz groff-1.21-sol10-x86-local.gz bzip2-1.0.6-sol10-x86-local.gz
mget make-3.82-sol10-x86-local.gz sed-4.2.1-sol10-x86-local.gz gdb-6.8-sol10-x86-local.gz coreutils-8.9-sol10-x86-local.gz cmake-2.6.0-sol10-x86-local.gz
quitsu
gunzip -v python-2.5.1-sol10-x86-local.gz
pkgadd -d python-2.5.1-sol10-x86-local
gunzip -v libintl-3.4.0-sol10-x86-local.gz libgcc-3.4.6-sol10-x86-local.gz libiconv-1.13.1-sol10-x86-local.gz sudo-1.7.4p4-sol10-x86-local.gz
pkgadd -d libintl-3.4.0-sol10-x86-local
pkgadd -d libgcc-3.4.6-sol10-x86-local
pkgadd -d libiconv-1.13.1-sol10-x86-local
pkgadd -d sudo-1.7.4p4-sol10-x86-local
mkdir -p /usr/local/var/lib/
/usr/local/sbin/visudocd /tmp
gunzip -v *.gz
sudo pkgadd -d autoconf-2.68-sol10-x86-local
sudo pkgadd -d autogen-5.9.8-sol10-x86-local
sudo pkgadd -d automake-1.9-sol10-intel-local
sudo pkgadd -d binutils-2.21-sol10-x86-local
sudo pkgadd -d gcc-4.5.1-sol10-x86-local
sudo pkgadd -d groff-1.21-sol10-x86-local
sudo pkgadd -d libsigsegv-2.9-sol10-x86-local
sudo pkgadd -d make-3.82-sol10-x86-local
sudo pkgadd -d m4-1.4.15-sol10-x86-local
sudo pkgadd -d md5-6142000-sol10-intel-local
sudo pkgadd -d openssl-1.0.0c-sol10-x86-local
sudo pkgadd -d perl-5.12.2-sol10-x86-local
sudo pkgadd -d tcl-8.5.9-sol10-x86-local
sudo pkgadd -d tk-8.5.9-sol10-x86-local
sudo pkgadd -d zlib-1.2.5-sol10-x86-local
sudo pkgadd -d bzip2-1.0.6-sol10-x86-local
sudo pkgadd -d libtool-2.4-sol10-x86-local
sudo pkgadd -d sed-4.2.1-sol10-x86-local
sudo pkgadd -d gdb-6.8-sol10-x86-local
sudo pkgadd -d coreutils-8.9-sol10-x86-local
sudo pkgadd -d gmp-4.2.1-sol10-x86-local
sudo pkgadd -d cmake-2.6.0-sol10-x86-local


cd /tmp
wget http://tmrc.mit.edu/mirror/twisted/Twisted/10.2/Twisted-10.2.0.tar.bz2
bunzip2 Twisted-10.2.0.tar.bz2
gtar -xf Twisted-10.2.0.tar
cd Twisted-10.2.0
sudo python setup.py installcd /tmp
wget http://launchpad.net/bzr/2.2/2.2.2/+download/bzr-2.2.2.tar.gz
gunzip -v bzr-2.2.2.tar.gz
gtar -xf bzr-2.2.2.tar
cd bzr-2.2.2
sudo python setup.py installsudo useradd -d /export/home/buildbot -m buildbotcd /tmp
wget http://samba.org/ftp/ccache/ccache-3.1.4.tar.gz
gunzip ccache-3.1.4.tar.gz
gtar xvf ccache-3.1.4.tar
cd ccache-3.1.4
./configure --prefix /usr
make
sudo make installsudo cp /etc/inet/ntp.server /etc/inet/ntp.conf
sudo vi /etc/inet/ntp.conf
#
# Comment out the following lines:
#server 127.127.XType.0
#fudge 127.127.XType.0 stratum 0
#broadcast 224.0.1.1 ttl 4
#
# Add in the following lines:
server 0.us.pool.ntp.org
server 1.us.pool.ntp.org
server 2.us.pool.ntp.org
server 3.us.pool.ntp.org
# save the file and quit back the command prompt
sudo touch /var/ntp/ntp.drift
sudo ntpdate 0.us.pool.ntp.org
sudo svcadm enable svc:/network/ntpcd
mkdir src
cd src/
bzr branch lp:maria trunk
cd trunk/
BUILD/compile-solaris-amd64cd /tmp
wget http://buildbot.googlecode.com/files/buildbot-slave-0.8.3.tar.gz
gunzip -v buildbot-slave-0.8.3.tar.gz
gtar -xf buildbot-slave-0.8.3.tar
cd buildbot-slave-0.8.3/
sudo python setup.py installsudo su - buildbot
buildslave create-slave --usepty=0 /export/home/buildbot/maria-slave \
hasky.askmonty.org:9989 ${slavename} ${password}
echo '${contact-email-address}' > /export/home/buildbot/maria-slave/info/admin
echo 'A host running Solaris 10 x86.' > /export/home/buildbot/maria-slave/info/host
exitsudo su - buildbot
buildslave start maria-slaveStart each VM booting from the server install iso one at a time and perform the following install steps:
Once running you can connect to the VNC server from your local host with:
Replace ${remote-host} with the host the vm is running on.
Note: When you activate the install, vncviewer may disconnect with a complaint about the rect being too large. This is fine. The Fedora installer has just resized the vnc screen. Simply reconnect.
Install, picking default options mostly, with the following notes:
The Installer will throw up a "Storage Device Warning", choose "Yes, discard any data"
Set the hostname to fedora16-amd64 (or fedora16-i386)
Click the "Configure Network" button on the Hostname screen.
Edit System eth0 to "connect automatically"
Apply and then close the "Network Connections" window
When partitioning disks, choose "Use All Space"
uncheck the "Use LVM" checkbox
do not check the "Encrypt system" checkbox
Minimal install
Customize Later
When the install is finished, you will be prompted to reboot. Go ahead and do so, but it will fail. Kill the VM (after the reboot fails) and start it up again:
Until the extra user is installed you must connect via VNC as before. SSH is preferred, so that's what we'll do first. Login as root.
After logging in as root, install proper ssh and then create a local user:
Enable password-less sudo and serial console:
Still logged in as root, add to /boot/grub/menu.lst:
Editing /boot/grub/menu.lst:
Logout as root, and then, from the VM host server:
Create a .ssh folder:
Copy over the authorized keys file:
Set permissions on the .ssh folder correctly:
Create the buildbot user:
su to the local buildbot user and ssh to the vm to put the key in known_hosts:
For i386:
For amd64:
Upload the ttyS0 file and put it where it goes:
Update the VM:
Once logged in:
After updating, shut down the VM:
Also:
MariaDB.local.repo points at a local directory on the VM. MariaDB.repo
points at the real MariaDB YUM repository.
The MariaDB upgrade testing VMs were not built. Once we have MariaDB Fedora 16 RPMs, then I will attempt building this VM. For now, the placeholder text below is copied from the page.
This page is licensed: CC BY-SA / Gnu FDL
qemu-img create -f qcow2 /kvm/vms/vm-fedora16-i386-serial.qcow2 10G
qemu-img create -f qcow2 /kvm/vms/vm-fedora16-amd64-serial.qcow2 10Gkvm -m 1024 -hda /kvm/vms/vm-fedora16-i386-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-16-i386-DVD.iso -redir tcp:2263::22 -boot d -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora16-amd64-serial.qcow2 -cdrom /kvm/iso/fedora/Fedora-16-x86_64-DVD.iso -redir tcp:2264::22 -boot d -smp 2 -cpu qemu64 -net nic,model=virtio -net uservncviewer -via ${remote-host} localhostkvm -m 1024 -hda /kvm/vms/vm-fedora16-i386-serial.qcow2 -redir tcp:2263::22 -boot c -smp 2 -cpu qemu32,-nx -net nic,model=virtio -net user
kvm -m 1024 -hda /kvm/vms/vm-fedora16-amd64-serial.qcow2 -redir tcp:2264::22 -boot c -smp 2 -cpu qemu64 -net nic,model=virtio -net userssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa root@localhost/sbin/chkconfig --level 35 network on
ifup eth0
yum install openssh-server openssh-clients
adduser ${username}
usermod -a -G wheel ${username}
passwd ${username}visudo
# Uncomment the line "%wheel ALL=(ALL) NOPASSWD: ALL"
# Comment out this line:
# Defaults requirettysudo vi /etc/default/grub
# Add/edit these entries:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8"
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
grub2-mkconfig -o /boot/grub2/grub.cfgssh -t -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"
ssh -t -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "mkdir -v .ssh"scp -P 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/
scp -P 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/authorized_keys localhost:.ssh/ssh -t -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"
ssh -t -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/buildbot.id_dsa localhost "chmod -R go-rwx .ssh"ssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no localhost 'chmod -R go-rwx .ssh; sudo adduser buildbot; sudo usermod -a -G wheel buildbot; sudo mkdir ~buildbot/.ssh; sudo cp -vi .ssh/authorized_keys ~buildbot/.ssh/; sudo chown -vR buildbot:buildbot ~buildbot/.ssh; sudo chmod -vR go-rwx ~buildbot/.ssh'sudo su - buildbot
ssh -p 2263 buildbot@localhost
# exit, then exit againsudo su - buildbot
ssh -p 2264 buildbot@localhost
# exit, then exit againscp -P 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
scp -P 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/ttyS0 buildbot@localhost:
ssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mv -vi ttyS0 /etc/event.d/;'
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost 'sudo mv -vi ttyS0 /etc/event.d/;'ssh -p 2263 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhost
ssh -p 2264 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no buildbot@localhostsudo yum updatesudo shutdown -h nowfor i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/build/')" \
"sudo yum -y groupinstall 'Development Tools'" \
"sudo yum -y install wget tree gperf readline-devel ncurses-devel zlib-devel pam-devel libaio-devel openssl-devel perl perl\(DBI\)" \
"sudo yum -y remove systemtap-sdt-dev" \
"sudo mkdir -vp /usr/src/redhat/SOURCES /usr/src/redhat/SPECS /usr/src/redhat/RPMS /usr/src/redhat/SRPMS" \
"wget http://www.cmake.org/files/v2.8/cmake-2.8.8.tar.gz;tar -zxvf cmake-2.8.8.tar.gz;cd cmake-2.8.8;./configure;make;sudo make install"; \
donefor i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/install/')" \
"sudo yum -y update" \
"sudo yum -y install libaio perl perl-Time-HiRes perl-DBI" \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
"sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
donefor i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade/')" \
"sudo yum -y update" \
'sudo yum -y install mysql-server' \
'sudo systemctl enable mysqld.service' \
'sudo systemctl start mysqld.service' \
'mysql -uroot -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
"sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/"; \
donefor i in '/kvm/vms/vm-fedora16-amd64-serial.qcow2 2264 qemu64' '/kvm/vms/vm-fedora16-i386-serial.qcow2 2263 qemu64' ; do \
set $i; \
runvm --user=buildbot --logfile=kernel_$2.log --base-image=$1 --port=$2 --cpu=$3 "$(echo $1 | sed -e 's/serial/upgrade2/')" \
'sudo yum -y update' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.repo buildbot@localhost:/tmp/" \
'sudo rpm --verbose --import http://downloads.mariadb.org/repo/RPM-GPG-KEY-MariaDB' \
'sudo mv -vi /tmp/MariaDB.repo /etc/yum.repos.d/' \
'sudo yum -y remove mysql-libs' \
'sudo yum -y install MariaDB-server MariaDB-client MariaDB-test' \
'sudo yum -y install cronie cronie-anacron crontabs.noarch postfix' \
'sudo /etc/init.d/mysqld start' \
'mysql -uroot -prootpass -e "create database mytest; use mytest; create table t(a int primary key); insert into t values (1); select * from t"' \
'sudo rm -v /etc/yum.repos.d/MariaDB.repo' \
"= scp -P $2 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /kvm/vms/MariaDB.local.repo buildbot@localhost:/tmp/" \
'sudo mv -vi /tmp/MariaDB.local.repo /etc/yum.repos.d/'; \
doneOne type of build we do in BuildBot is to build and test MariaDB binary packages for the platforms we release on. We build and test packages for Debian (4 and 5), Ubuntu (8.04 to 10.04), Centos 5, and generic Linux; amd64 and i386 architectures. This testing is done with virtual machines run in KVM.
To better control the startup and shutdown of the virtual machines we use a small wrapper around KVM we developed called runvm. The purpose of this tool is to make it easy to boot up a virtual machine, run some build commands inside it, and shut it down cleanly.
This wrapper encapsulates the steps needed to boot up a virtual machine, run a series of commands inside it, and shut it down gracefully afterwards. Some special care is taken in the script to ensure that the virtual machine is always shut down after use (gracefully if possible), even in case of various failures or the loss of the parent process or controlling TTY. And if a conflicting virtual machine somehow manages to escape shutdown, runvm automatically attempts to terminate it before starting a new one. This extra robustness is important for fully automated testing as in our Buildbot setup, to ensure that the system can run unattended for longer periods of time.
Essentially, instead of a normal Buildbot session which would do something like this on the slave:
We instead use runvm to do the same inside a virtual machine running as a KVM guest with the build slave as host:
See the runvm Usage Examples or runvm --help sections below for more detailed examples, but this is the basic idea.
Here is an example command you could use to run a build inside a virtual machine using runvm:
In this example, ubuntu-hardy-amd64.qcow2 is a KVM image already
installed with compilers and set up for password-less ssh access (using public
key authentication). Port 2222 on the host side is forwarded to the ssh service
(port 22) on the guest side (so by specifying different --port
options it is easy to run multiple runvm invocations in parallel;
in our Buildbot setup we run 3 builds in parallel this way).
Note the use of the scp command, prefixed with an equals sign
"=" Commands prefixed in this way are run on the host side rather
than the guest side; this is a convenient way to copy data in or results out of
the virtual machine while the runvm session is running.
Using runvm in this way we are able to easily and flexibly manage
a large number of virtual machines for automated builds with very little
overhead and complexity. In fact we have around 70 distinct virtual machines!
The only resource they take is a little disk space (around 37 GByte). And the
virtual machines images are also simple to set up, requiring only a minimal
install; no need to set up networking bridges or IP addresses, or to install a
Buildbot client. All the complex logic runs on the host system, which only
needs to be installed once.
By keeping the virtual images simple, builds and tests run in a minimal environment, which is useful to detect any missing dependencies or other problems that do not show themselves on normal developer machines with a full desktop install (we even do install testing on a separate virtual machine from the one used to build, with compilers etc. not installed on the one used to test installation).
A further refinement of example one (above) is to create a new
temporary virtual machine image before each step as a copy of a reference
image, run the build, and throw away the temporary image after the build. This
avoids any possibility of a previous build influencing a following build in any
way (and thus also simplifies the build setup, as we can install stuff freely
without any need to do cleanup). It also avoids having to fix a broken image,
like needing to manually run fsck after a crash or similar issue. We use this technique for most of our binary package builds in Buildbot.
To use this copy-and-discard technique with runvm, the --base-image option is useful:
This will run the build in a temporary copy tmp.qcow2 of the
reference image ubuntu-hardy-i386.qcow2, without modifying the
reference image in any way. This uses the copy-on-write feature of the qcow2
image format (see qemu-img(1)), so it even takes only very little
time (fraction of a second) and minimal space (only changed blocks are written
to the new image).
The above two examples show basically how the package testing in our Buildbot
setup is done. There are some further details of course, like more options for
the build commmands and extra care to get logfiles out to debug problems; the
full details are available in our .
But the basic principle is just a number of runvm commands like the examples above.
The runvm tool is available under GPL on Lauchpad in the project . In the bzr repository
it is found as .
If someone finds it useful or has suggestions for improvements, please drop us
a line on the
mailing list.
Since it might be useful, here is the output from runvm --help (check the latest version of the tool for up-to-date output):
This page is based on a by Kristian Nielsen, the primary developer of runvm.
This page is licensed: CC BY-SA / Gnu FDL
./configure && makerunvm image.qcow2 "./configure" "make"runvm --port=2222 ubuntu-hardy-i386.qcow2 \
"= scp -P 2222 mariadb-5.1.41-rc.tar.gz localhost:" \
"tar zxf mariadb-5.1.41-rc.tar.gz" \
"cd mariadb-5.1.41-rc && ./configure" \
"cd mariadb-5.1.41-rc && make"runvm --port=2222 --base-image=ubuntu-hardy-i386.qcow2 tmp.qcow2 \
"= scp -P 2222 mariadb-5.1.41-rc.tar.gz localhost:" \
"tar zxf mariadb-5.1.41-rc.tar.gz" \
"cd mariadb-5.1.41-rc && ./configure" \
"cd mariadb-5.1.41-rc && make"Usage: ./runvm <options> image.qcow2 [command ...]
Boot the given KVM virtual machine image and wait for it to come up.
Run the list of commands one at a time, aborting on receiving an error.
When all commands are run (or one of them failed), shutdown the virtual
machine and exit.
Commands are by default run inside the virtual machine using ssh(1). By
prefixing a command with an equals sign '=', it will instead be run on the
host system (for example to copy files into or out of the virtual machine
using scp(1)).
Some care is taken to ensure that the virtual machine is shutdown
gracefully and not left running even in case the controlling tty is
closed or the parent process killed. If a previous virtual machine is
already running on a conflicting port, an attempt is made to shut it
down first. For this purpose, a PID file is created in $HOME/.runvm/
Available options:
-p, --port=N Forward this port on the host side to the ssh port (port
22) on the guest side. Must be different for each runvm
instance running in parallel to avoid conflicts. The
default is 2222.
To copy files in/out of the guest use a command prefixed
with '=' calling scp(1) with the -P option using the port
specified here, like this:
runvm img.qcow2 "=scp -P 2222 file.txt localhost:"
-u, --user=USER Name of the account to ssh into in the guest. Defaults to
the name of the user invoking runvm on the host.
-m, --memory=N Amount of memory (in megabytes) to allocate to the guest.
Defaults to 2047.
--smp=N Number of CPU cores to allocate to the guest.
Defaults to 2.
-c, --cpu=NAME Type of CPU to emulate for KVM, see qemu(1) for details.
For example:
--cpu=qemu64 For 64-bit amd64 emulation
--cpu=qemu32 For 32-bit x86 emulation
--cpu=qemu32,-nx 32-bit and disable "no-execute"
The default is qemu32,-nx
--netdev=NAME Network device to emulate. The 'virtio' device has good
performance but may not have driver support in all
operating systems, if so another can be specified.
The default is virtio.
--kvm=OPT Pass additional option OPT to kvm. Specify multiple times
to pass more than one option. For example
runvm --kvm=-cdrom --kvm=mycd.iso img.qcow2 ...
--initial-sleep=SECS
Wait this many seconds before starting to poll the guest
ssh port for it to be up. Default 15.
--startup-timeout=SECS
Wait at most this many seconds for the guest OS to respond
to ssh. If this time is exceeded assume it has failed to
boot correctly. Default 300.
--shutdown-timeout=SECS
Wait at most this many seconds for the guest OS to
shutdown gracefully after sending a shutdown command. If
this time is exceeded, assume the guest has failed to
shutdown gracefully and kill it forcibly. Default 120.
--kvm-retries=N If the guest fails to come up, retry the boot this many
times before giving up. This helps if the virtual machine
sometimes crashes during boot. Default 3.
-l, --logfile=FILE File to redirect the output from kvm into. This includes
any (error) messages from kvm, and also includes anything
the guest writes to the kvm emulated serial port (it can
be useful to set the guest to send boot loader and kernel
messages to the serial console and log them with this
option). Default is to not log this output anywhere.
-b, --base-image=IMG
Instead of booting an existing image, create a new
copy-on-write image based on IMG. This uses the -b option
of qemu-img(1). IMG is not modified in any way. This way,
the booted image can be discarded after use, so that each
use of IMG is using the same reference image with no risk
of "polution" between different invocations.
Note that this DELETES any existing image of the same
name as the one specified on the command line to boot! It
will be replaced with the image created as a copy of IMG,
with any modifications done during the runvm session.