Category Archives: Tech Notes

Techie notes

Pure-ftp and virtual user on Ubuntu

Pureftp is a simple and secure ftp server that supports virtual users that can be chrooted. That was I was looking for when I needed to provide some external access to my home server. However, as often, on Ubuntu things work a bit different then on other platforms and the documentation I found was only partial.

The basic installation is simple, then the documentation mention you need to create one user that can be used by all the virtual users.

# Install pure-ftpd
sudo apt-get install pure-ftpd
# add group for virtual users
sudo groupadd ftpgroup
# add ftp user for virtual user
sudo useradd -g ftpgroup -d /dev/null -s /etc

Before creating any virtual users you need to change the settings. The way the settings work is that the /etc/pure-ftpd/conf contains files that specify the settings. To enable the ChrootEveryone option you need to create a file with that name with the contents “on”.

# Enable ChrootEveryone option
sudo echo on > /etc/pure-ftpd/conf/ChrootEveryone

What took me the longest to figure out is that the virtual users authentication is not enabled by default. To enable this you need to create a link in the auth folder to the PureDB.

# Enable the virtual user authentication
sudo ln -s /etc/pure-ftpd/conf/PureDB /etc/pure-ftpd/auth/40PureDB
# And restart pure-ftpd
sudo /etc/init.d/pure-ftpd restart
Restarting ftp server: Running: /usr/sbin/pure-ftpd -l puredb:/etc/pure-ftpd/pureftpd.pdb -l pam -8 UTF-8 -O clf:/var/log/pure-ftpd/transfer.log -u 1000 -E -A -B

Make sure that the -l puredb:/etc/pure-ftpd/pureftpd.pdb is present, this is the option for the virtual users authentication and the - A as this is the chroot option.

After this you can pretty much follow the standard procedures:

# Create the user dir
mkdir /srv/ftp/joe
# Change the owner/group to the ftp user
sudo chown ftpuser:ftpgroup /srv/ftp/joe
# Create user joe with home directory /srv/ftp/joe
pure-pw useradd joe -u ftpuser -d /srv/ftp/joe
# Write changes to the database
pure-pw mkdb

After this you can login with the newly created user.

Setup ssh key pair for jenkins under windows

It is always tricky to setup jenkins properly when running under Windows. There are a few things you need to take care of. In this post I will assume you are using the bash shell delivered together with git.

  • Create a new user account for jenkins
  • Set a password for this user account
  • Add the HOME environment variable for Jenkins
  • Start a bask shell under the new user.
  • Generate the ssh key
  • Add you public key to your target server
  • Make a new ssh connection to your server to add it to the known hosts

ssh uses the HOME environment variable to locate the .ssh folder that contains the ssh keys. If this folder is not present it will default to C: which is not a good place to store your keys. So it is very important to add  this HOME variable for a stable functioning of jenkins. If you can not actually log in using the jenkins account you can still set the variable using the registry. The Windows runas command will be used to allow us to add the environment to the user jenkins even when we are not logged in as such.

runas /user:jenkins reg add KEY  /v HOME /t REG_SZ /d "C:\Documents and Settings\jenkins"

Next we start a bash shell to check the HOME var and create the ssh key.

runas /user:jenkins "C:\Program Files\Git\Bin\bash.exe"

This will open a new bash shell that is logged in as jenkins. In this bash shell you can now check the HOME variable, generate your ssh key and add the server to you known_hosts

$ echo $HOME
/c/Documents and Settings/jenkins
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/c/Documents and Settings/jenkins/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /c/Documents and Settings/jenkins/.ssh/id_rsa.
Your public key has been saved in /c/Documents and Settings/jenkins/.ssh/id_rsa.pub.
$ ssh -p 29418 jenkins@gerrit.bruin.sg gerrit ls-projects

Now the ssh has been setup properly you need to change the service user from the Local System account to log in as the user Jenkins.
Lastly, restart the service and Jenkins should be using the correct ssh credentials.

How to edit the project.config for all projects in Gerrit

While I was evaluating Gerrit 2.6 I ended up installing the building the latest snapshot version (2.7RC) myself. All was fine until I was about to setup Jenkins to verify my commits. I was surprised to find the label “Verified” missing from the reviews. After some googling I found that in version 2.7 this has been removed and needed to be added manually in the project.config. After searching my drives I could not find this project.config only to realize that this is a file in the meta config. And this below is how you edit it:

mkdir tmp
cd tmp
git init
git remote add origin ssh://admin@remote.site.com:29418/All-Projects
git fetch origin refs/meta/config:refs/remotes/origin/meta/config
git checkout meta/config

Now you should have the project.config in you temp folder. Add the label as below:

[label "Verified"]
       function = MaxWithBlock
       value = -1 Fails
       value =  0 No score
       value = +1 Verified

Next you need to commit and push back to Gerrit. Ofcourse make sure you have the appropriate rights.

git commit -a
git push origin meta/config:meta/config

Cross flashing firmware for Supermicro AOC-S2308L-8i

I was trying to install Openindiana using the Supermico AOC S2308L-L8i HBA but I was unable to get Openindiana to recognize the card.

At this point I decided to cross flash the card to the LSI SAS 9207-8i firmware that uses the same controller.

As a back up I downloaded the original Supermicro firware from there ftp server.

In the original firmware there is a nice script for UEFI boot that shows you how to flash the card.
This is the original script:

echo -off 
sas2flash.efi -o -e 7
cls
sas2flash.efi -f <del datetime="2013-04-16T15:43:11+00:00">2308IT14.ROM</del>9207-8.BIN
cls 
sas2flash.efi -b mptsas2.rom
cls
sas2flash.efi -o -sasaddhi 5003048

To flash using the LSI firmware you just need to replace the 2308IT14.ROM with the 9207-8.BIN. Copy the script and the 9207-8.ROM and mptsas2.bin into the same folder and you are good to go.

Setting up ssl Subversion Mirror on Ubuntu 12.04

These are some quick notes on how to setup a subversion mirror on ubuntu 12.04

First install apache2, subversion and the binding

apt-get install apache2
apt-get install subversion
apt-get install libapache2-svn

create dir /etc/apache2/ssl and copy you ssl files to this folder.

Update /etc/apache2/sites-available/default-ssl or create a new site.

Make these changes to the site config.

SSLCertificateFile /etc/apache2/ssl/my-site.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/my-site.com.key
SSLCertificateChainFile /etc/apache2/ssl/gd_bundle.crt

Enable ssl module, the ssl site and then restart apache2:

a2enmod ssl
a2ensite default-ssl
service apache2 restart

Now you should ssl working and it is time to add the subversion proxy.

Add the proxy module to apache2:

a2enmod proxy
a2enmod proxy_http
a2enmod rewrite

Update default-sll, after ServerName add

       SSLProxyEngine on

Before BrowserMatch add:

       SSLProtocol -ALL +SSLv3

    <Location /svn>
        DAV svn
        SVNParentPath /www1/svn
        SVNMasterURI https://masterip/svn
        AuthType Basic
        AuthName "Subversion repository"
        AuthUserFile /www1/etc/local-authentication
        Require valid-user
    </Location>

Proxy for a Trac instance:

    ProxyPass /trac https://masterip/trac
    ProxyPassReverse /trac https://masterip/trac

Make sure you proxy https to https. Mixing http and https will cause problem later on when performing copies in subversion commits and they will fail.

Ubuntu house keeping

Ubuntu logo ubuntu.comI am running a small Ubuntu server that I have just given 8GB of virtual HD space. I thought that would be enough. However, as I was trying to transfer some data to it I ran out of space. This left me wondering what happened.

I went to search where the data was stored and it was in /var/cache/apt. It contained 560MB. This is what I did to clean it.

# sudo apt-get autoclean
Reading package lists... Done
Building dependency tree
Reading state information... Done
Del libapt-inst1.4 0.8.16~exp12ubuntu10.5 [99.8 kB]
Del linux-image-server 3.2.0.33.36 [2,652 B]
Del linux-headers-virtual 3.2.0.33.36 [2,642 B]
Del perl 5.14.2-6ubuntu2.1 [4,416 kB]
Del libapt-pkg4.12 0.8.16~exp12ubuntu10.5 [939 kB]
Del perl-modules 5.14.2-6ubuntu2.1 [3,396 kB]
Del firefox-locale-en 17.0+build2-0ubuntu0.12.04.1 [482 kB]
Del perl-base 5.14.2-6ubuntu2.1 [1,498 kB]
Del apt 0.8.16~exp12ubuntu10.5 [1,100 kB]
Del linux-libc-dev 3.2.0-33.52 [867 kB]
Del linux-headers-server 3.2.0.33.36 [2,648 B]
Del apt-utils 0.8.16~exp12ubuntu10.5 [190 kB]
Del coreutils 8.13-3ubuntu3.1 [2,216 kB]
Del apt-transport-https 0.8.16~exp12ubuntu10.5 [16.3 kB]
Del linux-server 3.2.0.33.36 [1,728 B]
Del firefox-locale-en 16.0.2+build1-0ubuntu0.12.04.1 [481 kB]

Cache reduced a little bit to 545MB, clearly that was not what I was looking for. Time to wipe out the cache!

sudo apt-get clean

Almost enough space now, but not quite. What else was taking up space. The /usr/src contained 1.2GB of linux headers of all previous kernels that were installed.
Then I came across this post by Ubuntu Genius where he gave this nice one line command:

dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge

or interactive:

sudo apt-get remove --purge $(dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d')

Just make sure you have rebooted since your last kernel upgrade!

This cleared another 2.5GB!!!!

Mount error (12) after upgrade to Ubuntu 12.04

I recently upgrade an Ubuntu server from 10.04 to 12.04 and I encountered this error when trying to mount a cifs folder on a Windows 7 machine.

mount error(12): Cannot allocate memory
Refer to the mount.cifs(8) manual page (e.g.man mount.cifs)

I suspected it was related to the recent upgrade but it turns out that the error was caused by the Windows 7 system. I only mount this folder on rare occasions when I want to back up some folders to this system. Therefore I can not be sure if the problem was caused by my recent server update or some recent Windows 7 updates.

I found the solution in this post and some more details here:

These registery items need to be changed to make Windows 7 act more as a file server and prevent the error:

  • HKLMSYSTEMCurrentControlSetControlSession ManagerMemory ManagementLargeSystemCache change to 1
  • HKLMSYSTEMCurrentControlSetServicesLanmanServerParametersSize change to 3

Setting up permalinks on WordPress

This site runs on a Ubuntu 12.04 LAMP server behind another Ubuntu Apache proxy using a virtual host. When I tried to set up the permalinks it didn’t work straight away. The .htaccess file was correct and mod_rewrite was enabled. It was the virtual host that had some issue.
The website has been configured under /var/www/blog.your-site.name and these are the steps you should take to make sure the permalinks work:

  • Open .htacces in /var/www/blog.your-site.name and change it to look like this:
    # BEGIN WordPress
    <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteBase /
    RewriteRule ^index.php$ - [L]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule . /index.php [L]
    </IfModule>
    
  • Enable mod_rewrite for Apache:
    sudo ae2enmod rewrite
  • Create your host configuration in /etc/apache2/sites-available/blog.your-site.name and make it look like this:

    <VirtualHost *:80>
        ServerName blog.your-site.name
        ServerAdmin admin@your-site.name
    
        DocumentRoot /var/www/blog.your-site.name
        
            Options FollowSymLinks
            AllowOverride None
            Order deny,allow
            Deny from all
        
        
            Options Indexes FollowSymLinks -MultiViews
            AllowOverride all
            Order allow,deny
            allow from all
        
    
    
        ErrorLog ${APACHE_LOG_DIR}/blog.your-site.name-error.log
    
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
    
        CustomLog ${APACHE_LOG_DIR}/blog.your-site.name-access.log combined
    
    </VirtualHost>
    

That should do the job.

Ubuntu mirror problems

Ubuntu logo ubuntu.comWhen I started what I thought would be a simple update of a few of my Ubuntu server virtual machines I didn’t know how much trouble the mirrors would give me.
For a start, the sg. mirror failed to load completely. Then I found some info about ec2 mirrors, but they gave me Hash Sum mismatch problems.
Then I came across the mirror:// schema, that worked great and I managed to upgrade one system. Funny enough, I would expected the upgrade from 10.04 to result in a 12.04 but I ended up with 11.10?? Not so sure if I started with a different version or the failed update and upgrades caused this.

Now my problems really started. Trying to upgrade from 11.10 Oneric to 12.04 Precise seemed impossible. If I would use the mirror://mirrors.ubuntu.com/mirrors.txt the mirrors would fail and no package information was downloaded. Next, using the ec2 it wouldn’t even try to download as the Hash would fail.

In the end I logged into a Ubuntu server I manage in the Netherlands and download the http://mirrors.ubuntu.com/mirrors.txt. This is the list I downloaded from the Netherlands:

http://ftp.telfort.nl/pub/mirror/ubuntu/
http://ubuntu.mirror.cambrium.nl/ubuntu/
http://ftp.snt.utwente.nl/pub/os/linux/ubuntu/
http://nl3.archive.ubuntu.com/ubuntu/
http://nl.archive.ubuntu.com/ubuntu/
http://ubuntu.mirror.atratoip.net/ubuntu/
http://ubuntu.tiscali.nl/
http://osmirror.rug.nl/ubuntu/
http://mirror.nl.leaseweb.net/ubuntu/
http://mirror.i3d.net/pub/ubuntu/
http://ftp.tudelft.nl/archive.ubuntu.com/
http://mirrors.nl.eu.kernel.org/ubuntu/

Compared to the one from Singapore:

http://linux.ntuoss.org/ubuntu/
http://mirror.nus.edu.sg/ubuntu/
http://download.nus.edu.sg/mirror/ubuntu/
http://ubuntu.oss.eznetsols.org/ubuntu/
http://archive.ubuntu.com/ubuntu/

The tudelft seemed the most reliable source and when I replaced my mirrors with this one I managed to finally upgrade the server to 12.04.

So much for Ubuntu being the most user friendly Linux distro around. And shame on the mirrors in Singapore for being of such poor standard.