In this tutorial, I will be going over setting up a Garmin GPS 18 VLC to work with NTP on FreeBSD 8.0. This is the result of many hours of researching and troubleshooting to get everything up and running on FreeBSD. It is based loosely on http://www.satsignal.eu/ntp/FreeBSD-GPS-PPS.htm but as I found out there are a few differences with FreeBSD 8.0. This guide hopefully will help anyone else wanting to operate an NTP server synced to a GPS receiver on a more current version of FreeBSD.
A quick bit of background.
If you know what PPS stands for and how NTP syncs to a GPS unit, skip this, otherwise read on. GPS satellites have atomic clocks built in them. Using a GPS receiver we can take advantage of that and synchronise a server to provide time to a network or the Internet.
GPS units output their data in what are called “sentences”. These sentences contain information about location, time, status of the fix and others. The time that is contained in these sentences is only accurate to that second. This alone is not good enough to synchronise the time with to be a stratum 1 time server. Some GPS units also provide a Pule Per Second (PPS) output. This PPS is a logic high and low that is switched off usually on the DCD line of an RS-232 serial port. The duration is usually around 200ms and this is what we will configure for the Garmin. This PPS has a claimed accuracy that is dependant on the unit. For the Garmin, it is 1 microsecond and is one of the better GPS units around.
Why FreeBSD?
FreeBSD is generally accepted as the best operating system for time keeping. Pulse Per Second (PPS) is supported without patching the kernel sources and has support for nano-second accuracies instead of micro-second accuracies found in 2.6 Linux kernels. This means you should expect accuracies around +/- 3 us. When I was using Linux, I found the accuracy to be about +/- 30 us and sometimes worse depending on the fluctuation of ambient temperature. Take my advice, the small learning curve from Linux to FreeBSD is definitely worth the results.
Hardware
A Garmin GPS 18 LVC is used to sync NTP. This seems to be quite a common unit used to sync NTP and is the primary reason I chose it. It is also fairly cheap as I picked up mine for $100. You will need to wire it up to a serial port as follows.
The 5 volts from a USB port is enough to power the GPS unit. I just cut up a USB cable that I had lying around and fed it into the shell of the serial port. It is also a good idea to fit a plug and socket to the cable of the GPS so you can detach the unit. I just used a 6 pin mini DIN plug and socket which is shown in the pictures below.
Garmin on the roof
Cable comes through the false ceiling. One of these days I’ll make it neat 
Mini DIN plug and socket. Just don’t short out any of the wires like I did initially.
Serial port on the back of the NTP server. Power from the USB port comes directly in through the back shell of the serial.
The server at the top is the NTP server. It’s the old router box that we used before moving everything to VMWare. It’s a whitebox Xeon 2.8, 2GB DDR RAM and 2 x 250GB SATA HDDs in RAID1. Very over-speced, but putting old hardware to good use.
Software
Post-installation
The installation of FreeBSD is quite simple so I won’t go over this. I will assume that you have just finished your installation and you have just logged into the terminal. FreeBSD by default does not permit root logins from SSH. We will open this up for the moment so we can work remotely. Close this again if you wish when we have finished. I would highly recommend doing so if SSH will be accessible from the Internet.
It is important to know the implications of allowing remote root logins. I have only allowed them here as the server is contained in my LAN and there are other mechanisms in place to secure the server. Using sudo or su with a normal user account and denying remote root logins is a better practice.
Open /etc/ssh/sshd_config
freebsd# vi /etc/ssh/sshd_config
Uncomment the line PermitRootLogin and save.
PermitRootLogin yes
Now restart SSHd
freebsd# /etc/rc.d/sshd restart
Stopping sshd.
Starting sshd.
You should now be able to login with the root account through SSH.
Package management
Package management in FreeBSD is great once you know a few basics. There are both packages and ports. Packages are pre built binaries of ports and are useful if you don’t want to compile and want a “generalised” distribution of that software. The ports concept is much more feature rich and allows the user to customise the features of the software during compile time. Anyone who has used Gentoo will be familiar with this idea – Portage is based on the concept of FreeBSD ports.
If you chose a minimal installation of FreeBSD, you will need to fetch the ports tree.
freebsd# portsnap fetch extract
Looking up portsnap.FreeBSD.org mirrors... 3 mirrors found.
Fetching public key from portsnap2.FreeBSD.org... done.
Fetching snapshot tag from portsnap2.FreeBSD.org... done.
Fetching snapshot metadata... done.
Fetching snapshot generated at Mon Jan 25 11:36:41 EST 2010:
9cf1fbc128d851da28f72f41aff6d52068036f4e5c169d 7% of 61 MB 516 kBps
...
...
/usr/ports/x11/yeahconsole/
/usr/ports/x11/yelp/
/usr/ports/x11/zenity/
Building new INDEX files... done.
In future, if you want to update the locally cached copy of the ports database use the following command
freebsd# portsnap fetch update
Looking up portsnap.FreeBSD.org mirrors... 3 mirrors found.
Fetching snapshot tag from portsnap2.freebsd.org... done.
Ports tree hasn't changed since last snapshot.
No updates needed.
Ports tree is already up to date.
Use the following to search the ports tree
freebsd# cd /usr/ports
freebsd# make search name="bash" | less
This will search for any package with bash in its name. I’ve piped it through less so that we can scroll up and down through the packages. Press q to stop this and get back to the shell.
...
...
Port: bash-4.0.35
Path: /usr/ports/shells/bash
Info: The GNU Project's Bourne Again SHell
Maint: obrien@FreeBSD.org
B-deps: bison-2.4.1,1 gettext-0.17_1 libiconv-1.13.1 m4-1.4.13,1
R-deps: gettext-0.17_1 libiconv-1.13.1
WWW: http://cnswww.cns.cwru.edu/~chet/bash/bashtop.html
...
...
Install Bash
Coming from a Linux background, I am used to bash. One thing that annoys me with the default FreeBSD shell csh is that it doesn’t list matches when I double tap tab. I use the feature a lot in bash. There are a lot of other great shells, but for anyone coming from a Linux background, bash is the safest.
Notice in the above the path to where bash is. We need to change to this directory and then do the install.
freebsd# cd /usr/ports/shells/bash
freebsd# make config install clean
===> No options to configure
===> Vulnerability check disabled, database not found
=> bash-4.0.tar.gz doesn't seem to exist in /usr/ports/distfiles/bash.
=> Attempting to fetch from http://ftp.gnu.org/gnu/bash/.
bash-4.0.tar.gz 16% of 6084 kB 205 kBps
If you get any prompts about options for packages (such as M4), for the moment leave the defaults. FreeBSD will now go about fetching the sources and compiling the software. Depending on the hardware and Internet connection speed, expect anywhere from 10-30 minutes for bash to install as well as its dependencies.
...
===> Cleaning for m4-1.4.13,1
===> Cleaning for libtool-2.2.6b
===> Cleaning for libiconv-1.13.1
===> Cleaning for bash-4.0.35
freebsd#
Now change the default shell using the chsh command:
freebsd# chsh -s /usr/local/bin/bash
chsh: user information updated
When you log in the next time, your shell will now be bash. If you want to use bash now, type /usr/local/bin/bash.
Install Vim
Vi is installed with a minimal installation of FreeBSD. Vim is an improved version of Vi which gives us things like syntax highlighting. If you prefer other editors such as nano, pico or emacs, following these same steps should still give you enough information to be able to install them.
First search for vim in the root of the ports tree:
[root@freebsd /usr/ports]# make search name="vim" | less
Scroll down and find vim.
...
...
Port: vim-7.2.344
Path: /usr/ports/editors/vim
Info: Vi "workalike", with many additional features
Maint: obrien@FreeBSD.org
B-deps: gettext-0.17_1 glib-1.2.10_13 gtk-1.2.10_21 ...
R-deps: gettext-0.17_1 glib-1.2.10_13 gtk-1.2.10_21 ...
WWW: http://www.vim.org/
...
...
Change to this directory and do the install.
[root@freebsd /usr/ports]# cd editors/vim
[root@freebsd /usr/ports/editors/vim]# make config install clean
This is going to take a while. Python and Perl are dependencies which are pretty large ports themselves. Once its done create the ~/.vimrc file with the following contents. This will enable syntax highlighting as well as a few other handy things.
set nocompatible " must be the first line
filetype on
filetype indent on
filetype plugin on
set laststatus=2
set statusline=%<%f\%h%m%r%=%-20.(line=%l\ \ col=%c%V\ \ totlin=%L%)\ \ \%h%m%r%=%-40(bytval=0x%B,%n%Y%)\%P
syntax on
Now we will have a much friendlier Vim with similar configuration as a majority of Linuxes.
Install NTP
We search for and install NTP like the rest of the ports. First lets search for it.
[root@freebsd /usr/ports]# make search name="ntp" | less
Find NTP and note its location in the ports tree.
Port: ntp-4.2.6p1.r2
Path: /usr/ports/net/ntp
Info: The Network Time Protocol Distribution
Maint: cy@FreeBSD.org
B-deps: autoconf-2.62 autoconf-wrapper-20071109...
R-deps:
WWW: http://www.ntp.org/
Now install it.
[root@freebsd /usr/ports/net/ntp]# make config install clean
Leave the default options for NTP if prompted.
For now, don’t worry about configuring ntp. We will do this a bit later on as there are a couple of other things we need to do before we get ntp up and running.
Enable PPS support for FreeBSD
Pulse Per Second is supported natively in the FreeBSD kernel sources. PPS kernel support will give us better accuracy as the PPS process is offloaded to the kernel and not run in user space. NTP can be run with or without kernel PPS support but seeing as we are using FreeBSD and not Linux for the reason that FreeBSD will give us better accuracy, we may as well make the most of what we have got.
Although PPS is supported, we need to recompile the kernel. WAIT! Don’t go. I can almost hear the sigh and the backspace key being pressed, taking you back to Google to see if there is a better way. Recompiling the FreeBSD kernel basically as simple as installing a port. Forget about initrds, updating bootloaders, selecting the correct drivers, making modules and installing them!
Lets install the kernel sources. We do this by running sysinstall then choosing Configure, then Distributions then src then select base and sys.
Once this is installed, change to the sources and we will make a custom configuration for our kernel.
[root@freebsd /usr/ports/net/ntp]# cd /usr/src/sys/i386/conf/
[root@freebsd /usr/src/sys/i386/conf]# cp GENERIC PPSGENERIC
Now open up PPSGENERIC in Vim (or whatever text editor you are using) and add the following line to enable PPS support.
# PPS Support
options PPS_SYNC
Now we change back to the /usr/src directory and we can begin building our new custom kernel
[root@freebsd /usr/src/sys/i386/conf]# cd /usr/src/
[root@freebsd /usr/src]# make buildkernel KERNCONF=PPSGENERIC
...
some time later...
...
ld -Bshareable -d -warn-common -o zlib.ko.debug zlib.kld
objcopy --only-keep-debug zlib.ko.debug zlib.ko.symbols
objcopy --strip-debug --add-gnu-debuglink=zlib.ko.symbols zlib.ko.debug zlib.ko
--------------------------------------------------------------
>>> Kernel build for PPSGENERIC completed on Mon Jan 25 18:41:50 EST 2010
--------------------------------------------------------------
[root@freebsd /usr/src]# make installkernel KERNCONF=PPSGENERIC
Now reboot and you will be running your new kernel!
Configure the Garmin GPS 18 LVC
The default configuration of the Garmin GPS 18 LVC gives a very verbose output. What we want to do is modify the sentences that it will send to NTP to be as minimal as possible. When I was running Linux, I used the stock configuration and it was only later that I realised that the GPS time was being rejected as the time it took to send over all the sentences was too long for the software to handle.
We use a Garmin software utility called SNSRCFG.exe to configure the GPS unit. You can get it from the Garmin website here. You will need to use a Windows PC, I’ve tested both Windows 7 and Windows XP, both work fine. I used a USB to serial adapter which I didn’t have any problems with. You can configure the unit directly through FreeBSD and Linux, but using this utility is by far the easiest.
Start the program and select GPS 18 PC/LVC for the device. Press F6 to enter the sensor configuration and make sure the baud rate is set to 4800 and the PPS Length is set to 200. Although you can specify the speed of the serial device in the NTP config file, I could *not* get the unit to respond on any baud rate other than 4800.
Click OK and now press F7 so we can select our sentences. Select only GPGGA and GPRMC. Technically GPRMC is only required for NTP to work, but GPGGA can also be used. Having only these sentences will make communications with NTP far quicker than the default configuration before. Note that this does not have anything to do with the overall accuracy! The microsecond accuracy is achieved using the PPS and is not dependant on GPS sentences.
Once we have made these selections, go to Comm > Setup and make sure the correct serial port is selected. Now click the connect button
Once we have connected, we can then upload our configuration to the GPS unit. Note that it is greyed out here. I don’t actually have access to the GPS unit as I am writing this. If you get a message about downloading the configuration from the device, cancel this and proceed to the upload.
Once this is done, you can connect the GPS unit back to your FreeBSD server.
Serial and Symlinks
I had a hard time understanding serial ports in FreeBSD 8.0. When looking up documentation on serial ports and names, all the documentation was pointing to /dev/ttyxx and /dev/cuadx. I eventually figured out I was after /dev/cuaux. In my case I use /dev/cuau0 as the first serial port.
We also want to disable the serial lines that can be used to log in to a console. This could potentially lock the serial port from being used by NTP.
Open /etc/ttys and comment out the following lines:
# Serial terminals
# The 'dialup' keyword identifies dialin lines to login, fingerd etc.
# ttyu0 "/usr/libexec/getty std.9600" dialup off secure
# ttyu1 "/usr/libexec/getty std.9600" dialup off secure
# ttyu2 "/usr/libexec/getty std.9600" dialup off secure
# ttyu3 "/usr/libexec/getty std.9600" dialup off secure
Now we also need to make a symlink from our serial port to a pseudo GPS device, /dev/gps1. NTP will look for this device when it is started. We don’t tell NTP what serial port to expect a GPS device on, NTP expects it to be /dev/gpsX and in our case when we configure the ntp.conf file, it will look specifically for /dev/gps1.
We create this link in /etc/devfs.conf. Add the following to that file.
link cuau0 gps1
For simplicity sake, reboot the server to apply the changes.
Configure NTP
Now we can finally configure NTP to use the GPS device as a time source. A basic line in a ntp.conf file would look something like this
server 192.168.0.123
This would add 192.168.0.123 as a time source to NTP. How we add the GPS unit to NTP is very simmilar. We still use a numerical “IP” address but this IP address is treated differently by the NTP software. These special IP addresses are clock drivers and there are quite a few different types. If you are interested, have a look here at all the different types supported.
We will be using type 20, “Generic NMEA GPS Receiver (NMEA)” clock driver. This uses both the information from the GPS sentences and the PPS to derive the time, precice to 1us. It has the special IP address 127.127.20.x where x is the number of the GPS unit in /dev/gpsx. Open /etc/ntp.conf and make the following edits and additions.
# Comment out the freebsd servers
#server 0.freebsd.pool.ntp.org iburst maxpoll 9
#server 1.freebsd.pool.ntp.org iburst maxpoll 9
#server 2.freebsd.pool.ntp.org iburst maxpoll 9
#
# Garmin GPS 18 LVC
server 127.127.20.1 mode 0 minpoll 4 maxpoll 4 prefer
fudge 127.127.20.1 flag1 1 flag3 1 refid GPS
#
# By default we don't want eveyone to be able to query and modify
# the server. This is different from serving out NTP time to clients
restrict default noquery nomodify
#
# Allow this machine access to query NTP's statistics etc...
restrict 127.0.0.1
The mode 0 option means that NTP will use any available GPS sentences to get the time, minpoll and maxpoll is a multiple of 4 of how often the time source will be queried (so 16 seconds for us). As far as I can tell, in our case seeing as we are using the PPS, the update is acually every second. The difference is that when we query NTP about our time sources, information about the GPS unit gets updated every 16 seconds. The reachability of the GPS device and a GPS fix will be updated every 16 seconds. Prefer means that this is the prefered time source. The maxpoll statement is not needed in ntp > 4.2.6 as it will be the same as minpoll for local clock drivers. Thanks to Dave Hart for the info.
The fudge flags are extra settings that we can apply to the time source. If you read this page you can see all the flags that can be set on this time source. I will only go over what we have here. Flag1 1 enables PPS processing which we defiantely want turned on. Flag3 1 enables kernel PPS support. If this is set to 0, NTP will handle the PPS. If we didn’t have PPS support in the kernel, we would set this to 0.
Starting NTP
Before we start NTP, we quickly want to sync the system clock just to make sure it is not too far away from the actual time that it will receive from the GPS unit. NTP will fail if the offset is too great.
[root@freebsd ~]# ntpdate pool.ntp.org
25 Jan 20:45:21 ntpdate[1032]: step time server 202.174.101.10 offset -3.497534 sec
Next add ntpd to /etc/rc.conf so it starts upon boot.
ntpd_enable="YES"
Now we can finally start NTP.
[root@freebsd ~]# /etc/rc.d/ntpd start
Starting ntpd.
A bit of an anticlimax really 
. All that matters now is that the GPS unit is providing NTP with the time information. We can check this by doing an ntpq -p.
[root@freebsd ~]# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*GPS_NMEA(1) .GPS. 0 l 5 16 377 0.000 -0.001 0.002
If all is well, the reach entry should slowly climb up to 377. This will take a few minutes but as long as it is not 0 and stays at 0 as this would indicate that there is a problem somewhere. Notice the offset – this is how far the time source is away from the local clock. Remember with NTP, we are keeping the servers local clock in sync as much as possible.
Troubleshooting
Check you can access the device and verify the sentences
We can use a utility called cu to check. I found that I had to log into another session and run killall cu to end this process as Ctrl+c didn’t work. Also stop NTP before performing the following.
[root@freebsd ~]# cu -l /dev/gps1 -s 4800
Connected
$GPRMC,101003,A,1111.1111,S,22222.2222,E,000.0,250.1,250110,011.8,E*6D
You should be seeing both $GPRMC and $GPGGA sentences. An important value to lookout for in the $GPRMC sentince is the third value along: A. This means that the GPS unit has a fix. If it is U, it means that it does not have a proper GPS fix.
Verify the baud rate
Make sure the baud rate is set to 4800. I originally had the baud rate set to 9600 but I could not get NTP to detect the device. There are extra flags you can pass to NTP to change the baud rate that it will connect with, but I still could not get it to work. I was only after I changed it back to 4800 that it started to work.
Double check the wiring
I know this is a simple one, but double check everything! If you soldered a mini DIN connector, make sure there are no shorts around this. My PPS line was originally shorting to the ground which made for some interesting results. Once this was fixed, everything was up and running perfectly. Use a multimeter and verify everything.
Don’t use USB serial ports
You should only be using a physical serial port. Driver support for USB serial adapters is flaky at best, even on Windows. When I was using Linux with a USB to serial adapter to test the GPS unit, I could not get the PPS to work. It was only later that I realised the problem was the USB to serial adapter (not sure if it was the adapter itself of the Linux driver). In anycase, you will lose precision with a USB to serial adapter.
High offset and jitter
When I started to graph the offset, I noticed that NTP was floating between 20us +/-. It was after I turned the climate controll system off in the building that the offset began to float around 3us +/- which is quite precise. As NTP is used to discipline the local clock, which is usually a crystal oscillator. The tinyest variations in temperature changes the frequency of which the oscillator functions. NTP can adjust for these variations, but if they happen to quickly, then the offset can be eratic.
Referneces
http://www.satsignal.eu/ntp/FreeBSD-GPS-PPS.htm
http://time.qnan.org/
http://www.wraith.sf.ca.us/ntp/
http://www.linux.com/archive/feed/142718
http://www.freebsd.org/doc/handbook/shells.html
http://www.freebsd.org/doc/handbook/kernelconfig-building.html
January 25 2010 | Tutorials | 5 Comments »
When I deploy new servers through VMWare ESX, I usually copy an existing base that I have already setup. I keep the base image VMDK size small so it is quick to copy. When I have copied the image and am setting up a new server, I adjust the size of the VMDK in VMWare and then use gparted to fill up the space that is added to the end of the virtual disk.
This works fine for normal partitions, but as I found out today, gparted doesn’t play so nicely with LVM. Anyway, to cut a long story short, these are the commands that I had to use to fill up the remaining space of the root partition. I could have created a new partition and then added this to the volume group, but I wanted to keep things clean. After all, LVM is supposed to make things easier.
Ill assume that you want to grow your root partition. Ill also assume that you have a basic LVM structure like the default CentOS 5 partitioning layout. Also, make sure you have a backup of the data on the partitions you will be messing with. When doing these kind of operations, it is very easy for data loss to happen. I did this and it worked for me but I can’t garuntee that it will for you. The main thing to watch out for is the partitioning layout I am using in this example.
/dev/sda1 = /boot
/dev/sda2 = VolGroup00
/dev/VolGroup00/LogVol00 = /
/dev/VolGroup00/LogVol01 = swap
First increase the size of your VMDK through the appropriate tool. I use VI Client to do this for ESX. Its a bit different for VMWare Workstation and VMWare Server. Then go through these commands:
[root@linux~]# fdisk /dev/sda
The number of cylinders for this disk is set to 5221.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help):
We now need to delete the sda2 partition and re-add it. When we re-add it, we can change the number of cylinders to fill the partition up with all the remaining space. Press p to print the current partitions.
Command (m for help): p
Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 5221 41833260 8e Linux LVM
Command (m for help):
We want to delete partition 2 and re-add it again so we can fill up the remaining space. Press d and then select partition 2.
Command (m for help): d
Partition number (1-4): 2
Command (m for help):
Now we will re-create the partition. Set the partition number to 2, for the starting cylinder, set to whatever was the starting cylinder before. The default should be set as this anyway. For the end cylinder, leave the default as this will have the value of the last available cylinder.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (14-5221, default 14):
Using default value 14
Last cylinder or +size or +sizeM or +sizeK (14-5221, default 5221):
Using default value 5221
Command (m for help):
Finally we want to change the type to LVM (8E)
Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)
Command (m for help):
Save and quit with w
Command (m for help): w
You will now need to reboot. After the reboot we use the pvresize command to fill out the extra space. As fair as I can tell, this resizes the amount of space that a LVM volume group can use on a partition and needs to be run if you resize its partition.
pvresize /dev/sda2
Now we need to resize the logical volume, LogVol00. We use lvresize for this. There is a funny looking argument that is passed to resize. We don’t say that we want to fill the rest of the volume, we say that we want to add 100% of the free space to the volume.
lvresize -l +100%FREE /dev/VolGroup00/LogVol00
Finally we want to resize the actual underlying file system. I am using ext3 for the root so I use the resize2fs command
resize2fs /dev/VolGroup00/LogVol00
There is no reboot needed after this. Do a df and see the results!
December 08 2009 | Linux | No Comments »
In this rather large tutorial, I will go over setting up a Linux server to be used for user web space, shell access, FTP access any anything else that is PAM aware. All user accounts will reside in Active Directory. There is no password syncronisation or dirty scripts to pull it all together.
Overview
This tutorial is based on the setup that I deployed recently at my work. We were wanting to give students web space but didn’t want to use IIS and also didn’t want to manually manage user accounts on a Linux server. I have used Samba/Winbind to join other web servers to our domain so that administrators can edit websites through SMB/CIFS instead of using FTP which had worked fine. This guide is basically an extension of that.
This guide will follow these steps listed below:
- Step 1: Setup NTP to syncronise time
- Step 2: Setup Kerberos
- Step 3: Setup Samba
- Step 4: Join the server to the domain
- Step 5: Edit the nsswitch file
- Step 6: Edit the fstab
- Step 7: Add pam_winbind PAM module
- Step 8: Edit the user skeleton directory
- Step 9: Add the pam_mkhomedir module
- Step 10: Setup Apache
- Step 11: Setup vsftpd
- Step 12: Optional: Setup quotas
- Step 13: Optional: Setup IPTables
- Step 14: Optional: Setup sudo
Quota’s, IPTables and sudo are optional steps but I reccomend that you at least read over why you might want them.
In the end you will have a server that does the following:
- Users will be able to login using SSH with their Active Directory credentials
- Users will be able to upload files to their website using FTP with their Active Directory credentials
- Users will be able to upload files to their website through SMB/CIFS. If they are already logged onto the domain, they will not need to login again to access this.
- Users’ home directories will be automatically created
- Depending upon their AD group, users will have quotas automatically assigned.
- Depending upon their AD group, users may or may not be able to access the Internet whilst using their shell
- Depending upon their AD group, users will be able to sudo (useful for domain admins group to be able to sudo, as they don’t need to know the root password of the box)
- And more! Anything that is PAM aware will be able to use AD credentials.
Step 0: Base CentOS 5 Install
This tutorial will be using CentOS. Why? Because it is stable, mature and pretty well supported. For servers, I run nothing else. We will also make a few assumptions regarding the setup:
- Your domain is known as example.com
- The short name/NT name of your domain is EXAMPLE
- The host name for this server will be webs, with an IP of 10.0.0.50
- Your have 2 domain controllers, with names dc1 and dc2. IP’s are 10.0.0.10 and 10.0.0.20
- Your server has just one partition, the root partition. (This is only relevant for quota support)
Setup CentOS 5 on a server using as small a footprint as possible. You don’t need X or any of the development stuff. We will install the bits and pieces that you will need using yum along the way. I will assume that you have networking setup and your DNS servers are your DNS servers for your domain (usually your domain controller/s).
file: /etc/resolv.conf
search example.com
nameserver 10.0.0.10
nameserver 10.0.0.20
Also be sure to edit your hosts file and make sure this server name is listed in it.
file: /etc/hosts
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain localhost6
10.0.0.50 webs.example.com webs
Step 1: Setup NTP client
Install NTP using yum
# yum install ntp
Now ideally you would synchronise the time by editing the /etc/ntp.conf file and adding your domain controllers in as NTP servers. I have tried this a few times but I always end up with the clock out. It is very important to keep the clock in sync. If you have a clock skew of more than 5 minutes from your domain, you will loose authentication completely. I do something that isn’t ideal but it works. I set the ntpclient to syncronise the time every 5 minutes using cron. I have read that this is not recommended as the sudden change in time can upset things, but I have only read this and I haven’t come across any issues as of yet. If you choose to go down my route, add the following into your /etc/crontab.
file: /etc/crontab
*/5 * * * * root /usr/sbin/ntpdate -s dc1.example.com
Step 2: Setup Kerberos
Kerberos is an authentication scheme originally created at MIT. Windows 2000/2003/2008 domains use Kerberos and we need to set it up on the Linux server to be able to join it to AD. Edit your /etc/krb5.conf file to contain the following:
file: /etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = EXAMPLE.COM
dns_lookup_realm = false
dns_lookup_kdc = false
[realms]
EXAMPLE.COM = {
kdc = dc1.example.com:88
kdc = dc2.example.com:88
default_domain = example.com }
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
[kdc]
profile = /var/kerberos/krb5kdc/kdc.conf
[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}
If you only have one domain controller, don’t add in the line for the second one. That is it for Kerberos.
Step 3: Setup Samba
Samba is awesome. Winbind (part of samba) will be doing all the hard work to provide authentication details to SSH, FTP etc… from AD. Before we do anything though, we need to install it.
# yum install samba
It is a fairly large download, about 25MB. Once it has finished downloading and installing, move the example configuration file it comes with.
# mv /etc/samba/smb.conf /etc/samba/smb.conf.example
Now create /etc/samba/smb.conf and add the following contents, editing the parts that are suitable.
file: /etc/samba/smb.conf
[global]
# General name options
workgroup = EXAMPLE
netbios name = webs
server string =
idmap uid = 10000-20000
idmap gid = 10000-20000
security = ads
encrypt passwords = yes
realm = example.com
password server = dc1.example.com
os level = 10
# Winbind Stuff - Active Directory
winbind enum users = yes
winbind enum groups = yes
winbind nested groups = yes
winbind use default domain = yes
template shell = /bin/bash
template homedir = /home/%D/%U
obey pam restrictions = yes
# Disabled printing
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes
# Extended ACL support
map acl inherit = yes
nt acl support = yes
[homes]
path = /home/%D/%U
browsable = no
writable = yes
[userhomes$]
path = /home/EXAMPLE
comment = User home directories
valid users = @"EXAMPLE\Domain Admins"
writable = yes
create mask = 775
directory mask = 775
admin users = @"EXAMPLE\Domain Admins"
This file is pretty self explainatory. There is a bit of magic in the winbind section that sets the shell and home directory for new users and also makes sure that Samba uses PAM (obey pam restrictions, which will be important later on). The [homes] share allows users to connect to their home directories which contains their web space. The second share is optional, [userhomes$]. This is a hidden share that is only available to domain admins that will allow them to browse all users home directories.
Finally we will want to start Samba and winbind and also make sure they start upon boot.
# /etc/init.d/smb start
# /etc/init.d/winbind start
# chkconfig smb on
# chkconfig winbind on
Step 4: Join server to the domain
This step will go over joining the server to the AD domain. Just like we would join a Windows XP machine, Server 2003, Server 2008 etc… member server to provide authentication services, so to do we join the server to provide the same services. The process is slightly more detailed but is still fairly easy.
First, we use kinit to get a Kerberos ticket and to test Kerberos is working
# kinit Administrator@EXAMPLE.COM
Password for Administrator@EXAMPLE.COM
Now do the join
# net ads join -U Administrator
Administrator's password
Joined 'WEBS' to realm 'EXAMPLE.COM'
Now test that we can get users and groups from AD.
# wbinfo -u
and
# wbinfo -g
If you can see users and groups of your domain, great! If not, you most likely got an error with the join. Double check some key areas such as your /etc/resolv.conf, hosts file and make sure that winbind is running. Also check your /var/log/samba/winbindd.log for any clues
Step 5: Edit the nsswitch.conf file
The nsswitch.conf file is the configuration file for the Name Service Switch. Basically it allows you to add and order the methods for looking up account information on the server. This is different from PAM. PAM gives us the authentication, but the Name Service Switch provides the server with account information typically found in a normal /etc/passwd file and /etc/group file. For example, if you want to chown a file to a domain user, nsswitch will lookup the databases configured, look in the local /etc/passwd file and then will use winbind to lookup that user on the domain if they don’t exist in the passwd file.
Edit the /etc/nsswitch.conf file so that winbind is added as a lookup method for passwd and group.
file: /etc/nsswitch.conf
passwd: files winbind
shadow: files
group: files winbind
Leave the rest untouched.
Step 6: Edit the fstab
This is not really nesessary, but will allow you to set file permissions through Windows on files hosted on the server. We need to add the acl flag to the mount options of the file system where the user homes are. In this instance, there is only one partition, /. Remove the defaults option and replace it with rw,acl
file: /etc/fstab
LABEL=/ / ext3 rw,acl 1 1
Now we need to remount the file system using the following command:
# mount -o remount /
This should take a few seconds. Now your file system is mounted with extended ACL support. Not only will it mean that samba can set permissions, but you will also be able to set more than the 3 default permissions (owner, group, others) on files and folders.
Step 7: Add winbind as a PAM module
This will give us the authentication to the server and will allow PAM aware programs (SSH, vsftpd) to know who we are and authenticate us. We need to add it to the auth, account and session parts of PAM. Open the /etc/pam.d/system-auth file and make the following changes:
file: /etc/pam.d/system-auth
#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth sufficient pam_winbind.so
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth required pam_deny.so
account sufficient pam_winbind.so
account required pam_unix.so
account sufficient pam_succeed_if.so uid < 500 quiet
account required pam_permit.so
password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok
password required pam_deny.so
session sufficient pam_winbind.so
session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
Take note of the warning up the top. authconfig will regenerate the file if it is run. Luckily, it seems the only way this is run is if you run it yourself. Upgrading CentOS versions (5.0 – 5.1 etc…) does not run authconfig again which I thought it might. Just remember what you did or write it down somewhere else incase it does get wiped.
Before you get ahead of yourself and try to login with SSH, we need to do a couple more things.
Step 8: Edit the user skeleton directory
The directory contained in /etc/skel is the directory where all new user accounts are sourced from. At the very least we need to add a folder to it so that all users will have a public_html directory. If you want to edit any other settings or add any default files or folders that all new user accounts will have, add them in here.
# mkdir /etc/skel/public_html
This will make sure that all new user accounts will have a public_html directory. This directory is the web root for the user.
Step 9: Add the pam_mkhomedir module
The pam_mkhomedir module will automatically create the user’s home directory if it does not already exist. This saves us having to write scripts that would get all the users out of AD and create their home directories for example. We add it to the system-auth file.
file: /etc/pam.d/system-auth
#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth sufficient pam_winbind.so
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth required pam_deny.so
account sufficient pam_winbind.so
account required pam_unix.so
account sufficient pam_succeed_if.so uid < 500 quiet
account required pam_permit.so
password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok
password required pam_deny.so
session required pam_mkhomedir.so skel=/etc/skel umask=0022 silent
session sufficient pam_winbind.so
session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
Add the section in bold to the file and save it. At this stage, AD users should be able to login using SSH and have their home directory automatically created for them. They should also be able to access their home directory through SMB/CIFS using a UNC path and if they are already logged into the domain they will not need to enter their credentials in again. Any other PAM aware applications that you also have installed on your server should also use AD to authenticate as well. In the next section, we will setup Apache to serve pages out of user home directories and also setup FTP for users to be able to access their home directories and upload files to their website.
Step 10: Setup Apache
We will now install and configure Apache to serve pages out of the users public_html directory. First, install Apache with yum and set it to start upon boot. Don’t start it yet though, we need to make a few changes to its config.
# yum install httpd
# chkconfig httpd on
Now open up /etc/httpd/conf/httpd.conf and scroll down to about line 350. Make the following changes in bold to the file and save it.
<IfModule mod_userdir.c>
#
# UserDir is disabled by default since it can confirm the presence
# of a username on the system (depending on home directory
# permissions).
#
# To enable requests to /~user/ to serve the user's public_html
# directory, remove the "UserDir disable" line above, and uncomment
# the following line instead:
#
UserDir public_html
</IfModule>
<Directory "/home/EXAMPLE/*/public_html">
# Allow indexes
Options +Indexes
</Directory>
This will enable user directories and also enable directory indexes for all users. Directory indexes will display a list of files and folders if there is no index.html, index.php etc… file. Some people choose not to enable this as it can be considered a bit of a security threat. This is true in some situations, but for the purpose of this server, I feel it makes it more functional having them turned on.
Now we will go ahead and start Apache.
# /etc/init.d/httpd start
Apache should start with no problems. Now go ahead and login as a domain user through SSH or SMB/CIFS and put something in their public_html directory. Apache serves user directories out of http://servername/~username. So in our setup, assuming we logged in as the user, user1, the URL would be http://webs/~user1. This would point to the root of user1’s public_html directory.
Step 11: Setup vsftpd
Vsftpd – Very Secure File Transfer Protocol Daemon, is the standard FTP server that ships with CentOS. Make sure it is installed via yum.
# yum install vsftpd
We need to edit the config file to change a few settings and to make sure that it uses PAM for its user database. Anything in bold you will need to change from the defaults.
file: /etc/vsftpd/vsftpd.conf
# Example config file /etc/vsftpd/vsftpd.conf
#
# The default compiled in settings are fairly paranoid. This sample file
# loosens things up a bit, to make the ftp daemon more usable.
# Please see vsftpd.conf.5 for all compiled in defaults.
#
# READ THIS: This example file is NOT an exhaustive list of vsftpd options.
# Please read the vsftpd.conf.5 manual page to get a full idea of vsftpd's
# capabilities.
#
# Allow anonymous FTP? (Beware - allowed by default if you comment this out).
anonymous_enable=NO
#
# Uncomment this to allow local users to log in.
local_enable=YES
#
# Uncomment this to enable any form of FTP write command.
write_enable=YES
#
# Default umask for local users is 077. You may wish to change this to 022,
# if your users expect that (022 is used by most other ftpd's)
local_umask=022
#
# Uncomment this to allow the anonymous FTP user to upload files. This only
# has an effect if the above global write enable is activated. Also, you will
# obviously need to create a directory writable by the FTP user.
#anon_upload_enable=YES
#
# Uncomment this if you want the anonymous FTP user to be able to create
# new directories.
#anon_mkdir_write_enable=YES
#
# Activate directory messages - messages given to remote users when they
# go into a certain directory.
dirmessage_enable=YES
#
# Activate logging of uploads/downloads.
xferlog_enable=YES
#
# Make sure PORT transfer connections originate from port 20 (ftp-data).
#connect_from_port_20=YES
#
# If you want, you can arrange for uploaded anonymous files to be owned by
# a different user. Note! Using "root" for uploaded files is not
# recommended!
#chown_uploads=YES
#chown_username=whoever
#
# You may override where the log file goes if you like. The default is shown
# below.
#xferlog_file=/var/log/vsftpd.log
#
# If you want, you can have your log file in standard ftpd xferlog format
xferlog_std_format=YES
#
# You may change the default value for timing out an idle session.
#idle_session_timeout=600
#
# You may change the default value for timing out a data connection.
#data_connection_timeout=120
#
# It is recommended that you define on your system a unique user which the
# ftp server can use as a totally isolated and unprivileged user.
#nopriv_user=ftp
#
# Enable this and the server will recognise asynchronous ABOR requests. Not
# recommended for security (the code is non-trivial). Not enabling it,
# however, may confuse older FTP clients.
#async_abor_enable=YES
#
# By default the server will pretend to allow ASCII mode but in fact ignore
# the request. Turn on the below options to have the server actually do ASCII
# mangling on files when in ASCII mode.
# Beware that on some FTP servers, ASCII support allows a denial of service
# attack (DoS) via the command "SIZE /big/file" in ASCII mode. vsftpd
# predicted this attack and has always been safe, reporting the size of the
# raw file.
# ASCII mangling is a horrible feature of the protocol.
#ascii_upload_enable=YES
#ascii_download_enable=YES
#
# You may fully customise the login banner string:
ftpd_banner=Welcome to example.com WEBS FTP server!
#
# You may specify a file of disallowed anonymous e-mail addresses. Apparently
# useful for combatting certain DoS attacks.
#deny_email_enable=YES
# (default follows)
#banned_email_file=/etc/vsftpd/banned_emails
#
# You may specify an explicit list of local users to chroot() to their home
# directory. If chroot_local_user is YES, then this list becomes a list of
# users to NOT chroot().
#chroot_list_enable=YES
# (default follows)
#chroot_list_file=/etc/vsftpd/chroot_list
#
# You may activate the "-R" option to the builtin ls. This is disabled by
# default to avoid remote users being able to cause excessive I/O on large
# sites. However, some broken FTP clients such as "ncftp" and "mirror" assume
# the presence of the "-R" option, so there is a strong case for enabling it.
#ls_recurse_enable=YES
#
# When "listen" directive is enabled, vsftpd runs in standalone mode and
# listens on IPv4 sockets. This directive cannot be used in conjunction
# with the listen_ipv6 directive.
listen=YES
#
# This directive enables listening on IPv6 sockets. To listen on IPv4 and IPv6
# sockets, you must run two copies of vsftpd whith two configuration files.
# Make sure, that one of the listen options is commented !!
#listen_ipv6=YES
pam_service_name=vsftpd
userlist_enable=YES
tcp_wrappers=YES
chroot_local_user=YES
session_support=YES
Save and close the file. An important line in the file is session_support=YES. This is not talking about an FTP session, but rather to make sure to use PAM session support. This means that it will run the pam_mkhomedir module and any other session modules that we configure. Now start proftpd and also don’t forget to add it to start on boot.
# /etc/init.d/vsftpd start
# chkconfig vsftpd on
Thats it! You can stop now and you will have a fully functional shell/web server for users of your Active Directory. If you continue I will go over setting up quotas, securing access using IPTables and allowing the Domain Admins group root access.
Step12: Optional – Setup quotas
Quotas will limit the amount that users are allowed to store on the server. This prevents users using their webspace to host large files which is probably what we want as administrators. The quotas can be based on a group so that in this case, anyone who is a member of the group “students” will be alloted 100MB. Note that this support of quotas based on groups is not Linux “group quotas” which will allot a group of users an overall quota. Also note that this support of quotas based on groups is not natively supported, but will can be by using a script that is executing upon login using PAM.
But before we go into details, we need to setup quota support on the file system. This guide assumes that you have just one partition, your root partition. If you have your /home on another partition, then set the quota options to your /home partition. We will edit the /etc/fstab file to enable the support and then remount the file system.
file: /etc/fstab
LABEL=/ / ext3 rw,acl,usrquota 1 1
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SWAP-sda2 swap swap defaults 0 0
# mount -o remount /
Add the usrquota option to enable user quotas. Now we need to create the file that will store the user quotas.
# touch /aquota.user
# chmod 600 /aquota.user
# chown root:root /aquota.user
Now we run quotacheck to get the most recent state of our quotas.
quotacheck -vgum /
Quota support is now enabled. Now there are a few ways to apply quotas to users. We can use the command line programs to manually apply quotas to users (very undesirable), write a script that fetches all the users in AD and applies a quota to each user every hour or so (also undesirable), use the pam_setquota module, or use the pam_script module and run a script at the PAM session stage. I couldn’t get the pam_setquota module to compile (latest source I could get was 2006), so I used the pam_script module and wrote a script that would run whenever a user logged in, used samba or used FTP. Create the following script in /usr/local/bin/quota_set.sh.
file: /usr/local/bin/quota_set.sh
#!/bin/bash
# Sets the users quota to 100MB if they are a student
#
# Get the user name
user=$1
# Will return a string if the user is a student
retval=`groups $user`
# If the user is a student, then set the quota to 100MB
if [[ $retval =~ " student " ]]
then
# Set the quota of the user to 100MB, with a 120MB hard limit
setquota -u $user 100000 120000 0 0 -a /
fi
Read through the comments to see how it works. Change the bolded value to match the group of your users. In this instance, all students are members of the AD group “student”. Note that anyone with the string ” student ” (note the spaces between student) will have this quota applied to them. Also feel free to change the amount of quota users get (the 100000 and 120000 values, where 1 = 1kB).
Also don’t forget to make it executable.
# chmod +x /usr/local/bin/quota_set.sh
Now we need to add this as a parameter of the pam_script module. First lets make sure that pam_script is installed.
# yum install pam_script
Now edit the /etc/pam.d/system-auth file to include this module
file: /etc/pam.d/system-auth
#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth sufficient pam_winbind.so
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth required pam_deny.so
account sufficient pam_winbind.so
account required pam_unix.so
account sufficient pam_succeed_if.so uid < 500 quiet
account required pam_permit.so
password requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok
password required pam_deny.so
session required pam_mkhomedir.so skel=/etc/skel umask=0022 silent
session required pam_script.so runas=root onsessionopen=/usr/local/bin/quota_set.sh
session sufficient pam_winbind.so
session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
When a user opens a session (using FTP, logging into SSH etc…) this script will be run and will set the user’s quota. Note that the quota can be changed later down the track as the script will run everytime a PAM session is created.
Step 13: Optional – Setup IPTables
This step will be dependant on the hostility of the users. This server was originally comissioned for use in a school environment. The possibility for users to use this server as a tunnel out to the Internet was not just a possibility, but a ceartainty. But what about PHP proxy scripts you might ask? Well that can be solved as well. We will use IPTables user matching module to allow outgoing connetions to only ceartain user groups. The idea of matching users based on their group might sounnd really cool, and that’s because it is.
CentOS luckily has this support already enabled. Below is the firewall script that I use and it works quite well. Create a file /etc/sysconfig/firewall.sh and add the following:
file: /etc/sysconfig/firewall.sh
#!/bin/bash
# Flush tables
iptables -F
# Set defaults
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP
########## INPUT RULES ##########
# Stateful inpection input
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Allow HTTP
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
# Allow SSH from internal network
iptables -A INPUT -s 192.168.0.0/24 -p tcp --dport 22 -j ACCEPT
# Allow FTP
iptables -A INPUT -p tcp --dport 21 -j ACCEPT
# Allow ping
iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT
######### OUTPUT RULES ##########
# Set some default outgoing rules that are allowed
iptables -A OUTPUT -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp --dport 53 -j ACCEPT
# Only allow domain admins and root to make new outgoing connetions that
# are not pre-defined
#
# Need to do this so students don't use this server to tunnel to the
# internet, access web, ssh to different servers etc...
#
# Accepted users
#
iptables -A OUTPUT -m owner --gid-owner domain\ admins -j ACCEPT
iptables -A OUTPUT -m owner --gid-owner root -j ACCEPT
iptables -A OUTPUT -p tcp --sport 80 -m owner --uid-owner apache -j ACCEPT
#
# Rejected users
#
iptables -A OUTPUT -m owner --gid-owner domain\ users -j REJECT
iptables -A OUTPUT -m owner --gid-owner apache -j REJECT
# Finally accept new connections if no other match is made
iptables -A OUTPUT -m state --state NEW -j ACCEPT
The important lines are at the end of the file bolded. By default, we want to make sure that the AD group “Domain Admins” is allowed to make new connections as well as the “root” group. Also, the rule matching the Apache user with a source port of 80 is important as we want to make sure Apache is allowed to send out packets from its sessions but we want to restrict it from sending out new connections which would be the Apache user attempting to access a web site to proxy to the user. We then need to REJECT users that haven’t matched already. The AD group “Domain Users” will be denied access as well as Apache from making outgoing connections.
Now we want to make the file executable and add it to the /etc/rc.local. There is a “correct” way to do firewalling/IPTables in CentOS but I prefer to write a firewall script and run it on boot.
# chmod 700 /etc/sysconfig/firewall.sh
# chown root:root /etc/sysconfig/firewall.sh
# echo "/etc/sysconfig/firewall.sh" >> /etc/rc.local
Apply the firewall and make sure everything still works
# /etc/sysconfig/firewall.sh
NOTE: You might notice that all users are allowed to ping. This is because an ICMP ping packet is generated in the kernel space and not in userland programs, therefore the owner of the packet is root and not the user. Try using links or wget to confirm that the user matching works as these packets will be owned by the user that is requesting them.
Step 14: Optional – Setup sudo
Sudo, or super user do, allows you run or do things as root using your normal user account. You don’t need to know the root password, but just have to be in the /etc/sudoers file. You can allow users to do everything, or just a select few things. What we want to do is allow Domain Admins full access to the server, like they have full access to other member servers (by default).
We can’t edit the sudoers file directly, but rather have to edit it using the visudo command. Add these lines after you have run visudo
# visudo
file: /etc/sudoers
## Allows domain admins root privilages using their password
%Domain\ Admins ALL=(ALL) ALL
Now Domain Admins will have root prviliges using their current credentials.
Conclusion
You should now have a web/shell/FTP server that will authenticate off Active Directory. If you chose to add in the extra steps, your server will be even more secure against possible misuse. There are more things that you can do such as emailing users who have gone over their quota and searching for and deleting media files (mp3, avi, etc…). I’ll leave these things up to you if you wish to implement them.
Feel free to comment with suggestions, questions or to correct any mistakes.
August 31 2009 | Linux and Windows | No Comments »
Solid state hard drives are the new craze – and for good reason too. They offer a linear access speed regardless of where the data is located on the drive, a improved MTBF (Mean Time Between Failure) and have a lightening fast access/seek time. We are looking at using SSD drives for our new servers and I wondered how reliable they were compared to normal hard drives. For example, would a single SSD drive be more reliable that 2 SATA hard drives in RAID 1? The answer is a definite “no” and by a long shot.
Hourly Reliability
I did a few calculations and plotted two different graphs. The first one is the probabilities of failure within the same hour. It can be assumed that 2 drives dieing in the same hour in a RAID1 array would destory the array.
Daily Reliability
The next graph shows the probability of a drive failing in a 24 hour period.
RAID 1 Failure
It really shows how well RAID1 is against drive failure. 1 SSD versus 2 SATA in RAID1 is no where near as reliable. The reason why there is such a massive difference is that for a RAID1 drive to fail, both drives have to fail within the period of bringing up a replacement drive. In other words, drive 1 and drive 2 need to fail. To calculate this, we bring the probability of a single drive failing to the exponent of the number of drives in the RAID1 array.
RAID 0 Failure
It is interesting to see that the SSD’s in RAID0 have a failure rate less than a single SATA drive. The extra speed gained by SSD’s in RAID0 is quite a small cost in terms of reliability. To calculate the probability of a RAID0 failing it is simply if drive 1 or drive 2 fails.
May 07 2009 | Uncategorized | No Comments »
The host cache in Skype keeps a database of peers that Skype talked to upon last running. A host cache is one of several bootstrapping technologies that peer-to-peer networks use to connect a peer into the overlay network.
The host cache is kept in the shared.xml file located in the users home directory. If you look at how the host cache is stored, it looks like a jumble of hex.
...41C8010500410502004C6E771C823B0001040002B981EDCE043
B981EDCE04000400050041050200180818ADAFD40001040002BBA6
8CCE040003BBA68CCE040004000500410502004A38D3323D990001
040002B981EDCE040003B981EDCE04000400050041050200972FBC
6464420001040002B981EDCE040003B981EDCE0400040005004105
02005169EEF23F930001040002BD81EDCE040003BD81EDCE040004
000500410502005C0F17B769F90001020002C481EDCE040003FA81
EDCE040004000500410502005C4A838623360001020002BB81EDCE
040003FA81EDCE04000400050041050200440AAA7D5EF100010200
02BC81EDCE040003FA81EDCE040004000500410502003AAC048...
A pattern does start to emerge and I wrote a tool to extract the IP address and port of each peer listed in the host cache. It is written in perl and requires the XML::Simple module. You can download the tool here.
I’ve only tested this with 2 different shared.xml files so let me know if you have any problems with it.
May 01 2009 | Scribbles | 2 Comments »
The Windows XP sysprep tool is quite limited in the hardware configurations that it supports. With a few hacks and tweaks you can successfully deploy the same syspreped image to both Intel and AMD hardware and uni-processor as well as multi-processor CPU’s.
Before starting, there are a few assumptions
- The master hardware is a uni-processor Intel PC
- It is as old as possible (P4 vintage is good if you can get your hands on them, its what I used)
- You are competent using sysprep already and simply want to consolidate on the number of images
Uni/Multi-Processor
This part is easy as there is enough documentation around. Unfortunately I cannot remember exactly where I found the solution so I cannot give credit. Add the following line to your unattended section of sysprep
[unattended]
UpdateHAL="ACPIAPIC_MP,%WINDIR%\Inf\Hal.inf"
Intel and AMD
This is a bit harder to achieve. If you image back an image created on an Intel PC to an AMD PC, it will BSOD before mini-setup runs. This is due to the Intel Power Managment driver that runs only if the PC is an Intel. To get around this, the driver needs to be disabled before the sysprep tool is run on the master and then re-enabled after the image is deployed on the target PC’s only if the PC is Intel. This does not break the Intel image and the driver will be re-enabled on the target PC’s if it is identified as an Intel.
First, you will need to create a batch script that disables the Intel driver and then runs sysprep. Create a file called SYSPREP.BAT with the following and put it in your Sysprep directory (EG: C:\sysprep).
@echo off
cd C:\SYSPREP
echo Enabling image for AMD and Intel processors
reg add "HKLM\SYSTEM\ControlSet001\Services\intelppm" /v Start /t REG_DWORD /d 4 /f
echo Running sysprep and shutting down.
sysprep.exe -forceshutdown -mini -reseal -quiet -activated
Next we need to create a VBScript that will be run after the image has been deployed to the target PC and mini-setup has run. This will check for the CPU manufacturer throught WMI and adjust the Intel driver accordingly. Create a file called checkforintel.vbs and enter the following text. You can also download the script here.
' CheckForIntel.vbs
' Checks if the processor is an Intel and re-enables the power managment driver if it is.
' Written by Ryan D - based off the WMI sample script by Guy Thomas http://computerperformance.co.uk/
'
' ===Version History===
' 1.0 - Initial release
' 1.1 - Changed the way that the script checks for Intel machines. Now it looks at the CPU type and looks
' to see if the string "GenuineIntel" is present.
' --------------------------------------------------------------'
option explicit
const HKEY_LOCAL_MACHINE = &H80000002
dim objWMIService, objItem, colItems, strComputer, compModel, strKeyPath, strValueName, strValue, oReg
strKeyPath = "SYSTEM\ControlSet001\Services\intelppm"
strValueName = "Start"
strValue = "1"
strComputer = "."
' WMI connection to Root CIM and get the computer type
set objWMIService = GetObject("winmgmts:\\" _
& strComputer & "\root\cimv2")
set colItems = objWMIService.ExecQuery(_
"Select Manufacturer from Win32_Processor")
'Loop through the results and store the type in compModel
for each objItem in colItems
compModel = objItem.Manufacturer
next
'Get a registry object
Set oReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & strComputer & "\root\default:StdRegProv")
'Check the computer type. If the processor is an Intel, then re-enable the driver
if compModel = "GenuineIntel" then
oReg.SetDWORDValue HKEY_LOCAL_MACHINE,strKeyPath,strValueName,strValue
else
end if
' Exit
WSCript.Quit
Save this file to C:\WINDOWS\PostGhost\checkforintel.vbs. This needs to be set to automatically run after the target PC has finished mini-setup. Make the following alterations to your sysprep.inf
[Unattended]
OemSkipEula=Yes
UpdateInstalledDrivers=yes
[GuiUnattended]
AdminPassword="adminpasshere"
EncryptedAdminPassword=NO
AutoLogon=Yes
AutoLogonCount=1
OEMSkipRegional=1
TimeZone=255
OemSkipWelcome=1
[GuiRunOnce]
Command0="C:\WINDOWS\PostGhost\checkforintel.vbs"
The admin password and and other settings are to ensure that mini-setup will run unattended. The vital parts of the process are bolded. These settings will make sure that the target PC will auto login and run the script. You may also want to add in another script so that the PC automatically reboots so it is not logged in as an administrator.
Conclusion
After following these steps, you should now have a master image that is able to be deployed on all hardware regardless of CPU brand or type of CPU. I will write a tutorial later which covers the whole sysprep process including:
- Building in mass storage drivers
- Building in drivers for all your hardware platforms
- Debugging drivers for hardware platforms
- Automatically renaming the PC’s based on MAC address
- Automatically joining the Active Directory domain
EDIT: Added the suggestion from Bastian to look for the “GenuineIntel” string instead of AMD computer models so each model doesn’t have to be entered into the script manually. Thanks for the advice : D
February 05 2009 | Windows | 26 Comments »
Background
I bought a 500GB WD MyBook World edition with the intention of using it to backup various servers on my network. I did a bit of research and found out that it ran Linux and also had a pretty decent community following; here and here. It isn’t anywhere near what the NSLU2 had (I was planning to purchase an NSLU2 and found out they were no longer in production) but it was still enough to convince me of my purchase. The plan was to mount the servers file systems locally on the MyBook and then use rsnapshot to take snapshots. The MyBook would sit in my cupboard doing its thing each night backing up files.
After purchasing the box it was pretty quick to enable SSH. I hit a brick wall when I tried to compile the CIFS module on the MyBook. It only had GCC 3.4 and the kernel was compiled with GCC 4.1 which would mean the strings wouldn’t match and wouldn’t load itself in. After many, many frustrating hours setting up a cross compiling ARM toolchain on my laptop, I managed to compile the module and eventially loaded it into the kernel.
Doing it yourself
SSH into the MyBook and issue the following commands.
# wget http://files.doylenet.net/linux/mybook/modules/2.6.17.14/cifs.ko
Make the directory for CIFS and copy it accross
# mkdir /lib/modules/2.6.17.14/kernel/fs/cifs
# cp cifs.ko /lib/modules/2.6.17.14/kernel/fs/cifs/
Now we would normally use depmod to add the module to the modules.dep file and find any dependancies that module requires, but the MyBook doesn’t have it installed and I couldn’t be bothered compiling it, so we can add the line that is required manually. Don’t forget the double “>”’s!!! I cannot stress this enough. If you don’t use >> then the entire file will get overwritten and you will brick your MyBook!
# echo "/lib/modules/2.6.17.14/kernel/fs/cifs/cifs.ko:" >> \
# /lib/modules/2.6.17.14/modules.dep
Now use modprobe to load the module into the kernel
# modprobe cifs
Finally we mount the CIFS share using the mount.cifs program (its part of samba). Add /usr/local/samba/sbin into your PATH if you want to use the “mount -f cifs” style, but the way shown below works fine.
# /usr/local/samba/sbin/mount.cifs //server/share /mnt \
# -o username=someuser,password=somepass
That should be it! Remeber to pass the ro (read only) option if you are using this for backup purposes. And remember, I take NO responsibility if you brick your MyBook. This worked for my 500GB MyBook World. I assume it will work for other models but won’t garuntee anything.
Conclusion
It was painful getting this module to compile so it would load cleanly into the kernel. I had a lot of trouble with buildroot (what the MyBook is based off) but eventially found a version that compiled for me. Ill blog a bit later on getting NFS mounted onto the MyBook as well. I’ve got the module ready but I am having a couple of issues getting nfs-utils to compile.
October 18 2008 | Uncategorized | 23 Comments »
I recently had to replace a failed drive in my Linux server (in fact, the server that this blog is hosted). It is setup as 2 x 200GB PATA hard drives configured in Linux software RAID1.
Once you have identified that the RAID has failed (you will get an email about the event if you have set your server up properly), make sure you have a disk of equal of greater size. I only had a spare 250GB HDD spare, so I used that.
The following commands assume that hdc was the failed drive and that hda is the drive that is still working
# sfdisk -d /dev/hda | sfdisk /dev/hdc
# mdadm /dev/md0 -a /dev/hdc1
# mdadm /dev/md1 -a /dev/hdc2
# mdadm /dev/md2 -a /dev/hdc3
I have 3 partitions, md0 is the boot partition, md1 is swap and md2 is my root partition. Modify your configuration to suit. You can then view the rebuilding by executing
# watch -n .5 ‘cat /proc/mdstat’
You will also want to copy over the boot record so you will be able to boot the server from hdc incase hda fails next. Pretty much every linux uses grub now so I will show how to use that.
# grub
grub> root (hd1,0)
grub> setup (hd1)
grub> quit
And that should be it. That is what I did on my system and it worked fine. That said I don’t take any responsibility for breaking anyones RAID.
October 08 2008 | Uncategorized | No Comments »
I’ve recently completed a small project for Uni; an image based CAPTCHA mechasisim I have called Jaci (Just Another Captcha Implementation). It requires you to drag and drop relevent images onto eachother in order to pass the test. The images are not static and uses Google Image Search for the source of the images.
The main motivation behind the work is a dislike for current CAPTCHA mechasisims. I have good vision and still find the common distorted word captcha frustrating at times. This test is useless for the dislexic and vision impared and creates some accessability problems that will be more and more prevailant as CAPTCHA type mechanisims are intergrated into everyday life. To read more about it, check it out here http://ryandoyle.net/devel/jaci
September 06 2008 | Uncategorized | No Comments »
I recently needed to find a function that would be able to bias random numbers. Out of a set of 1000 random numbers, I wanted more of these to be smaller instead of true random numbers (or as true as random number generators are). I had a look at simple parabolic and exponential functions and eventially devised the following equation.
Where b = factor of bias and c = max. integer of random function. The higher value of b, the more biased the function will be towards lower numbers. c is defined as when f(x) = x. As stated previously, c is the highest possible integer of the random function you are using. If your RNG is generating a maximum number of 1000, then c = 1000.
Shown is a plot of several b values
September 01 2008 | Scribbles | No Comments »
Next »