= !SilverFile General Server Setup = == Installation of Ubuntu 9.04 Server 32bit == We are selecting this version for the following reasons: * GRUB installation doesn't work in older releases. * 64 bit version does not seem to support the VIA padlock engine (as of 9.10 64 bit still doesn't work) === Partitioning (During Install) === We feel that 20GB and 6GB of swap is more than sufficient, leaving ample room for client files. * / 20GB Primary Ext3 Bootable = Yes (leave all other defaults) * swap 6GB swap * /FILES/ Primary Ext3 Bootable = No (leave all other defaults) === Primary User - sf === Add sf user with sudo privileges during install === Install SSH Deamon === {{{ > sudo apt-get install ssh }}} === Config SSH === edit /etc/ssh/sshd_config {{{ Port 2222 PermitRootLogin no PasswordAuthentication no }}} Change port line to 2222 (or whatever port) restart sshd to check {{{ /etc/init.d/ssh restart }}} === Networking === '''Edit /etc/network/interfaces''' {{{ auto eth0 iface eth0 inet static address 192.168.1.27 netmask 255.255.255.0 gateway 192.168.1.1 }}} '''Edit /etc/resolv.conf''' {{{ nameserver 68.6.16.30 nameserver 68.2.16.30 or use google's public DNS servers: nameserver 8.8.8.8 nameserver 8.8.4.4 }}} === VIA Padlock and OpenSSL === ==== Openssl Installation ==== {{{ > sudo apt-get install openssl }}} ==== Padlock Verification ==== Next, verify engine: {{{ > openssl engine (padlock) VIA PadLock (no-RNG, ACE) (dynamic) Dynamic engine loading support }}} The response string should include '`(padlock) VIA PadLock (no-RNG, ACE)`'. ==== Make default engine Padlock ==== {{{ > vim /etc/ssl/openssl.cnf }}} Add the following under oid_section = new_oids {{{ ... oid_section = new_oids # Enable Via Padlock by default openssl_conf = openssl_def [openssl_def] engines = openssl_engines [openssl_engines] padlock = padlock_engine [padlock_engine] default_algorithms = ALL }}} === Install Duplicity / S3tools / Rsync === Duplicity is our preferred backup method. Install s3tools for S3. Rsync for app syncing. {{{ > sudo apt-get install duplicity python-boto s3cmd rsync }}} === Install Apache for Django === [https://help.ubuntu.com/7.10/server/C/httpd.html#https-configuration Great Ubuntu Apache/SSL How-To] {{{ > sudo apt-get install apache2 libapache2-mod-python > sudo ln -s /usr/sbin/apache2ctl /usr/sbin/apachectl (old habits die hard) }}} === Install Django === The platform for our app. {{{ # sudo apt-get install python-setuptools # sudo easy_install pip # sudo pip install django }}} === Install MySQL === Install MySQL with python db support mysqldb. {{{ > sudo apt-get install mysql-server python-mysqldb }}} You will be asked to supply a mysql root password during installation. Generate a random password and save it. {{{ > mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.0.67-0ubuntu6 (Ubuntu) r Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> CREATE DATABASE silverfile CHARACTER SET utf8; Query OK, 1 row affected (0.00 sec) mysql> GRANT ALL ON silverfile.* TO sf@localhost IDENTIFIED BY '<>'; Query OK, 0 rows affected (0.00 sec) }}} === Install !SilverFile App === '''Create app directory:''' {{{ mkdir -p /usr/wwwapps/sf-app/ }}} ''' Grab files from jump. Files are located in: /usr/silverfile on jump.silverfilecorp.com ''' {{{ cd /usr/wwwapps/sf-app/ scp -P 2222 -r @jump.silverfilecorp.com:/usr/silverfile/sf-app/dist/files-0.2.tar.gz files.tar.gz scp -P 2222 -r @jump.silverfilecorp.com:/usr/silverfile/sf-app/utils /usr/wwwapps/sf-app/ scp -P 2222 -r @jump.silverfilecorp.com:/usr/silverfile/production /usr/wwwapps/sf-app/ }}} ''' Uncompress files tar/gzip ''' {{{ tar -zxvf files.tar.gz mv files-0.2/files /usr/wwwapps/sf-app/files rm files.tar.gz rm -r files-0.2 }}} '''Create a unique secret key for production/settings.py:''' {{{ cd /usr/wwwapps/sf-app/files/ python manage.py shell (InteractiveConsole) >>> import utils.django >>> fout = open("key", "w") >>> fout.write(utils.django.new_secret_key()) >>> fout.close() >>> exit () }}} '''Copy secret key from "key" into production/settings.py''' {{{ vim /usr/wwwapps/sf-app/files/key (vim commands) dd (cut the key to clipboard) :split /usr/wwwapps/sf-app/production/settings.py (opens setting.py, page down to old secret key) alt p (paste new secret key) :wq (save and close settings.py :q! (quit without saving key file) }}} '''Copy DB credentials into settings.py''' {{{ DATABASE_PASSWORD = '<>' }}} '''Make sure settings.py has the proper permissions (add read for other)''' {{{ chmod 754 /usr/wwwapps/sf-app/production/settings.py chmod 755 /usr/wwwapps/sf-app/production }}} '''Initialize App Database''' {{{ cd files/ python manage.py syncdb --pythonpath=.. --settings=production.settings Create Django superuser when prompted (sf) }}} === Configure SSL === {{{ > sudo a2enmod ssl }}} copy cert file (''mydomain.com''.crt) to /etc/ssl/certs copy key file (''mydomain.com''.key) to /etc/ssl/private === Configure Apache for !SilverFile App === Django app is turned "on" by default. Two locations (webspaces) are turned off, and are served directly. These are: * site_media (css, js, images, etc...) * site_files (these are all the client files) See sf-apps/files/examples {{{ > sudo ln -s /usr/wwwapps/sf-app/production/apache/files.conf /etc/apache2/sites-available/silverfile > mkdir /usr/wwwapps/logs/ > touch /usr/wwwapps/logs/silverfile.access > touch /usr/wwwapps/logs/silverfile.error > sudo ln -s /etc/apache2/sites-available/silverfile /etc/apache2/sites-enabled/silverfile }}} Example virtual host conf file: {{{ # Edit here: # ServerName sfxxx.silverfilecorp.com ServerName 127.0.0.1 # ServerAlias 127.0.0.1 ErrorLog "/usr/wwwapps/logs/silverfile.error" CustomLog "/usr/wwwapps/logs/silverfile.access" common # Edit Here: # SSLEngine on # SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire # SSLCertificateFile /etc/ssl/certs/docs.silverfilecorp.com.crt # SSLCertificateKeyFile /etc/ssl/private/docs.silverfilecorp.com.key # Django app SetHandler python-program PythonHandler django.core.handlers.modpython SetEnv DJANGO_SETTINGS_MODULE production.settings PythonPath "['/usr/wwwapps/sf-app'] + sys.path" PythonOption django.root PythonDebug On # Site media files - css, js, img Alias /site_media /usr/wwwapps/sf-app/production/media SetHandler none allow from all # Admin media files - css, js, img Alias /media /var/lib/python-support/python2.5/django/contrib/admin/media # or # Alias /media /usr/local/lib/python2.6/dist-packages/django/contrib/admin/media SetHandler none allow from all # Client Files Alias /docs /FILES SetEnv DJANGO_SETTINGS_MODULE production.settings PythonOption DJANGO_SETTINGS_MODULE production.settings PythonPath "['/usr/wwwapps/sf-app'] + sys.path" PythonAccessHandler files.common.modpython PythonDebug On SetHandler none allow from all }}} '''Edit /etc/init.d/apache2''' We change LANG=C to LC_ALL=en_US.UTF-8 for our locale: {{{ ENV="env -i LC_ALL=en_US.UTF-8 PATH=/usr/local/bin:/usr/bin:/bin" }}} '''Redirect port 80 traffic to secure port 443''' Save the original default http file: {{{ # cd /etc/apache2/site-available # mv default default.orig }}} Creat a new default file: {{{ #vim default }}} Insert: {{{ RewriteEngine on RewriteCond %{SERVER_PORT} ^80$ RewriteRule ^(.*)$ https://%{SERVER_NAME}$1 [L,R] RewriteLog "/var/log/apache2/rewrite.log" RewriteLogLevel 2 }}} Save and exit. Restart apache. It's a good idea to double check the symbolic link to 000-default in /etc/apache2/sites-enabled == 3Ware RAID monitor == As root: {{{ cd /usr/local/bin/ scp -P 2222 @jump.silverfilecorp.com:/usr/silverfile/scripts/tw_cli tw_cli scp -P 2222 @jump.silverfilecorp.com:/usr/silverfile/scripts/tw_status tw_status chmod 755 tw_cli chmod 700 tw_status }}} Note- Make sure that CLI var in tw_status is correct. RAID client number (c1 etc..) may differ. '''If RAID is rebuilt or after error condition, client may increment up''' i.e. from c4 to c5- make sure to change this! Add crontab to monitor RAID {{{ ########################### # monitor raid array ########################### */10 * * * * /usr/local/bin/tw_status CRON }}} tw_status will send alerts to monitor@silverfilecorp.com == SMTP through google accounts == Make sure sendmail isn't installed: {{{ sudo apt-get remove sendmail sudo apt-get install ssmtp mailx mailutils }}} Make sure sendmail libraries are linked to ssmtp: {{{ sudo rm /usr/lib/sendmail sudo ln -s /usr/lib/ssmtp /usr/lib/sendmail }}} Edit /etc/ssmtp/ssmtp.conf: {{{ root=monitor@silverfilecorp.com mailhub=smtp.gmail.com:587 UseSTARTTLS=yes UseTLS=yes AuthUser=monitor@silverfilecorp.com AuthPass=<< monitor password >> }}} == Users and Permissions on /FILES == '''Add nologin to shells''' {{{ echo "/usr/sbin/nologin" >> /etc/shells }}} '''Add fileuser with nologin''' {{{ useradd -s /usr/sbin/nologin fileuser passwd fileuser < fileuser passwd selected by firm > }}} '''Create fileusers group''' {{{ addgroup fileusers adduser sf fileusers adduser fileuser fileusers adduser www-data fileusers }}} '''Set up appropriate permissions on /FILES''' {{{ chown -R fileuser /FILES chgrp -R fileusers /FILES chmod -R 660 /FILES find /FILES -type d -exec chmod 770 {} \; find /FILES -type f -exec chmod 550 {} \; }}} == Samba Installation == '''Installation''' {{{ sudo apt-get install samba }}} '''Add a fileuser as samba user''' {{{ sudo smbpasswd -a fileuser }}} '''Copy samba config files from repo''' {{{ cd /etc/samba/ mv smb.conf smb.conf.org scp -P 2222 @jump.silverfilecorp.com:/usr/silverfile/scripts/smb.conf.master smb.conf.master scp -P 2222 @jump.silverfilecorp.com:/usr/silverfile/scripts/recycle.conf recycle.conf chmod 644 smb.conf.master chmod 644 recycle.conf testparm -s smb.conf.master > smb.conf }}} '''Create smbusers file''' fileuser = {{{ fileuser = matt }}} == Firewall == Rules allow ssh(2222), https, and Samba (from LAN) {{{ ufw allow from 174.143.232.60 to any port 2222 ufw allow from 67.23.24.209 to any port 2222 ufw allow from 192.168.0.0/24 to any app Samba ufw allow "Apache Secure" ufw logging on ufw enable ufw status }}} SSH rules above add access from jump.silverfilecorp.com(174.143.232.60), monitor.silverfilecorp.com (67.23.24.209). == Java == For versions prior to 10.04: {{{ sudo apt-get install sun-java6-bin }}} The default repositories in 10.04 no longer contain the java files {{{ vim /etc/apt/sources.list uncomment the Canonical's 'partner' repositories: deb http://archive.canonical.com/ubuntu lucid partner dev-src htt;://archive.canonical.con/ubuntu lucid partner save and exit the file apt-get update apt-get install sun-java6-bin }}} == Dynamic DNS == Optional... Create an account at www.dyndns.com. Use the machine name with 'dyndns.org' domain for the alias setup. Create a CName (Alias) record at godaddy. sf00x.silverfilecorp.com sf00x.dyndns.org Install ddclient on the sf box {{{ sudo apt-get install ddclient }}} You'll be prompted for answers which will complete the config. == Setting up a Software RAID 1 - (DEPRECATED) == We are using the hardware raid cards now, so this is deprecated. Per recommendations from our friends at !MonkeyBrains, we'll set up a software RAID 1 and monitor it with mdadm. The plan is to sync up RAID health with SNMP monitoring. This is a very good tutorial on [http://ubuntuadministrator.com/?p=3 setting up a software RAID 1] please follow it for the step by step RAID install. The idea is to create 3 partitions: * / (root) where all the OS etc files go * /FILES where all the documents go * Swap The necessary swap partition On a 500 GB drive I propose doing this : * /FILES = 430 GB * Swap = 4 GB (swap is conventionally 2X RAM, a swap this size may not be necessary with 2 GB of RAM and for use as a file server * / = In order to create the software RAID, you first create regular primary partitions on the first disk (SDA) as in the following: {{{ select: Partition Disk Manually select: Device SDA1 Create new empty partition table on this device: yes Select Free Space (pri/log): select: Create new primary partition Mount point: / Bootable Flag: on Use as: Select Physical Volume For RAID select: Done Setting up partition }}} And then you create an MD device from each partition. Again, see the [http://ubuntuadministrator.com/?p=3ubuntu RAID tutorial] as it explains exactly how to do this step by step. == Check Software RAID Status == {{{ mdadm --detail /dev/md0 }}}