= !SilverFile General Server Setup = == Installation of Ubuntu 9.04 Server 32bit == We are selecting this version for the following reasons: * GRUB installation doesn't work in older releases. * 64 bit version does not seem to support the VIA padlock engine === Partitioning (During Install) === We feel that 20GB and 6GB of swap is more than sufficient, leaving ample room for client files. * / 20GB Primary Ext3 Bootable = Yes (leave all other defaults) * swap 6GB swap * /FILES/ Primary Ext3 Bootable = No (leave all other defaults) === Primary User - sf === Add sf user with sudo privileges during install === Install SSH Deamon === {{{ > sudo apt-get install ssh }}} === Config SSH === edit /etc/ssh/sshd_config {{{ Port 2222 PermitRootLogin no PasswordAuthentication no }}} Change port line to 2222 (or whatever port) restart sshd to check {{{ /etc/init.d/ssh restart }}} === Networking === '''Edit /etc/network/interfaces''' {{{ auto eth0 iface eth0 inet static address 192.168.1.27 netmask 255.255.255.0 gateway 192.168.1.1 }}} '''Edit /etc/resolv.conf''' {{{ nameserver 68.6.16.30 nameserver 68.2.16.30 }}} === VIA Padlock and OpenSSL === ==== Openssl Installation ==== {{{ > sudo apt-get install openssl }}} ==== Padlock Verification ==== Next, verify engine: {{{ > openssl engine (padlock) VIA PadLock (no-RNG, ACE) (dynamic) Dynamic engine loading support }}} The response string should include '`(padlock) VIA PadLock (no-RNG, ACE)`'. ==== Make default engine Padlock ==== {{{ > vim /etc/ssl/openssl.cnf }}} Add the following under oid_section = new_oids {{{ ... oid_section = new_oids # Enable Via Padlock by default openssl_conf = openssl_def [openssl_def] engines = openssl_engines [openssl_engines] padlock = padlock_engine [padlock_engine] default_algorithms = ALL }}} === Install Duplicity / S3tools / Rsync === Duplicity is our preferred backup method. Install s3tools for S3. Rsync for app syncing. {{{ > sudo apt-get install duplicity python-boto s3cmd rsync }}} === Install Apache for Django === [https://help.ubuntu.com/7.10/server/C/httpd.html#https-configuration Great Ubuntu Apache/SSL How-To] {{{ > sudo apt-get install apache2 libapache2-mod-python > sudo ln -s /usr/sbin/apache2ctl /usr/sbin/apachectl (old habits die hard) }}} === Install Django === The platform for our app. {{{ > sudo apt-get install python-django }}} === Install MySQL === Install MySQL with python db support mysqldb. {{{ > sudo apt-get install mysql-server python-mysqldb }}} You will be asked to supply a mysql root password during installation. Generate a random password and save it. {{{ > mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.0.67-0ubuntu6 (Ubuntu) r Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> CREATE DATABASE silverfile CHARACTER SET utf8; Query OK, 1 row affected (0.00 sec) mysql> GRANT ALL ON silverfile.* TO sf@localhost IDENTIFIED BY '<>'; Query OK, 0 rows affected (0.00 sec) }}} === Install !SilverFile App === ''' Create distribution package ''' On instance with access to mercurial repository, hg clone and/or hg update to latest version {{{ .../sf-app/ > scons sdist }}} Move the dist/..tar.gz file to the new instance '''Create app directory:''' {{{ mkdir -p /usr/wwwapps/sf-app/ }}} '''Create Initial Copy From Mercurial Repo (as hank) - DEPRECATED:''' {{{ /usr/bin/rsync -av --timeout=300 --delete -e '/usr/bin/ssh -p 2240 -o ConnectTimeout=3' \ hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/configs \ hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/files \ hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/utils \ hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/third_party \ hank@dev.silverfilecorp.com:/usr/svn/sfexport/app_skins/production \ /usr/wwwapps/sf-app/ }}} '''Create symbolic links in production media - DEPRECATED:''' {{{ ln -s /usr/wwwapps/sf-app/files/media/js/admin-path.js /usr/wwwapps/sf-app/production/media/js/admin-path.js ln -s /usr/wwwapps/sf-app/files/media/js/path-suggest.js /usr/wwwapps/sf-app/production/media/js/path-suggest.js ln -s /usr/wwwapps/sf-app/third_party/jquery-autocomplete/jquery.autocomplete-current.min.js /usr/wwwapps/sf-app/production/media/js/path-suggest.js ln -s /usr/wwwapps/sf-app/third_party/jquery-json/jquery.json-current.min.js /usr/wwwapps/sf-app/production/media/js/jquery.json-current.min.js ln -s /usr/wwwapps/sf-app/third_party/jquery/jquery-current.min.js /usr/wwwapps/sf-app/production/media/js/jquery-current.min.js ln -s /usr/wwwapps/sf-app/files/media/css/admin.css /usr/wwwapps/sf-app/production/media/css/admin.css ln -s /usr/wwwapps/sf-app/files/media/css/admin-path.css /usr/wwwapps/sf-app/production/media/css/admin-path.css ln -s /usr/wwwapps/sf-app/files/media/css/jquery.autocomplete-current.css /usr/wwwapps/sf-app/production/media/css/jquery.autocomplete-current.css }}} '''Create a unique secret key for production/settings.py:''' {{{ cd /usr/wwwapps/sf-app/files/ python manage.py shell (InteractiveConsole) >>> import utils.django >>> fout = open("key", "w") >>> fout.write(utils.django.new_secret_key()) >>> fout.close() }}} Copy secret key from "key" into production/settings.py '''Copy DB credentials into settings.py''' {{{ DATABASE_PASSWORD = '<>' }}} '''Initialize App Database''' {{{ cd files/ python manage.py syncdb --pythonpath=.. --settings=production.settings }}} === Configure SSL === {{{ > sudo a2enmod ssl }}} copy cert file (''mydomain.com''.crt) to /etc/ssl/certs copy key file (''mydomain.com''.key) to /etc/ssl/private === Configure Apache for !SilverFile App === Django app is turned "on" by default. Two locations (webspaces) are turned off, and are served directly. These are: * site_media (css, js, images, etc...) * site_files (these are all the client files) See sf-apps/files/examples {{{ > sudo ln -s /usr/wwwapps/sf-app/production/apache/files.conf /etc/apache2/sites-available/silverfile > mkdir /usr/wwwapps/logs/ > touch /usr/wwwapps/logs/silverfile.access > touch /usr/wwwapps/logs/silverfile.error > sudo ln -s /etc/apache2/sites-available/silverfile /etc/apache2/sites-enabled/silverfile }}} Example virtual host conf file: {{{ # Edit here: # ServerName sfxxx.silverfilecorp.com ServerName 127.0.0.1 # ServerAlias 127.0.0.1 ErrorLog "/usr/wwwapps/logs/silverfile.error" CustomLog "/usr/wwwapps/logs/silverfile.access" common # Edit Here: # SSLEngine on # SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire # SSLCertificateFile /etc/ssl/certs/docs.silverfilecorp.com.crt # SSLCertificateKeyFile /etc/ssl/private/docs.silverfilecorp.com.key # Django app SetHandler python-program PythonHandler django.core.handlers.modpython SetEnv DJANGO_SETTINGS_MODULE production.settings PythonPath "['/usr/wwwapps/sf-app'] + sys.path" PythonOption django.root PythonDebug On # Site media files - css, js, img Alias /site_media /usr/wwwapps/sf-app/production/media SetHandler none allow from all # Admin media files - css, js, img Alias /media /var/lib/python-support/python2.5/django/contrib/admin/media SetHandler none allow from all # Client Files Alias /docs /FILES SetEnv DJANGO_SETTINGS_MODULE production.settings PythonOption DJANGO_SETTINGS_MODULE production.settings PythonPath "['/usr/wwwapps/sf-app'] + sys.path" PythonAccessHandler files.common.modpython PythonDebug On SetHandler none allow from all }}} '''Edit /etc/init.d/apache2''' We change LANG=C to LC_ALL=en_US.UTF-8 for our locale: {{{ ENV="env -i LC_ALL=en_US.UTF-8 PATH=/usr/local/bin:/usr/bin:/bin" }}} == 3Ware RAID monitor == As root: {{{ cd /usr/local/bin/ scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third-party/3ware-twcli/tw_cli tw_cli scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third-party/3ware-twcli/tw_status tw_status chmod 755 tw_cli chmod 700 tw_status }}} Note- Make sure that CLI var in tw_status is correct. RAID client number (c1 etc..) may differ. Add crontab to monitor RAID {{{ ########################### # monitor raid array ########################### */10 * * * * /usr/local/bin/tw_status CRON }}} tw_status will send alerts to monitor@silverfilecorp.com == SMTP through google accounts == Make sure sendmail isn't installed: {{{ sudo apt-get remove sendmail sudo apt-get install ssmtp mailx mailutils }}} Make sure sendmail libraries are linked to ssmtp: {{{ sudo rm /usr/lib/sendmail sudo ln -s /usr/lib/ssmtp /usr/lib/sendmail }}} Edit /etc/ssmtp/ssmtp.conf: {{{ root=monitor@silverfilecorp.com mailhub=smtp.gmail.com:587 UseSTARTTLS=yes UseTLS=yes AuthUser=monitor@silverfilecorp.com AuthPass=<< monitor password >> }}} == Users and Permissions on /FILES == '''Add nologin to shells''' {{{ echo "/usr/sbin/nologin" >> /etc/shells }}} '''Add fileuser with nologin''' {{{ useradd -s /usr/sbin/nologin fileuser passwd fileuser < fileuser passwd selected by firm > }}} '''Create fileusers group''' {{{ addgroup fileusers adduser sf fileusers adduser fileuser fileusers adduser www-data fileusers }}} '''Set up appropriate permissions on /FILES''' {{{ chown -R fileuser /FILES chgrp -R fileusers /FILES chmod -R 660 /FILES find /FILES -type d -exec chmod 770 {} \; find /FILES -type f -exec chmod 550 {} \; }}} == Samba Installation == '''Installation''' {{{ sudo apt-get install samba }}} '''Add a fileuser as samba user''' {{{ sudo smbpasswd -a fileuser }}} '''Copy samba config files from repo''' {{{ cd /etc/samba/ mv smb.conf smb.conf.org scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third-party/samba/smb.conf.master smb.conf.master scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third-party/samba/recycle.conf recycle.conf chmod 644 smb.conf.master chmod 644 recycle.conf testparm -s smb.conf.master > smb.conf }}} '''Create smbusers file''' fileuser = {{{ fileuser = matt }}} == Firewall == Rules allow ssh(2222), https, and Samba (from LAN) {{{ ufw allow from 174.143.232.60 to any port 2222 ufw allow from 98.173.61.70 to any port 2222 ufw allow from 208.69.40.208/29 to any port 2222 ufw allow from 192.168.0.0/24 to any app Samba ufw allow "Apache Secure" ufw logging on ufw enable ufw status }}} SSH rules above add access from jump.silverfilecorp.com(174.143.232.60), SF/BajaBound office (98.173.61.70), and BajaBound Network (208.69.40.208/29) == Java == {{{ sudo apt-get install sun-java6-bin }}} == Setting up a Software RAID 1 - (DEPRECATED) == We are using the hardware raid cards now, so this is deprecated. Per recommendations from our friends at !MonkeyBrains, we'll set up a software RAID 1 and monitor it with mdadm. The plan is to sync up RAID health with SNMP monitoring. This is a very good tutorial on [http://ubuntuadministrator.com/?p=3 setting up a software RAID 1] please follow it for the step by step RAID install. The idea is to create 3 partitions: * / (root) where all the OS etc files go * /FILES where all the documents go * Swap The necessary swap partition On a 500 GB drive I propose doing this : * /FILES = 430 GB * Swap = 4 GB (swap is conventionally 2X RAM, a swap this size may not be necessary with 2 GB of RAM and for use as a file server * / = In order to create the software RAID, you first create regular primary partitions on the first disk (SDA) as in the following: {{{ select: Partition Disk Manually select: Device SDA1 Create new empty partition table on this device: yes Select Free Space (pri/log): select: Create new primary partition Mount point: / Bootable Flag: on Use as: Select Physical Volume For RAID select: Done Setting up partition }}} And then you create an MD device from each partition. Again, see the [http://ubuntuadministrator.com/?p=3ubuntu RAID tutorial] as it explains exactly how to do this step by step. == Check Software RAID Status == {{{ mdadm --detail /dev/md0 }}}