SilverFile General Server Setup

Installation of Ubuntu 9.04 Server 32bit

We are selecting this version for the following reasons:

• GRUB installation doesn't work in older releases.
• 64 bit version does not seem to support the VIA padlock engine (as of 9.10 64 bit still doesn't work)

Partitioning (During Install)

We feel that 20GB and 6GB of swap is more than sufficient, leaving ample room for client files.

• / 20GB Primary Ext3 Bootable = Yes (leave all other defaults)
• swap 6GB swap
• /FILES/ Primary Ext3 Bootable = No (leave all other defaults)

Primary User - sf

Add sf user with sudo privileges during install

Install SSH Deamon

> sudo apt-get install ssh

Config SSH

edit /etc/ssh/sshd_config

Port 2222
PermitRootLogin no
PasswordAuthentication no

Change port line to 2222 (or whatever port) restart sshd to check

/etc/init.d/ssh restart

Networking

Edit /etc/network/interfaces

auto eth0
iface eth0 inet static
address 192.168.1.27
netmask 255.255.255.0
gateway 192.168.1.1

Edit /etc/resolv.conf

nameserver 68.6.16.30
nameserver 68.2.16.30

VIA Padlock and OpenSSL

Openssl Installation

> sudo apt-get install openssl

Padlock Verification

Next, verify engine:

> openssl engine
(padlock) VIA PadLock (no-RNG, ACE)
(dynamic) Dynamic engine loading support

The response string should include '(padlock) VIA PadLock (no-RNG, ACE)'.

Make default engine Padlock

> vim /etc/ssl/openssl.cnf

Add the following under oid_section = new_oids

...
oid_section             = new_oids

# Enable Via Padlock by default
openssl_conf = openssl_def

[openssl_def]
engines = openssl_engines

[openssl_engines]
padlock = padlock_engine

[padlock_engine]
default_algorithms = ALL

Install Duplicity / S3tools / Rsync

Duplicity is our preferred backup method. Install s3tools for S3. Rsync for app syncing.

> sudo apt-get install duplicity python-boto s3cmd rsync

Install Apache for Django

​Great Ubuntu Apache/SSL How-To

> sudo apt-get install apache2 libapache2-mod-python
> sudo ln -s /usr/sbin/apache2ctl /usr/sbin/apachectl (old habits die hard)

Install Django

The platform for our app.

> sudo apt-get install python-django

Install MySQL

Install MySQL with python db support mysqldb.

> sudo apt-get install mysql-server python-mysqldb

You will be asked to supply a mysql root password during installation. Generate a random password and save it.

> mysql -uroot -p

Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.0.67-0ubuntu6 (Ubuntu)
r
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> CREATE DATABASE silverfile CHARACTER SET utf8;
Query OK, 1 row affected (0.00 sec)

mysql> GRANT ALL ON silverfile.* TO sf@localhost IDENTIFIED BY '<<databasepwd>>';
Query OK, 0 rows affected (0.00 sec)

Install SilverFile App

Create app directory:

mkdir -p /usr/wwwapps/sf-app/

Grab files from jump. Files are located in: /usr/silverfile on jump.silverfilecorp.com

cd /usr/wwwapps/sf-app/
scp -P 2222 -r <user>@jump.silverfilecorp.com:/usr/silverfile/sf-app/dist/files-0.2.tar.gz files.tar.gz
scp -P 2222 -r <user>@jump.silverfilecorp.com:/usr/silverfile/sf-app/utils utils
scp -P 2222 -r <user>@jump.silverfilecorp.com:/usr/silverfile/production production

Uncompress files tar/gzip

tar -zxvf files.tar.gz 
mv files-0.2/files /usr/wwwapps/sf-app/files 
rm files.tar.gz
rm files-0.2

Create a unique secret key for production/settings.py:

cd /usr/wwwapps/sf-app/files/
python manage.py shell
(InteractiveConsole)
>>> import utils.django
>>> fout = open("key", "w")
>>> fout.write(utils.django.new_secret_key())
>>> fout.close()

Copy secret key from "key" into production/settings.py

Copy DB credentials into settings.py

DATABASE_PASSWORD = '<<databasepwd>>'

Initialize App Database

cd files/
python manage.py syncdb --pythonpath=.. --settings=production.settings

Create Django superuser when prompted (sf)

Configure SSL

> sudo a2enmod ssl

copy cert file (mydomain.com.crt) to /etc/ssl/certs copy key file (mydomain.com.key) to /etc/ssl/private

Configure Apache for SilverFile App

Django app is turned "on" by default. Two locations (webspaces) are turned off, and are served directly. These are:

• site_media (css, js, images, etc...)
• site_files (these are all the client files)

See sf-apps/files/examples

> sudo ln -s /usr/wwwapps/sf-app/production/apache/files.conf /etc/apache2/sites-available/silverfile
> mkdir /usr/wwwapps/logs/
> touch /usr/wwwapps/logs/silverfile.access
> touch /usr/wwwapps/logs/silverfile.error
> sudo ln -s /etc/apache2/sites-available/silverfile /etc/apache2/sites-enabled/silverfile

Example virtual host conf file:

<VirtualHost *:80>
  # Edit here:
  # ServerName sfxxx.silverfilecorp.com
  ServerName 127.0.0.1
  # ServerAlias 127.0.0.1
  
  ErrorLog "/usr/wwwapps/logs/silverfile.error"
  CustomLog "/usr/wwwapps/logs/silverfile.access" common
  
  # Edit Here:
  # SSLEngine on
  # SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
  
  # SSLCertificateFile /etc/ssl/certs/docs.silverfilecorp.com.crt
  # SSLCertificateKeyFile /etc/ssl/private/docs.silverfilecorp.com.key

  # Django app
  <Location />
    SetHandler python-program
    PythonHandler django.core.handlers.modpython
    SetEnv DJANGO_SETTINGS_MODULE production.settings
    PythonPath "['/usr/wwwapps/sf-app'] + sys.path"
    PythonOption django.root
    PythonDebug On
  </Location>
    
  # Site media files - css, js, img 
  Alias /site_media /usr/wwwapps/sf-app/production/media
  <Location /site_media/>
    SetHandler none
    allow from all
  </Location>
  
  # Admin media files - css, js, img
  Alias /media /var/lib/python-support/python2.5/django/contrib/admin/media
  <Location /media/>
    SetHandler none
    allow from all
  </Location>
  
  # Client Files
  Alias /docs /FILES
  <Location /docs/>
    SetEnv DJANGO_SETTINGS_MODULE production.settings
    PythonOption DJANGO_SETTINGS_MODULE production.settings
    PythonPath "['/usr/wwwapps/sf-app'] + sys.path"
    PythonAccessHandler files.common.modpython
    PythonDebug On
    
    SetHandler none
    allow from all
  </Location>
</VirtualHost>

Edit /etc/init.d/apache2

We change LANG=C to LC_ALL=en_US.UTF-8 for our locale:

ENV="env -i LC_ALL=en_US.UTF-8 PATH=/usr/local/bin:/usr/bin:/bin"

3Ware RAID monitor

As root:

cd /usr/local/bin/
scp -P 2222 <user>@jump.silverfilecorp.com:/usr/silverfile/scripts/tw_cli tw_cli
scp -P 2222 <user>@jump.silverfilecorp.com:/usr/silverfile/scripts/tw_status tw_status
chmod 755 tw_cli
chmod 700 tw_status

Note- Make sure that CLI var in tw_status is correct. RAID client number (c1 etc..) may differ. If RAID is rebuilt or after error condition, client may increment up i.e. from c4 to c5- make sure to change this!

Add crontab to monitor RAID

###########################
# monitor raid array
###########################
*/10 * * * * /usr/local/bin/tw_status CRON

tw_status will send alerts to monitor@…

SMTP through google accounts

Make sure sendmail isn't installed:

sudo apt-get remove sendmail
sudo apt-get install ssmtp mailx mailutils

Make sure sendmail libraries are linked to ssmtp:

sudo rm /usr/lib/sendmail
sudo ln -s /usr/lib/ssmtp /usr/lib/sendmail

Edit /etc/ssmtp/ssmtp.conf:

root=monitor@silverfilecorp.com
mailhub=smtp.gmail.com:587
UseSTARTTLS=yes
UseTLS=yes
AuthUser=monitor@silverfilecorp.com
AuthPass=<< monitor password >> 

Users and Permissions on /FILES

Add nologin to shells

echo "/usr/sbin/nologin" >> /etc/shells

Add fileuser with nologin

useradd -s /usr/sbin/nologin fileuser
passwd fileuser 
< fileuser passwd selected by firm >

Create fileusers group

addgroup fileusers
adduser sf fileusers
adduser fileuser fileusers
adduser www-data fileusers

Set up appropriate permissions on /FILES

chown -R fileuser /FILES
chgrp -R fileusers /FILES
chmod -R 660 /FILES

find /FILES -type d -exec chmod 770 {} \;
find /FILES -type f -exec chmod 550 {} \;

Samba Installation

Installation

sudo apt-get install samba

Add a fileuser as samba user

sudo smbpasswd -a fileuser

Copy samba config files from repo

cd /etc/samba/
mv smb.conf smb.conf.org
scp -P 2222 <user>@jump.silverfilecorp.com:/usr/silverfile/scripts/smb.conf.master smb.conf.master
scp -P 2222 <user>@jump.silverfilecorp.com:/usr/silverfile/scripts/recycle.conf recycle.conf
chmod 644 smb.conf.master
chmod 644 recycle.conf
testparm -s smb.conf.master > smb.conf

Create smbusers file fileuser = <name of principal attorney, etc>

fileuser = matt

Firewall

Rules allow ssh(2222), https, and Samba (from LAN)

ufw allow from 174.143.232.60 to any port 2222
ufw allow from 98.173.61.70 to any port 2222
ufw allow from 208.69.40.208/29 to any port 2222
ufw allow from 192.168.0.0/24 to any app Samba
ufw allow "Apache Secure"
ufw logging on
ufw enable
ufw status

SSH rules above add access from jump.silverfilecorp.com(174.143.232.60), SF/BajaBound office (98.173.61.70), and BajaBound? Network (208.69.40.208/29)

Java

sudo apt-get install sun-java6-bin

Setting up a Software RAID 1 - (DEPRECATED)

We are using the hardware raid cards now, so this is deprecated.

Per recommendations from our friends at MonkeyBrains, we'll set up a software RAID 1 and monitor it with mdadm. The plan is to sync up RAID health with SNMP monitoring.

This is a very good tutorial on ​setting up a software RAID 1 please follow it for the step by step RAID install.

The idea is to create 3 partitions:

• / (root) where all the OS etc files go
• /FILES where all the documents go
• Swap The necessary swap partition

On a 500 GB drive I propose doing this :

• /FILES = 430 GB
• Swap = 4 GB (swap is conventionally 2X RAM, a swap this size may not be necessary with 2 GB of RAM and for use as a file server
• / = <leftover space>

In order to create the software RAID, you first create regular primary partitions on the first disk (SDA) as in the following:

select: Partition Disk Manually
select: Device SDA1
Create new empty partition table on this device: yes
Select Free Space (pri/log): <per size of the disk>
select: Create new primary partition
Mount point: /
Bootable Flag: on
Use as: Select Physical Volume For RAID
select: Done Setting up partition

And then you create an MD device from each partition.

Again, see the ​RAID tutorial as it explains exactly how to do this step by step.

Check Software RAID Status

mdadm --detail /dev/md0