wiki:servers/setup-condensed
close Warning: Can't synchronize with repository "(default)" (/usr/svn/silverfile does not appear to be a Subversion repository.). Look in the Trac log for more information.

Version 36 (modified by hank, 13 years ago) (diff)

work on how-to

SilverFile General Server Setup

Installation of Ubuntu 9.04 Server 32bit

We are selecting this version for the following reasons:

  • GRUB installation doesn't work in older releases.
  • 64 bit version does not seem to support the VIA padlock engine

Partitioning (During Install)

We feel that 20GB and 6GB of swap is more than sufficient, leaving ample room for client files.

  • / 20GB Primary Ext3 Bootable = Yes (leave all other defaults)
  • swap 6GB swap
  • /FILES/ Primary Ext3 Bootable = No (leave all other defaults)

Primary User - sf

Add sf user with sudo privileges during install

Install SSH Deamon

> sudo apt-get install ssh

Config SSH

edit /etc/ssh/sshd_config

Port 2222
PermitRootLogin no
PasswordAuthentication no

Change port line to 2222 (or whatever port) restart sshd to check

/etc/init.d/ssh restart

VIA Padlock and OpenSSL

Openssl Installation

> sudo apt-get install openssl

Padlock Verification

Next, verify engine:

> openssl engine
(padlock) VIA PadLock (no-RNG, ACE)
(dynamic) Dynamic engine loading support

The response string should include '(padlock) VIA PadLock (no-RNG, ACE)'.

Make default engine Padlock

> vim /etc/ssl/openssl.cnf

Add the following under oid_section = new_oids

...
oid_section             = new_oids

# Enable Via Padlock by default
openssl_conf = openssl_def

[openssl_def]
engines = openssl_engines

[openssl_engines]
padlock = padlock_engine

[padlock_engine]
default_algorithms = ALL

Install Duplicity / S3tools / Rsync

Duplicity is our preferred backup method. Install s3tools for S3. Rsync for app syncing.

> sudo apt-get install duplicity python-boto s3cmd rsync

Install Django

The platform for our app.

> sudo apt-get install python-django

Install Apache for Django

Great Ubuntu Apache/SSL How-To

> sudo apt-get install apache2 libapache2-mod-python
> sudo ln -s /usr/sbin/apache2ctl apachectl (old habits die hard)

Install MySQL

Install MySQL with python db support mysqldb.

OLD: > sudo apt-get install mysql-server mysql-python
CORRECTED: > sudo apt-get install mysql-server python-mysqldb
> mysql -uroot -p

Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.0.67-0ubuntu6 (Ubuntu)
r
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> CREATE DATABASE silverfile CHARACTER SET utf8;
Query OK, 1 row affected (0.00 sec)

mysql> GRANT ALL ON silverfile.* TO sf@localhost IDENTIFIED BY '<<databasepwd>>';
Query OK, 0 rows affected (0.00 sec)

Install SilverFile? App

On Dev Server export SF svn repo to create a clean copy:

rm -rf /usr/svn/sfexport
svn export /home/devsilverfile/trunk /usr/svn/sfexport

Create app directory:

mkdir -p /usr/wwwapps/sf-app/

Create Initial Copy From Mercurial Repo (as hank):

/usr/bin/rsync -av --timeout=300 --delete -e '/usr/bin/ssh -p 2240 -o ConnectTimeout=3' \
  hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/configs \
  hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/files \
  hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/utils \
  hank@dev.silverfilecorp.com:/usr/hg/repos/sf-app/third_party \
  hank@dev.silverfilecorp.com:/usr/svn/sfexport/app_skins/production \
  /usr/wwwapps/sf-app/

Create a unique secret key for production/settings.py: cd /usr/wwwapps/sf-app/files/

python manage.py shell
(InteractiveConsole)
>>> import utils.django
>>> utils.django.new_secret_key()

Copy secret key into settings.py

Copy DB credentials into settings.py

Initialize App Database

python manage.py syncdb

Configure SSL

> sudo a2enmod ssl

copy cert file (mydomain.com.crt) to /etc/ssl/certs copy key file (mydomain.com.key) to /etc/ssl/private

Configure Apache for SilverFile App

Django app is turned "on" by default. Two locations (webspaces) are turned off, and are served directly. These are:

  • site_media (css, js, images, etc...)
  • site_files (these are all the client files)

See sf-apps/files/examples

> sudo ln -s /usr/wwwapps/sf-app/production/apache/files.conf /etc/apache2/sites-available/silverfile
> mkdir /usr/wwwapps/logs/
> touch /usr/wwwapps/logs/silverfile.access
> touch /usr/wwwapps/logs/silverfile.error

Example virtual host conf file:

<VirtualHost *:80>
  # Edit here:
  # ServerName sfxxx.silverfilecorp.com
  ServerName 127.0.0.1
  # ServerAlias 127.0.0.1
  
  ErrorLog "/usr/wwwapps/logs/silverfile.error"
  CustomLog "/usr/wwwapps/logs/silverfile.access" common
  
  # Edit Here:
  # SSLEngine on
  # SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
  
  # SSLCertificateFile /etc/ssl/certs/docs.silverfilecorp.com.crt
  # SSLCertificateKeyFile /etc/ssl/private/docs.silverfilecorp.com.key

  # Django app
  <Location />
    SetHandler python-program
    PythonHandler django.core.handlers.modpython
    SetEnv DJANGO_SETTINGS_MODULE production.settings
    PythonPath "['/usr/wwwapps/sf-app'] + sys.path"
    PythonOption django.root
    PythonDebug On
  </Location>
    
  # Site media files - css, js, img 
  Alias /site_media /usr/wwwapps/sf-app/production/media
  <Location /site_media/>
    SetHandler none
    allow from all
  </Location>
  
  # Admin media files - css, js, img
  Alias /media /var/lib/python-support/python2.5/django/contrib/admin/media
  <Location /media/>
    SetHandler none
    allow from all
  </Location>
  
  # Client Files
  Alias /docs /FILES
  <Location /docs/>
    SetEnv DJANGO_SETTINGS_MODULE production.settings
    PythonOption DJANGO_SETTINGS_MODULE production.settings
    PythonPath "['/usr/wwwapps/sf-app'] + sys.path"
    PythonAccessHandler files.common.modpython
    PythonDebug On
    
    SetHandler none
    allow from all
  </Location>
</VirtualHost>

3Ware RAID monitor

As root:

cd /usr/local/bin/
scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third_party/3ware_twcli/tw_cli tw_cli
scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third_party/3ware_twcli/tw_status tw_status
chmod 755 tw_cli
chmod 700 tw_status

SMTP through google accounts

As root:

sudo apt-get install ssmtp mailx

Edit /etc/ssmtp/ssmtp.conf:

root=monitor@silverfilecorp.com
mailhub=smtp.gmail.com:587
UseSTARTTLS=yes
UseTLS=yes
AuthUser=monitor@silverfilecorp.com
AuthPass=<< monitor password >> 

Samba Installation

Installation

sudo apt-get install samba

Add a fileuser as samba user

sudo smbpasswd -a filuser

Copy samba config files from repo

cd /etc/samba/
scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third_party/samba/smb.conf.master smb.conf.master
scp -P 2240 hank@dev.silverfilecorp.com:/usr/svn/sfexport/third_party/samba/recycle.conf recycle.conf

Create smbusers file fileuser = <name of principal attorney, etc>

fileuser = matt

Users and Permissions on /FILES

Add nologin to shells

echo "/usr/sbin/nologin" >> /etc/shells

Add fileuser with nologin

useradd -s /usr/sbin/nologin fileuser
passwd fileuser 
< fileuser passwd selected by firm >

Create fileusers group

addgroup fileusers
adduser sf fileusers
adduser fileuser fileusers
adduser www-data fileusers

Set up appropriate permissions on /FILES

chown -R fileuser /FILES
chgrp -R fileusers /FILES
chown -R 660 /FILES

find /FILES -type d -exec chmod 770 {} \;
find /FILES -type d -exec chmod 550 {} \;

Java

sudo apt-get install sun-java6-bin

Setting up a Software RAID 1 - (DEPRECATED)

We are using the hardware raid cards now, so this is deprecated.

Per recommendations from our friends at MonkeyBrains, we'll set up a software RAID 1 and monitor it with mdadm. The plan is to sync up RAID health with SNMP monitoring.

This is a very good tutorial on setting up a software RAID 1 please follow it for the step by step RAID install.

The idea is to create 3 partitions:

  • / (root) where all the OS etc files go
  • /FILES where all the documents go
  • Swap The necessary swap partition

On a 500 GB drive I propose doing this :

  • /FILES = 430 GB
  • Swap = 4 GB (swap is conventionally 2X RAM, a swap this size may not be necessary with 2 GB of RAM and for use as a file server
  • / = <leftover space>

In order to create the software RAID, you first create regular primary partitions on the first disk (SDA) as in the following:

select: Partition Disk Manually
select: Device SDA1
Create new empty partition table on this device: yes
Select Free Space (pri/log): <per size of the disk>
select: Create new primary partition
Mount point: /
Bootable Flag: on
Use as: Select Physical Volume For RAID
select: Done Setting up partition

And then you create an MD device from each partition.

Again, see the RAID tutorial as it explains exactly how to do this step by step.

Check Software RAID Status

mdadm --detail /dev/md0