wiki:administration/backup_method
close Warning: Can't synchronize with repository "(default)" (/usr/svn/silverfile does not appear to be a Subversion repository.). Look in the Trac log for more information.

Version 3 (modified by hank, 13 years ago) (diff)

--

SilverFile Backup Methodology

Introduction

Data integrity is a key feature of the SilverFile service. To this end, we propose a combination of backup methods to protect files/data from hardware failure and user error. The following describes these methods:

Disk Mirroring (RAID)

We are using two physical hard drives in a RAID mirror configuration (RAID 1). Data is replicated in real time between the two drives so that if one drive fails data is still preserved on the other drive. The RAID is created during installation of the Ubuntu OS. RAID Installation How-To

  • RAID Monitoring [BR] Disk/RAID monitoring is done with mdadm. Email alerts are sent if the RAID status is negative.
  • More Here

Data Mirroring (Between separate servers)

Data is replicated to a backup server on an hourly basis via rsync. Whatever file composition is in the primary Silver File server becomes reflected in the mirror in the backup Silver File server.

Incremental Backup

While the above methods for data preservation are useful to safeguard against data loss due to hardware failure, they are not as useful for protecting against loss due to user error. For instance if someone accidentally deletes a file or folder and is unable to "undo" this, it is useful to be able to revert back to certain backup points in order to retrieve the data.

We propose the following backup points. All backup points will be at midnight.

  • 1 Day (or midnight prior to current time)
  • 2 Day (or 2 midnights prior to current time)
  • 1 Week (or 7 midnights prior to current time)
  • 2 Week (or 14 midnights prior to current time)

2 weeks will become the farthest point back to which we can revert.

Incremental Backup (Local)

rdiff-backup Rsnapshot

Incremental Backup (Remote)

Duplicity

duplicity

s3cmd

S3cmd is part of s3tools, which are a group of tools for interacting with the Amazon S3.

yum install s3cmd
s3cmd --configure

Create a new S3 bucket with s3cmd:

s3cmd mb s3://herron-steele/

List all buckets in S3 account:

s3cmd ls