SysBackup Final Checkout and First Backup

Once you have finished your Installation and Job Configuration you can run the SysBackup utility. Read this note to see some sample output.


Docs & FAQ Pages
Download SysBackup
SysBackup on SourceForge
SysBackup on GitHub
SysBackup on Ohloh
Installation Overview
System Requirements
Basic Installation
SSH Remote Storage
First Run
Be Prepared!



At this point you should be able to dump the sysbackup configurtion without any problems:

$ sudo /opt/sysbackup/sysbackup --dump-config

You should now be able to run the backup and see the desired results. Check the Sample Output below to see the output from a test run on a dev VM.

Troubleshooting

There are quite a few sanity checks that take place immediately after you run the SysBackup script. If you have SSH Remote servers configured you might experience a noticable delay before the configuration dump is produced.

The code is designed to exit with a fatal error if the sanity checks find any problem that might block the script from generating the complete backupset. As a result your first few attampts to run SysBackup may result in a bunch of complicated error messages. Hopefully you will find the messages to be sufficiently helpful and you will correct the minor problems very quickly.

Problems that occur on a remote storage server are sometimes difficult to debug. Don't forget to check the OpenSSH Secure Log on the remote server, as well as the error messages logged to /var/log/remote-exec.log (also on the remote server.)

At the time of this writing the lowest version of PHP that has been tested by the author with SysBackup is 5.1.6. If you get any Function Not Found errors and the offending function name is listed in the PHP documentation, please email the author at sam [at] azertech [dot] net with your PHP version and the function name.

Sample Output

Here is the sample output from a run of SysBackup on a test VM:


$ sudo /opt/sysbackup/sysbackup --dump-config --exec
System Backups 3.01(alpha)

Basic Application Settings:
  req app_title                "System Backups"
  req app_version              "3.01(alpha)"
  req app_conffile             "/etc/sysbackup/sysbackup.conf"
  req app_confdir              "/etc/sysbackup/conf.d/"
  req app_plugindir            "/etc/sysbackup/plugin.d/"
  req admin_email              "sysbackup[at]azertech.net"

Plugins Loaded:
  System Home Directories 0.01a (homedirs)
    produces backups of User Home Directories
  Mail Folders 0.01a (maildirs)
    produces backups of Mail Folders consisting of one file per user.
  Mailman Mailing List Server 0.01a (mailman)
    produces backups of a Mailman Installation
  MySQL Database 0.01a (mysql)
    produces backups of a MySQL server (local or remote)
  System Package List 0.01a (syspackages)
    creates a backup of the System Package List for reference purposes
  System Directories 0.01a (sysdirs)
    produces backups of important System Directories

Jobs Configured:
  Event         Index                       Plugin(JobID)               
  genbackups    1000.System.Packages        syspackages(genpackagelist) 
  genbackups    1100.System.SystemDirs      sysdirs(systemdirs)         
  genbackups    1200.System.SystemMail.0000 maildirs(sysmail-dirs)    
  genbackups    1200.System.SystemMail.0001 sysdirs(sysmail-files)     
  genbackups    1900.System.SystemWWW       homedirs(sysweb)            
  genbackups    2000.Daemons.MySQL          mysql(mysqldb-localhost) 
  genbackups    2000.Daemons.MySQL-Config   sysdirs(MySQLConfig)     
  auxops        5000.RemoteXfer             internal(xfer-queued)    

Plugins Not Loaded (disabled):
  Sample do-nothing Plugin 0.01a (dummy)

Jobs Not Configured:
  systemdirs in '/etc/sysbackup/conf.d/dummy.conf' (disabled)
  homedirs in '/etc/sysbackup/conf.d/homedirs.conf' (disabled)
  mailmanlists in '/etc/sysbackup/conf.d/mailman.conf' (disabled by run conditions)
  mailmanconfig in '/etc/sysbackup/conf.d/mailman.conf' (disabled by run conditions)
  mailmansystem in '/etc/sysbackup/conf.d/mailman.conf' (disabled by run conditions)
  mailmanlists in '/etc/sysbackup/conf.d/mailman.conf' (disabled by run conditions)

Local Fileio VFS Plugin
  req hostname                 "dev.azertech.net"                                       
  req keeplocal                bool(true)                                              
  req path_temp                "/home/sysbackup/temp/"                                
  req path_backup              "/home/sysbackup/dev.azertech.net/daily/"               
  req path_rotation            "/home/sysbackup/dev.azertech.net/"                     

SSH Remote Fileio VFS Plugin
  req hostname                 "nas.azertech.net"                                          
  req keeplocal                bool(true)                                              
  req path_temp                "/home/sysbackup/temp/"                                 
  req path_backup              "/home/sysbackup/dev.azertech.net/daily/"                
  req path_rotation            "/home/sysbackup/dev.azertech.net/"                      

  req enabled                  bool(true)
  req usehardlinks             bool(true)
  req remconf_file             "/etc/sysbackup-nas.conf"
  def ssh_binary               "/usr/bin/ssh"
  int ssh_localexec_keypath    "/etc/sysbackup/keys/nas-localexec.openssh"
  opt ssh_localexec_options    string(blank)
  int ssh_localexec_userid     "sysbackup"
  def scp_binary               "/usr/bin/scp"
  int scp_filecopy_keypath     "/etc/sysbackup/keys/nas-filecopy.openssh"
  opt scp_filecopy_options     "-B -p -q"
  int scp_filecopy_userid      "sysbackup"
  int remote_server            "nas.azertech.net"

Compression Options:
  int tar_name                 "tar"
  int tar_bin                  "/bin/tar"
  int tar_ext                  ".tgz"
  int tar_defopts              "--preserve-order --preserve-permissions --atime-preserve=system"
  opt tar_opts1                string(blank)
  opt tar_opts2                string(blank)
  opt tar_opts3                string(blank)
  int tar_cmd                  "%BINARY% %DEFOPTS% %OPTS1% --create %OPTS2% '%SOURCE%' %OPTS3%"
  int gzip_name                "gzip"
  int gzip_bin                 "/bin/gzip"
  int gzip_ext                 ".gz"
  int gzip_defopts             "--no-name"
  opt gzip_opts1               string(blank)
  int gzip_filter              "%BINARY% %DEFOPTS% %OPTS1%             >'%TARGET%'"
  int gzip_file                "%BINARY% %DEFOPTS% %OPTS1% <'%SOURCE%' >'%TARGET%'"


genbackups: Generate the Backup Set
  1000.System.Packages: syspackages(genpackagelist)
    etc          61KiB dev.azertech.net.packagelist.gz
  --------dev.azertech.net: 0.5692s 61KiB/1

  1100.System.SystemDirs: sysdirs(systemdirs)
    etc           9MiB etc.tgz
    etc          69MiB var-log.tgz
    etc          71MiB root.tgz
  --------dev.azertech.net: 36.3283s 149MiB/3

  1200.System.SystemMail.0000: maildirs(sysmail-dirs)
    mail         49KiB *var-spool-mail-demo.gz
    mail        484KiB *var-spool-mail-root.gz
  --------dev.azertech.net: 2.1870s  *533KiB/2

  1200.System.SystemMail.0001: sysdirs(sysmail-files)
    mail           67B *etc-aliases.gz
  --------dev.azertech.net: 0.4256s  *67B/1

  1900.System.SystemWWW: homedirs(sysweb)
    var-www       146B *var-www-index.html.gz
    var-www       133B *var-www-usr--share--doc.tgz
  --------dev.azertech.net: 1.3715s  *536B/2

  2000.Daemons.MySQL: mysql(mysqldb-localhost)
    mysql       123KiB localhost-information_schema.sql.gz
    mysql       132KiB *localhost-mysql.sql.gz
  --------dev.azertech.net: 1:09.9145s 68MiB/7 *68MiB/6

  2000.Daemons.MySQL-Config: sysdirs(MySQLConfig)
    mysql         3KiB *etc-mysql.tgz
    mysql       228KiB var-log-mysql.tgz
  --------dev.azertech.net: 1.9456s 231KiB/2 *3KiB/1

Summary Stats
  --------dev.azertech.net: 2:03.3295s 218MiB/20 *69MiB/14 (~6GiB/h)


auxops: Auxiliary Operations (ie: copy to NAS)
  5000.RemoteXfer: internal(xfer-queued)
       61KiB etc/dev.azertech.net.packagelist.gz        nas:0.8068s
        9MiB etc/etc.tgz                                nas:1.8601s
       69MiB etc/var-log.tgz                            nas:3.9200s
       71MiB etc/root.tgz                               nas:4.1335s
       49KiB mail/var-spool-mail-demo.gz                nas:*0.9164s
      484KiB mail/var-spool-mail-root.gz                nas:*0.8114s
         67B mail/etc-aliases.gz                        nas:*0.8193s
        146B var-www/var-www-index.html.gz              nas:*0.8327s
        133B var-www/var-www-usr--share--doc.tgz        nas:*0.8761s
      123KiB mysql/localhost-information_schema.sql.gz  nas:0.7971s
      132KiB mysql/localhost-mysql.sql.gz               nas:*0.8912s
        3KiB mysql/etc-mysql.tgz                        nas:*0.8724s
      228KiB mysql/var-log-mysql.tgz                    nas:0.7949s
  ------------dev.azertech.net: 24.5852s 
  -sysbackup[at]nas.azertech.net: 24.1898s 218MiB/20 *69MiB/14

Summary Stats
  ------------dev.azertech.net: 2:27.9149s 218MiB/20 *69MiB/14 (~5GiB/h)
  -sysbackup[at]nas.azertech.net: 24.1901s 218MiB/20 *69MiB/14 (~32GiB/h)

NOTES: The output above has been manually edited for privacy and various output lines have been deleted. As such the total numbers of bytes, the tarball counts and the estimated throughputs are not correct. The stars (*) in the above output indicate that a file was hardlinked instead of being copied. This does not save any time on the local filesystem but it saves a great deal of time on the remote filesystems under SSH as only a few filesystem commands go over the network in cases where tarballs are hardlinked instead of copied.

Going Further

In practice there are many backup scripts that you can download from the internet - and many highly sophisticated backup tools available - but none of them are able to take the time to review your server and your requirements to find out what you really need. For example: is your server running 24x7? If so, how do you backup your databases or your mail services, for example? One good way to do it is to stop all services for a few seconds in the middle of the night so that you can take snapshots of the filesystem using, for example, LVM. Then you can restart all the services and run your backups off of the snapshots.

It takes time to organize all this and get it running smoothly. Please take a moment to contact the author, Sam Azer, with your questions, comments and suggestions. Hopefully system administrators will find SysBackup to be a helpful tool in their toolbox.