Migrating data from versions prior to 3.9

This document describes the process of manually migrating data from version prior to 3.9 to the latest version of CloudShark.

Please contact support@qacafe.com before beginning the migration for assistance and to ensure your new CloudShark system has a valid license.

The general procedure is to stop CloudShark on the existing system, and then save the database contents and user data to a new location. This data can then be copied and imported onto the new CloudShark system.

In the commands below, the /backups directory is used to store the data. After the data has been exported the directory can be copied to a new CloudShark system. This could instead be a remote NFS service or another storage mechanism.

A note about older versions

CloudShark prior to 2.6 tracked a hash for each capture file using MD5. The latest version requires the SHA-1 hash and CloudShark will not open any files imported without it. In some migrations from earlier than CloudShark 3.0 the SHA-1 was not transferred to every capture. If you may be in this situation, you will need to run the following commands to generate this before exporting the database:

  • cd /var/www/cloudshark/current/
  • sudo -u cloudshark bin/run CloudShark::Utils.update_missing_sha1

Export Data From Existing System

On the existing CloudShark system, the following commands will gather and export the data to migrate to the new CloudShark system.

In step 2 the ‘cloudshark.activities table has been excluded from the database export. This table can be quite big, especially on long-running systems Excluding this table will result in an empty activity log available to an admin user. If you need to keep that data, please leave that part of the command out.

  1. systemctl stop cloudshark
  2. mysqldump -uroot -pqacafe123 --ignore-table=cloudshark.activities cloudshark > /backups/cloudshark.sql
  3. mysqldump -uroot -pqacafe123 mysql > /backups/mysql.sql
  4. cd /var/www/cloudshark/shared/uploads/ && tar -czpf /backups/cloudshark_data.tar.gz
  5. cd /var/www/cloudshark/shared/ && tar -czpf /backups/cloudshark_private.tar.gz private/
  6. cp /var/www/cloudshark/shared/config/cloudshark.conf /backups/
  7. systemctl start cloudshark

TLS Certificates

If you have added a custom TLS certificate and private key to the existing CloudShark system you would like to use on the new CloudShark system they can be copied to the following locations:

  • /usr/cloudshark/etc/tls-certificate/cloudshark.crt
  • /usr/cloudshark/etc/tls-certificate/cloudshark.key

The default configuration on the new CloudShark system will use the TLS certificate and private key in these locations.

Additional Customizations

If you have connected CloudShark to an external database or other shared services, or have modified the default nginx configuration, please also review this section for some required steps to replicate those changes on your new system.

External Services

If you have modified the default ‘services.conf’ file on the existing CloudShark system to point to an external service such as the database, this file will have to be copied to the new ‘services.conf’ location manually.

On the existing CloudShark system this file is located at:

  • /var/www/cloudshark/shared/config/services.conf

On the new CloudShark system edit the following file to configure CloudShark to use the same external services:

  • /usr/cloudshark/etc/services.conf

Nginx

Any custom configuration changes made to nginx on the existing CloudShark system in the following files should be rewritten into a single /usr/cloudshark/etc/nginx.conf file on the new CloudShark system manually:

  • /var/www/cloudshark/shared/config/nginx-custom-headers.conf
  • /var/www/cloudshark/shared/config/nginx-logging.conf
  • /var/www/cloudshark/shared/config/nginx-network.conf
  • /var/www/cloudshark/shared/config/nginx-ssl.conf

Import Data to the New System

Once the data from the existing CloudShark system has been copied to the new CloudShark system, the cloudshark-full service must be stopped to ensure that the data is imported correctly. The following commands will stop cloudshark, import the data, and start cloudshark again:

  1. systemctl stop cloudshark-full
  2. tar -pxzf /backups/cloudshark_data.tar.gz -C /usr/cloudshark/data/
  3. tar -pxzf /backups/cloudshark_private.tar.gz -C /usr/cloudshark/data/
  4. cp /backups/cloudshark.conf /usr/cloudshark/etc/
  5. mysql -uroot mysql < /backups/mysql.sql
  6. mysql -uroot cloudshark < /backups/cloudshark.sql
  7. ./cloudshark-installer-v4.0.0.el8.run --keep --noexec
  8. ./cloudshark-installer-v4-0-0/database-upgrade
  9. systemctl start cloudshark-full