Large File Support

Large File Support allows CloudShark to handle extremely large capture files by loading them in a restricted preview mode. This prevents system degradation and keeps CloudShark responsive when working with captures that contain millions of packets.

Overview

Very large capture files can consume significant system resources and cause performance issues when loading all packets into memory. Large File Support addresses this by introducing a configurable packet limit. When a capture file exceeds this limit, CloudShark automatically opens it in Large File Preview mode, loading only a subset of packets while keeping the system responsive.

Configuring the Packet Limit

The packet limit can be configured in the CloudShark administration settings. Available options are:

  • 10,000 packets
  • 25,000 packets
  • 50,000 packets
  • 100,000 packets (default)
  • 250,000 packets
  • 500,000 packets

The packet limit determines the threshold at which files will be opened in Large File Preview mode. Choose a limit appropriate for your system resources and typical use cases.

Large File Preview Mode

When a capture file exceeds the configured packet limit, CloudShark opens it in Large File Preview mode with the following behavior:

  • Only the first N packets (based on your configured limit) are loaded and displayed
  • A dialog appears informing users they are in preview mode
  • The system remains responsive and avoids performance degradation
  • Users are presented with tools to work with the large file effectively

Disabled Analysis Tools

Most analysis tools are disabled in Large File Preview mode for several important reasons:

  • Incomplete data: Analysis tools would only process the limited subset of packets that have been loaded, producing incomplete and potentially misleading results
  • Inaccurate statistics: Metrics, graphs, and statistics would not reflect the entire capture, leading to incorrect conclusions
  • Resource consumption: Running analysis tools on even a subset of packets would consume system resources, defeating the purpose of preview mode
  • Performance impact: Analysis operations would slow down the system and make it difficult to quickly review the preview and decide how to proceed
  • User confusion: Displaying partial results without clear indication they’re incomplete could lead users to make decisions based on inaccurate data

The preview mode is designed to give users a quick look at the beginning of the capture so they can determine how to best work with the file using the Split PCAP tool. Once splits are created, all analysis tools become available on the smaller, manageable files.

Working with Large Files

When users open a file in Large File Preview mode, they see a dialog explaining:

  • Only the first portion of the file has been loaded to keep the system responsive
  • They can use the Split PCAP tool to extract smaller subsets of packets
  • After splitting, they can analyze specific portions or use Deep Search across all splits

The dialog provides:

  • View previous splits link: Access previously created splits from this file
  • Split PCAP button: Launch the Split PCAP tool to break the file into manageable pieces

Use Cases

Large File Support is particularly useful for:

  • Preventing system overload: Automatically protect your CloudShark instance from performance degradation when users upload very large captures
  • Long-term captures: Handle multi-hour or multi-day captures that contain millions of packets
  • High-bandwidth captures: Work with captures from 10G, 40G, or 100G network links that generate massive amounts of data
  • Resource management: Maintain system responsiveness for all users, even when some are working with extremely large files

Workflow

The typical workflow for large files is:

  1. User uploads or opens a very large capture file
  2. CloudShark detects the file exceeds the packet limit
  3. Large File Preview mode loads only the configured number of packets
  4. User reviews the preview to understand the capture content
  5. User uses Split PCAP to extract relevant portions or divide the file into manageable chunks
  6. User analyzes the splits individually or uses Deep Search across all splits

This workflow combines the performance benefits of working with smaller files while maintaining complete fidelity to the original capture. Unlike tools that summarize traffic or reduce data to flow patterns, CloudShark’s Split PCAP feature preserves all the original packets with no loss of information. Users get the full-fidelity network data that PCAPs provide, while still being able to access and search the entire original capture through CloudShark’s Deep Search functionality.

Best Practices

  • Set appropriate limits: Choose a packet limit based on your system resources and typical capture sizes
  • Ensure adequate disk space: The original large file remains on disk after splitting, and the splits create additional files. Plan for sufficient free disk storage to accommodate both the original captures and their splits
  • Educate users: Make sure users understand how to use Split PCAP when they encounter Large File Preview mode
  • System resources: Ensure your CloudShark instance has adequate RAM and storage for your typical workload
  • Split PCAP - Break large captures into smaller, manageable pieces
  • Deep Search - Search across multiple capture files simultaneously