Chapter 3 - Understanding Spectra Advanced Bucket Management Concepts
Spectra S3 Overview
The second command is called Create Bulk Get. The Create Bulk Get command is actually
an HTTP PUT command because it too contains a payload for the BlackPearl system. This
payload is a list of objects that the client wants to get from the BlackPearl system. It is not
necessary for the request payload to contain the size of the files because the BlackPearl
system already knows the sizes of the objects (files). The response to the request is again an
ordered list of the objects and information about the objects, including if they are already in
the cache and ready to be retrieved from cache by a GET command.
Knowing the files that the client wants to retrieve, the BlackPearl system can make the best
use of its resources in retrieving the objects. For example, if the list of objects spans across
four different tapes and there are four tape drives available, those four tapes can all be
loaded into drives and the objects can be read back in parallel, greatly improving the speed
at which the client can get all of the objects. Without the Create Bulk Get request, the client
would be asking the BlackPearl system for those objects in a less efficient manner.
Storing large amounts of bulk data on tape has historically presented challenges. Spectra S3
addresses these challenges:
• Tape drives are sequential block storage devices, with data laid out in a sequential
manner along the full length of the tape. This makes it inefficient to retrieve data out of
order. Spectra S3 plans and queues a large amount of data to be efficiently written to
tape; it logically groups data on tape in a way that reflects how the client is likely to
read it back.
• Because of the mechanical nature of the tape media and drives, tape drives demand a
large amount of data to be available, via a fast connection. When data is not efficiently
streamed for the tape drives to write (due to slow data buffering or a slow connection to
the drive), the result is poor write performance. This poor performance is due to a
phenomenon referred to as “shoe-shining”. When a drive is sent a small amount of data,
it writes the data and then is forced to stop. Because the tape cannot stop
instantaneously, the drive overshoots a small amount and the tape is not in position for
the next write operation. To compensate, the tape drive rewinds to get back to the correct
position for the next write. If the next write also has a small amount of data, then the
drive writes the next portion and again stops, overshoots, and rewinds, causing a back
and forth “shoe-shining” like action. Spectra S3 caches data on the BlackPearl Converged
Storage System before starting the transfer to tape, which prevents the shoe-shining
behavior from occurring.
• Classically, different tape storage devices wrote data to tape in unique ways, locking you
into a proprietary and single vendor solution to retrieve previously written data. Spectra
S3 writes data to tape using the open source Linear Tape File System
®
(LTFS). With
LTFS, data is always accessible with any LTFS enabled system.
For more information on the Spectra S3 interface, see the
Spectra BlackPearl Spectra S3 API
November 2020
User Guide-BlackPearl Converged Storage System
- 68 -
