The engineering teams took a fresh look at the challenges teams face with growing storage needs and designed an innovative and unchained solution to those needs with cutting edge technology, data lifecycle and provenance tools and analytics to make any IT administrator exuberant about storage.
The Hormel Institute, University of MN, Deploys Quantum ATFS to Accelerate Cancer Research
The Hormel Institute, a leading cancer research department at the University of Minnesota since 1942, is at the forefront of cancer research—with a focus on identifying the molecular mechanisms of cancer development and best ways to prevent, diagnose, and treat the disease. Currently, the Institute includes more than 130 leading cancer scientists working across dozens of cancer research projects—each one contributing multiple terabytes of data to the Institute’s data storage system each week.
The Challenge
One aspect of the Institute’s research efforts involves the use microscopes to study molecules. These powerful cryoelectromagnetic microscopes continuously capture images of molecules over a period of a few days to a week. These images serve as the foundation for analysis performed by the Institute’s researchers. The data created by the microscope are high resolution images of molecules, which need to be kept indefinitely for future research. Once data is captured, the images are processed, creating 3D images of the molecule. These images are used to identify effective ways for cancer drugs to achieve more desirable results. After the microscope captures the images, it writes data into a shared storage repository where it can be accessed by researchers. Once processing has been completed, data needs to be archived for either the life of the researcher or according to the terms of the grant. Prior to the introduction of ATFS into the data environment, the Institute struggled to deliver data provenance, long-term retention, isolation of data by owner, real-time status updates, and workflow automation.
The Solution
The Hormel Institute chose to implement ATFS after an extensive proof of concept, proving that ATFS’s unique architecture and feature-set could help address its operational and organizational challenges. ATFS’s unique architecture enables the Institute to accelerate research and manage data according to grant or project requirements—all while realizing an increase in ROI. The Institute’s new ATFS storage solution includes four main components:
- The Zero-touch Classification Engine. This engine organizes data as it enters the system, capturing file metadata and applying user-defined tags based on business variables such as projects, data owners, or funding sources. This information can then be used to automate tasks, provide real-time status updates, determine data placement across media types and geographies—all while ensuring compliance with retention requirements. Additionally, the software is agile and can adjust to the use of different media, network, and other system components to maintain an optimal ratio of cost-value.
- The Policy Orchestration Engine. Users can manage the data automatically according to its lifecycle requirements and simplify risk and compliance management by placing data according to policy. The engine can also deliver flexibility and customization according to user-defined variables—so one faculty member can keep data in flash for two weeks before archiving to bulk disk, while another faculty member places all their data in the cloud to enable public review and easy remote team access.
- Flash Performance at Archive Pricing. The ATFS system has been designed to achieve optimal results in performance, cost, and capacity. The combination of NVMe, nearline disk, and cloud (public or private) delivers performance at a fraction of the cost compared to more traditional architectures. The hardware configurations can be modified to reflect changes in requirements for performance or capacity.
- Hassle-Free Implementation. The system the Institute implemented consisted of 51.2 TB of NVMe flash and 2 PB of bulk storage using nearline disk. As the system ingests data, categorization tags are applied via the zero-touch classification engine and metadata is captured. Tags applied to data represent a user-defined category. The data orchestrator drives data placement, lifecycle, retention, and workflows.
Results
By adopting the ATFS storage system, the Hormel Institute was able to implement data retention and deletion policies to assure data provenance and silo research findings. The new solution also helped the Institute to realize an improved ROI while increasing performance and data mobility. Moving forward, the Institute believes these important updates will expedite research that can contribute to the eradication of cancer.