Finding files in a sea of instrument and analytically generated data can be challenging at best and impossible at worst.
Read More
Knowing the location of a dataset is only a fraction of the battle. Operating on files how and where they are needed can be tedious and loaded with risk.
Read More
Sewing together scientific instruments, file systems, cloud providers, customers, users, computing, security, auditing is no trivial task.
Read More
Much of today’s research does not happen in a computing vacuum. Projects can span various cloud services and even multiple on-premises infrastructures. Operend simplifies the management of this data flow that typically requires high levels of expertise and is time consuming.
Files are often not the ultimate end point of scientific research. Data often needs to be queried, analyzed and mined via data lake or data warehouse infrastructures. Operend can not only serve as queryable data repository but also provides for painless integration with many of these data building blocks, greatly reducing the effort of typical ETL workflows.
An inordinate amount of time and effort in informatics is spent not only on acquiring and organizing files for analysis but also effectively disseminating and managing the results for subsequent use. Operend was built to deliver data where and how an researcher needs and keeps things in order for the next phases of the data's life.
Files Files everywhere! Finding files in a sea of instrument and analytically generated data can be challenging at best and impossible at worst. Directory and file naming conventions may suffice on a certain scale but things get messy fast. Seemingly simple tasks such as acquiring files for a specific study or locating data sets to analyze specific conditions may take days. Simple cloud storage, data lakes and other similar emerging techniques move toward solving some of these issues by centralizing the actual home of this data but the lack of structure may still make it difficult to locate the exact files/data sets of interested. Operend capitalizes on everything we may know about a file such as the instrument of origin or the date it was run, clinical or phenotypic traits, analytics pipeline that generated it or the customer it was generated for. The possibilities are literally endless.
Knowing the location of a dataset is only a fraction of the battle. Integrating and using this data can be tedious and loaded with risk. Copying or linking files can not only consume valuable storage but also pose security risks. Migrations to cloud or other such platforms often require re-tooling of analytic workflows and other such tasks to get to productivity. Operend is built with the philosophy that users should be able to access the data they need in the ways they are accustom to with minimal disruption while, still providing for migration paths for the future.
Sewing together scientific instruments, file systems, cloud providers, customers, users, computing, security, auditing is no trivial task. Once the sewing is complete, the job of overseeing and managing this system is not only expensive but error prone unless proper accompanying software and systems are in place. Operend provides a central point of managment cutting down on the admin and scripting time.