Pipeline Pilot
Pipeline Pilot is a desktop software application developed by Dassault Systèmes, focused on extract, transform, and load (ETL) processes and data analytics.
≈ 3 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
Pipeline Pilot is a desktop software application developed by Dassault Systèmes, focused on extract, transform, and load (ETL) processes and data analytics. Since its inception, the software has evolved to offer broader capabilities in various scientific and industrial applications.
Pipeline Pilot uses a visual and dataflow programming interface, allowing users to design workflows for data processing. The software's functionality spans several domains, including cheminformatics, QSAR, next-generation sequencing, image analysis, and text analytics. Pipeline Pilot was initially developed by SciTegic, a company that was acquired by BIOVIA in 2004.
In 2014, BIOVIA became part of Dassault Systèmes.
Overview
The interface, known as the Pipeline Pilot Professional Client, allows users to create workflows by selecting and arranging individual data processing units called "components." These components perform a variety of functions such as loading, filtering, joining, or modifying data.
Additional components can carry out more complex tasks, such as constructing regression models, training neural networks, or generating reports in formats like PDF. Pipeline Pilot follows a component-based architecture where components serve as nodes in a workflow, connected by "pipes" that represent data flow in a directed graph.
This framework enables the processing of data as it moves between the components. Users have the flexibility to work with pre-installed components or develop custom ones within workflows, referred to as "protocols." Protocols, which consist of linked components, can be saved, reused, and shared, enabling streamlined data processing.
The interface visualizes the connections between components, simplifying complex data workflows by presenting them as sequences of operations.
Component collections
Pipeline Pilot offers several add-ons called "collections," which are groups of specialized functions aimed at specific domains, such as genetic information processing or polymer analysis. These collections are available to users for an additional licensing fee. The collections are organized into two main groups: science-specific and generic.
The science-specific collections focus on areas like chemistry, biology, and materials modeling, while the generic collections provide tools for reporting, data analysis, and document search. Below is an overview of the available collections:
Custom scripts
Pipeline Pilot is commonly used for processing large and complex datasets, often exceeding 1TB in size. In its early development, Pipeline Pilot introduced a scripting language called "PilotScript," which allows users to write basic scripts that can be integrated into a protocol.
Over time, support for additional programming languages was added, including Python, .NET, Matlab, Perl, SQL, Java, VBScript, and R. These languages can be used through APIs that execute commands without requiring the graphical user interface. PilotScript, a language modeled on PLSQL, is used within specific components like the "Custom Manipulator (PilotScript)" or "Custom Filter (PilotScript)."
An example of a simple PilotScript command is shown below, where a property named "Hello" is added to each record passing through the component with the value "Hello World!": Hello := "Hello World!";
Text derived from https://en.wikipedia.org/w/index.php?oldid=1315099131 under CC BY-SA 4.0; modifications have been made.