LONI Pipeline

LONI Pipeline
Name	LONI Pipeline
Developer	University of Southern California Laboratory of Neuro Imaging
Released	2000s
Programming language	Java (programming language), C++
Operating system	Microsoft Windows, macOS, Linux
Genre	Scientific workflow system, Neuroimaging
License	Proprietary software / academic

LONI Pipeline is a graphical workflow environment developed by the Laboratory of Neuro Imaging at the University of Southern California to design, execute, and manage complex data-processing pipelines for neuroimaging and biomedical research. The system integrates diverse tools, coordinates distributed computation, and exposes visual programming constructs to researchers working with large datasets from projects such as the Alzheimer's Disease Neuroimaging Initiative, Human Connectome Project, and multi-site clinical consortia. It has been used in conjunction with major neuroinformatics resources and imaging modalities, facilitating reproducible analysis and provenance tracking across heterogeneous computing environments.

Overview

The platform provides a visual pipeline builder that assembles modules representing software tools, scripts, and services into directed acyclic graphs to perform end-to-end processing. Its design emphasizes interoperability with established packages like FSL (software), AFNI, SPM (software), FreeSurfer, and connectors to data repositories such as XNAT, ADNI, and institutional archives. The Pipeline supports parameterization, conditional branching, and metadata capture to enable reproducible workflows for cohorts managed by groups including the National Institutes of Health, National Institute of Mental Health, and international collaborations.

Architecture and Components

The architecture separates the graphical authoring environment from execution engines and resource managers. Core components include a visual workflow editor, a pipeline execution server, module repositories, and client-server communication layers that interface with systems like Sun Grid Engine, PBS (software), SLURM, and cloud platforms. Modules encapsulate command-line tools or scripts and expose inputs, outputs, and parameters; the system includes wrappers for environments such as Docker, remote execution via SSH, and integration points with databases like PostgreSQL for provenance. Security and user management align with institutional identity systems such as Shibboleth and LDAP.

Workflow Design and Execution

Users build directed graphs by dragging modules into a canvas and connecting data flows to represent processing stages. The engine translates graphs into task graphs, schedules jobs across compute nodes, and streams intermediate artifacts to shared storage or object stores akin to Amazon S3 when deployed in cloud configurations. Features include parallelization strategies, checkpointing, retry policies, and monitoring dashboards; provenance capture records module versions, parameter values, and execution metadata suited to compliance with reporting standards promoted by organizations like the Open Science Framework and the International Neuroinformatics Coordinating Facility. Integration with job schedulers allows scaling from laboratory workstations to high-performance clusters such as those provided by XSEDE.

Supported Data Types and Tools

The system handles a range of biomedical imaging formats and related data types, including DICOM, NIfTI, volumetric MRI, diffusion tensor imaging, functional MRI, and structural connectomics outputs. It includes converters and validators for formats handled by Dicompyler, dcm2niix, and supports surface data used by Caret and Connectome Workbench. Commonly wrapped toolkits include ANTS (software), MRtrix, Camino (diffusion MRI), and scripting languages such as Python (programming language), MATLAB, and R (programming language). The Pipeline also links to statistical packages and machine learning libraries leveraged in projects with collaborators at institutions like Stanford University, Massachusetts Institute of Technology, and Harvard University.

Performance and Scalability

Performance considerations address parallel throughput, I/O bottlenecks, and resource allocation across heterogeneous clusters and cloud instances. Empirical deployments demonstrate linear scaling for embarrassingly parallel tasks across clusters managed by SLURM or Sun Grid Engine, while tightly coupled operations remain bounded by single-node CPU, memory, and storage constraints. Strategies to improve scalability include data partitioning, streaming intermediate artifacts, leveraging containerized execution to reduce environment setup overhead, and employing shared high-performance file systems such as Lustre (file system) and GPFS in institutional centers. Benchmarking exercises have been reported in consortium publications involving compute resources from agencies like the National Science Foundation.

Use Cases and Applications

Principal applications involve large-scale neuroimaging preprocessing, morphometric analyses, diffusion tractography, functional connectivity mapping, and multimodal integration for studies of Alzheimer disease, schizophrenia, autism spectrum disorder, and developmental cohorts. The platform has supported multi-site harmonization efforts, reproducible pipelines for consortia such as the Alzheimer's Disease Neuroimaging Initiative and the Human Connectome Project, and translational pipelines used in collaborations with clinical centers and biobanks. It is also used in teaching workflows in computational neuroscience courses at universities and in method-comparison studies published in journals associated with societies like the Organization for Human Brain Mapping.

Development History and Licensing

Development originated within the Laboratory of Neuro Imaging at the University of Southern California in the early 2000s, evolving through successive versions to incorporate web services, cluster integration, and container support. Funding and collaboration involved federal entities such as the National Institutes of Health and academic partners across North America and Europe. Licensing models have combined academic distribution and institutional licensing; deployments in research centers often follow institutional agreements, while commercial or clinical users negotiate terms with the developing laboratory. The software has influenced subsequent workflow systems and remains part of the neuroinformatics ecosystem alongside tools from projects like Nipype and enterprise workflow managers.

Category:Neuroimaging