From 80974c7fc1b6db5000a6a03776ccb87ac3b0d2a6 Mon Sep 17 00:00:00 2001 From: Dennis Klein Date: Thu, 9 Dec 2021 15:35:03 +0100 Subject: [PATCH] Update --- readme.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/readme.md b/readme.md index 205e322c..76cb59bd 100644 --- a/readme.md +++ b/readme.md @@ -1,13 +1,13 @@ ## FairMQ in a nutshell -Next-generation Particle Physics Experiments at [GSI](https://www.gsi.de)/[FAIR](https://www.gsi.de/forschungbeschleuniger/fair) and [CERN](https://home.web.cern.ch/) are facing [unprecedented data processing challenges](https://doi.org/10.1051/epjconf/201921405010). The expected data rates require a non-trivial amount of high performance compute (HPC) resources, i.e: thousands of CPU/GPU cores per experiment. Online data processing/compression are crucial to stay within storage capacity limits. The complexity of tasks that need to be performed during the online (synchronous) data processing is significantly higher than ever before. Complex tasks that usually run offline (e.g: calibration, track finding, etc) have to now run online in a high performance and high throughput environment. +Next-generation Particle Physics Experiments at [GSI](https://www.gsi.de)/[FAIR](https://www.gsi.de/forschungbeschleuniger/fair) and [CERN](https://home.web.cern.ch/) are facing [unprecedented data processing challenges](https://doi.org/10.1051/epjconf/201921405010). Expected data rates require a non-trivial amount of high performance compute (HPC) resources in the order of thousands of CPU/GPU cores per experiment. Online (synchronous) data processing (compression) is crucial to stay within storage capacity limits. The complexity of tasks that need to be performed during the online data processing is significantly higher than ever before. Most complex tasks like calibration and track finding classically ran in an offline (asynchronous) environment and have to now run online in a high performance and high throughput environment under more strict resource constraints. The [FairMQ C++ library](https://github.com/FairRootGroup/FairMQ/) is designed to aid the implementation of such large-scale online data processing workflows by - providing an **asynchronous message passing abstraction** that integrates different existing data transport technologies (no need to re-invent the wheel), - providing a **reasonably efficient data transport service** (zero-copy, high throughput - [TCP](https://zeromq.org/), [SHMEM](https://doi.org/10.1051/epjconf/201921405029), and [RDMA](https://doi.org/10.1051/epjconf/201921405022) implementations available), - being **data format agnostic** (suitable data formats are usually experiment-specific), and -- providing other **basic building blocks** such as a simple state machine based execution framework and a plugin mechanism to integrate with external config/control systems. +- providing further **basic building blocks** such as a simple state machine based execution framework and a plugin mechanism to integrate with external config/control systems. FairMQ is not an end-user application, but a library and framework used by experiment software experts to implement higher-level experiment-specific applications. @@ -15,4 +15,4 @@ FairMQ is not an end-user application, but a library and framework used by exper The screenshot shows a visualization of the data processing workflow on a single [Alice](https://home.cern/science/experiments/alice) event processing node (The "O2 Framework debug GUI" tool in the screenshot is part of the [AliceO2 project](https://aliceo2group.github.io/AliceO2/)). Data logically flows along the yellow edges (in this case via the FairMQ shmem data transport) through the various processing stages of which some are implemented as GPU and others as CPU algorithms. -Although FairMQ was initially designed for the synchronous (online) data processing, it has been successfully used to parallelize asynchronous (offline) [simulation](https://doi.org/10.1051/epjconf/201921402029) and [analysis](https://doi.org/10.1051/epjconf/201921405045) workloads. +Initially designed with the online data processing in mind, FairMQ has been successfully used to parallelize offline [simulation](https://doi.org/10.1051/epjconf/201921402029) and [analysis](https://doi.org/10.1051/epjconf/201921405045) workloads as well.