Compare commits

...

47 Commits

Author SHA1 Message Date
Dennis Klein
8867e15d8c build: ABI version is defined to be equal to the API version 2022-05-30 09:15:25 +02:00
Dennis Klein
6b2c3c4486 fix(tidy): Regex 2022-05-28 15:14:25 +02:00
Dennis Klein
adf743750f fix(tidy): Only emit diagnostic if source location is valid 2022-05-28 15:14:25 +02:00
Dennis Klein
f3385173b2 fix(zeromq): Leaking monitor socket messages 2022-05-28 15:14:25 +02:00
Dennis Klein
f798f18b95 fix: Remove long obsolete hotfix version component 2022-05-28 15:14:25 +02:00
Dennis Klein
d1dd9d7824 ci: Update macOS builds 2022-05-28 15:14:25 +02:00
Dennis Klein
df742272ed build: Bump bundled GTest to @a1cc8c55 2022-05-28 15:14:25 +02:00
Dennis Klein
58b78995b4 build(fairmq-tidy): Add missing EXPERIMENTAL tag 2022-05-28 15:14:25 +02:00
Dennis Klein
efd6523112 feat(Parts)!: Refine and tweak
* Optimize appending another Parts container
* Remove redundant/verbose comments
* Change r-value args to move-only types into l-value args for
  readability
* BREAKING CHANGE: Remove `AtRef(int)` and `AddPart(Message*)` member functions
* Add various const overloads
* Add `Empty()` and `Clear()` member functions
* Add `noexcept` where applicable
2022-05-28 15:14:25 +02:00
Dennis Klein
6fc2839d02 build: Use kebab-case library names in install tree 2022-05-28 15:14:25 +02:00
Dennis Klein
c585b0d486 feat(plugins): Allow kebab-case plugin names, e.g. libfairmq-plugin-pmix
Camel+snake-case plugin names are still allowed! e.g. `libFairMQPlugin_pmix`
2022-05-28 15:14:25 +02:00
Dennis Klein
933ed8de6e test: Increase robustness of the test suite for high -j 2022-05-28 15:14:25 +02:00
Dennis Klein
08d64ad463 test(channel): Increase sleep time
The logic of the GetNumberOfConnectedPeers test case relies on sleeping
a certain time. We have observed the 10ms sleep time to sometimes be too
short. Increasing it to 100ms should improve test stability.
2022-05-28 15:14:25 +02:00
Dennis Klein
56a7b6e9cb build!: Create a single library again
BREAKING CHANGE: Removes exported targets FairMQ::Tools and
FairMQ::StateMachine. However, it is unlikely those were used
by anyone.
2022-05-28 15:14:25 +02:00
Dennis Klein
8d575a23a5 fix: Use namespaced typenames/headers 2022-05-28 15:14:25 +02:00
Dennis Klein
18f50871ef feat: Deprecate non-namespaced headers
For more details see https://github.com/FairRootGroup/FairMQ/discussions/423
2022-05-28 15:14:25 +02:00
Dennis Klein
c81b03ff29 feat: Deprecate non-namespaced typenames
For more details see https://github.com/FairRootGroup/FairMQ/discussions/423
2022-05-28 15:14:25 +02:00
Dennis Klein
a9ffa2a8af feat!: Remove deprecated components sdk, sdk_commands, dds_plugin
BREAKING CHANGE: Components have been moved to ODC project, see
https://github.com/FairRootGroup/FairMQ/discussions/392 for details.
2022-05-28 15:14:25 +02:00
Alexey Rybalchenko
2500771689 shm: ResetContent(): reset data after recreating the metadata 2022-05-28 14:46:21 +02:00
Alexey Rybalchenko
d2aa3b6bb0 shm: open managament data as read only during cleanup 2022-05-28 14:46:21 +02:00
Alexey Rybalchenko
00df117c7c Shm::Monitor: add nullptr check for segment info 2022-05-28 14:46:21 +02:00
Dennis Klein
69faa63c5b docs: Update README 2022-03-21 18:22:07 +01:00
Dennis Klein
b7474ae138 build: Deprecate components dds_plugin, sdk, sdk_commands 2022-03-21 18:22:07 +01:00
Dennis Klein
b426bf39d7 fix: Update metadata 2022-03-21 18:22:07 +01:00
Dennis Klein
6780b7452c fix(control): Honor SIGINT and SIGTERM in more places
* Queue next transition for long-running states (fix #421)
* Add *OrCustom/Push/Locked family of functions to StateQueue to enable
  composition with custom signals
2022-03-21 16:28:43 +01:00
Dennis Klein
27277b11b4 fix(Device): Warning about narrowing conversion 2022-03-21 16:28:43 +01:00
Dennis Klein
cb5029f826 fix(Device): Spawn rate logger thread only if needed 2022-03-21 16:28:43 +01:00
Dennis Klein
5d45d89269 feat: Remove --max-run-time option
BREAKING CHANGE: was introduced in 1.4.0 release but appears unused
2022-03-21 16:28:43 +01:00
Dennis Klein
eb9ddc81cf ci: Run thread sanitizer with clang++ 2022-03-21 16:28:43 +01:00
Dennis Klein
f5891d5ae3 ci: Add thread sanitizer check and bump all checks to Fedora 35 2022-03-21 16:28:43 +01:00
Dennis Klein
3b2ad1f6f4 ci: Add Fedora 35 build 2022-03-21 16:28:43 +01:00
Giulio Eulisse
fa0bf96eb2 Skip error message only for tcp 2022-02-28 19:13:00 +01:00
Alexey Rybalchenko
29827f0426 Shm: bring back thread-safety for fRegions (intra-process) 2022-02-16 23:17:04 +01:00
Alexey Rybalchenko
8efe7adf0e Shm: fix number of region events 2022-02-16 23:17:04 +01:00
Alexey Rybalchenko
b747a8787c shm: check region size when opening existing 2022-02-08 09:09:25 +01:00
Alexey Rybalchenko
1a75141fc4 shm: allow monitor::ResetContent to cleanup after a crash 2022-02-02 10:49:00 +01:00
Alexey Rybalchenko
2f82eb4f09 shm: monitor: disable number of msgs in the ack queue output 2022-02-02 10:49:00 +01:00
Alexey Rybalchenko
92a56c26bc shm: remove UR queues on ResetContent 2022-02-02 10:49:00 +01:00
Alexey Rybalchenko
4f9aeda8ec shm: Add size to UnmanagedRegion debug output 2022-02-02 10:49:00 +01:00
Giulio Eulisse
ad894c79cf GUI Controller
provide a controller which can be used to control state
transitions from an external GUI.
2022-01-25 18:02:25 +01:00
Alexey Rybalchenko
5f33401d41 Parallelize more tests 2022-01-25 11:55:38 +01:00
Alexey Rybalchenko
f4d39d224b Avoid fixed ports in the test suites 2022-01-25 11:55:38 +01:00
Alexey Rybalchenko
bfd08bb33f Don't use to-be-deprecated names 2022-01-24 06:40:24 +01:00
Alexey Rybalchenko
f15f669853 use [[maybe_unused]] for values used in assertions 2022-01-24 06:40:24 +01:00
Alexey Rybalchenko
f6bade32bb modify keep-alive example executable a bit, make it configurable 2022-01-12 19:54:49 +01:00
Alexey Rybalchenko
ddf9bc7272 shm: keep mng segment around when skipping cleanup 2022-01-12 19:54:49 +01:00
Alexey Rybalchenko
f79a0714b4 shm: fix double unlock() 2022-01-12 19:54:49 +01:00
220 changed files with 1823 additions and 8322 deletions

View File

@@ -1,5 +1,5 @@
################################################################################
# Copyright (C) 2018-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH #
# Copyright (C) 2018-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH #
# #
# This software is distributed under the terms of the #
# GNU Lesser General Public Licence (LGPL) version 3, #
@@ -9,7 +9,7 @@
# Project ######################################################################
cmake_minimum_required(VERSION 3.15 FATAL_ERROR)
cmake_policy(VERSION 3.15...3.20)
cmake_policy(VERSION 3.15...3.22)
list(PREPEND CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake)
include(GitHelper)
@@ -29,16 +29,10 @@ fairmq_build_option(BUILD_TESTING "Build tests."
DEFAULT OFF REQUIRES "BUILD_FAIRMQ")
fairmq_build_option(BUILD_OFI_TRANSPORT "Build experimental OFI transport."
DEFAULT OFF REQUIRES "BUILD_FAIRMQ")
fairmq_build_option(BUILD_SDK_COMMANDS "Build the FairMQ SDK commands."
DEFAULT OFF)
fairmq_build_option(BUILD_DDS_PLUGIN "Build DDS plugin."
DEFAULT OFF REQUIRES "BUILD_FAIRMQ;BUILD_SDK_COMMANDS")
fairmq_build_option(BUILD_PMIX_PLUGIN "Build PMIx plugin."
DEFAULT OFF REQUIRES "BUILD_FAIRMQ;BUILD_SDK_COMMANDS")
DEFAULT OFF REQUIRES "BUILD_FAIRMQ")
fairmq_build_option(BUILD_EXAMPLES "Build FairMQ examples."
DEFAULT ON REQUIRES "BUILD_FAIRMQ")
fairmq_build_option(BUILD_SDK "Build the FairMQ controller SDK."
DEFAULT OFF REQUIRES "BUILD_DDS_PLUGIN;BUILD_SDK_COMMANDS")
fairmq_build_option(BUILD_TIDY_TOOL "Build the fairmq-tidy tool."
DEFAULT OFF)
fairmq_build_option(BUILD_DOCS "Build FairMQ documentation."
@@ -57,7 +51,7 @@ include(FairMQDependencies)
# Targets ######################################################################
if(BUILD_FAIRMQ OR BUILD_SDK)
if(BUILD_FAIRMQ)
add_subdirectory(fairmq)
endif()
@@ -92,9 +86,6 @@ endif()
if(BUILD_TESTING)
list(APPEND PROJECT_PACKAGE_COMPONENTS tests)
endif()
if(BUILD_DDS_PLUGIN)
list(APPEND PROJECT_PACKAGE_COMPONENTS dds_plugin)
endif()
if(BUILD_PMIX_PLUGIN)
list(APPEND PROJECT_PACKAGE_COMPONENTS pmix_plugin)
endif()
@@ -107,12 +98,6 @@ endif()
if(BUILD_DOCS)
list(APPEND PROJECT_PACKAGE_COMPONENTS docs)
endif()
if(BUILD_SDK)
list(APPEND PROJECT_PACKAGE_COMPONENTS sdk)
endif()
if(BUILD_SDK_COMMANDS)
list(APPEND PROJECT_PACKAGE_COMPONENTS sdk_commands)
endif()
if(BUILD_TIDY_TOOL)
list(APPEND PROJECT_PACKAGE_COMPONENTS tidy_tool)
endif()
@@ -125,11 +110,6 @@ if(BUILD_FAIRMQ)
DESTINATION ${PROJECT_INSTALL_CMAKEMODDIR}
)
endif()
if(BUILD_SDK OR BUILD_DDS_PLUGIN)
install(FILES cmake/Findasio.cmake
DESTINATION ${PROJECT_INSTALL_CMAKEMODDIR}
)
endif()
if(BUILD_DOCS)
install(DIRECTORY ${CMAKE_BINARY_DIR}/doxygen/html
DESTINATION ${PROJECT_INSTALL_DATADIR}/docs

View File

@@ -4,19 +4,19 @@ Upstream-Contact: Mohammad Al-Turany <m.al-turany@gsi.de>
Source: https://github.com/FairRootGroup/FairMQ
Files: *
Copyright: 2012-2021, GSI Helmholtzzentrum fuer Schwerionenforschung GmbH
Copyright: 2012-2021, [see AUTHORS file]
Copyright: 2012-2021, [see CONTRIBUTORS file]
Copyright: 2012-2022, GSI Helmholtzzentrum fuer Schwerionenforschung GmbH
Copyright: 2012-2022, [see AUTHORS file]
Copyright: 2012-2022, [see CONTRIBUTORS file]
Comment: The copyright of individual contributors is documented in the
Git history.
License: LGPL-3.0-only
Files: extern/googletest
Copyright: 2008-2021, Google Inc.
Copyright: 2008-2022, Google Inc.
License: GOOGLE
Files: extern/asio
Copyright: 2003-2021, Christopher M. Kohlhoff (chris at kohlhoff dot com)
Copyright: 2003-2022, Christopher M. Kohlhoff (chris at kohlhoff dot com)
License: BSL-1.0
Files: extern/PicoSHA2

View File

@@ -42,11 +42,8 @@ endif()
ctest_start(Continuous)
list(APPEND options "-DDISABLE_COLOR=ON" "-DBUILD_EXAMPLES=ON" "-DBUILD_TESTING=ON")
if(HAS_ASIO AND HAS_DDS)
list(APPEND options "-DBUILD_SDK_COMMANDS=ON" "-DBUILD_SDK=ON" "-DBUILD_DDS_PLUGIN=ON")
endif()
if(HAS_PMIX)
list(APPEND options "-DBUILD_SDK_COMMANDS=ON" "-DBUILD_PMIX_PLUGIN=ON")
list(APPEND options "-DBUILD_PMIX_PLUGIN=ON")
endif()
if(HAS_ASIO AND HAS_ASIOFI)
list(APPEND options "-DBUILD_OFI_TRANSPORT=ON")
@@ -72,6 +69,9 @@ endif()
if(ENABLE_SANITIZER_THREAD)
list(APPEND options "-DENABLE_SANITIZER_THREAD=ON")
endif()
if(CMAKE_CXX_COMPILER)
list(APPEND options "-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}")
endif()
if(CMAKE_CXX_FLAGS)
list(APPEND options "-DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}")
endif()

10
Jenkinsfile vendored
View File

@@ -15,9 +15,9 @@ def jobMatrix(String type, List specs) {
ver = spec.ver
} else { // == 'check'
job = "${spec.name}"
selector = 'fedora-34-x86_64'
selector = 'fedora-35-x86_64'
os = 'fedora'
ver = '34'
ver = '35'
}
def label = "${job}"
@@ -96,8 +96,9 @@ pipeline{
[os: 'fedora', ver: '32', arch: 'x86_64', compiler: 'gcc-10', extra: all],
[os: 'fedora', ver: '33', arch: 'x86_64', compiler: 'gcc-10', extra: all],
[os: 'fedora', ver: '34', arch: 'x86_64', compiler: 'gcc-11', extra: all],
[os: 'macos', ver: '11', arch: 'x86_64', compiler: 'apple-clang-12', extra: '-DHAS_ASIO=ON'],
[os: 'macos', ver: '11', arch: 'arm64', compiler: 'apple-clang-13', extra: '-DHAS_ASIO=ON'],
[os: 'fedora', ver: '35', arch: 'x86_64', compiler: 'gcc-11', extra: all],
[os: 'macos', ver: '12', arch: 'x86_64', compiler: 'apple-clang-13', extra: '-DHAS_ASIO=ON'],
[os: 'macos', ver: '12', arch: 'arm64', compiler: 'apple-clang-13', extra: '-DHAS_ASIO=ON'],
])
def all_debug = "${all} -DCMAKE_BUILD_TYPE=Debug"
@@ -106,6 +107,7 @@ pipeline{
[name: 'static-analyzers', extra: "${all_debug} -DRUN_STATIC_ANALYSIS=ON"],
[name: '{address,leak,ub}-sanitizers',
extra: "${all_debug} -DENABLE_SANITIZER_ADDRESS=ON -DENABLE_SANITIZER_LEAK=ON -DENABLE_SANITIZER_UNDEFINED_BEHAVIOUR=ON -DCMAKE_CXX_FLAGS='-O1 -fno-omit-frame-pointer'"],
[name: 'thread-sanitizer', extra: "${all_debug} -DENABLE_SANITIZER_THREAD=ON -DCMAKE_CXX_COMPILER=clang++"],
])
parallel(builds + checks)

View File

@@ -1,12 +1,9 @@
<!-- {#mainpage} -->
# FairMQ [![license](https://alfa-ci.gsi.de/shields/badge/license-LGPL--3.0-orange.svg)](COPYRIGHT) [![build status](https://alfa-ci.gsi.de/buildStatus/icon?job=FairRootGroup/FairMQ/dev)](https://alfa-ci.gsi.de/blue/organizations/jenkins/FairRootGroup%2FFairMQ/branches) [![Coverity Badge](https://alfa-ci.gsi.de/shields/coverity/scan/fairrootgroup-fairmq.svg)](https://scan.coverity.com/projects/fairrootgroup-fairmq)
# FairMQ [![license](https://alfa-ci.gsi.de/shields/badge/license-LGPL--3.0-orange.svg)](COPYRIGHT)
C++ Message Queuing Library and Framework
| Release | Version | Docs |
| :---: | :--- | :--- |
| `stable` | [![release](https://alfa-ci.gsi.de/shields/github/release/FairRootGroup/FairMQ.svg)](https://github.com/FairRootGroup/FairMQ/releases/latest) | [API](https://fairrootgroup.github.io/FairMQ/latest), [Book](https://github.com/FairRootGroup/FairMQ/blob/master/README.md#documentation) |
| `testing` | [![dev tag](https://alfa-ci.gsi.de/shields/github/tag/FairRootGroup/FairMQ.svg)](https://github.com/FairRootGroup/FairMQ/tags) | [Book](https://github.com/FairRootGroup/FairMQ/blob/dev/README.md#documentation) |
Docs: [Book](https://github.com/FairRootGroup/FairMQ/blob/dev/README.md#documentation)
Find all FairMQ releases [here](https://github.com/FairRootGroup/FairMQ/releases).
@@ -24,11 +21,13 @@ FairMQ provides multiple implementations for its API (so-called "transports",
e.g. `zeromq`, `shmem` and `ofi` (in development)) to cover a variety of use cases
(e.g. inter-thread, inter-process, inter-node communication) and machines (e.g. Ethernet, Infiniband).
In addition to this core functionality FairMQ provides a framework for creating "devices" - actors which
are communicating through message passing. FairMQ does not only allow the user to use different transport but also to mix them; i.e: A Device can communicate using different transport on different channels at the same time. Device execution is modelled as a simple state machine that
shapes the integration points for the user task. Devices also incorporate a plugin system for runtime configuration and control.
Next to the provided devices and plugins (e.g. [DDS](https://github.com/FairRootGroup/DDS))
the user can extend FairMQ by developing his own plugins to integrate his devices with external
configuration and control services.
are communicating through message passing. FairMQ does not only allow the user to use different transport
but also to mix them; i.e: A Device can communicate using different transport on different channels at the
same time. Device execution is modelled as a simple state machine that shapes the integration points for
the user task. Devices also incorporate a plugin system for runtime configuration and control.
Next to the provided [devices](https://github.com/FairRootGroup/FairMQ/tree/master/fairmq/devices) and
[plugins](https://github.com/FairRootGroup/FairMQ/tree/master/fairmq/plugins) the user can extend FairMQ
by developing his own plugins to integrate his devices with external configuration and control services.
FairMQ has been developed in the context of its mother project [FairRoot](https://github.com/FairRootGroup/FairRoot) -
a simulation, reconstruction and analysis framework.
@@ -47,14 +46,15 @@ cmake --build fairmq_build --target install
Please consult the [manpages of your CMake version](https://cmake.org/cmake/help/latest/manual/cmake.1.html) for more options.
If dependencies are not installed in standard system directories, you can hint the installation location via `-DCMAKE_PREFIX_PATH=...` or per dependency via `-D{DEPENDENCY}_ROOT=...`. `{DEPENDENCY}` can be `GTEST`, `BOOST`, `FAIRLOGGER`, `ZEROMQ`, `OFI`, `PMIX`, `ASIO`, `ASIOFI` or `DDS` (`*_ROOT` variables can also be environment variables).
If dependencies are not installed in standard system directories, you can hint the installation location via
`-DCMAKE_PREFIX_PATH=...` or per dependency via `-D{DEPENDENCY}_ROOT=...` (`*_ROOT` variables can also be environment variables).
## Usage
FairMQ ships as a CMake package, so in your `CMakeLists.txt` you can discover it like this:
```cmake
find_package(FairCMakeModules 0.2 REQUIRED)
find_package(FairCMakeModules 1.0 REQUIRED)
include(FairFindPackage2)
find_package2(FairMQ)
find_package2_implicit_dependencies()
@@ -71,18 +71,17 @@ list(PREPEND CMAKE_PREFIX_PATH /path/to/fairmq_install)
Optionally, you can require certain FairMQ package components and a minimum version:
```cmake
find_package(FairMQ 1.4.0 COMPONENTS dds_plugin)
find_package(FairMQ 1.4.50 COMPONENTS ofi_transport)
```
When building FairMQ, CMake will print a summary table of all available package components.
## Dependencies
* [asio](https://github.com/chriskohlhoff/asio) (optionally bundled)
* [asio](https://github.com/chriskohlhoff/asio)
* [asiofi](https://github.com/FairRootGroup/asiofi)
* [Boost](https://www.boost.org/)
* [CMake](https://cmake.org/)
* [DDS](http://dds.gsi.de)
* [Doxygen](http://www.doxygen.org/)
* [FairCMakeModules](https://github.com/FairRootGroup/FairCMakeModules) (optionally bundled)
* [FairLogger](https://github.com/FairRootGroup/FairLogger)
@@ -92,7 +91,7 @@ When building FairMQ, CMake will print a summary table of all available package
Which dependencies are required depends on which components are built.
Supported platforms: Linux and MacOS.
Supported platform is Linux. macOS is supported on a best-effort basis.
## CMake options
@@ -102,7 +101,6 @@ On command line:
* `-DBUILD_TESTING=OFF` disables building of tests.
* `-DBUILD_EXAMPLES=OFF` disables building of examples.
* `-DBUILD_OFI_TRANSPORT=ON` enables building of the experimental OFI transport.
* `-DBUILD_DDS_PLUGIN=ON` enables building of the DDS plugin.
* `-DBUILD_PMIX_PLUGIN=ON` enables building of the PMIx plugin.
* `-DBUILD_DOCS=ON` enables building of API docs.
* You can hint non-system installations for dependent packages, see the #installation-from-source section above
@@ -161,6 +159,4 @@ After the `find_package(FairMQ)` call the following CMake variables are defined:
1. [Usage](docs/Plugins.md#71-usage)
2. [Development](docs/Plugins.md#72-development)
3. [Provided Plugins](docs/Plugins.md#73-provided-plugins)
1. [DDS](docs/Plugins.md#731-dds)
2. [PMIx](docs/Plugins.md#732-pmix)
8. [Controller SDK](docs/SDK.md)
2. [PMIx](docs/Plugins.md#731-pmix)

View File

@@ -12,7 +12,7 @@ include(FairCMakeModules)
include(FairFindPackage2)
include(FairMQBundlePackageHelper)
if(BUILD_FAIRMQ OR BUILD_SDK)
if(BUILD_FAIRMQ)
set(THREADS_PREFER_PTHREAD_FLAG TRUE)
find_package2(PUBLIC Threads REQUIRED)
set(Threads_PREFIX "<system>")
@@ -23,28 +23,18 @@ if(BUILD_OFI_TRANSPORT)
find_package2(PRIVATE OFI REQUIRED)
endif()
if(BUILD_SDK_COMMANDS)
find_package2(PRIVATE Flatbuffers REQUIRED)
endif()
if(BUILD_DDS_PLUGIN OR BUILD_SDK)
find_package2(PRIVATE DDS REQUIRED VERSION 3.5.13.7)
set(DDS_Boost_COMPONENTS system log log_setup regex filesystem thread)
set(DDS_Boost_VERSION 1.67)
endif()
if(BUILD_PMIX_PLUGIN)
find_package2(PRIVATE PMIx REQUIRED VERSION 2.1.4)
endif()
if(BUILD_FAIRMQ OR BUILD_SDK OR BUILD_TIDY_TOOL)
if(BUILD_FAIRMQ OR BUILD_TIDY_TOOL)
find_package2(PUBLIC FairLogger REQUIRED VERSION 1.6.0)
find_package2(PUBLIC Boost REQUIRED VERSION 1.66
COMPONENTS container program_options filesystem date_time regex
)
endif()
if(BUILD_OFI_TRANSPORT OR BUILD_SDK OR BUILD_DDS_PLUGIN)
if(BUILD_OFI_TRANSPORT)
set(__old ${CMAKE_FIND_PACKAGE_PREFER_CONFIG})
set(CMAKE_FIND_PACKAGE_PREFER_CONFIG ON)
find_package2(PUBLIC asio REQUIRED VERSION 1.18)
@@ -68,7 +58,7 @@ if(BUILD_TESTING)
endif()
find_package2(BUNDLED GTest REQUIRED)
if(GTest_BUNDLED)
set(GTest_VERSION "Apr 28 2021 @f5e592d8")
set(GTest_VERSION "Apr 8 2022 @a1cc8c55")
set(GTest_PREFIX "<bundled>")
endif()
endif()
@@ -101,8 +91,6 @@ if(PROJECT_PACKAGE_DEPENDENCIES)
if(NOT asiofi_PREFIX AND asiofi_ROOT)
set(asiofi_PREFIX ${asiofi_ROOT})
endif()
elseif(${dep} STREQUAL DDS)
set(DDS_PREFIX "${DDS_INSTALL_PREFIX}")
elseif(${dep} STREQUAL Boost)
if(TARGET Boost::headers)
get_target_property(boost_include Boost::headers INTERFACE_INCLUDE_DIRECTORIES)
@@ -116,14 +104,6 @@ if(PROJECT_PACKAGE_DEPENDENCIES)
get_filename_component(Doxygen_PREFIX ${doxygen_bin} DIRECTORY)
get_filename_component(Doxygen_PREFIX ${Doxygen_PREFIX}/.. ABSOLUTE)
unset(doxygen_bin)
elseif(${dep} STREQUAL Flatbuffers)
if(TARGET flatbuffers::flatbuffers)
get_target_property(flatbuffers_include flatbuffers::flatbuffers INTERFACE_INCLUDE_DIRECTORIES)
else()
get_target_property(flatbuffers_include flatbuffers::flatbuffers_shared INTERFACE_INCLUDE_DIRECTORIES)
endif()
get_filename_component(Flatbuffers_PREFIX ${flatbuffers_include}/.. ABSOLUTE)
unset(flatbuffers_include)
elseif(NOT ${dep}_PREFIX)
# try to guess
if(TARGET ${dep}::${dep})

View File

@@ -1,5 +1,5 @@
################################################################################
# Copyright (C) 2018-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH #
# Copyright (C) 2018-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH #
# #
# This software is distributed under the terms of the #
# GNU Lesser General Public Licence (LGPL) version 3, #
@@ -33,12 +33,6 @@ macro(fairmq_summary_components)
set(ofi_summary "${BRed} NO${CR} EXPERIMENTAL (default, enable with ${BMagenta}-DBUILD_OFI_TRANSPORT=ON${CR})")
endif()
message(STATUS " ${BWhite}ofi_transport${CR} ${ofi_summary}")
if(BUILD_DDS_PLUGIN)
set(dds_summary "${BGreen}YES${CR} (disable with ${BMagenta}-DBUILD_DDS_PLUGIN=OFF${CR})")
else()
set(dds_summary "${BRed} NO${CR} (default, enable with ${BMagenta}-DBUILD_DDS_PLUGIN=ON${CR})")
endif()
message(STATUS " ${BWhite}dds_plugin${CR} ${dds_summary}")
if(BUILD_PMIX_PLUGIN)
set(pmix_summary "${BGreen}YES${CR} EXPERIMENTAL (disable with ${BMagenta}-DBUILD_PMIX_PLUGIN=OFF${CR})")
else()
@@ -57,22 +51,10 @@ macro(fairmq_summary_components)
set(docs_summary "${BRed} NO${CR} (default, enable with ${BMagenta}-DBUILD_DOCS=ON${CR})")
endif()
message(STATUS " ${BWhite}docs${CR} ${docs_summary}")
if(BUILD_SDK)
set(sdk_summary "${BGreen}YES${CR} EXPERIMENTAL (disable with ${BMagenta}-DBUILD_SDK=OFF${CR})")
else()
set(sdk_summary "${BRed} NO${CR} EXPERIMENTAL (default, enable with ${BMagenta}-DBUILD_SDK=ON${CR})")
endif()
message(STATUS " ${BWhite}sdk${CR} ${sdk_summary}")
if(BUILD_SDK_COMMANDS)
set(sdk_commands_summary "${BGreen}YES${CR} (disable with ${BMagenta}-DBUILD_SDK_COMMANDS=OFF${CR})")
else()
set(sdk_commands_summary "${BRed} NO${CR} (default, enable with ${BMagenta}-DBUILD_SDK_COMMANDS=ON${CR})")
endif()
message(STATUS " ${BWhite}sdk_commands${CR} ${sdk_commands_summary}")
if(BUILD_TIDY_TOOL)
set(sdk_tidy_summary "${BGreen}YES${CR} (disable with ${BMagenta}-DBUILD_TIDY_TOOL=OFF${CR})")
set(sdk_tidy_summary "${BGreen}YES${CR} EXPERIMENTAL (disable with ${BMagenta}-DBUILD_TIDY_TOOL=OFF${CR})")
else()
set(sdk_tidy_summary "${BRed} NO${CR} (default, enable with ${BMagenta}-DBUILD_TIDY_TOOL=ON${CR})")
set(sdk_tidy_summary "${BRed} NO${CR} EXPERIMENTAL (default, enable with ${BMagenta}-DBUILD_TIDY_TOOL=ON${CR})")
endif()
message(STATUS " ${BWhite}tidy_tool${CR} ${sdk_tidy_summary}")
endmacro()

View File

@@ -16,17 +16,16 @@ Here is an overview of the device/channel options and when they are applied:
| Property | Applied in |
| --- | --- |
| `severity` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<runFairMQDevice.h>`)) |
| `file-severity` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<runFairMQDevice.h>`)) |
| `verbosity` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<runFairMQDevice.h>`)) |
| `color` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<runFairMQDevice.h>`)) |
| `log-to-file` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<runFairMQDevice.h>`)) |
| `severity` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<fairmq/runDevice.h>`)) |
| `file-severity` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<fairmq/runDevice.h>`)) |
| `verbosity` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<fairmq/runDevice.h>`)) |
| `color` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<fairmq/runDevice.h>`)) |
| `log-to-file` | immidiately (if `fair::mq::DeviceRunner` is used (also the case when using `<fairmq/runDevice.h>`)) |
| `id` | at the end of `fair::mq::State::InitializingDevice` |
| `io-threads` | at the end of `fair::mq::State::InitializingDevice` |
| `transport` | at the end of `fair::mq::State::InitializingDevice` |
| `network-interface` | at the end of `fair::mq::State::InitializingDevice` |
| `init-timeout` | at the end of `fair::mq::State::InitializingDevice` |
| `max-run-time` | at the end of `fair::mq::State::InitializingDevice` |
| `shm-segment-size` | at the end of `fair::mq::State::InitializingDevice` |
| `shm-monitor` | at the end of `fair::mq::State::InitializingDevice` |
| `ofi-size-hint` | at the end of `fair::mq::State::InitializingDevice` |

View File

@@ -6,7 +6,7 @@
For unit testing it is often not feasible to boot up a full-blown distributed system with dozens of processes.
In some scenarios it is useful to not even instantiate a `FairMQDevice` at all. Please see [this example](../test/protocols/_push_pull_multipart.cxx) for single and multi threaded unit test without a device instance. If you store your transport factories and channels on the heap, pls make sure, you destroy the channels before you destroy the related transport factory for proper shutdown. Channels provide all the `Send/Receive` and `New*Message/New*Poller` APIs provided by the device too.
In some scenarios it is useful to not even instantiate a `fair::mq::Device` at all. Please see [this example](../test/protocols/_push_pull_multipart.cxx) for single and multi threaded unit test without a device instance. If you store your transport factories and channels on the heap, pls make sure, you destroy the channels before you destroy the related transport factory for proper shutdown. Channels provide all the `Send/Receive` and `New*Message/New*Poller` APIs provided by the device too.
## 4.2 Static Analysis

View File

@@ -2,7 +2,7 @@
# 1. Device
The components encapsulating the tasks are called **devices** and derive from the common base class `FairMQDevice`. FairMQ provides ready to use devices to organize the dataflow between the components (without touching the contents of a message), providing functionality like merging and splitting of the data stream (see subdirectory `devices`).
The components encapsulating the tasks are called **devices** and derive from the common base class `fair::mq::Device`. FairMQ provides ready to use devices to organize the dataflow between the components (without touching the contents of a message), providing functionality like merging and splitting of the data stream (see subdirectory `devices`).
## 1.1 Topology

View File

@@ -35,7 +35,7 @@ Plugin Manager:
see man ld.so(8) for details.
-P [ --plugin ] arg List of plugin names to load in
order,e.g. if the file is called
'libFairMQPlugin_example.so', just list
'libfairmq-plugin-example.so', just list
'example' or 'd:example' here.To load a
prelinked plugin, list 'p:example'
here.
@@ -54,37 +54,7 @@ A more complete example which may serve as a start including example CMake code
## 7.3 Provided Plugins
### 7.3.1 DDS
When launching a FairMQ topology via [DDS](http://dds.gsi.de/) the DDS plugin enables FairMQ devices to interact with DDS' custom command and property subsystems - enable the plugin by passing `-P dds` on the command line.
Via the property subsystem a FairMQ topology may exchange channel connection data (essentially to do service discovery) needed to connect/bind all FairMQ channels appropriately. DDS is highly optimized for this use case. See [examples/dds](examples/dds/README.md) for more details.
Via the custom command subsystem a FairMQ device can receive a number of commands. FairMQ provides a convenient command line tool `fairmq-dds-command-ui` that allows interactive or scripted control of a running FairMQ topology managed via DDS. If one develops directly against the custom command DDS API, the following table lists all the commands the DDS plugin currently understands:
| Custom Command | Response | Error | Description |
| --- | --- | --- | --- |
| `check-state` | `<ID>: <STATE> (pid: <PID>)` | n/a | Query current device state, see state machine for possible states |
| `dump-config` | `(<ID>: <PKEY> -> <PVALUE>\n)+` | n/a | Query current device config (list property key/value pairs) |
| `INIT DEVICE` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `BIND` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `CONNECT` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `INIT TASK` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `RUN` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `STOP` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `RESET TASK` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `RESET DEVICE` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `END` | `<ID>: queued <CMD> transition` | `<ID>: could not queue <CMD> transition` | Initiate state transition |
| `subscribe-to-heartbeats` | `heartbeat-subscription: <ID>,OK` | n/a | Subscribe to heartbeats |
| on heartbeat subscription | `heartbeat: <ID>,<PID>` | n/a | Heartbeat every 100ms |
| `unsubscribe-from-heartbeats` | `heartbeat-unsubscription: <ID>,OK` | n/a | Unsubscribe from heartbeats |
| `subscribe-to-state-changes` | `state-changes-subscription: <ID>,OK` | n/a | Subscribe to state changes |
| on state changes subscription | `state-change: <ID>,<STATE>` | n/a | State change notification |
| `unsubscribe-from-state-changes` | `state-changes-unsubscription: <ID>,OK` | n/a | Unsubscribe from state changes |
If unknown commands are received the plugin will print a warning.
### 7.3.2 PMIx
### 7.3.1 PMIx
The [PMIx](https://pmix.org/) plugin enables launching a FairMQ topology with any PMIx capable launcher, e.g. the [Open Run-Time Environment (ORTE) of OpenMPI](https://www.open-mpi.org/doc/v4.0/man1/mpirun.1.php) or the [Slurm workload manager](https://slurm.schedmd.com/srun.html). This plugin is not (yet) very mature and serves as a proof of concept at the moment.

View File

@@ -1,34 +0,0 @@
← [Back](../README.md)
# 8. Controller SDK
The FairMQ Controller Software Development Kit (`-DBUILD_SDK=ON`) contains a (as of today still experimental) set of C++ APIs that provide essential functionality to the implementer of a global controller.
The FairMQ core library only provides two local controllers - `static` (a fixed sequence of state transitions) and `interactive` (a read-eval-print-loop which reads keyboard commands from standard input). A local controller only knows how steer a single [FairMQ device](Device.md) - in fact, it runs in a thread within the device process.
A global controller has knowledge about the full topology of connected FairMQ devices. Its responsibility is to facilitate the lifecycle of a distributed FairMQ-based application (*executing a topology*), such as
* allocating/releasing compute resources from a resource management system,
* launching/setting up the run-time environment and the FairMQ devices,
* driving the device state machines in lock-step across the full topology,
* pushing the device configuration,
* monitoring (some aspects of the application's) operation,
* and handling/reporting (some) error cases.
The low-level hook to integrate FairMQ devices with such a global contoller is the [plugin mechanism](Plugins.md) in the FairMQ core library. The FairMQ Controller SDK provides C++ APIs that communicate to the endpoints exposed by such a FairMQ plugin.
At the moment, the Controller SDK only supports [DDS](https://dds.gsi.de) as resource manager and run-time environment. A second implementation based on [PMIx](https://pmix.org/) (targeting its implementation in [Slurm](https://slurm.schedmd.com/documentation.html) and [OpenRTE](https://www-lb.open-mpi.org/papers/euro-pvmmpi-2005-orte/)) is in development.
The following section give a short overview on the APIs provided.
## RMS and run-time environment
The classes [`fair::mq::sdk::DDSEnvironment`](../fairmq/sdk/DDSEnvironment.h), [`fair::mq::sdk::DDSSession`](../fairmq/sdk/DDSSession.h), and [`fair::mq::sdk::DDSTopology`](../fairmq/sdk/DDSTopology.h) are thin wrappers of most of the synchronous APIs exposed by DDS ([`dds::tools_api`](http://dds.gsi.de/doc/api-docs/DDS/html/namespacedds_1_1tools__api.html) and [`dds::topology_api`](http://dds.gsi.de/doc/api-docs/DDS/html/namespacedds_1_1topology__api.html)). E.g. they allow to [start a DDS session](https://github.com/FairRootGroup/FairMQ/blob/077eb0ef691940d764cfd1852bf3981dc812ddbd/main.cpp#L26-L28), [allocate resources](https://github.com/FairRootGroup/FairMQ/blob/077eb0ef691940d764cfd1852bf3981dc812ddbd/main.cpp#L34) and [launch a topology](https://github.com/FairRootGroup/FairMQ/blob/077eb0ef691940d764cfd1852bf3981dc812ddbd/main.cpp#L39) from a C++ program.
## Driving the global state machine
The class [`fair::mq::sdk::Topology`](../fairmq/sdk/Topology.h) adds a FairMQ-specific view on an existing DDS session that is executing a topology of FairMQ devices. One can e.g. [initiate a state transition on all devices in the topology simultaneously](https://github.com/FairRootGroup/FairMQ/blob/077eb0ef691940d764cfd1852bf3981dc812ddbd/main.cpp#L48-L49). This topology transition completes once a topology-wide barrier is passed (all devices completed the transition). This effectively exposes the device state machine as a topology state machine. The implementation is based on remote procedure calls over the [DDS intercom service](http://dds.gsi.de/doc/api-docs/DDS/html/namespacedds_1_1intercom__api.html) between the controller and the DDS plugin shipped with FairMQ (`-DBUILD_DDS_PLUGIN=ON`).
For future versions of the SDK new APIs are planned to inspect and modify the device configurations and also operate only on subsets of a given topology.
← [Back](../README.md)

View File

@@ -23,10 +23,10 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-1-1.sh.in ${CMAKE_CUR
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-1-1.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-1-1.sh)
add_test(NAME Example.1-1.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-1-1.sh zeromq)
set_tests_properties(Example.1-1.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received: ")
set_tests_properties(Example.1-1.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received: ")
add_test(NAME Example.1-1.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-1-1.sh shmem)
set_tests_properties(Example.1-1.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received: ")
set_tests_properties(Example.1-1.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received: ")
# install

View File

@@ -30,7 +30,7 @@ struct Sampler : fair::mq::Device
// create message object with a pointer to the data buffer, its size,
// custom deletion function (called when transfer is done),
// and pointer to the object managing the data buffer
FairMQMessagePtr msg(NewMessage(
fair::mq::MessagePtr msg(NewMessage(
const_cast<char*>(text->c_str()),
text->length(),
[](void* /*data*/, void* object) { delete static_cast<std::string*>(object); },

View File

@@ -27,7 +27,7 @@ struct Sink : fair::mq::Device
fMaxIterations = fConfig->GetProperty<uint64_t>("max-iterations");
}
bool HandleData(FairMQMessagePtr& msg, int)
bool HandleData(fair::mq::MessagePtr& msg, int)
{
LOG(info) << "Received: \"" << std::string(static_cast<char*>(msg->GetData()), msg->GetSize()) << "\"";

View File

@@ -8,20 +8,23 @@ if [[ $1 =~ ^[a-z]+$ ]]; then
transport=$1
fi
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan="data"
chanAddr="/tmp/fmq_$session""_""$chan""_""$transport"
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; wait $SAMPLER_PID; wait $SINK_PID;' TERM
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; wait $SAMPLER_PID; wait $SINK_PID; rm $chanAddr' TERM
SAMPLER="fairmq-ex-1-1-sampler"
SAMPLER+=" --id sampler1"
SAMPLER+=" --rate 1"
SAMPLER+=" --transport $transport"
SAMPLER+=" --verbosity veryhigh"
SAMPLER+=" --session $SESSION"
SAMPLER+=" --session $session"
SAMPLER+=" --shm-segment-size 100000000"
SAMPLER+=" --control static --color false"
SAMPLER+=" --max-iterations 1"
SAMPLER+=" --channel-config name=data,type=push,method=bind,address=tcp://*:5555,rateLogging=0"
SAMPLER+=" --channel-config name=$chan,type=push,method=bind,address=ipc://$chanAddr,rateLogging=0"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER &
SAMPLER_PID=$!
@@ -29,13 +32,16 @@ SINK="fairmq-ex-1-1-sink"
SINK+=" --id sink1"
SINK+=" --transport $transport"
SINK+=" --verbosity veryhigh"
SINK+=" --session $SESSION"
SINK+=" --session $session"
SINK+=" --shm-segment-size 100000000"
SINK+=" --control static --color false"
SINK+=" --max-iterations 1"
SINK+=" --channel-config name=data,type=pull,method=connect,address=tcp://localhost:5555,rateLogging=0"
SINK+=" --channel-config name=$chan,type=pull,method=connect,address=ipc://$chanAddr,rateLogging=0"
@CMAKE_CURRENT_BINARY_DIR@/$SINK &
SINK_PID=$!
# wait for sampler and sink to finish
wait $SAMPLER_PID
wait $SINK_PID
rm $chanAddr

View File

@@ -29,10 +29,10 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/ex-1-n-1.json ${CMAKE_CURRENT_BINARY_
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-1-n-1.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-1-n-1.sh)
add_test(NAME Example.1-n-1.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-1-n-1.sh zeromq)
set_tests_properties(Example.1-n-1.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received: ")
set_tests_properties(Example.1-n-1.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received: ")
add_test(NAME Example.1-n-1.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-1-n-1.sh shmem)
set_tests_properties(Example.1-n-1.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received: ")
set_tests_properties(Example.1-n-1.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received: ")
# install

View File

@@ -20,7 +20,7 @@ struct Processor : fair::mq::Device
OnData("data1", &Processor::HandleData);
}
bool HandleData(FairMQMessagePtr& msg, int)
bool HandleData(fair::mq::MessagePtr& msg, int)
{
LOG(info) << "Received data, processing...";
@@ -32,7 +32,7 @@ struct Processor : fair::mq::Device
// its size,
// custom deletion function (called when transfer is done),
// and pointer to the object managing the data buffer
FairMQMessagePtr msg2(NewMessage(const_cast<char*>(text->c_str()),
fair::mq::MessagePtr msg2(NewMessage(const_cast<char*>(text->c_str()),
text->length(),
[](void* /*data*/, void* object) { delete static_cast<std::string*>(object); },
text));

View File

@@ -28,7 +28,7 @@ struct Sampler : fair::mq::Device
{
// Initializing message with NewStaticMessage will avoid copy
// but won't delete the data after the sending is completed.
FairMQMessagePtr msg(NewStaticMessage(fText));
fair::mq::MessagePtr msg(NewStaticMessage(fText));
LOG(info) << "Sending \"" << fText << "\"";

View File

@@ -27,7 +27,7 @@ struct Sink : fair::mq::Device
fMaxIterations = fConfig->GetProperty<uint64_t>("max-iterations");
}
bool HandleData(FairMQMessagePtr& msg, int)
bool HandleData(fair::mq::MessagePtr& msg, int)
{
LOG(info) << "Received: \"" << std::string(static_cast<char*>(msg->GetData()), msg->GetSize()) << "\"";

View File

@@ -8,20 +8,25 @@ if [[ $1 =~ ^[a-z]+$ ]]; then
transport=$1
fi
ex2config="@CMAKE_CURRENT_BINARY_DIR@/ex-1-n-1.json"
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan1="data1"
chan2="data2"
chan1Addr="/tmp/fmq_$session""_""$chan1""_""$transport"
chan2Addr="/tmp/fmq_$session""_""$chan2""_""$transport"
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; kill -TERM $PROCESSOR1_PID; kill -TERM $PROCESSOR2_PID; wait $SAMPLER_PID; wait $SINK_PID; wait $PROCESSOR1_PID; wait $PROCESSOR2_PID;' TERM
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; kill -TERM $PROCESSOR1_PID; kill -TERM $PROCESSOR2_PID; wait $SAMPLER_PID; wait $SINK_PID; wait $PROCESSOR1_PID; wait $PROCESSOR2_PID; rm $chan1Addr; rm $chan2Addr' TERM
SAMPLER="fairmq-ex-1-n-1-sampler"
SAMPLER+=" --id sampler1"
SAMPLER+=" --transport $transport"
SAMPLER+=" --verbosity veryhigh"
SAMPLER+=" --session $SESSION"
SAMPLER+=" --session $session"
SAMPLER+=" --severity debug"
SAMPLER+=" --shm-segment-size 100000000"
SAMPLER+=" --control static --color false"
SAMPLER+=" --max-iterations 2"
SAMPLER+=" --mq-config $ex2config"
SAMPLER+=" --channel-config name=$chan1,type=push,method=bind,address=ipc://$chan1Addr,rateLogging=0"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER &
SAMPLER_PID=$!
@@ -29,10 +34,12 @@ PROCESSOR1="fairmq-ex-1-n-1-processor"
PROCESSOR1+=" --id processor1"
PROCESSOR1+=" --transport $transport"
PROCESSOR1+=" --verbosity veryhigh"
PROCESSOR1+=" --session $SESSION"
PROCESSOR1+=" --session $session"
PROCESSOR1+=" --severity debug"
PROCESSOR1+=" --shm-segment-size 100000000"
PROCESSOR1+=" --control static --color false"
PROCESSOR1+=" --mq-config $ex2config"
PROCESSOR1+=" --config-key processor"
PROCESSOR1+=" --channel-config name=$chan1,type=pull,method=connect,address=ipc://$chan1Addr,rateLogging=0"
PROCESSOR1+=" name=$chan2,type=push,method=connect,address=ipc://$chan2Addr,rateLogging=0"
@CMAKE_CURRENT_BINARY_DIR@/$PROCESSOR1 &
PROCESSOR1_PID=$!
@@ -40,10 +47,12 @@ PROCESSOR2="fairmq-ex-1-n-1-processor"
PROCESSOR2+=" --id processor2"
PROCESSOR2+=" --transport $transport"
PROCESSOR2+=" --verbosity veryhigh"
PROCESSOR2+=" --session $SESSION"
PROCESSOR2+=" --session $session"
PROCESSOR2+=" --severity debug"
PROCESSOR2+=" --shm-segment-size 100000000"
PROCESSOR2+=" --control static --color false"
PROCESSOR2+=" --mq-config $ex2config"
PROCESSOR2+=" --config-key processor"
PROCESSOR2+=" --channel-config name=$chan1,type=pull,method=connect,address=ipc://$chan1Addr,rateLogging=0"
PROCESSOR2+=" name=$chan2,type=push,method=connect,address=ipc://$chan2Addr,rateLogging=0"
@CMAKE_CURRENT_BINARY_DIR@/$PROCESSOR2 &
PROCESSOR2_PID=$!
@@ -51,10 +60,12 @@ SINK="fairmq-ex-1-n-1-sink"
SINK+=" --id sink1"
SINK+=" --transport $transport"
SINK+=" --verbosity veryhigh"
SINK+=" --session $SESSION"
SINK+=" --session $session"
SINK+=" --severity debug"
SINK+=" --shm-segment-size 100000000"
SINK+=" --control static --color false"
SINK+=" --max-iterations 2"
SINK+=" --mq-config $ex2config"
SINK+=" --channel-config name=$chan2,type=pull,method=bind,address=ipc://$chan2Addr,rateLogging=0"
@CMAKE_CURRENT_BINARY_DIR@/$SINK &
SINK_PID=$!
@@ -69,3 +80,5 @@ kill -SIGINT $PROCESSOR2_PID
# wait for everything to finish
wait $PROCESSOR1_PID
wait $PROCESSOR2_PID
rm $chan1Addr; rm $chan2Addr

View File

@@ -15,16 +15,16 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-builtin-devices.sh.in
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-builtin-devices.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-builtin-devices.sh)
add_test(NAME Example.BuiltinDevices.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-builtin-devices.sh zeromq)
set_tests_properties(Example.BuiltinDevices.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
set_tests_properties(Example.BuiltinDevices.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
add_test(NAME Example.BuiltinDevices.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-builtin-devices.sh shmem)
set_tests_properties(Example.BuiltinDevices.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
set_tests_properties(Example.BuiltinDevices.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
add_test(NAME Example.BuiltinDevices.multipart.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-builtin-devices.sh zeromq true 2)
set_tests_properties(Example.BuiltinDevices.multipart.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
set_tests_properties(Example.BuiltinDevices.multipart.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
add_test(NAME Example.BuiltinDevices.multipart.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-builtin-devices.sh shmem true 2)
set_tests_properties(Example.BuiltinDevices.multipart.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
set_tests_properties(Example.BuiltinDevices.multipart.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Configured maximum number of iterations reached")
# install

View File

@@ -2,8 +2,6 @@
export FAIRMQ_PATH=@FAIRMQ_BIN_DIR@
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
transport="zeromq"
multipart="false"
numParts="1"
@@ -20,8 +18,22 @@ if [[ $3 =~ ^[0-9]+$ ]]; then
numParts=$3
fi
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan1="data1"
chan2="data2"
chan3="data3"
chan4="data4"
chan5="data5"
chan1Addr="/tmp/fmq_$session""_""$chan1""_""$transport"
chan2Addr1="/tmp/fmq_$session""_""$chan2""_1""_""$transport"
chan2Addr2="/tmp/fmq_$session""_""$chan2""_2""_""$transport"
chan3Addr1="/tmp/fmq_$session""_""$chan3""_1""_""$transport"
chan3Addr2="/tmp/fmq_$session""_""$chan3""_2""_""$transport"
chan4Addr="/tmp/fmq_$session""_""$chan4""_""$transport"
chan5Addr="/tmp/fmq_$session""_""$chan5""_""$transport"
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SPLITTER_PID; kill -TERM $PROXY1_PID; kill -TERM $PROXY2_PID; kill -TERM $MERGER_PID; kill -TERM $MULTIPLIER_PID; kill -TERM $SINK_PID;' TERM
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SPLITTER_PID; kill -TERM $PROXY1_PID; kill -TERM $PROXY2_PID; kill -TERM $MERGER_PID; kill -TERM $MULTIPLIER_PID; kill -TERM $SINK_PID; rm $chan1Addr; rm $chan2Addr1; rm $chan2Addr2; rm $chan3Addr1; rm $chan3Addr2; rm $chan4Addr; rm $chan5Addr' TERM
SAMPLER="fairmq-bsampler"
SAMPLER+=" --id bsampler1"
@@ -30,14 +42,15 @@ SAMPLER+=" --transport $transport"
SAMPLER+=" --color false"
SAMPLER+=" --control static"
SAMPLER+=" --verbosity veryhigh"
SAMPLER+=" --shm-segment-size 100000000"
SAMPLER+=" --severity debug"
SAMPLER+=" --msg-size 100000"
SAMPLER+=" --multipart $multipart"
SAMPLER+=" --num-parts $numParts"
SAMPLER+=" --msg-rate 1"
SAMPLER+=" --max-iterations 0"
SAMPLER+=" --out-channel data1"
SAMPLER+=" --channel-config name=data1,type=push,method=bind,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5555"
SAMPLER+=" --out-channel $chan1"
SAMPLER+=" --channel-config name=$chan1,type=push,method=bind,sndBufSize=50,rcvBufSize=50,address=ipc://$chan1Addr"
@FAIRMQ_BIN_DIR@/$SAMPLER &
SAMPLER_PID=$!
@@ -48,11 +61,12 @@ SPLITTER+=" --transport $transport"
SPLITTER+=" --color false"
SPLITTER+=" --control static"
SPLITTER+=" --verbosity veryhigh"
SPLITTER+=" --shm-segment-size 100000000"
SPLITTER+=" --multipart $multipart"
SPLITTER+=" --in-channel data1"
SPLITTER+=" --out-channel data2"
SPLITTER+=" --channel-config name=data1,type=pull,method=connect,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5555"
SPLITTER+=" name=data2,type=push,method=bind,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5556,address=tcp://localhost:5557"
SPLITTER+=" --in-channel $chan1"
SPLITTER+=" --out-channel $chan2"
SPLITTER+=" --channel-config name=$chan1,type=pull,method=connect,sndBufSize=50,rcvBufSize=50,address=ipc://$chan1Addr"
SPLITTER+=" name=$chan2,type=push,method=bind,sndBufSize=50,rcvBufSize=50,address=ipc://$chan2Addr1,address=ipc://$chan2Addr2"
@FAIRMQ_BIN_DIR@/$SPLITTER &
SPLITTER_PID=$!
@@ -63,11 +77,12 @@ PROXY1+=" --transport $transport"
PROXY1+=" --color false"
PROXY1+=" --control static"
PROXY1+=" --verbosity veryhigh"
PROXY1+=" --shm-segment-size 100000000"
PROXY1+=" --multipart $multipart"
PROXY1+=" --in-channel data2"
PROXY1+=" --out-channel data3"
PROXY1+=" --channel-config name=data2,type=pull,method=connect,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5556"
PROXY1+=" name=data3,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5558"
PROXY1+=" --in-channel $chan2"
PROXY1+=" --out-channel $chan3"
PROXY1+=" --channel-config name=$chan2,type=pull,method=connect,sndBufSize=50,rcvBufSize=50,address=ipc://$chan2Addr1"
PROXY1+=" name=$chan3,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=ipc://$chan3Addr1"
@FAIRMQ_BIN_DIR@/$PROXY1 &
PROXY1_PID=$!
@@ -78,11 +93,12 @@ PROXY2+=" --transport $transport"
PROXY2+=" --color false"
PROXY2+=" --control static"
PROXY2+=" --verbosity veryhigh"
PROXY2+=" --shm-segment-size 100000000"
PROXY2+=" --multipart $multipart"
PROXY2+=" --in-channel data2"
PROXY2+=" --out-channel data3"
PROXY2+=" --channel-config name=data2,type=pull,method=connect,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5557"
PROXY2+=" name=data3,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5559"
PROXY2+=" --in-channel $chan2"
PROXY2+=" --out-channel $chan3"
PROXY2+=" --channel-config name=$chan2,type=pull,method=connect,sndBufSize=50,rcvBufSize=50,address=ipc://$chan2Addr2"
PROXY2+=" name=$chan3,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=ipc://$chan3Addr2"
@FAIRMQ_BIN_DIR@/$PROXY2 &
PROXY2_PID=$!
@@ -93,11 +109,12 @@ MERGER+=" --transport $transport"
MERGER+=" --color false"
MERGER+=" --control static"
MERGER+=" --verbosity veryhigh"
MERGER+=" --shm-segment-size 100000000"
MERGER+=" --multipart $multipart"
MERGER+=" --in-channel data3"
MERGER+=" --out-channel data4"
MERGER+=" --channel-config name=data3,type=pull,method=bind,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5558,address=tcp://localhost:5559"
MERGER+=" name=data4,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5560"
MERGER+=" --in-channel $chan3"
MERGER+=" --out-channel $chan4"
MERGER+=" --channel-config name=$chan3,type=pull,method=bind,sndBufSize=50,rcvBufSize=50,address=ipc://$chan3Addr1,address=ipc://$chan3Addr2"
MERGER+=" name=$chan4,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=ipc://$chan4Addr"
@FAIRMQ_BIN_DIR@/$MERGER &
MERGER_PID=$!
@@ -108,11 +125,12 @@ MULTIPLIER+=" --transport $transport"
MULTIPLIER+=" --color false"
MULTIPLIER+=" --control static"
MULTIPLIER+=" --verbosity veryhigh"
MULTIPLIER+=" --shm-segment-size 100000000"
MULTIPLIER+=" --multipart $multipart"
MULTIPLIER+=" --in-channel data4"
MULTIPLIER+=" --out-channel data5"
MULTIPLIER+=" --channel-config name=data4,type=pull,method=bind,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5560"
MULTIPLIER+=" name=data5,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5561,address=tcp://localhost:5561"
MULTIPLIER+=" --in-channel $chan4"
MULTIPLIER+=" --out-channel $chan5"
MULTIPLIER+=" --channel-config name=$chan4,type=pull,method=bind,sndBufSize=50,rcvBufSize=50,address=ipc://$chan4Addr"
MULTIPLIER+=" name=$chan5,type=push,method=connect,sndBufSize=50,rcvBufSize=50,address=ipc://$chan5Addr,address=ipc://$chan5Addr"
@FAIRMQ_BIN_DIR@/$MULTIPLIER &
MULTIPLIER_PID=$!
@@ -126,8 +144,8 @@ SINK+=" --verbosity veryhigh"
SINK+=" --severity debug"
SINK+=" --multipart $multipart"
SINK+=" --max-iterations 2"
SINK+=" --in-channel data5"
SINK+=" --channel-config name=data5,type=pull,method=bind,sndBufSize=50,rcvBufSize=50,address=tcp://localhost:5561"
SINK+=" --in-channel $chan5"
SINK+=" --channel-config name=$chan5,type=pull,method=bind,sndBufSize=50,rcvBufSize=50,address=ipc://$chan5Addr"
@FAIRMQ_BIN_DIR@/$SINK &
SINK_PID=$!
@@ -146,3 +164,5 @@ wait $PROXY1_PID
wait $PROXY2_PID
wait $MERGER_PID
wait $MULTIPLIER_PID
rm $chan1Addr; rm $chan2Addr1; rm $chan2Addr2; rm $chan3Addr1; rm $chan3Addr2; rm $chan4Addr; rm $chan5Addr

View File

@@ -24,10 +24,10 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-copypush.sh.in ${CMAK
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-copypush.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-copypush.sh)
add_test(NAME Example.CopyPush.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-copypush.sh zeromq)
set_tests_properties(Example.CopyPush.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received message: ")
set_tests_properties(Example.CopyPush.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received message: ")
add_test(NAME Example.CopyPush.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-copypush.sh shmem)
set_tests_properties(Example.CopyPush.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received message: ")
set_tests_properties(Example.CopyPush.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received message: ")
# install

View File

@@ -27,10 +27,10 @@ struct Sampler : fair::mq::Device
{
// NewSimpleMessage creates a copy of the data and takes care of its destruction (after the transfer takes place).
// Should only be used for small data because of the cost of an additional copy
FairMQMessagePtr msg(NewSimpleMessage(fCounter++));
fair::mq::MessagePtr msg(NewSimpleMessage(fCounter++));
for (int i = 0; i < fNumDataChannels - 1; ++i) {
FairMQMessagePtr msgCopy(NewMessage());
fair::mq::MessagePtr msgCopy(NewMessage());
msgCopy->Copy(*msg);
Send(msgCopy, "data", i);
}

View File

@@ -27,7 +27,7 @@ struct Sink : fair::mq::Device
fMaxIterations = fConfig->GetProperty<uint64_t>("max-iterations");
}
bool HandleData(FairMQMessagePtr& msg, int)
bool HandleData(fair::mq::MessagePtr& msg, int)
{
LOG(info) << "Received message: \"" << *(static_cast<uint64_t*>(msg->GetData())) << "\"";

View File

@@ -8,19 +8,24 @@ if [[ $1 =~ ^[a-z]+$ ]]; then
transport=$1
fi
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan="data"
chanAddr1="/tmp/fmq_$session""_""$chan""_1""_""$transport"
chanAddr2="/tmp/fmq_$session""_""$chan""_2""_""$transport"
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK1_PID; kill -TERM $SINK2_PID; wait $SAMPLER_PID; wait $SINK1_PID; wait $SINK2_PID;' TERM
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK1_PID; kill -TERM $SINK2_PID; wait $SAMPLER_PID; wait $SINK1_PID; wait $SINK2_PID; rm $chanAddr1; rm $chanAddr2' TERM
SAMPLER="fairmq-ex-copypush-sampler"
SAMPLER+=" --id sampler1"
SAMPLER+=" --transport $transport"
SAMPLER+=" --verbosity veryhigh"
SAMPLER+=" --session $SESSION"
SAMPLER+=" --severity debug"
SAMPLER+=" --shm-segment-size 100000000"
SAMPLER+=" --session $session"
SAMPLER+=" --control static --color false"
SAMPLER+=" --max-iterations 1"
SAMPLER+=" --channel-config name=data,type=push,method=bind,rateLogging=0,address=tcp://*:5555,address=tcp://*:5556"
SAMPLER+=" --channel-config name=$chan,type=push,method=bind,rateLogging=0,address=ipc://$chanAddr1,address=ipc://$chanAddr2"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER &
SAMPLER_PID=$!
@@ -28,10 +33,12 @@ SINK1="fairmq-ex-copypush-sink"
SINK1+=" --id sink1"
SINK1+=" --transport $transport"
SINK1+=" --verbosity veryhigh"
SINK1+=" --session $SESSION"
SINK1+=" --severity debug"
SINK1+=" --shm-segment-size 100000000"
SINK1+=" --session $session"
SINK1+=" --control static --color false"
SINK1+=" --max-iterations 1"
SINK1+=" --channel-config name=data,type=pull,method=connect,rateLogging=0,address=tcp://localhost:5555"
SINK1+=" --channel-config name=$chan,type=pull,method=connect,rateLogging=0,address=ipc://$chanAddr1"
@CMAKE_CURRENT_BINARY_DIR@/$SINK1 &
SINK1_PID=$!
@@ -39,10 +46,12 @@ SINK2="fairmq-ex-copypush-sink"
SINK2+=" --id sink2"
SINK2+=" --transport $transport"
SINK2+=" --verbosity veryhigh"
SINK2+=" --session $SESSION"
SINK2+=" --severity debug"
SINK2+=" --shm-segment-size 100000000"
SINK2+=" --session $session"
SINK2+=" --control static --color false"
SINK2+=" --max-iterations 1"
SINK2+=" --channel-config name=data,type=pull,method=connect,rateLogging=0,address=tcp://localhost:5556"
SINK2+=" --channel-config name=$chan,type=pull,method=connect,rateLogging=0,address=ipc://$chanAddr2"
@CMAKE_CURRENT_BINARY_DIR@/$SINK2 &
SINK2_PID=$!
@@ -50,3 +59,5 @@ SINK2_PID=$!
wait $SAMPLER_PID
wait $SINK1_PID
wait $SINK2_PID
rm $chanAddr1; rm $chanAddr2

View File

@@ -20,7 +20,7 @@ struct Processor : fair::mq::Device
OnData("data1", &Processor::HandleData);
}
bool HandleData(FairMQMessagePtr& msg, int)
bool HandleData(fair::mq::MessagePtr& msg, int)
{
LOG(info) << "Received data, processing...";
@@ -32,7 +32,7 @@ struct Processor : fair::mq::Device
// its size,
// custom deletion function (called when transfer is done),
// and pointer to the object managing the data buffer
FairMQMessagePtr msg2(NewMessage(const_cast<char*>(text->c_str()),
fair::mq::MessagePtr msg2(NewMessage(const_cast<char*>(text->c_str()),
text->length(),
[](void* /*data*/, void* object) { delete static_cast<std::string*>(object); },
text));

View File

@@ -24,7 +24,7 @@ struct Sampler : fair::mq::Device
{
// NewSimpleMessage creates a copy of the data and takes care of its destruction (after the transfer takes place).
// Should only be used for small data because of the cost of an additional copy
FairMQMessagePtr msg(NewSimpleMessage("Data"));
fair::mq::MessagePtr msg(NewSimpleMessage("Data"));
LOG(info) << "Sending \"Data\"";

View File

@@ -25,7 +25,7 @@ struct Sink : fair::mq::Device
fIterations = fConfig->GetValue<uint64_t>("iterations");
}
bool HandleData(FairMQMessagePtr& msg, int)
bool HandleData(fair::mq::MessagePtr& msg, int)
{
LOG(info) << "Received: \"" << std::string(static_cast<char*>(msg->GetData()), msg->GetSize()) << "\"";

View File

@@ -23,14 +23,14 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-multipart.sh.in ${CMA
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-multipart.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multipart.sh)
add_test(NAME Example.Multipart.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multipart.sh zeromq)
set_tests_properties(Example.Multipart.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received message with 7 parts")
set_tests_properties(Example.Multipart.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received message with 7 parts")
add_test(NAME Example.Multipart.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multipart.sh shmem)
set_tests_properties(Example.Multipart.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received message with 7 parts")
set_tests_properties(Example.Multipart.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received message with 7 parts")
if(BUILD_OFI_TRANSPORT)
add_test(NAME Example.Multipart.ofi COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multipart.sh ofi)
set_tests_properties(Example.Multipart.ofi PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received message with 7 parts")
set_tests_properties(Example.Multipart.ofi PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received message with 7 parts")
endif()
# install

View File

@@ -5,7 +5,7 @@ A topology of two devices - Sampler and Sink, communicating with PUSH-PULL patte
The Sampler sends a multipart message to the Sink, consisting of two message parts - header and body.
Each message part is a regular FairMQMessage. To combine them into a multi-part message use `FairMQParts`. Add messages to `FairMQParts` with `AddPart` method.
Each message part is a regular fair::mq::Message. To combine them into a multi-part message use `fair::mq::Parts`. Add messages to `fair::mq::Parts` with `AddPart` method.
All parts are guaranteed to be delivered together. The Receive call in the sink will recive the entire parts structure.

View File

@@ -35,15 +35,15 @@ struct Sampler : fair::mq::Device
}
LOG(info) << "Sending header with stopFlag: " << header.stopFlag;
FairMQParts parts;
fair::mq::Parts parts;
// NewSimpleMessage creates a copy of the data and takes care of its destruction (after the transfer takes place).
// Should only be used for small data because of the cost of an additional copy
parts.AddPart(NewSimpleMessage(header));
parts.AddPart(NewMessage(1000));
// create more data parts, testing the FairMQParts in-place constructor
FairMQParts auxData{ NewMessage(500), NewMessage(600), NewMessage(700) };
// create more data parts, testing the fair::mq::Parts in-place constructor
fair::mq::Parts auxData{ NewMessage(500), NewMessage(600), NewMessage(700) };
assert(auxData.Size() == 3);
parts.AddPart(std::move(auxData));
assert(auxData.Size() == 0);

View File

@@ -20,7 +20,7 @@ struct Sink : fair::mq::Device
OnData("data", &Sink::HandleData);
}
bool HandleData(FairMQParts& parts, int)
bool HandleData(fair::mq::Parts& parts, int)
{
LOG(info) << "Received message with " << parts.Size() << " parts";

View File

@@ -8,19 +8,28 @@ if [[ $1 =~ ^[a-z]+$ ]]; then
transport=$1
fi
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan="data"
chanAddr=""
chanIpcFile="/tmp/fmq_$session""_""$chan""_""$transport"
if [ $transport = "ofi" ]; then
chanAddr="tcp://127.0.0.1:5656"
else
chanAddr="ipc://""$chanIpcFile"
fi
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; wait $SAMPLER_PID; wait $SINK_PID;' TERM
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; wait $SAMPLER_PID; wait $SINK_PID; rm $chanIpcFile' TERM
SAMPLER="fairmq-ex-multipart-sampler"
SAMPLER+=" --id sampler1"
SAMPLER+=" --transport $transport"
SAMPLER+=" --verbosity veryhigh"
SAMPLER+=" --session $SESSION"
SAMPLER+=" --session $session"
SAMPLER+=" --shm-segment-size 100000000"
SAMPLER+=" --max-iterations 1"
SAMPLER+=" --control static --color false"
SAMPLER+=" --channel-config name=data,type=pair,method=connect,rateLogging=0,address=tcp://127.0.0.1:5555,linger=1000"
SAMPLER+=" --channel-config name=$chan,type=pair,method=connect,rateLogging=0,address=$chanAddr,linger=1000"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER &
SAMPLER_PID=$!
@@ -28,11 +37,14 @@ SINK="fairmq-ex-multipart-sink"
SINK+=" --id sink1"
SINK+=" --transport $transport"
SINK+=" --verbosity veryhigh"
SINK+=" --session $SESSION"
SINK+=" --session $session"
SINK+=" --shm-segment-size 100000000"
SINK+=" --control static --color false"
SINK+=" --channel-config name=data,type=pair,method=bind,rateLogging=0,address=tcp://127.0.0.1:5555"
SINK+=" --channel-config name=$chan,type=pair,method=bind,rateLogging=0,address=$chanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$SINK &
SINK_PID=$!
wait $SAMPLER_PID
wait $SINK_PID
rm $chanIpcFile

View File

@@ -26,7 +26,7 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-multiple-channels.sh.
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-multiple-channels.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multiple-channels.sh)
add_test(NAME Example.MultipleChannels.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multiple-channels.sh zeromq)
set_tests_properties(Example.MultipleChannels.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received messages from both sources.")
set_tests_properties(Example.MultipleChannels.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received messages from both sources.")
# install

View File

@@ -5,4 +5,4 @@ This example demonstrates how to work with multiple channels and multiplex betwe
A topology of three devices - **Sampler**, **Sink** and **Broadcaster**. The Sampler sends data to the Sink via the PUSH-PULL pattern. The Broadcaster device sends a message to both Sampler and Sink containing a string "OK" every second. The Broadcaster sends the message via PUB pattern. Both Sampler and Sink, besides doing their PUSH-PULL job, listen via SUB to the Broadcaster.
The multiplexing between their data channels and the broadcast channels happens with `FairMQPoller`.
The multiplexing between their data channels and the broadcast channels happens with `fair::mq::Poller`.

View File

@@ -22,7 +22,7 @@ struct Broadcaster : fair::mq::Device
// NewSimpleMessage creates a copy of the data and takes care of its destruction (after the transfer takes place).
// Should only be used for small data because of the cost of an additional copy
FairMQMessagePtr msg(NewSimpleMessage("OK"));
fair::mq::MessagePtr msg(NewSimpleMessage("OK"));
LOG(info) << "Sending OK";

View File

@@ -7,7 +7,7 @@
********************************************************************************/
#include <fairmq/Device.h>
#include <FairMQPoller.h>
#include <fairmq/Poller.h>
#include <fairmq/runDevice.h>
#include <chrono>
@@ -26,13 +26,13 @@ struct Sampler : fair::mq::Device
void Run() override
{
FairMQPollerPtr poller(NewPoller("data", "broadcast"));
fair::mq::PollerPtr poller(NewPoller("data", "broadcast"));
while (!NewStatePending()) {
poller->Poll(100);
if (poller->CheckInput("broadcast", 0)) {
FairMQMessagePtr msg(NewMessage());
fair::mq::MessagePtr msg(NewMessage());
if (Receive(msg, "broadcast") > 0) {
LOG(info) << "Received broadcast: \"" << std::string(static_cast<char*>(msg->GetData()), msg->GetSize()) << "\"";
@@ -42,7 +42,7 @@ struct Sampler : fair::mq::Device
if (poller->CheckOutput("data", 0)) {
std::this_thread::sleep_for(std::chrono::seconds(1));
FairMQMessagePtr msg(NewSimpleMessage(fText));
fair::mq::MessagePtr msg(NewSimpleMessage(fText));
if (Send(msg, "data") > 0) {
LOG(info) << "Sent \"" << fText << "\"";

View File

@@ -27,7 +27,7 @@ struct Sink : fair::mq::Device
fMaxIterations = fConfig->GetProperty<uint64_t>("max-iterations");
}
bool HandleBroadcast(FairMQMessagePtr& msg, int /*index*/)
bool HandleBroadcast(fair::mq::MessagePtr& msg, int /*index*/)
{
LOG(info) << "Received broadcast: \"" << std::string(static_cast<char*>(msg->GetData()), msg->GetSize()) << "\"";
fReceivedBroadcast = true;
@@ -35,7 +35,7 @@ struct Sink : fair::mq::Device
return CheckIterations();
}
bool HandleData(FairMQMessagePtr& msg, int /*index*/)
bool HandleData(fair::mq::MessagePtr& msg, int /*index*/)
{
LOG(info) << "Received message: \"" << std::string(static_cast<char*>(msg->GetData()), msg->GetSize()) << "\"";
fReceivedData = true;

View File

@@ -8,18 +8,25 @@ if [[ $1 =~ ^[a-z]+$ ]]; then
transport=$1
fi
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; kill -TERM $BROADCASTER_PID; wait $SAMPLER_PID; wait $SINK_PID; wait $BROADCASTER_PID;' TERM
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
dataChan="data"
broadcastChan="broadcast"
dataChanAddr="/tmp/fmq_$session""_""$dataChan""_""$transport"
broadcastChanAddr="/tmp/fmq_$session""_""$broadcastChan""_""$transport"
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; kill -TERM $BROADCASTER_PID; wait $SAMPLER_PID; wait $SINK_PID; wait $BROADCASTER_PID; rm $dataChanAddr; rm $broadcastChanAddr' TERM
SINK="fairmq-ex-multiple-channels-sink"
SINK+=" --id sink1"
SINK+=" --session $session"
SINK+=" --transport $transport"
SINK+=" --verbosity veryhigh"
SINK+=" --verbosity veryhigh --severity debug"
SINK+=" --shm-segment-size 100000000"
SINK+=" --max-iterations 1"
SINK+=" --control static --color false"
SINK+=" --channel-config name=data,type=pull,method=connect,rateLogging=0,address=tcp://localhost:5555"
SINK+=" name=broadcast,type=sub,method=connect,rateLogging=0,address=tcp://localhost:5005"
SINK+=" --channel-config name=$dataChan,type=pull,method=connect,rateLogging=0,address=ipc://$dataChanAddr"
SINK+=" name=$broadcastChan,type=sub,method=connect,rateLogging=0,address=ipc://$broadcastChanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$SINK &
SINK_PID=$!
@@ -27,21 +34,25 @@ sleep 1
SAMPLER="fairmq-ex-multiple-channels-sampler"
SAMPLER+=" --id sampler1"
SAMPLER+=" --session $session"
SAMPLER+=" --transport $transport"
SAMPLER+=" --verbosity veryhigh"
SAMPLER+=" --verbosity veryhigh --severity debug"
SAMPLER+=" --shm-segment-size 100000000"
SAMPLER+=" --max-iterations 1"
SAMPLER+=" --control static --color false"
SAMPLER+=" --channel-config name=data,type=push,method=bind,rateLogging=0,address=tcp://*:5555"
SAMPLER+=" name=broadcast,type=sub,method=connect,rateLogging=0,address=tcp://localhost:5005"
SAMPLER+=" --channel-config name=$dataChan,type=push,method=bind,rateLogging=0,address=ipc://$dataChanAddr"
SAMPLER+=" name=$broadcastChan,type=sub,method=connect,rateLogging=0,address=ipc://$broadcastChanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER &
SAMPLER_PID=$!
BROADCASTER="fairmq-ex-multiple-channels-broadcaster"
BROADCASTER+=" --id broadcaster1"
BROADCASTER+=" --session $session"
BROADCASTER+=" --transport $transport"
BROADCASTER+=" --verbosity veryhigh"
BROADCASTER+=" --verbosity veryhigh --severity debug"
BROADCASTER+=" --shm-segment-size 100000000"
BROADCASTER+=" --control static --color false"
BROADCASTER+=" --channel-config name=broadcast,type=pub,method=bind,rateLogging=0,address=tcp://*:5005"
BROADCASTER+=" --channel-config name=$broadcastChan,type=pub,method=bind,rateLogging=0,address=ipc://$broadcastChanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$BROADCASTER &
BROADCASTER_PID=$!
@@ -53,3 +64,5 @@ kill -SIGINT $BROADCASTER_PID
# wait for broadcaster to finish
wait $BROADCASTER_PID
rm $dataChanAddr; rm $broadcastChanAddr

View File

@@ -25,7 +25,7 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-multiple-transports.s
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-multiple-transports.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multiple-transports.sh)
add_test(NAME Example.MultipleTransports COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-multiple-transports.sh)
set_tests_properties(Example.MultipleTransports PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received messages from both sources.")
set_tests_properties(Example.MultipleTransports PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received messages from both sources.")
# install

View File

@@ -28,7 +28,7 @@ struct Sampler1 : fair::mq::Device
bool ConditionalRun() override
{
// Creates a message using the transport of channel data1
FairMQMessagePtr msg(NewMessageFor("data1", 0, 1000000));
fair::mq::MessagePtr msg(NewMessageFor("data1", 0, 1000000));
// in case of error or transfer interruption, return false to go to IDLE state
// successfull transfer will return number of bytes transfered (can be 0 if sending an empty message).
@@ -54,7 +54,7 @@ struct Sampler1 : fair::mq::Device
uint64_t numAcks = 0;
while (!NewStatePending()) {
FairMQMessagePtr ack(NewMessageFor("ack", 0));
fair::mq::MessagePtr ack(NewMessageFor("ack", 0));
if (Receive(ack, "ack") < 0) {
break;
}

View File

@@ -22,7 +22,7 @@ struct Sampler2 : fair::mq::Device
bool ConditionalRun() override
{
FairMQMessagePtr msg(NewMessage(1000));
fair::mq::MessagePtr msg(NewMessage(1000));
// in case of error or transfer interruption, return false to go to IDLE state
// successfull transfer will return number of bytes transfered (can be 0 if sending an empty message).

View File

@@ -26,11 +26,11 @@ struct Sink : fair::mq::Device
}
// handler is called whenever a message arrives on "data", with a reference to the message and a sub-channel index (here 0)
bool HandleData1(FairMQMessagePtr& /*msg*/, int /*index*/)
bool HandleData1(fair::mq::MessagePtr& /*msg*/, int /*index*/)
{
fNumIterations1++;
// Creates a message using the transport of channel ack
FairMQMessagePtr ack(NewMessageFor("ack", 0));
fair::mq::MessagePtr ack(NewMessageFor("ack", 0));
if (Send(ack, "ack") < 0) {
return false;
}
@@ -40,7 +40,7 @@ struct Sink : fair::mq::Device
}
// handler is called whenever a message arrives on "data", with a reference to the message and a sub-channel index (here 0)
bool HandleData2(FairMQMessagePtr& /*msg*/, int /*index*/)
bool HandleData2(fair::mq::MessagePtr& /*msg*/, int /*index*/)
{
fNumIterations2++;
// return true if want to be called again (otherwise go to IDLE state)

View File

@@ -2,46 +2,57 @@
export FAIRMQ_PATH=@FAIRMQ_BIN_DIR@
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan1="data1"
chan2="data2"
ackChan="ack"
chan1Addr="/tmp/fmq_$session""_""$chan1"
chan2Addr="/tmp/fmq_$session""_""$chan2"
ackChanAddr="/tmp/fmq_$session""_""$ackChan"
trap 'kill -TERM $SAMPLER1_PID; kill -TERM $SAMPLER2_PID; kill -TERM $SINK_PID; wait $SAMPLER1_PID; wait $SAMPLER2_PID; wait $SINK_PID; @CMAKE_BINARY_DIR@/fairmq/fairmq-shmmonitor --cleanup --session $SESSION;' TERM
trap 'kill -TERM $SAMPLER1_PID; kill -TERM $SAMPLER2_PID; kill -TERM $SINK_PID; wait $SAMPLER1_PID; wait $SAMPLER2_PID; wait $SINK_PID; @CMAKE_BINARY_DIR@/fairmq/fairmq-shmmonitor --cleanup --session $SESSION; rm $chan1Addr; rm $chan2Addr; rm $ackChanAddr' TERM
SINK="fairmq-ex-multiple-transports-sink"
SINK+=" --id sink1"
SINK+=" --verbosity veryhigh"
SINK+=" --session $SESSION"
SINK+=" --verbosity veryhigh --severity debug"
SINK+=" --shm-segment-size 100000000"
SINK+=" --session $session"
SINK+=" --max-iterations 1"
SINK+=" --control static --color false"
SINK+=" --transport shmem"
SINK+=" --channel-config name=data1,type=pull,method=connect,address=tcp://127.0.0.1:5555"
SINK+=" name=data2,type=pull,method=connect,address=tcp://127.0.0.1:5556,transport=zeromq"
SINK+=" name=ack,type=pub,method=connect,address=tcp://127.0.0.1:5557,transport=zeromq"
SINK+=" --channel-config name=$chan1,type=pull,method=connect,address=ipc://$chan1Addr"
SINK+=" name=$chan2,type=pull,method=connect,address=ipc://$chan2Addr,transport=zeromq"
SINK+=" name=$ackChan,type=pub,method=connect,address=ipc://$ackChanAddr,transport=zeromq"
@CMAKE_CURRENT_BINARY_DIR@/$SINK &
SINK_PID=$!
SAMPLER1="fairmq-ex-multiple-transports-sampler1"
SAMPLER1+=" --id sampler1"
SAMPLER1+=" --session $SESSION"
SAMPLER1+=" --verbosity veryhigh"
SAMPLER1+=" --session $session"
SAMPLER1+=" --verbosity veryhigh --severity debug"
SAMPLER1+=" --shm-segment-size 100000000"
SAMPLER1+=" --max-iterations 1"
SAMPLER1+=" --control static --color false"
SAMPLER1+=" --transport shmem"
SAMPLER1+=" --channel-config name=data1,type=push,method=bind,address=tcp://127.0.0.1:5555"
SAMPLER1+=" name=ack,type=sub,method=bind,address=tcp://127.0.0.1:5557,transport=zeromq"
SAMPLER1+=" --channel-config name=$chan1,type=push,method=bind,address=ipc://$chan1Addr"
SAMPLER1+=" name=$ackChan,type=sub,method=bind,address=ipc://$ackChanAddr,transport=zeromq"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER1 &
SAMPLER1_PID=$!
SAMPLER2="fairmq-ex-multiple-transports-sampler2"
SAMPLER2+=" --id sampler2"
SAMPLER2+=" --session $SESSION"
SAMPLER2+=" --verbosity veryhigh"
SAMPLER2+=" --session $session"
SAMPLER2+=" --verbosity veryhigh --severity debug"
SAMPLER2+=" --shm-segment-size 100000000"
SAMPLER2+=" --max-iterations 1"
SAMPLER2+=" --control static --color false"
SAMPLER2+=" --transport zeromq"
SAMPLER2+=" --channel-config name=data2,type=push,method=bind,address=tcp://127.0.0.1:5556"
SAMPLER2+=" --channel-config name=$chan2,type=push,method=bind,address=ipc://$chan2Addr"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER2 &
SAMPLER2_PID=$!
wait $SAMPLER1_PID
wait $SAMPLER2_PID
wait $SINK_PID
rm $chan1Addr; rm $chan2Addr; rm $ackChanAddr

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2020 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2020-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -24,7 +24,7 @@ namespace bpo = boost::program_options;
struct TFBuffer
{
FairMQParts parts;
fair::mq::Parts parts;
chrono::steady_clock::time_point start;
chrono::steady_clock::time_point end;
};
@@ -43,7 +43,7 @@ struct Receiver : fair::mq::Device
fMaxTimeframes = GetConfig()->GetValue<int>("max-timeframes");
}
bool HandleData(FairMQParts& parts, int /* index */)
bool HandleData(fair::mq::Parts& parts, int /* index */)
{
Header& h = *(static_cast<Header*>(parts.At(0)->GetData()));
// LOG(info) << "Received sub-time frame #" << h.id << " from Sender" << h.senderIndex;
@@ -93,7 +93,7 @@ struct Receiver : fair::mq::Device
unordered_map<uint16_t, TFBuffer> fBuffer;
unordered_set<uint16_t> fDiscardedSet;
int fNumSenders = 0;
unsigned int fNumSenders = 0;
int fBufferTimeoutInMs = 5000;
int fMaxTimeframes = 0;
int fTimeframeCounter = 0;
@@ -107,7 +107,7 @@ void addCustomOptions(bpo::options_description& options)
("max-timeframes", bpo::value<int>()->default_value(0), "Maximum number of timeframes to receive (0 - unlimited)");
}
std::unique_ptr<fair::mq::Device> getDevice(FairMQProgOptions& /* config */)
std::unique_ptr<fair::mq::Device> getDevice(fair::mq::ProgOptions& /* config */)
{
return std::make_unique<Receiver>();
}

View File

@@ -28,11 +28,11 @@ struct Sender : fair::mq::Device
void Run() override
{
FairMQChannel& dataInChannel = GetChannel("sync", 0);
fair::mq::Channel& dataInChannel = GetChannel("sync", 0);
while (!NewStatePending()) {
Header h;
FairMQMessagePtr id(NewMessage());
fair::mq::MessagePtr id(NewMessage());
if (dataInChannel.Receive(id) > 0) {
h.id = *(static_cast<uint16_t*>(id->GetData()));
h.senderIndex = fIndex;
@@ -40,7 +40,7 @@ struct Sender : fair::mq::Device
continue;
}
FairMQParts parts;
fair::mq::Parts parts;
parts.AddPart(NewSimpleMessage(h));
parts.AddPart(NewMessage(fSubtimeframeSize));
@@ -66,7 +66,7 @@ void addCustomOptions(bpo::options_description& options)
("subtimeframe-size", bpo::value<int>()->default_value(1000), "Subtimeframe size in bytes")
("num-receivers", bpo::value<int>()->required(), "Number of EPNs");
}
std::unique_ptr<fair::mq::Device> getDevice(FairMQProgOptions& /* config */)
std::unique_ptr<fair::mq::Device> getDevice(fair::mq::ProgOptions& /* config */)
{
return std::make_unique<Sender>();
}

View File

@@ -19,7 +19,7 @@ struct Synchronizer : fair::mq::Device
{
bool ConditionalRun() override
{
FairMQMessagePtr msg(NewSimpleMessage(fTimeframeId));
fair::mq::MessagePtr msg(NewSimpleMessage(fTimeframeId));
if (Send(msg, "sync") > 0) {
if (++fTimeframeId == UINT16_MAX - 1) {
@@ -37,7 +37,7 @@ struct Synchronizer : fair::mq::Device
};
void addCustomOptions(bpo::options_description& /* options */) {}
std::unique_ptr<fair::mq::Device> getDevice(FairMQProgOptions& /* config */)
std::unique_ptr<fair::mq::Device> getDevice(fair::mq::ProgOptions& /* config */)
{
return std::make_unique<Synchronizer>();
}

View File

@@ -30,10 +30,10 @@ struct QCDispatcher : fair::mq::Device
});
}
bool HandleData(FairMQMessagePtr& msg, int)
bool HandleData(fair::mq::MessagePtr& msg, int)
{
if (fDoQC.load() == true) {
FairMQMessagePtr msgCopy(NewMessage());
fair::mq::MessagePtr msgCopy(NewMessage());
msgCopy->Copy(*msg);
if (Send(msg, "qc") < 0) {
return false;

View File

@@ -9,12 +9,12 @@
#include <fairmq/Device.h>
#include <fairmq/runDevice.h>
class QCTask : public FairMQDevice
class QCTask : public fair::mq::Device
{
public:
QCTask()
{
OnData("qc", [](FairMQMessagePtr& /*msg*/, int) {
OnData("qc", [](fair::mq::MessagePtr& /*msg*/, int) {
LOG(info) << "received data";
return false;
});

View File

@@ -16,7 +16,7 @@ struct Sampler : fair::mq::Device
{
bool ConditionalRun() override
{
FairMQMessagePtr msg(NewMessage(1000));
fair::mq::MessagePtr msg(NewMessage(1000));
if (Send(msg, "data1") < 0) {
return false;

View File

@@ -14,7 +14,7 @@
struct Sink : fair::mq::Device
{
Sink() { OnData("data2", &Sink::HandleData); }
bool HandleData(FairMQMessagePtr& /*msg*/, int /*index*/) { return true; }
bool HandleData(fair::mq::MessagePtr& /*msg*/, int /*index*/) { return true; }
};
namespace bpo = boost::program_options;

View File

@@ -13,7 +13,7 @@
namespace bpo = boost::program_options;
class Builder : public FairMQDevice
class Builder : public fair::mq::Device
{
public:
Builder() = default;
@@ -24,7 +24,7 @@ class Builder : public FairMQDevice
OnData("rb", &Builder::HandleData);
}
bool HandleData(FairMQMessagePtr& msg, int /*index*/)
bool HandleData(fair::mq::MessagePtr& msg, int /*index*/)
{
if (Send(msg, fOutputChannelName) < 0) {
return false;

View File

@@ -18,9 +18,9 @@ struct Processor : fair::mq::Device
OnData("bp", &Processor::HandleData);
}
bool HandleData(FairMQMessagePtr& msg, int /*index*/)
bool HandleData(fair::mq::MessagePtr& msg, int /*index*/)
{
FairMQMessagePtr msg2(NewMessageFor("ps", 0, msg->GetSize()));
fair::mq::MessagePtr msg2(NewMessageFor("ps", 0, msg->GetSize()));
if (Send(msg2, "ps") < 0) {
return false;
}

View File

@@ -22,7 +22,7 @@ struct Readout : fair::mq::Device
fMsgSize = fConfig->GetProperty<int>("msg-size");
fMaxIterations = fConfig->GetProperty<uint64_t>("max-iterations");
fRegion = FairMQUnmanagedRegionPtr(NewUnmanagedRegionFor("rb",
fRegion = fair::mq::UnmanagedRegionPtr(NewUnmanagedRegionFor("rb",
0,
10000000,
[this](const std::vector<fair::mq::RegionBlock>& blocks) { // callback to be called when message buffers no longer needed by transport
@@ -36,7 +36,7 @@ struct Readout : fair::mq::Device
bool ConditionalRun() override
{
FairMQMessagePtr msg(NewMessageFor("rb", // channel
fair::mq::MessagePtr msg(NewMessageFor("rb", // channel
0, // sub-channel
fRegion, // region
fRegion->GetData(), // ptr within region
@@ -71,7 +71,7 @@ struct Readout : fair::mq::Device
int fMsgSize = 10000;
uint64_t fMaxIterations = 0;
uint64_t fNumIterations = 0;
FairMQUnmanagedRegionPtr fRegion = nullptr;
fair::mq::UnmanagedRegionPtr fRegion = nullptr;
std::atomic<uint64_t> fNumUnackedMsgs = 0;
};

View File

@@ -21,7 +21,7 @@ struct Sender : fair::mq::Device
OnData(fInputChannelName, &Sender::HandleData);
}
bool HandleData(FairMQMessagePtr& msg, int /*index*/)
bool HandleData(fair::mq::MessagePtr& msg, int /*index*/)
{
if (Send(msg, "sr") < 0) {
return false;

View File

@@ -26,10 +26,10 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-region.sh.in ${CMAKE_
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-region.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-region.sh)
add_test(NAME Example.Region.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-region.sh zeromq)
set_tests_properties(Example.Region.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received [0-9*] acks")
set_tests_properties(Example.Region.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received [0-9*] acks")
add_test(NAME Example.Region.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-region.sh shmem)
set_tests_properties(Example.Region.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received [0-9*] acks")
set_tests_properties(Example.Region.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received [0-9*] acks")
# install

View File

@@ -14,13 +14,18 @@
#include <fairlogger/Logger.h>
#include <boost/algorithm/string.hpp>
#include <boost/program_options.hpp>
#include <csignal>
#include <chrono>
#include <map>
#include <string>
#include <thread>
using namespace std;
using namespace boost::program_options;
namespace
{
@@ -32,19 +37,69 @@ void signalHandler(int /* signal */)
gStopping = 1;
}
struct ShmRemover
struct ShmManager
{
ShmRemover(std::string _shmId) : shmId(std::move(_shmId)) {}
~ShmRemover()
ShmManager(uint64_t _shmId, const vector<string>& _segments, const vector<string>& _regions)
: shmId(fair::mq::shmem::makeShmIdStr(_shmId))
{
// This will clean all segments, regions and any other shmem objects belonging to this shmId
for (const auto& s : _segments) {
vector<string> segmentConf;
boost::algorithm::split(segmentConf, s, boost::algorithm::is_any_of(","));
if (segmentConf.size() != 2) {
LOG(error) << "incorrect format for --segments. Expecting pairs of <id>,<size>.";
fair::mq::shmem::Monitor::Cleanup(fair::mq::shmem::ShmId{shmId});
throw runtime_error("incorrect format for --segments. Expecting pairs of <id>,<size>.");
}
uint16_t id = stoi(segmentConf.at(0));
uint64_t size = stoull(segmentConf.at(1));
auto ret = segments.emplace(id, fair::mq::shmem::Segment(shmId, id, size, fair::mq::shmem::rbTreeBestFit));
fair::mq::shmem::Segment& segment = ret.first->second;
LOG(info) << "Created segment " << id << " of size " << segment.GetSize() << ", starting at " << segment.GetData() << ". Locking...";
segment.Lock();
LOG(info) << "Done.";
LOG(info) << "Zeroing...";
segment.Zero();
LOG(info) << "Done.";
}
for (const auto& r : _regions) {
vector<string> regionConf;
boost::algorithm::split(regionConf, r, boost::algorithm::is_any_of(","));
if (regionConf.size() != 2) {
LOG(error) << "incorrect format for --regions. Expecting pairs of <id>,<size>.";
fair::mq::shmem::Monitor::Cleanup(fair::mq::shmem::ShmId{shmId});
throw runtime_error("incorrect format for --regions. Expecting pairs of <id>,<size>.");
}
uint16_t id = stoi(regionConf.at(0));
uint64_t size = stoull(regionConf.at(1));
auto ret = regions.emplace(id, make_unique<fair::mq::shmem::UnmanagedRegion>(shmId, id, size));
fair::mq::shmem::UnmanagedRegion& region = *(ret.first->second);
LOG(info) << "Created unamanged region " << id << " of size " << region.GetSize() << ", starting at " << region.GetData() << ". Locking...";
region.Lock();
LOG(info) << "Done.";
LOG(info) << "Zeroing...";
region.Zero();
LOG(info) << "Done.";
}
}
void ResetContent()
{
fair::mq::shmem::Monitor::ResetContent(fair::mq::shmem::ShmId{shmId});
}
~ShmManager()
{
// clean all segments, regions and any other shmem objects belonging to this shmId
fair::mq::shmem::Monitor::Cleanup(fair::mq::shmem::ShmId{shmId});
}
std::string shmId;
map<uint16_t, fair::mq::shmem::Segment> segments;
map<uint16_t, unique_ptr<fair::mq::shmem::UnmanagedRegion>> regions;
};
int main(int /* argc */, char** /* argv */)
int main(int argc, char** argv)
{
fair::Logger::SetConsoleColor(true);
@@ -52,47 +107,28 @@ int main(int /* argc */, char** /* argv */)
signal(SIGTERM, signalHandler);
try {
const string session = "default"; // to_string(fair::mq::tools::UuidHash());
// generate shmId out of session id + user id (geteuid).
const string shmId = fair::mq::shmem::makeShmIdStr(session);
uint64_t shmId = 0;
vector<string> segments;
vector<string> regions;
const uint16_t s1id = 0;
const uint64_t s1size = 100000000;
const uint16_t s2id = 1;
const uint64_t s2size = 200000000;
options_description desc("Options");
desc.add_options()
("shmid", value<uint64_t>(&shmId)->required(), "Shm id")
("segments", value<vector<string>>(&segments)->multitoken()->composing(), "Segments, as <id>,<size> <id>,<size> <id>,<size> ...")
("regions", value<vector<string>>(&regions)->multitoken()->composing(), "Regions, as <id>,<size> <id>,<size> <id>,<size> ...")
("help,h", "Print help");
const uint16_t r1id = 0;
const uint64_t r1size = 100000000;
const uint16_t r2id = 1;
const uint64_t r2size = 200000000;
variables_map vm;
store(parse_command_line(argc, argv, desc), vm);
// cleanup when done
ShmRemover shmRemover(shmId);
if (vm.count("help")) {
LOG(info) << "ShmManager" << "\n" << desc;
return 0;
}
// managed segments
fair::mq::shmem::Segment segment1(shmId, s1id, s1size, fair::mq::shmem::rbTreeBestFit);
segment1.Lock();
segment1.Zero();
LOG(info) << "Created segment " << s1id << " of size " << segment1.GetSize() << " starting at " << segment1.GetData();
notify(vm);
fair::mq::shmem::Segment segment2(shmId, s2id, s2size, fair::mq::shmem::rbTreeBestFit);
segment2.Lock();
segment2.Zero();
LOG(info) << "Created segment " << s2id << " of size " << segment2.GetSize() << " starting at " << segment2.GetData();
// unmanaged regions
fair::mq::shmem::UnmanagedRegion region1(shmId, r1id, r1size);
region1.Lock();
region1.Zero();
LOG(info) << "Created region " << r1id << " of size " << region1.GetSize() << " starting at " << region1.GetData();
fair::mq::shmem::UnmanagedRegion region2(shmId, r2id, r2size);
region2.Lock();
region2.Zero();
LOG(info) << "Created region " << r2id << " of size " << region2.GetSize() << " starting at " << region2.GetData();
// for a "soft reset" call (shmem should not be in active use by (no messages in flight) devices during this call):
// fair::mq::shmem::Monitor::ResetContent(fair::mq::shmem::ShmId{shmId});
ShmManager shmManager(shmId, segments, regions);
while (!gStopping) {
std::this_thread::sleep_for(std::chrono::milliseconds(50));

View File

@@ -23,7 +23,7 @@ struct Sampler : fair::mq::Device
fLinger = fConfig->GetProperty<uint32_t>("region-linger");
fMaxIterations = fConfig->GetProperty<uint64_t>("max-iterations");
GetChannel("data", 0).Transport()->SubscribeToRegionEvents([](FairMQRegionInfo info) {
GetChannel("data", 0).Transport()->SubscribeToRegionEvents([](fair::mq::RegionInfo info) {
LOG(info) << "Region event: " << info.event << ": "
<< (info.managed ? "managed" : "unmanaged")
<< ", id: " << info.id
@@ -36,7 +36,7 @@ struct Sampler : fair::mq::Device
regionCfg.linger = fLinger; // delay in ms before region destruction to collect outstanding events
regionCfg.lock = true; // mlock region after creation
regionCfg.zero = true; // zero region content after creation
fRegion = FairMQUnmanagedRegionPtr(NewUnmanagedRegionFor("data", // region is created using the transport of this channel...
fRegion = fair::mq::UnmanagedRegionPtr(NewUnmanagedRegionFor("data", // region is created using the transport of this channel...
0, // ... and this sub-channel
10000000, // region size
[this](const std::vector<fair::mq::RegionBlock>& blocks) { // callback to be called when message buffers no longer needed by transport
@@ -50,7 +50,7 @@ struct Sampler : fair::mq::Device
bool ConditionalRun() override
{
FairMQMessagePtr msg(NewMessageFor("data", // channel
fair::mq::MessagePtr msg(NewMessageFor("data", // channel
0, // sub-channel
fRegion, // region
fRegion->GetData(), // ptr within region
@@ -93,7 +93,7 @@ struct Sampler : fair::mq::Device
uint32_t fLinger = 100;
uint64_t fMaxIterations = 0;
uint64_t fNumIterations = 0;
FairMQUnmanagedRegionPtr fRegion = nullptr;
fair::mq::UnmanagedRegionPtr fRegion = nullptr;
std::mutex fMtx;
uint64_t fNumUnackedMsgs = 0;
};

View File

@@ -9,22 +9,25 @@ if [[ $1 =~ ^[a-z]+$ ]]; then
fi
msgSize="1000000"
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan="data"
chanAddr="/tmp/fmq_$session""_""$chan""_""$transport"
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; wait $SAMPLER_PID; wait $SINK_PID; @CMAKE_BINARY_DIR@/fairmq/fairmq-shmmonitor --cleanup --session $SESSION' TERM
trap 'kill -TERM $SAMPLER_PID; kill -TERM $SINK_PID; wait $SAMPLER_PID; wait $SINK_PID; @CMAKE_BINARY_DIR@/fairmq/fairmq-shmmonitor --cleanup --session $SESSION; rm $chanAddr' TERM
SAMPLER="fairmq-ex-region-sampler"
SAMPLER+=" --id sampler1"
SAMPLER+=" --transport $transport"
SAMPLER+=" --severity debug"
SAMPLER+=" --session $SESSION"
SAMPLER+=" --session $session"
SAMPLER+=" --shm-segment-size 100000000"
SAMPLER+=" --verbosity veryhigh"
SAMPLER+=" --control static --color false"
SAMPLER+=" --max-iterations 1"
SAMPLER+=" --msg-size $msgSize"
SAMPLER+=" --region-linger 500"
SAMPLER+=" --channel-config name=data,type=push,method=bind,address=tcp://127.0.0.1:7777"
SAMPLER+=" --channel-config name=$chan,type=push,method=bind,address=ipc://$chanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$SAMPLER &
SAMPLER_PID=$!
@@ -32,14 +35,17 @@ SINK="fairmq-ex-region-sink"
SINK+=" --id sink1"
SINK+=" --transport $transport"
SINK+=" --severity debug"
SINK+=" --session $SESSION"
SINK+=" --session $session"
SINK+=" --shm-segment-size 100000000"
SINK+=" --verbosity veryhigh"
SINK+=" --control static --color false"
SINK+=" --max-iterations 1"
SINK+=" --channel-config name=data,type=pull,method=connect,address=tcp://127.0.0.1:7777"
SINK+=" --channel-config name=$chan,type=pull,method=connect,address=ipc://$chanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$SINK &
SINK_PID=$!
# wait for sampler and sink to finish
wait $SAMPLER_PID
wait $SINK_PID
rm $chanAddr

View File

@@ -24,10 +24,10 @@ configure_file(${CMAKE_CURRENT_SOURCE_DIR}/fairmq-start-ex-req-rep.sh.in ${CMAKE
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/test-ex-req-rep.sh.in ${CMAKE_CURRENT_BINARY_DIR}/test-ex-req-rep.sh)
add_test(NAME Example.ReqRep.zeromq COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-req-rep.sh zeromq)
set_tests_properties(Example.ReqRep.zeromq PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received reply from server: ")
set_tests_properties(Example.ReqRep.zeromq PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received reply from server: ")
add_test(NAME Example.ReqRep.shmem COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test-ex-req-rep.sh shmem)
set_tests_properties(Example.ReqRep.shmem PROPERTIES TIMEOUT "30" RUN_SERIAL true PASS_REGULAR_EXPRESSION "Received reply from server: ")
set_tests_properties(Example.ReqRep.shmem PROPERTIES TIMEOUT "30" PASS_REGULAR_EXPRESSION "Received reply from server: ")
# install

View File

@@ -32,11 +32,11 @@ struct Client : fair::mq::Device
// its size,
// custom deletion function (called when transfer is done),
// and pointer to the object managing the data buffer
FairMQMessagePtr req(NewMessage(const_cast<char*>(text->c_str()), // data
fair::mq::MessagePtr req(NewMessage(const_cast<char*>(text->c_str()), // data
text->length(), // size
[](void* /*data*/, void* object) { delete static_cast<std::string*>(object); }, // deletion callback
text)); // object that manages the data
FairMQMessagePtr rep(NewMessage());
fair::mq::MessagePtr rep(NewMessage());
LOG(info) << "Sending \"" << fText << "\" to server.";

View File

@@ -26,7 +26,7 @@ struct Server : fair::mq::Device
fMaxIterations = fConfig->GetProperty<uint64_t>("max-iterations");
}
bool HandleData(FairMQMessagePtr& req, int)
bool HandleData(fair::mq::MessagePtr& req, int)
{
LOG(info) << "Received request from client: \"" << std::string(static_cast<char*>(req->GetData()), req->GetSize()) << "\"";
@@ -34,7 +34,7 @@ struct Server : fair::mq::Device
LOG(info) << "Sending reply to client.";
FairMQMessagePtr rep(NewMessage(const_cast<char*>(text->c_str()), // data
fair::mq::MessagePtr rep(NewMessage(const_cast<char*>(text->c_str()), // data
text->length(), // size
[](void* /*data*/, void* object) { delete static_cast<std::string*>(object); }, // deletion callback
text)); // object that manages the data

View File

@@ -8,19 +8,22 @@ if [[ $1 =~ ^[a-z]+$ ]]; then
transport=$1
fi
SESSION="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
session="$(@CMAKE_BINARY_DIR@/fairmq/fairmq-uuid-gen -h)"
chan="data"
chanAddr="/tmp/fmq_$session""_""$chan""_""$transport"
# setup a trap to kill everything if the test fails/timeouts
trap 'kill -TERM $CLIENT_PID; kill -TERM $SERVER_PID; wait $CLIENT_PID; wait $SERVER_PID;' TERM
trap 'kill -TERM $CLIENT_PID; kill -TERM $SERVER_PID; wait $CLIENT_PID; wait $SERVER_PID; rm $chanAddr' TERM
CLIENT="fairmq-ex-req-rep-client"
CLIENT+=" --id client"
CLIENT+=" --transport $transport"
CLIENT+=" --verbosity veryhigh"
CLIENT+=" --session $SESSION"
CLIENT+=" --session $session"
CLIENT+=" --shm-segment-size 100000000"
CLIENT+=" --control static --color false"
CLIENT+=" --max-iterations 1"
CLIENT+=" --channel-config name=data,type=req,method=connect,rateLogging=0,address=tcp://127.0.0.1:5005"
CLIENT+=" --channel-config name=$chan,type=req,method=connect,rateLogging=0,address=ipc://$chanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$CLIENT &
CLIENT_PID=$!
@@ -28,13 +31,16 @@ SERVER="fairmq-ex-req-rep-server"
SERVER+=" --id server"
SERVER+=" --transport $transport"
SERVER+=" --verbosity veryhigh"
SERVER+=" --session $SESSION"
SERVER+=" --session $session"
SERVER+=" --shm-segment-size 100000000"
SERVER+=" --control static --color false"
SERVER+=" --max-iterations 1"
SERVER+=" --channel-config name=data,type=rep,method=bind,rateLogging=0,address=tcp://127.0.0.1:5005"
SERVER+=" --channel-config name=$chan,type=rep,method=bind,rateLogging=0,address=ipc://$chanAddr"
@CMAKE_CURRENT_BINARY_DIR@/$SERVER &
SERVER_PID=$!
# wait for everything to finish
wait $CLIENT_PID
wait $SERVER_PID
rm $chanAddr

View File

@@ -1,12 +1,12 @@
################################################################################
# Copyright (C) 2012-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH #
# Copyright (C) 2012-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH #
# #
# This software is distributed under the terms of the #
# GNU Lesser General Public Licence (LGPL) version 3, #
# copied verbatim in the file "LICENSE" #
################################################################################
if(BUILD_FAIRMQ OR BUILD_SDK)
if(BUILD_FAIRMQ)
if(BUILD_TIDY_TOOL)
include(FairMQTidy)
@@ -23,135 +23,6 @@ if(BUILD_FAIRMQ OR BUILD_SDK)
DESTINATION ${PROJECT_INSTALL_INCDIR}
)
#########
# Tools #
#########
set(target Tools)
set(TOOLS_PUBLIC_HEADER_FILES
tools/CppSTL.h
tools/Exceptions.h
tools/InstanceLimit.h
tools/IO.h
tools/Network.h
tools/Process.h
tools/RateLimit.h
tools/Semaphore.h
tools/Strings.h
tools/Unique.h
tools/Version.h
Error.h
Tools.h
)
set(TOOLS_SOURCE_FILES
Error.cxx
tools/Network.cxx
tools/Process.cxx
tools/Semaphore.cxx
tools/Unique.cxx
)
add_library(${target}
${TOOLS_SOURCE_FILES}
${TOOLS_PUBLIC_HEADER_FILES}
)
target_compile_features(${target} PUBLIC cxx_std_17)
target_compile_definitions(${target} PUBLIC BOOST_ERROR_CODE_HEADER_ONLY)
target_include_directories(${target}
PUBLIC
$<BUILD_INTERFACE:${CMAKE_SOURCE_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
target_link_libraries(${target}
PRIVATE
FairLogger::FairLogger
Threads::Threads
Boost::boost
)
set_target_properties(${target} PROPERTIES
VERSION ${PROJECT_VERSION}
SOVERSION "${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}"
OUTPUT_NAME FairMQ${target}
)
if(BUILD_TIDY_TOOL AND RUN_FAIRMQ_TIDY)
fairmq_target_tidy(TARGET ${target})
endif()
install(
TARGETS ${target}
EXPORT ${PROJECT_EXPORT_SET}
RUNTIME DESTINATION ${PROJECT_INSTALL_BINDIR}
LIBRARY DESTINATION ${PROJECT_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${PROJECT_INSTALL_LIBDIR}
)
foreach(HEADER ${TOOLS_PUBLIC_HEADER_FILES})
get_filename_component(_path ${HEADER} DIRECTORY)
file(TO_CMAKE_PATH ${PROJECT_INSTALL_INCDIR}/${_path} _destination)
install(FILES ${HEADER}
DESTINATION ${_destination}
)
endforeach()
#################
# State Machine #
#################
set(target StateMachine)
set(FSM_PUBLIC_HEADER_FILES
StateMachine.h
States.h
StateQueue.h
)
set(FSM_SOURCE_FILES
StateMachine.cxx
States.cxx
)
add_library(${target}
${FSM_SOURCE_FILES}
${FSM_PUBLIC_HEADER_FILES}
)
target_compile_features(${target} PUBLIC cxx_std_17)
target_compile_definitions(${target} PUBLIC BOOST_ERROR_CODE_HEADER_ONLY)
target_include_directories(${target}
PUBLIC
$<BUILD_INTERFACE:${CMAKE_SOURCE_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
target_link_libraries(${target}
PUBLIC
FairLogger::FairLogger
PRIVATE
Boost::boost
Tools
)
set_target_properties(${target} PROPERTIES
VERSION ${PROJECT_VERSION}
SOVERSION "${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}"
OUTPUT_NAME FairMQ${target}
)
if(BUILD_TIDY_TOOL AND RUN_FAIRMQ_TIDY)
fairmq_target_tidy(TARGET ${target})
endif()
install(
TARGETS ${target}
EXPORT ${PROJECT_EXPORT_SET}
RUNTIME DESTINATION ${PROJECT_INSTALL_BINDIR}
LIBRARY DESTINATION ${PROJECT_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${PROJECT_INSTALL_LIBDIR}
)
foreach(HEADER ${FSM_PUBLIC_HEADER_FILES})
get_filename_component(_path ${HEADER} DIRECTORY)
file(TO_CMAKE_PATH ${PROJECT_INSTALL_INCDIR}/${_path} _destination)
install(FILES ${HEADER}
DESTINATION ${_destination}
)
endforeach()
endif()
if(BUILD_FAIRMQ)
##########################
# libFairMQ header files #
##########################
@@ -159,6 +30,7 @@ if(BUILD_FAIRMQ)
Channel.h
Device.h
DeviceRunner.h
Error.h
EventManager.h
FairMQChannel.h
FairMQDevice.h
@@ -184,17 +56,32 @@ if(BUILD_FAIRMQ)
Properties.h
PropertyOutput.h
Socket.h
StateMachine.h
States.h
StateQueue.h
SuboptParser.h
Tools.h
TransportFactory.h
Transports.h
UnmanagedRegion.h
options/FairMQProgOptions.h
runDevice.h
runFairMQDevice.h
shmem/Monitor.h
shmem/Common.h
shmem/UnmanagedRegion.h
shmem/Monitor.h
shmem/Segment.h
shmem/UnmanagedRegion.h
tools/CppSTL.h
tools/Exceptions.h
tools/IO.h
tools/InstanceLimit.h
tools/Network.h
tools/Process.h
tools/RateLimit.h
tools/Semaphore.h
tools/Strings.h
tools/Unique.h
tools/Version.h
)
set(FAIRMQ_PRIVATE_HEADER_FILES
@@ -239,6 +126,7 @@ if(BUILD_FAIRMQ)
Channel.cxx
Device.cxx
DeviceRunner.cxx
Error.cxx
JSONParser.cxx
MemoryResources.cxx
Plugin.cxx
@@ -246,6 +134,8 @@ if(BUILD_FAIRMQ)
PluginServices.cxx
ProgOptions.cxx
Properties.cxx
StateMachine.cxx
States.cxx
SuboptParser.cxx
TransportFactory.cxx
plugins/config/Config.cxx
@@ -253,6 +143,10 @@ if(BUILD_FAIRMQ)
shmem/Common.cxx
shmem/Manager.cxx
shmem/Monitor.cxx
tools/Network.cxx
tools/Process.cxx
tools/Semaphore.cxx
tools/Unique.cxx
)
if(BUILD_OFI_TRANSPORT)
@@ -331,8 +225,6 @@ if(BUILD_FAIRMQ)
Boost::filesystem
Boost::regex
FairLogger::FairLogger
Tools
StateMachine
PRIVATE # only libFairMQ links against private dependencies
libzmq
@@ -341,7 +233,7 @@ if(BUILD_FAIRMQ)
)
set_target_properties(${target} PROPERTIES
VERSION ${PROJECT_VERSION}
SOVERSION "${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}"
OUTPUT_NAME ${PROJECT_NAME_LOWER}
)
if(BUILD_TIDY_TOOL AND RUN_FAIRMQ_TIDY)
fairmq_target_tidy(TARGET ${target})
@@ -414,7 +306,7 @@ if(BUILD_FAIRMQ)
add_executable(fairmq-uuid-gen tools/runUuidGenerator.cxx)
target_link_libraries(fairmq-uuid-gen PUBLIC
Boost::program_options
Tools
FairMQ
)
if(BUILD_TIDY_TOOL AND RUN_FAIRMQ_TIDY)
fairmq_target_tidy(TARGET fairmq-uuid-gen)
@@ -452,20 +344,9 @@ if(BUILD_FAIRMQ)
endforeach()
endif()
if(BUILD_SDK_COMMANDS)
add_subdirectory(sdk/commands)
endif()
if(BUILD_SDK)
add_subdirectory(sdk)
endif()
####################
# external plugins #
####################
if(BUILD_DDS_PLUGIN)
add_subdirectory(plugins/DDS)
endif()
if(BUILD_PMIX_PLUGIN)
add_subdirectory(plugins/PMIx)
endif()

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2021-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -475,7 +475,6 @@ class Channel
} // namespace fair::mq
// using FairMQChannel [[deprecated("Use fair::mq::Channel")]] = fair::mq::Channel;
using FairMQChannel = fair::mq::Channel;
using FairMQChannel [[deprecated("Use fair::mq::Channel")]] = fair::mq::Channel;
#endif // FAIR_MQ_CHANNEL_H

View File

@@ -1,19 +1,19 @@
/********************************************************************************
* Copyright (C) 2012-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2012-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
* copied verbatim in the file "LICENSE" *
********************************************************************************/
#include <algorithm> // std::max
#include <algorithm> // std::max, std::any_of
#include <boost/algorithm/string.hpp> // join/split
#include <chrono>
#include <fairmq/Device.h>
#include <fairmq/Tools.h>
#include <future>
#include <iomanip>
#include <list>
#include <memory> // std::make_unique
#include <mutex>
#include <thread>
@@ -27,7 +27,6 @@ constexpr const char* Device::DefaultTransportName;
constexpr mq::Transport Device::DefaultTransportType;
constexpr const char* Device::DefaultNetworkInterface;
constexpr int Device::DefaultInitTimeout;
constexpr uint64_t Device::DefaultMaxRunTime;
constexpr float Device::DefaultRate;
constexpr const char* Device::DefaultSession;
@@ -83,7 +82,6 @@ Device::Device(ProgOptions* config, tools::Version version)
, fMultitransportProceed(false)
, fVersion(version)
, fRate(DefaultRate)
, fMaxRunRuntimeInS(DefaultMaxRunTime)
, fInitializationTimeoutInS(DefaultInitTimeout)
, fTransitioning(false)
{
@@ -215,7 +213,6 @@ void Device::InitWrapper()
Init();
fRate = fConfig->GetProperty<float>("rate", DefaultRate);
fMaxRunRuntimeInS = fConfig->GetProperty<uint64_t>("max-run-time", DefaultMaxRunTime);
fInitializationTimeoutInS = fConfig->GetProperty<int>("init-timeout", DefaultInitTimeout);
try {
@@ -293,7 +290,9 @@ void Device::BindWrapper()
Bind();
ChangeState(Transition::Auto);
if (!NewStatePending()) {
ChangeState(Transition::Auto);
}
}
void Device::ConnectWrapper()
@@ -330,7 +329,9 @@ void Device::ConnectWrapper()
Connect();
ChangeState(Transition::Auto);
if (!NewStatePending()) {
ChangeState(Transition::Auto);
}
}
void Device::AttachChannels(vector<Channel*>& chans)
@@ -430,15 +431,28 @@ void Device::InitTaskWrapper()
{
InitTask();
ChangeState(Transition::Auto);
if (!NewStatePending()) {
ChangeState(Transition::Auto);
}
}
void Device::RunWrapper()
{
LOG(info) << "fair::mq::Device running...";
// start the rate logger thread
future<void> rateLogger = async(launch::async, &Device::LogSocketRates, this);
unique_ptr<thread> rateLogger;
// Check if rate logging thread is needed
const bool rateLogging = any_of(fChannels.cbegin(), fChannels.cend(), [](auto ch) {
return any_of(ch.second.cbegin(), ch.second.cend(), [](auto sub) { return sub.fRateLogging > 0; });
});
if (rateLogging) {
rateLogger = make_unique<thread>(&Device::LogSocketRates, this);
}
tools::CallOnDestruction joinRateLogger([&](){
if (rateLogging && rateLogger->joinable()) { rateLogger->join(); }
});
// notify transports to resume transfers
for (auto& t : fTransports) {
@@ -481,8 +495,6 @@ void Device::RunWrapper()
PostRun();
cod.disable();
rateLogger.get();
}
void Device::HandleSingleChannelInput()
@@ -710,7 +722,6 @@ void Device::LogSocketRates()
chrono::time_point<chrono::high_resolution_clock> t0(chrono::high_resolution_clock::now());
chrono::time_point<chrono::high_resolution_clock> t1;
uint64_t secondsElapsed = 0;
while (!NewStatePending()) {
WaitFor(chrono::seconds(1));
@@ -743,7 +754,7 @@ void Device::LogSocketRates()
bytesOut.at(i) = bytesOutNew.at(i);
msgOut.at(i) = msgOutNew.at(i);
LOG(info) << setw(chanNameLen) << filteredChannelNames.at(i) << ": "
LOG(info) << setw(static_cast<int>(chanNameLen)) << filteredChannelNames.at(i) << ": "
<< "in: " << msgPerSecIn.at(i) << " (" << mbPerSecIn.at(i) << " MB) "
<< "out: " << msgPerSecOut.at(i) << " (" << mbPerSecOut.at(i) << " MB)";
}
@@ -753,9 +764,6 @@ void Device::LogSocketRates()
}
t0 = t1;
if (fMaxRunRuntimeInS > 0 && ++secondsElapsed >= fMaxRunRuntimeInS) {
ChangeState(Transition::Stop);
}
}
}
@@ -770,7 +778,9 @@ void Device::ResetTaskWrapper()
{
ResetTask();
ChangeState(Transition::Auto);
if (!NewStatePending()) {
ChangeState(Transition::Auto);
}
}
void Device::ResetWrapper()
@@ -784,7 +794,9 @@ void Device::ResetWrapper()
fChannels.clear();
fTransports.clear();
fTransportFactory.reset();
ChangeState(Transition::Auto);
if (!NewStatePending()) {
ChangeState(Transition::Auto);
}
}
Device::~Device()

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2021-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -565,7 +565,6 @@ class Device
static constexpr mq::Transport DefaultTransportType = mq::Transport::ZMQ;
static constexpr const char* DefaultNetworkInterface = "default";
static constexpr int DefaultInitTimeout = 120;
static constexpr uint64_t DefaultMaxRunTime = 0;
static constexpr float DefaultRate = 0.;
static constexpr const char* DefaultSession = "default";
@@ -636,15 +635,9 @@ class Device
} // namespace fair::mq
// using FairMQChannelMap [[deprecated("Use fair::mq::ChannelMap")]] = fair::mq::ChannelMap;
// using InputMsgCallback [[deprecated("Use fair::mq::InputMsgCallback")]] =
// fair::mq::InputMsgCallback;
// using InputMultipartCallback [[deprecated("Use fair::mq::InputMultipartCallback")]] =
// fair::mq::InputMultipartCallback;
// using FairMQDevice [[deprecated("Use fair::mq::Device")]] = fair::mq::Device;
using FairMQChannelMap = fair::mq::ChannelMap;
using InputMsgCallback = fair::mq::InputMsgCallback;
using InputMultipartCallback = fair::mq::InputMultipartCallback;
using FairMQDevice = fair::mq::Device;
using FairMQChannelMap [[deprecated("Use fair::mq::ChannelMap")]] = fair::mq::ChannelMap;
using InputMsgCallback [[deprecated("Use fair::mq::InputMsgCallback")]] = fair::mq::InputMsgCallback;
using InputMultipartCallback [[deprecated("Use fair::mq::InputMultipartCallback")]] = fair::mq::InputMultipartCallback;
using FairMQDevice [[deprecated("Use fair::mq::Device")]] = fair::mq::Device;
#endif /* FAIR_MQ_DEVICE_H */

View File

@@ -45,7 +45,7 @@ namespace fair::mq
* Each hook has access to all members of the DeviceRunner and really only differs by the point in
* time it is called.
*
* For an example usage of this class see the fairmq/runFairMQDevice.h header.
* For an example usage of this class see the <fairmq/runDevice.h> header.
*/
class DeviceRunner
{

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQCHANNEL_H_
#define FAIRMQCHANNEL_H_
#if 0
#ifndef FAIR_MQ_CHANNEL_H
#pragma GCC warning "Deprecated header: Use <fairmq/Channel.h> instead"
#endif
#endif
#include <fairmq/Channel.h>

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2012-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2012-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQDEVICE_H_
#define FAIRMQDEVICE_H_
#if 0
#ifndef FAIR_MQ_DEVICE_H
#pragma GCC warning "Deprecated header: Use <fairmq/Device.h> instead"
#endif
#endif
#include <fairmq/Device.h>

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2017-2018 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2017-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,6 +9,8 @@
#ifndef FAIRMQLOGGER_H_
#define FAIRMQLOGGER_H_
#pragma GCC warning "Deprecated header: Use <fairlogger/Logger.h> instead"
#include <fairlogger/Logger.h>
#endif /* FAIRMQLOGGER_H_ */

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQMESSAGE_H_
#define FAIRMQMESSAGE_H_
#if 0
#ifndef FAIR_MQ_MESSAGE_H
#pragma GCC warning "Deprecated header: Use <fairmq/Message.h> instead"
#endif
#endif
#include <fairmq/Message.h>

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQPARTS_H_
#define FAIRMQPARTS_H_
#if 0
#ifndef FAIR_MQ_PARTS_H
#pragma GCC warning "Deprecated header: Use <fairmq/Parts.h> instead"
#endif
#endif
#include <fairmq/Parts.h>

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQPOLLER_H_
#define FAIRMQPOLLER_H_
#if 0
#ifndef FAIR_MQ_POLLER_H
#pragma GCC warning "Deprecated header: Use <fairmq/Poller.h> instead"
#endif
#endif
#include <fairmq/Poller.h>

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQSOCKET_H_
#define FAIRMQSOCKET_H_
#if 0
#ifndef FAIR_MQ_SOCKET_H
#pragma GCC warning "Deprecated header: Use <fairmq/Socket.h> instead"
#endif
#endif
#include <fairmq/Socket.h>

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQTRANSPORTFACTORY_H_
#define FAIRMQTRANSPORTFACTORY_H_
#if 0
#ifndef FAIR_MQ_TRANSPORTFACTORY_H
#pragma GCC warning "Deprecated header: Use <fairmq/TransportFactory.h> instead"
#endif
#endif
#include <fairmq/TransportFactory.h>

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,11 +9,9 @@
#ifndef FAIRMQUNMANAGEDREGION_H_
#define FAIRMQUNMANAGEDREGION_H_
#if 0
#ifndef FAIR_MQ_UNMANAGEDREGION_H
#pragma GCC warning "Deprecated header: Use <fairmq/UnmanagedRegion.h> instead"
#endif
#endif
#include <fairmq/UnmanagedRegion.h>

View File

@@ -30,13 +30,13 @@
using namespace std;
using namespace fair::mq;
using namespace fair::mq::tools;
using namespace tools;
using namespace boost::property_tree;
namespace fair::mq
{
fair::mq::Properties PtreeParser(const ptree& pt, const string& id)
Properties PtreeParser(const ptree& pt, const string& id)
{
if (id.empty()) {
throw ParserError("no device ID provided. Provide with `--id` cmd option");
@@ -47,7 +47,7 @@ fair::mq::Properties PtreeParser(const ptree& pt, const string& id)
return helper::DeviceParser(pt.get_child("fairMQOptions"), id);
}
fair::mq::Properties JSONParser(const string& filename, const string& deviceId)
Properties JSONParser(const string& filename, const string& deviceId)
{
ptree pt;
LOG(debug) << "Parsing JSON from " << filename << " ...";
@@ -58,9 +58,9 @@ fair::mq::Properties JSONParser(const string& filename, const string& deviceId)
namespace helper
{
fair::mq::Properties DeviceParser(const ptree& fairMQOptions, const string& deviceId)
Properties DeviceParser(const ptree& fairMQOptions, const string& deviceId)
{
fair::mq::Properties properties;
Properties properties;
for (const auto& node : fairMQOptions) {
if (node.first == "devices") {
@@ -82,27 +82,27 @@ fair::mq::Properties DeviceParser(const ptree& fairMQOptions, const string& devi
return properties;
}
void ChannelParser(const ptree& tree, fair::mq::Properties& properties)
void ChannelParser(const ptree& tree, Properties& properties)
{
for (const auto& node : tree) {
if (node.first == "channels") {
for (const auto& cn : node.second) {
fair::mq::Properties commonProperties;
commonProperties.emplace("type", cn.second.get<string>("type", FairMQChannel::DefaultType));
commonProperties.emplace("method", cn.second.get<string>("method", FairMQChannel::DefaultMethod));
commonProperties.emplace("address", cn.second.get<string>("address", FairMQChannel::DefaultAddress));
commonProperties.emplace("transport", cn.second.get<string>("transport", FairMQChannel::DefaultTransportName));
commonProperties.emplace("sndBufSize", cn.second.get<int>("sndBufSize", FairMQChannel::DefaultSndBufSize));
commonProperties.emplace("rcvBufSize", cn.second.get<int>("rcvBufSize", FairMQChannel::DefaultRcvBufSize));
commonProperties.emplace("sndKernelSize", cn.second.get<int>("sndKernelSize", FairMQChannel::DefaultSndKernelSize));
commonProperties.emplace("rcvKernelSize", cn.second.get<int>("rcvKernelSize", FairMQChannel::DefaultRcvKernelSize));
commonProperties.emplace("sndTimeoutMs", cn.second.get<int>("sndTimeoutMs", FairMQChannel::DefaultSndTimeoutMs));
commonProperties.emplace("rcvTimeoutMs", cn.second.get<int>("rcvTimeoutMs", FairMQChannel::DefaultRcvTimeoutMs));
commonProperties.emplace("linger", cn.second.get<int>("linger", FairMQChannel::DefaultLinger));
commonProperties.emplace("rateLogging", cn.second.get<int>("rateLogging", FairMQChannel::DefaultRateLogging));
commonProperties.emplace("portRangeMin", cn.second.get<int>("portRangeMin", FairMQChannel::DefaultPortRangeMin));
commonProperties.emplace("portRangeMax", cn.second.get<int>("portRangeMax", FairMQChannel::DefaultPortRangeMax));
commonProperties.emplace("autoBind", cn.second.get<bool>("autoBind", FairMQChannel::DefaultAutoBind));
Properties commonProperties;
commonProperties.emplace("type", cn.second.get<string>("type", Channel::DefaultType));
commonProperties.emplace("method", cn.second.get<string>("method", Channel::DefaultMethod));
commonProperties.emplace("address", cn.second.get<string>("address", Channel::DefaultAddress));
commonProperties.emplace("transport", cn.second.get<string>("transport", Channel::DefaultTransportName));
commonProperties.emplace("sndBufSize", cn.second.get<int>("sndBufSize", Channel::DefaultSndBufSize));
commonProperties.emplace("rcvBufSize", cn.second.get<int>("rcvBufSize", Channel::DefaultRcvBufSize));
commonProperties.emplace("sndKernelSize", cn.second.get<int>("sndKernelSize", Channel::DefaultSndKernelSize));
commonProperties.emplace("rcvKernelSize", cn.second.get<int>("rcvKernelSize", Channel::DefaultRcvKernelSize));
commonProperties.emplace("sndTimeoutMs", cn.second.get<int>("sndTimeoutMs", Channel::DefaultSndTimeoutMs));
commonProperties.emplace("rcvTimeoutMs", cn.second.get<int>("rcvTimeoutMs", Channel::DefaultRcvTimeoutMs));
commonProperties.emplace("linger", cn.second.get<int>("linger", Channel::DefaultLinger));
commonProperties.emplace("rateLogging", cn.second.get<int>("rateLogging", Channel::DefaultRateLogging));
commonProperties.emplace("portRangeMin", cn.second.get<int>("portRangeMin", Channel::DefaultPortRangeMin));
commonProperties.emplace("portRangeMax", cn.second.get<int>("portRangeMax", Channel::DefaultPortRangeMax));
commonProperties.emplace("autoBind", cn.second.get<bool>("autoBind", Channel::DefaultAutoBind));
string name = cn.second.get<string>("name");
int numSockets = cn.second.get<int>("numSockets", 0);
@@ -128,7 +128,7 @@ void ChannelParser(const ptree& tree, fair::mq::Properties& properties)
}
}
void SubChannelParser(const ptree& channelTree, fair::mq::Properties& properties, const string& channelName, const fair::mq::Properties& commonProperties)
void SubChannelParser(const ptree& channelTree, Properties& properties, const string& channelName, const Properties& commonProperties)
{
// for each socket in channel
int i = 0;
@@ -137,7 +137,7 @@ void SubChannelParser(const ptree& channelTree, fair::mq::Properties& properties
if (node.first == "sockets") {
for (const auto& sn : node.second) {
// a sub-channel inherits relevant properties from the common channel ...
fair::mq::Properties newProperties(commonProperties);
Properties newProperties(commonProperties);
// ... and adds/overwrites its own properties
newProperties["type"] = sn.second.get<string>("type", boost::any_cast<string>(commonProperties.at("type")));
@@ -177,7 +177,7 @@ void SubChannelParser(const ptree& channelTree, fair::mq::Properties& properties
LOG(trace) << "\tNo sockets specified,";
LOG(trace) << "\tapplying common settings to the channel:";
fair::mq::Properties newProperties(commonProperties);
Properties newProperties(commonProperties);
for (auto& p : newProperties) {
LOG(trace) << "\t" << setw(13) << left << p.first << " : " << p.second;

View File

@@ -12,7 +12,7 @@
///
/// @author Mikolaj Krzewicki, mkrzewic@cern.ch
#include <fairmq/FairMQTransportFactory.h>
#include <fairmq/TransportFactory.h>
#include <fairmq/MemoryResources.h>
namespace fair::mq
@@ -28,13 +28,13 @@ template<typename ContainerT>
// pmr::polymorphic_allocator<typename
// ContainerT::value_type>,
// typename ContainerT::allocator_type>::value == true,
// FairMQMessagePtr>::type
FairMQMessagePtr getMessage(ContainerT &&container_, FairMQMemoryResource *targetResource = nullptr)
// MessagePtr>::type
MessagePtr getMessage(ContainerT &&container_, MemoryResource *targetResource = nullptr)
{
auto container = std::move(container_);
auto alloc = container.get_allocator();
auto resource = dynamic_cast<FairMQMemoryResource *>(alloc.resource());
auto resource = dynamic_cast<MemoryResource *>(alloc.resource());
if (!resource && !targetResource) {
throw std::runtime_error("Neither the container or target resource specified");
}

View File

@@ -1,10 +1,10 @@
/********************************************************************************
* Copyright (C) 2018 CERN and copyright holders of ALICE O2 *
* Copyright (C) 2018 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2018 CERN and copyright holders of ALICE O2 *
* Copyright (C) 2018-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
* copied verbatim in the file "LICENSE" *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
* copied verbatim in the file "LICENSE" *
********************************************************************************/
/// @brief Memory allocators and interfaces related to managing memory via the
@@ -12,7 +12,7 @@
///
/// @author Mikolaj Krzewicki, mkrzewic@cern.ch
#include <fairmq/FairMQTransportFactory.h>
#include <fairmq/TransportFactory.h>
#include <fairmq/MemoryResources.h>
void *fair::mq::ChannelResource::do_allocate(std::size_t bytes, std::size_t alignment)

View File

@@ -1,6 +1,6 @@
/********************************************************************************
* Copyright (C) 2018 CERN and copyright holders of ALICE O2 *
* Copyright (C) 2018-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2018-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -103,9 +103,7 @@ class ChannelResource : public MemoryResource
};
};
// using FairMQMemoryResource [[deprecated("Use fair::mq::MemoryResource")]] =
// MemoryResource;
using FairMQMemoryResource = MemoryResource;
using FairMQMemoryResource [[deprecated("Use fair::mq::MemoryResource")]] = MemoryResource;
} // namespace fair::mq

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2021-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -78,11 +78,8 @@ struct MessageBadAlloc : std::runtime_error
} // namespace fair::mq
// using fairmq_free_fn [[deprecated("Use fair::mq::FreeFn")]] = fair::mq::FreeFn;
// using FairMQMessage [[deprecated("Use fair::mq::Message")]] = fair::mq::Message;
// using FairMQMessagePtr [[deprecated("Use fair::mq::MessagePtr")]] = fair::mq::MessagePtr;
using fairmq_free_fn = fair::mq::FreeFn;
using FairMQMessage = fair::mq::Message;
using FairMQMessagePtr = fair::mq::MessagePtr;
using fairmq_free_fn [[deprecated("Use fair::mq::FreeFn")]] = fair::mq::FreeFn;
using FairMQMessage [[deprecated("Use fair::mq::Message")]] = fair::mq::Message;
using FairMQMessagePtr [[deprecated("Use fair::mq::MessagePtr")]] = fair::mq::MessagePtr;
#endif // FAIR_MQ_MESSAGE_H

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,84 +9,80 @@
#ifndef FAIR_MQ_PARTS_H
#define FAIR_MQ_PARTS_H
#include <fairmq/Message.h>
#include <memory> // unique_ptr
#include <vector>
#include <algorithm> // std::move
#include <fairmq/Message.h> // fair::mq::MessagePtr
#include <iterator> // std::back_inserter
#include <utility> // std::move, std::forward
#include <vector> // std::vector
namespace fair::mq {
/// fair::mq::Parts is a lightweight convenience wrapper around a vector of unique pointers to
/// fair::mq::Parts is a lightweight move-only convenience wrapper around a vector of unique pointers to
/// Message, used for sending multi-part messages
class Parts
struct Parts
{
private:
using container = std::vector<MessagePtr>;
using size_type = container::size_type;
using reference = container::reference;
using const_reference = container::const_reference;
using iterator = container::iterator;
using const_iterator = container::const_iterator;
public:
Parts() = default;
Parts() noexcept(noexcept(container())) = default;
Parts(const Parts&) = delete;
Parts(Parts&&) = default;
template<typename... Ts>
Parts(Ts&&... messages) { AddPart(std::forward<Ts>(messages)...); }
Parts& operator=(const Parts&) = delete;
Parts(Parts&&) = default;
Parts& operator=(Parts&&) = default;
~Parts() = default;
/// Adds part (Message) to the container
/// @param msg message pointer (for example created with NewMessage() method of Device)
void AddPart(Message* msg) { fParts.push_back(MessagePtr(msg)); }
template<typename... Ps>
Parts(Ps&&... parts)
{
AddPart(std::forward<Ps>(parts)...);
}
/// Adds part to the container (move)
/// @param msg unique pointer to Message
/// rvalue ref (move required when passing argument)
void AddPart(MessagePtr&& msg) { fParts.push_back(std::move(msg)); }
void AddPart(MessagePtr msg) { fParts.push_back(std::move(msg)); }
/// Add variable list of parts to the container (move)
template<typename... Ts>
void AddPart(MessagePtr&& first, Ts&&... remaining)
void AddPart(MessagePtr first, Ts&&... remaining)
{
AddPart(std::move(first));
AddPart(std::forward<Ts>(remaining)...);
}
/// Add content of another object by move
void AddPart(Parts&& other)
void AddPart(Parts parts)
{
container parts = std::move(other.fParts);
for (auto& part : parts) {
fParts.push_back(std::move(part));
if (fParts.empty()) {
fParts = std::move(parts.fParts);
} else {
fParts.reserve(parts.Size() + fParts.size());
std::move(std::begin(parts), std::end(parts), std::back_inserter(fParts));
}
}
/// Get reference to part in the container at index (without bounds check)
/// @param index container index
Message& operator[](const int index) { return *(fParts[index]); }
reference operator[](size_type index) { return fParts[index]; }
const_reference operator[](size_type index) const { return fParts[index]; }
/// Get reference to unique pointer to part in the container at index (with bounds check)
/// @param index container index
MessagePtr& At(const int index) { return fParts.at(index); }
reference At(size_type index) { return fParts.at(index); }
const_reference At(size_type index) const { return fParts.at(index); }
// ref version
Message& AtRef(const int index) { return *(fParts.at(index)); }
size_type Size() const noexcept { return fParts.size(); }
bool Empty() const noexcept { return fParts.empty(); }
void Clear() noexcept { fParts.clear(); }
/// Get number of parts in the container
/// @return number of parts in the container
int Size() const { return fParts.size(); }
// range access
iterator begin() noexcept { return fParts.begin(); }
const_iterator begin() const noexcept { return fParts.begin(); }
const_iterator cbegin() const noexcept { return fParts.cbegin(); }
iterator end() noexcept { return fParts.end(); }
const_iterator end() const noexcept { return fParts.end(); }
const_iterator cend() const noexcept { return fParts.cend(); }
container fParts;
// forward container iterators
using iterator = container::iterator;
using const_iterator = container::const_iterator;
auto begin() -> decltype(fParts.begin()) { return fParts.begin(); }
auto end() -> decltype(fParts.end()) { return fParts.end(); }
auto cbegin() -> decltype(fParts.cbegin()) { return fParts.cbegin(); }
auto cend() -> decltype(fParts.cend()) { return fParts.cend(); }
container fParts{};
};
} // namespace fair::mq
// using FairMQParts [[deprecated("Use fair::mq::Parts")]] = fair::mq::Parts;
using FairMQParts = fair::mq::Parts;
using FairMQParts [[deprecated("Use fair::mq::Parts")]] = fair::mq::Parts;
#endif /* FAIR_MQ_PARTS_H */

View File

@@ -7,7 +7,7 @@
********************************************************************************/
#include <fairmq/Plugin.h>
#include <FairMQLogger.h>
#include <fairlogger/Logger.h>
#include <utility>
using namespace std;

View File

@@ -1,16 +1,17 @@
/********************************************************************************
* Copyright (C) 2017 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2017-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
* copied verbatim in the file "LICENSE" *
********************************************************************************/
#include <fairmq/plugins/Builtin.h>
#include <fairmq/PluginManager.h>
#include <fairmq/tools/Strings.h>
#include <boost/program_options.hpp>
#include <algorithm>
#include <boost/program_options.hpp>
#include <fairlogger/Logger.h>
#include <fairmq/PluginManager.h>
#include <fairmq/plugins/Builtin.h>
#include <fairmq/tools/Strings.h>
#include <iterator>
#include <memory>
#include <sstream>
@@ -18,22 +19,23 @@
#include <utility>
using namespace std;
using fair::mq::Plugin;
using fair::mq::tools::ToString;
using fair::mq::tools::ToStrVector;
using fair::mq::Plugin;
namespace fs = boost::filesystem;
namespace po = boost::program_options;
namespace dll = boost::dll;
using boost::optional;
const std::string fair::mq::PluginManager::fgkLibPrefix = "FairMQPlugin_";
const std::string fair::mq::PluginManager::fgkLibPrefix = "fairmq-plugin-";
const std::string fair::mq::PluginManager::fgkLibPrefixAlt = "FairMQPlugin_";
std::vector<boost::dll::shared_library> fair::mq::PluginManager::fgDLLKeepAlive = std::vector<boost::dll::shared_library>();
std::vector<boost::dll::shared_library> fair::mq::PluginManager::fgDLLKeepAlive =
std::vector<boost::dll::shared_library>();
fair::mq::PluginManager::PluginManager()
: fPluginServices()
{
}
{}
fair::mq::PluginManager::PluginManager(const vector<string>& args)
: fPluginServices()
@@ -46,7 +48,8 @@ fair::mq::PluginManager::PluginManager(const vector<string>& args)
po::store(parsed, vm);
po::notify(vm);
} catch (const po::error& e) {
throw ProgramOptionsParseError{ToString("Error occured while parsing the 'Plugin Manager' program options: ", e.what())};
throw ProgramOptionsParseError{ToString(
"Error occured while parsing the 'Plugin Manager' program options: ", e.what())};
}
// Process plugin search paths
@@ -55,24 +58,38 @@ fair::mq::PluginManager::PluginManager(const vector<string>& args)
auto searchPaths = vector<fs::path>{};
if (vm.count("plugin-search-path")) {
for (const auto& path : vm["plugin-search-path"].as<vector<string>>()) {
if (path.substr(0, 1) == "<") { prepend.emplace_back(path.substr(1)); }
else if (path.substr(0, 1) == ">") { append.emplace_back(path.substr(1)); }
else { searchPaths.emplace_back(path); }
if (path.substr(0, 1) == "<") {
prepend.emplace_back(path.substr(1));
} else if (path.substr(0, 1) == ">") {
append.emplace_back(path.substr(1));
} else {
searchPaths.emplace_back(path);
}
}
}
// Set supplied options
SetSearchPaths(searchPaths);
for(const auto& path : prepend) { PrependSearchPath(path); }
for(const auto& path : append) { AppendSearchPath(path); }
if (vm.count("plugin")) { LoadPlugins(vm["plugin"].as<vector<string>>()); }
for (const auto& path : prepend) {
PrependSearchPath(path);
}
for (const auto& path : append) {
AppendSearchPath(path);
}
if (vm.count("plugin")) {
LoadPlugins(vm["plugin"].as<vector<string>>());
}
}
auto fair::mq::PluginManager::ValidateSearchPath(const fs::path& path) -> void
{
if (path.empty()) throw BadSearchPath{"Specified path is empty."};
if (path.empty()) {
throw BadSearchPath{"Specified path is empty."};
}
// we ignore non-existing search paths
if (fs::exists(path) && !fs::is_directory(path)) throw BadSearchPath{ToString(path, " is not a directory.")};
if (fs::exists(path) && !fs::is_directory(path)) {
throw BadSearchPath{ToString(path, " is not a directory.")};
}
}
auto fair::mq::PluginManager::SetSearchPaths(const vector<fs::path>& searchPaths) -> void
@@ -96,31 +113,38 @@ auto fair::mq::PluginManager::PrependSearchPath(const fs::path& path) -> void
auto fair::mq::PluginManager::ProgramOptions() -> po::options_description
{
auto plugin_options = po::options_description{"Plugin Manager"};
plugin_options.add_options()
("plugin-search-path,S", po::value<vector<string>>()->multitoken(), "List of plugin search paths.\n\n"
"* Override default search path, e.g.\n"
" -S /home/user/lib /lib\n"
"* Append(>) or prepend(<) to default search path, e.g.\n"
" -S >/lib </home/user/lib\n"
"* If you mix the overriding and appending/prepending syntaxes, the overriding paths act as default search path, e.g.\n"
" -S /usr/lib >/lib </home/user/lib /usr/local/lib results in /home/user/lib,/usr/local/lib,/usr/lib/,/lib\n"
"If nothing is found, the default dynamic library lookup is performed, see man ld.so(8) for details.")
("plugin,P", po::value<vector<string>>(), "List of plugin names to load in order,"
"e.g. if the file is called 'libFairMQPlugin_example.so', just list 'example' or 'd:example' here."
"To load a prelinked plugin, list 'p:example' here.");
plugin_options.add_options()("plugin-search-path,S",
po::value<vector<string>>()->multitoken(),
"List of plugin search paths.\n\n"
"* Override default search path, e.g.\n"
" -S /home/user/lib /lib\n"
"* Append(>) or prepend(<) to default search path, e.g.\n"
" -S >/lib </home/user/lib\n"
"* If you mix the overriding and appending/prepending syntaxes, "
"the overriding paths act as default search path, e.g.\n"
" -S /usr/lib >/lib </home/user/lib /usr/local/lib results in "
"/home/user/lib,/usr/local/lib,/usr/lib/,/lib\n"
"If nothing is found, the default dynamic library lookup is "
"performed, see man ld.so(8) for details.")(
"plugin,P",
po::value<vector<string>>(),
"List of plugin names to load in order,"
"e.g. if the file is called 'libfairmq-plugin-example.so', just list 'example' or "
"'d:example' here."
"To load a prelinked plugin, list 'p:example' here.");
return plugin_options;
}
auto fair::mq::PluginManager::LoadPlugin(const string& pluginName) -> void
{
if (pluginName.substr(0,2) == "p:") {
if (pluginName.substr(0, 2) == "p:") {
// Mechanism A: prelinked dynamic
LoadPluginPrelinkedDynamic(pluginName.substr(2));
} else if (pluginName.substr(0,2) == "d:") {
} else if (pluginName.substr(0, 2) == "d:") {
// Mechanism B: dynamic
LoadPluginDynamic(pluginName.substr(2));
} else if (pluginName.substr(0,2) == "s:") {
} else if (pluginName.substr(0, 2) == "s:") {
// Mechanism C: static (builtin)
LoadPluginStatic(pluginName.substr(2));
} else {
@@ -137,52 +161,76 @@ auto fair::mq::PluginManager::LoadPluginPrelinkedDynamic(const string& pluginNam
LoadSymbols(pluginName, dll::program_location());
fPluginOrder.push_back(pluginName);
} catch (boost::system::system_error& e) {
throw PluginLoadError(ToString("An error occurred while loading prelinked dynamic plugin ", pluginName, ": ", e.what()));
throw PluginLoadError(
ToString("An error occurred while loading prelinked dynamic plugin ",
pluginName,
": ",
e.what()));
}
}
}
auto fair::mq::PluginManager::LoadPluginDynamic(const string& pluginName) -> void
auto fair::mq::PluginManager::SearchPluginFile(const string& pluginName) const
-> boost::filesystem::path
{
// Search plugin and load, if found
if (fPluginFactories.find(pluginName) == fPluginFactories.end()) {
auto success = false;
for (const auto& searchPath : SearchPaths()) {
try {
LoadSymbols(pluginName, searchPath / ToString(LibPrefix(), pluginName),
dll::load_mode::append_decorations | dll::load_mode::rtld_global);
fPluginOrder.push_back(pluginName);
success = true;
break;
} catch (boost::system::system_error& e) {
if (string{e.what()}.find("No such file or directory") == string::npos) {
throw PluginLoadError(
ToString("An error occurred while loading dynamic plugin ",
pluginName, ": ", e.what()));
}
}
}
if (!success) {
try {
// LoadSymbols(pluginName,
// ToString(LibPrefix(), pluginName),
// dll::load_mode::search_system_folders | dll::load_mode::append_decorations);
// Not sure, why the above does not work. Workaround for now:
LoadSymbols(pluginName,
ToString("lib",
LibPrefix(),
pluginName,
boost::dll::detail::shared_library_impl::suffix().native()),
dll::load_mode::search_system_folders | dll::load_mode::rtld_global);
fPluginOrder.push_back(pluginName);
} catch (boost::system::system_error& e) {
throw PluginLoadError(
ToString("An error occurred while loading dynamic plugin ",
pluginName, ": ", e.what()));
for (const auto& searchPath : SearchPaths()) {
for (const auto& libPrefix : {fgkLibPrefix, fgkLibPrefixAlt}) {
auto const file =
searchPath
/ ToString("lib",
libPrefix,
pluginName,
boost::dll::detail::shared_library_impl::suffix().native());
auto const found = boost::filesystem::exists(file);
if (found) {
return file;
}
}
}
throw PluginNotFound(ToString("Could not find plugin file for plugin ", pluginName));
}
auto fair::mq::PluginManager::LoadPluginDynamic(const string& pluginName) -> void
{
if (fPluginFactories.find(pluginName) != fPluginFactories.end()) {
return; // already loaded, nothing to do
}
// Search in configured search path
try {
LoadSymbols(pluginName, SearchPluginFile(pluginName), dll::load_mode::rtld_global);
fPluginOrder.push_back(pluginName);
return;
} catch (PluginNotFound& e) {
// ignore
} catch (boost::system::system_error& e) {
// ignore
}
// Search with system logic (including RPATH, LD_LIBRARY_PATH)
for (const auto& libPrefix : {fgkLibPrefix, fgkLibPrefixAlt}) {
try {
// LoadSymbols(pluginName,
// ToString(LibPrefix(), pluginName),
// dll::load_mode::search_system_folders | dll::load_mode::append_decorations);
// Not sure, why the above does not work. Workaround for now:
LoadSymbols(pluginName,
ToString("lib",
libPrefix,
pluginName,
boost::dll::detail::shared_library_impl::suffix().native()),
dll::load_mode::search_system_folders | dll::load_mode::rtld_global);
fPluginOrder.push_back(pluginName);
return;
} catch (boost::system::system_error& e) {
// ignore
}
}
throw PluginLoadError(ToString("Could not load dynamic plugin ",
pluginName,
" from configured search paths (-S) nor with ",
"system lookup (system libdirs, RUNPATH, $LD_LIBRARY_PATH)."));
}
auto fair::mq::PluginManager::LoadPluginStatic(const string& pluginName) -> void
@@ -192,20 +240,25 @@ auto fair::mq::PluginManager::LoadPluginStatic(const string& pluginName) -> void
try {
if ("control" == pluginName) {
try {
fPluginProgOptions.insert({pluginName, plugins::ControlPluginProgramOptions().value()});
fPluginProgOptions.insert(
{pluginName, plugins::ControlPluginProgramOptions().value()});
} catch (const boost::bad_optional_access&) {
/* just ignore, if no prog options are declared */
}
catch (const boost::bad_optional_access& e) { /* just ignore, if no prog options are declared */ }
} else if ("config" == pluginName) {
try {
fPluginProgOptions.insert({pluginName, plugins::ConfigPluginProgramOptions().value()});
fPluginProgOptions.insert(
{pluginName, plugins::ConfigPluginProgramOptions().value()});
} catch (const boost::bad_optional_access&) {
/* just ignore, if no prog options are declared */
}
catch (const boost::bad_optional_access& e) { /* just ignore, if no prog options are declared */ }
} else {
LoadSymbols(pluginName, dll::program_location());
}
fPluginOrder.push_back(pluginName);
} catch (boost::system::system_error& e) {
throw PluginLoadError(ToString("An error occurred while loading static plugin ", pluginName, ": ", e.what()));
throw PluginLoadError(ToString(
"An error occurred while loading static plugin ", pluginName, ": ", e.what()));
}
}
}
@@ -225,11 +278,12 @@ auto fair::mq::PluginManager::InstantiatePlugin(const string& pluginName) -> voi
auto fair::mq::PluginManager::InstantiatePlugins() -> void
{
for(const auto& pluginName : fPluginOrder) {
for (const auto& pluginName : fPluginOrder) {
try {
InstantiatePlugin(pluginName);
} catch (std::exception& e) {
throw PluginInstantiationError(ToString("An error occurred while instantiating plugin ", pluginName, ": ", e.what()));
throw PluginInstantiationError(ToString(
"An error occurred while instantiating plugin ", pluginName, ": ", e.what()));
}
}
}

View File

@@ -66,6 +66,8 @@ class PluginManager
auto SearchPaths() const -> const std::vector<boost::filesystem::path>& { return fSearchPaths; }
struct BadSearchPath : std::invalid_argument { using std::invalid_argument::invalid_argument; };
auto SearchPluginFile(const std::string&) const -> boost::filesystem::path;
struct PluginNotFound : std::runtime_error { using std::runtime_error::runtime_error; };
auto LoadPlugin(const std::string& pluginName) -> void;
auto LoadPlugins(const std::vector<std::string>& pluginNames) -> void { for(const auto& pluginName : pluginNames) { LoadPlugin(pluginName); } }
struct PluginLoadError : std::runtime_error { using std::runtime_error::runtime_error; };
@@ -118,6 +120,7 @@ class PluginManager
auto InstantiatePlugin(const std::string& pluginName) -> void;
static const std::string fgkLibPrefix;
static const std::string fgkLibPrefixAlt;
std::vector<boost::filesystem::path> fSearchPaths;
static std::vector<boost::dll::shared_library> fgDLLKeepAlive; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
std::map<std::string, std::function<PluginFactory>> fPluginFactories;

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2021-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -41,9 +41,7 @@ struct PollerError : std::runtime_error
} // namespace fair::mq
// using FairMQPoller [[deprecated("Use fair::mq::Poller")]] = fair::mq::Poller;
// using FairMQPollerPtr [[deprecated("Use fair::mq::PollerPtr")]] = fair::mq::PollerPtr;
using FairMQPoller = fair::mq::Poller;
using FairMQPollerPtr = fair::mq::PollerPtr;
using FairMQPoller [[deprecated("Use fair::mq::Poller")]] = fair::mq::Poller;
using FairMQPollerPtr [[deprecated("Use fair::mq::PollerPtr")]] = fair::mq::PollerPtr;
#endif // FAIR_MQ_POLLER_H

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2014-2018 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2014-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -12,9 +12,7 @@
* Created on March 11, 2015, 10:20 PM
*/
#include "FairMQLogger.h"
#include <fairmq/ProgOptions.h>
#include <fairmq/tools/Strings.h>
#include <boost/any.hpp>
@@ -22,6 +20,7 @@
#include <algorithm>
#include <exception>
#include <fairlogger/Logger.h>
#include <iomanip>
#include <iostream>
#include <sstream>
@@ -351,7 +350,7 @@ void ProgOptions::DeleteProperty(const string& key)
vm.erase(key);
}
void ProgOptions::AddChannel(const string& name, const FairMQChannel& channel)
void ProgOptions::AddChannel(const string& name, const Channel& channel)
{
lock_guard<mutex> lock(fMtx);
unordered_map<string, int> existingChannels = GetChannelInfoImpl();

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2019-2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2019-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -13,7 +13,6 @@ namespace fair::mq {
class ProgOptions;
}
// using FairMQProgOptions [[deprecated("Use fair::mq::ProgOptions")]] = fair::mq::ProgOptions;
using FairMQProgOptions = fair::mq::ProgOptions;
using FairMQProgOptions [[deprecated("Use fair::mq::ProgOptions")]] = fair::mq::ProgOptions;
#endif /* FAIR_MQ_PROGOPTIONSFWD_H */

View File

@@ -1,26 +0,0 @@
/********************************************************************************
* Copyright (C) 2019 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
* copied verbatim in the file "LICENSE" *
********************************************************************************/
#ifndef FAIR_MQ_SDK_H
#define FAIR_MQ_SDK_H
// IWYU pragma: begin_exports
#include <fairmq/sdk/AsioAsyncOp.h>
#include <fairmq/sdk/AsioBase.h>
#include <fairmq/sdk/DDSAgent.h>
#include <fairmq/sdk/DDSEnvironment.h>
#include <fairmq/sdk/DDSInfo.h>
#include <fairmq/sdk/DDSSession.h>
#include <fairmq/sdk/DDSTask.h>
#include <fairmq/sdk/DDSTopology.h>
#include <fairmq/sdk/Error.h>
#include <fairmq/sdk/Topology.h>
#include <fairmq/sdk/Traits.h>
// IWYU pragma: end_exports
#endif // FAIR_MQ_SDK_H

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2021-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -103,9 +103,7 @@ struct SocketError : std::runtime_error
} // namespace fair::mq
// using FairMQSocket [[deprecated("Use fair::mq::Socket")]] = fair::mq::Socket;
// using FairMQSocketPtr [[deprecated("Use fair::mq::SocketPtr")]] = fair::mq::SocketPtr;
using FairMQSocket = fair::mq::Socket;
using FairMQSocketPtr = fair::mq::SocketPtr;
using FairMQSocket [[deprecated("Use fair::mq::Socket")]] = fair::mq::Socket;
using FairMQSocketPtr [[deprecated("Use fair::mq::SocketPtr")]] = fair::mq::SocketPtr;
#endif // FAIR_MQ_SOCKET_H

View File

@@ -187,9 +187,7 @@ struct Machine_ : public state_machine_def<Machine_>
{
unique_lock<mutex> lock(fStateMtx);
while (!fNewStatePending) {
fNewStatePendingCV.wait_for(lock, chrono::milliseconds(100));
}
fNewStatePendingCV.wait(lock, [this]{ return fNewStatePending.load(); });
LOG(state) << fState << " ---> " << fNewState;
fState = static_cast<State>(fNewState);

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2019 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2019-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -9,16 +9,14 @@
#ifndef FAIRMQSTATEQUEUE_H_
#define FAIRMQSTATEQUEUE_H_
#include <fairmq/States.h>
#include <queue>
#include <mutex>
#include <chrono>
#include <utility> // pair
#include <condition_variable>
#include <fairmq/States.h>
#include <mutex>
#include <queue>
#include <utility> // pair
namespace fair::mq
{
namespace fair::mq {
class StateQueue
{
@@ -33,41 +31,47 @@ class StateQueue
fair::mq::State WaitForNext()
{
std::unique_lock<std::mutex> lock(fMtx);
while (fStates.empty()) {
fCV.wait_for(lock, std::chrono::milliseconds(50));
}
fair::mq::State state = fStates.front();
if (state == fair::mq::State::Error) {
throw DeviceErrorState("Controlled device transitioned to error state.");
}
fStates.pop();
return state;
fCV.wait(lock, [this] { return Predicate(); });
return PopFrontUnsafe();
}
template<typename Rep, typename Period>
std::pair<bool, fair::mq::State> WaitForNext(std::chrono::duration<Rep, Period> const& duration)
template<typename Timeout>
std::pair<bool, fair::mq::State> WaitForNext(Timeout&& duration)
{
std::unique_lock<std::mutex> lock(fMtx);
fCV.wait_for(lock, duration);
if (fStates.empty()) {
return { false, fair::mq::State::Ok };
}
fair::mq::State state = fStates.front();
if (state == fair::mq::State::Error) {
throw DeviceErrorState("Controlled device transitioned to error state.");
}
fStates.pop();
return { true, state };
fCV.wait_for(lock, std::forward<Timeout>(duration), [this] { return Predicate(); });
return ReturnPairUnsafe();
}
void WaitForState(fair::mq::State state) { while (WaitForNext() != state) {} }
template<typename CustomPredicate>
std::pair<bool, fair::mq::State> WaitForNextOrCustom(CustomPredicate&& customPredicate)
{
std::unique_lock<std::mutex> lock(fMtx);
fCV.wait(lock, [this, cp = std::move(customPredicate)] { return Predicate() || cp(); });
return ReturnPairUnsafe();
}
template<typename CustomPredicate>
std::pair<bool, fair::mq::State> WaitForCustom(CustomPredicate&& customPredicate)
{
std::unique_lock<std::mutex> lock(fMtx);
fCV.wait(lock, [cp = std::move(customPredicate)] { return cp(); });
return ReturnPairUnsafe();
}
void WaitForState(fair::mq::State state)
{
while (WaitForNext() != state) {}
}
template<typename CustomPredicate>
void WaitForStateOrCustom(fair::mq::State state, CustomPredicate customPredicate)
{
auto next = WaitForNextOrCustom(customPredicate);
while (!customPredicate() && (next.first && next.second != state)) {
next = WaitForNextOrCustom(customPredicate);
}
}
void Push(fair::mq::State state)
{
@@ -75,7 +79,35 @@ class StateQueue
std::lock_guard<std::mutex> lock(fMtx);
fStates.push(state);
}
fCV.notify_all();
fCV.notify_one();
}
template<typename CustomSignal>
void Push(fair::mq::State state, CustomSignal&& signal)
{
{
std::lock_guard<std::mutex> lock(fMtx);
fStates.push(state);
signal();
}
fCV.notify_one();
}
template<typename CustomSignal>
void Notify(CustomSignal&& signal)
{
{
std::lock_guard<std::mutex> lock(fMtx);
signal();
}
fCV.notify_one();
}
template<typename CustomSignal>
void Locked(CustomSignal&& signal)
{
std::lock_guard<std::mutex> lock(fMtx);
signal();
}
void Clear()
@@ -88,8 +120,29 @@ class StateQueue
std::queue<fair::mq::State> fStates;
std::mutex fMtx;
std::condition_variable fCV;
// must be called under locked fMtx
fair::mq::State PopFrontUnsafe()
{
fair::mq::State state = fStates.front();
if (state == fair::mq::State::Error) {
throw DeviceErrorState("Controlled device transitioned to error state.");
}
fStates.pop();
return state;
}
// must be called under locked fMtx
std::pair<bool, fair::mq::State> ReturnPairUnsafe()
{
auto const pred = Predicate();
return {pred, pred ? PopFrontUnsafe() : fair::mq::State::Ok};
}
// must be called under locked fMtx
bool Predicate() { return !fStates.empty(); }
};
} // namespace fair::mq
} // namespace fair::mq
#endif /* FAIRMQSTATEQUEUE_H_ */

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2021-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -241,11 +241,7 @@ struct TransportFactoryError : std::runtime_error
} // namespace fair::mq
// using FairMQTransportFactory [[deprecated("Use fair::mq::TransportFactory")]] =
// fair::mq::TransportFactory;
// using FairMQTransportFactoryError [[deprecated("Use fair::mq::TransportFactoryError")]] =
// fair::mq::TransportFactoryError;
using FairMQTransportFactory = fair::mq::TransportFactory;
using FairMQTransportFactoryError = fair::mq::TransportFactoryError;
using FairMQTransportFactory [[deprecated("Use fair::mq::TransportFactory")]] = fair::mq::TransportFactory;
using FairMQTransportFactoryError [[deprecated("Use fair::mq::TransportFactoryError")]] = fair::mq::TransportFactoryError;
#endif // FAIR_MQ_TRANSPORTFACTORY_H

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2021 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2021-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -133,6 +133,7 @@ struct RegionConfig
bool removeOnDestruction = true; /// remove the region on object destruction
int creationFlags = 0; /// flags passed to the underlying transport on region creation
int64_t userFlags = 0; /// custom flags that have no effect on the transport, but can be retrieved from the region by the user
uint64_t size = 0; /// region size
std::string path = ""; /// file path, if the region is backed by a file
std::optional<uint16_t> id = std::nullopt; /// region id
uint32_t linger = 100; /// delay in ms before region destruction to collect outstanding events
@@ -140,23 +141,14 @@ struct RegionConfig
} // namespace fair::mq
// using FairMQRegionEvent [[deprecated("Use fair::mq::RegionBlock")]] = fair::mq::RegionEvent;
// using FairMQRegionInfo [[deprecated("Use fair::mq::RegionInfo")]] = fair::mq::RegionInfo;
// using FairMQRegionBlock [[deprecated("Use fair::mq::RegionBlock")]] = fair::mq::RegionBlock;
// using FairMQRegionCallback [[deprecated("Use fair::mq::RegionCallback")]] = fair::mq::RegionCallback;
// using FairMQRegionBulkCallback [[deprecated("Use fair::mq::RegionBulkCallback")]] = fair::mq::RegionBulkCallback;
// using FairMQRegionEventCallback [[deprecated("Use fair::mq::RegionEventCallback")]] = fair::mq::RegionEventCallback;
// using FairMQUnmanagedRegion [[deprecated("Use fair::mq::UnmanagedRegion")]] = fair::mq::UnmanagedRegion;
// using FairMQUnmanagedRegionPtr [[deprecated("Use fair::mq::UnmanagedRegionPtr")]] = fair::mq::UnmanagedRegionPtr;
// using FairMQRegionConfig [[deprecated("Use fair::mq::RegionConfig")]] = fair::mq::RegionConfig;
using FairMQRegionEvent = fair::mq::RegionEvent;
using FairMQRegionInfo = fair::mq::RegionInfo;
using FairMQRegionBlock = fair::mq::RegionBlock;
using FairMQRegionCallback = fair::mq::RegionCallback;
using FairMQRegionBulkCallback = fair::mq::RegionBulkCallback;
using FairMQRegionEventCallback = fair::mq::RegionEventCallback;
using FairMQUnmanagedRegion = fair::mq::UnmanagedRegion;
using FairMQUnmanagedRegionPtr = fair::mq::UnmanagedRegionPtr;
using FairMQRegionConfig = fair::mq::RegionConfig;
using FairMQRegionEvent [[deprecated("Use fair::mq::RegionEvent")]] = fair::mq::RegionEvent;
using FairMQRegionInfo [[deprecated("Use fair::mq::RegionInfo")]] = fair::mq::RegionInfo;
using FairMQRegionBlock [[deprecated("Use fair::mq::RegionBlock")]] = fair::mq::RegionBlock;
using FairMQRegionCallback [[deprecated("Use fair::mq::RegionCallback")]] = fair::mq::RegionCallback;
using FairMQRegionBulkCallback [[deprecated("Use fair::mq::RegionBulkCallback")]] = fair::mq::RegionBulkCallback;
using FairMQRegionEventCallback [[deprecated("Use fair::mq::RegionEventCallback")]] = fair::mq::RegionEventCallback;
using FairMQUnmanagedRegion [[deprecated("Use fair::mq::UnmanagedRegion")]] = fair::mq::UnmanagedRegion;
using FairMQUnmanagedRegionPtr [[deprecated("Use fair::mq::UnmanagedRegionPtr")]] = fair::mq::UnmanagedRegionPtr;
using FairMQRegionConfig [[deprecated("Use fair::mq::RegionConfig")]] = fair::mq::RegionConfig;
#endif // FAIR_MQ_UNMANAGEDREGION_H

View File

@@ -1,5 +1,5 @@
/********************************************************************************
* Copyright (C) 2018-2020 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* Copyright (C) 2018-2022 GSI Helmholtzzentrum fuer Schwerionenforschung GmbH *
* *
* This software is distributed under the terms of the *
* GNU Lesser General Public Licence (LGPL) version 3, *
@@ -11,17 +11,15 @@
#define FAIRMQ_VERSION "@PROJECT_VERSION@"
#define FAIRMQ_VERSION_DEC (@PROJECT_VERSION_MAJOR@ * 100000) \
+ (@PROJECT_VERSION_MINOR@ * 1000) \
+ (@PROJECT_VERSION_PATCH@ * 10) \
+ @PROJECT_VERSION_HOTFIX@
+ (@PROJECT_VERSION_PATCH@ * 10)
#define FAIRMQ_VERSION_MAJOR @PROJECT_VERSION_MAJOR@
#define FAIRMQ_VERSION_MINOR @PROJECT_VERSION_MINOR@
#define FAIRMQ_VERSION_PATCH @PROJECT_VERSION_PATCH@
#define FAIRMQ_VERSION_HOTFIX @PROJECT_VERSION_HOTFIX@
#define FAIRMQ_GIT_VERSION "@PROJECT_GIT_VERSION@"
#define FAIRMQ_GIT_DATE "@PROJECT_GIT_DATE@"
#define FAIRMQ_REPO_URL "https://github.com/FairRootGroup/FairMQ"
#define FAIRMQ_LICENSE "LGPL-3.0"
#define FAIRMQ_COPYRIGHT "2012-2021 GSI"
#define FAIRMQ_COPYRIGHT "2012-2022 GSI"
#define FAIRMQ_BUILD_TYPE "@CMAKE_BUILD_TYPE@"
#endif // FAIR_MQ_VERSION_H

View File

@@ -44,7 +44,7 @@ class BenchmarkSampler : public Device
void Run() override
{
// store the channel reference to avoid traversing the map on every loop iteration
FairMQChannel& dataOutChannel = GetChannel(fOutChannelName, 0);
Channel& dataOutChannel = GetChannel(fOutChannelName, 0);
LOG(info) << "Starting the benchmark with message size of " << fMsgSize << " and " << fMaxIterations << " iterations.";
auto tStart = std::chrono::high_resolution_clock::now();
@@ -53,7 +53,7 @@ class BenchmarkSampler : public Device
while (!NewStatePending()) {
if (fMultipart) {
FairMQParts parts;
Parts parts;
for (size_t i = 0; i < fNumParts; ++i) {
parts.AddPart(dataOutChannel.NewMessage(fMsgSize, fair::mq::Alignment{fMsgAlignment}));
@@ -71,7 +71,7 @@ class BenchmarkSampler : public Device
++fNumIterations;
}
} else {
FairMQMessagePtr msg(dataOutChannel.NewMessage(fMsgSize, fair::mq::Alignment{fMsgAlignment}));
MessagePtr msg(dataOutChannel.NewMessage(fMsgSize, fair::mq::Alignment{fMsgAlignment}));
if (fMemSet) {
std::memset(msg->GetData(), 0, msg->GetSize());
}

Some files were not shown because too many files have changed in this diff Show More