Commit Graph

196 Commits

Author SHA1 Message Date
Dennis Klein
0ff8eaf84d Fix package dependencies 2019-03-06 14:23:08 +01:00
Dennis Klein
03912e86f8 Drop protobuf dependencies 2019-03-06 14:23:08 +01:00
Dennis Klein
4d7a1c81c6 Depend on asiofi 2019-03-06 14:23:08 +01:00
Alexey Rybalchenko
fc94342db8 Update state machine
- Split INITIALIZING state into Init+Bind+Connect
 - Remove PAUSE state
 - Convert state/transitions to enum classes (CamelCase)
 - Transition to a state only once previous handler is complete
 - Add CompleteInit transition to notify Initializing state
   that config updates are complete
 - Deprecate WaitForEndOfState(transition) in favor of
   WaitForState(state)/WaitForNextState()
 - Update tests/plugins to new APIs
 - Deprecate CheckCurrentState() in favor of NewStatePending()
2019-02-25 12:19:50 +01:00
Dennis Klein
1191c3cda5 Add PMIx plugin
Proof of concept for now.
2019-02-11 11:12:30 +01:00
Dennis Klein
e2e476ba19 Remove obsolete dependency to boost signals v1 2019-01-29 16:01:12 +01:00
mkrzewic
310b9647b5 Adopt FairMQMessage backed memory resource collection from AliceO2
Add a pmr interface to FairMQTransportFactory

refactor

Port the unit tests for MemoryResources

clang format
2018-10-31 13:12:38 +01:00
Alexey Rybalchenko
985150437a Remove shmem prototype code - unused 2018-10-08 17:20:02 +02:00
Dennis Klein
60f27b94b2 Support BUILD_SHARED_LIBS flag
https://cmake.org/cmake/help/v3.0/variable/BUILD_SHARED_LIBS.html
2018-09-19 19:09:10 +02:00
Dennis Klein
adfa0e2c95 Fix ODR violation
fairmq/shmem/Monitor.cxx:39:27: error: redefinition of ‘volatile sig_atomic_t {anonymous}::gSignalStatus’
     volatile sig_atomic_t gSignalStatus = 0;
                           ^~~~~~~~~~~~~
In file included from fairmq/cotire/FairMQ__CXX_unity_20_29.cxx:4:
fairmq/plugins/Control.cxx:20:27: note: ‘volatile sig_atomic_t {anonymous}::gSignalStatus’ previously defined here
     volatile sig_atomic_t gSignalStatus = 0;
                           ^~~~~~~~~~~~~
2018-09-19 19:09:10 +02:00
Alexey Rybalchenko
b89c309768 Fix missing header install 2018-08-30 22:54:03 +02:00
Dennis Klein
c02fbed331 Require msgpack 3.1.0
Since 3.1.0 msgpack ships a proper CMake package exporting a target
for the header-only C++ library. Allows us to get rid of our custom
find module.
2018-08-30 17:39:56 +02:00
Dennis Klein
8b3e3bbe28 Fix coverage reporting for FAST_BUILD 2018-08-09 16:15:15 +02:00
Dennis Klein
70ffc0d8c6 Guard list operation for the case the list is empty 2018-08-09 16:15:15 +02:00
Dennis Klein
ff701006fd Reflect dev version in the installed artifacts 2018-08-09 16:15:15 +02:00
Dennis Klein
c8bd19b7a1 Add experimental FAST_BUILD option
Significantly reduces compile time for the FairMQ target with
precompiled headers and unity build. For maximum improvement, use
a multi-core-aware build tool, e.g. Ninja.

Leave it undocumented for now, let's first test it internally for a while.
2018-08-09 16:15:15 +02:00
Dennis Klein
9184d5bdae clean 2018-08-09 16:15:15 +02:00
Dennis Klein
ab6fd35a86 Add header-only target for msgpack 2018-08-09 16:15:15 +02:00
Dennis Klein
1e0159b775 Link against system threads library 2018-08-08 16:13:41 +02:00
Alexey Rybalchenko
ee8afd7d2b Fix race in plugin manager/services 2018-08-08 16:13:41 +02:00
Dennis Klein
e403d18cb9 Add codecov reports to PRs 2018-07-20 17:21:23 +02:00
Dennis Klein
96e2076300 Only install public header files
Resolves #15
2018-06-08 11:25:03 +02:00
Alexey Rybalchenko
8b88e67360 Refactor FairMQProgOptions 2018-06-06 16:04:08 +02:00
Alexey Rybalchenko
e4c349888d Improve compilation speed 2018-05-17 16:32:49 +02:00
Alexey Rybalchenko
bf5983d8f6 provide location for general fairmq devices/tools 2018-05-02 16:12:57 +02:00
Alexey Rybalchenko
f68b5425a6 prefix general executables with 'fairmq-' 2018-05-02 16:12:57 +02:00
Alexey Rybalchenko
bab7e13737 Refactor the examples after move from FairRoot 2018-05-02 16:12:57 +02:00
Dennis Klein
7bb4b0625e
Depend on ZeroMQ via CMake package 2018-04-23 17:38:47 +02:00
Dennis Klein
a01aac4467
Add library versioning 2018-04-18 02:25:44 +02:00
Dennis Klein
d98207b475
Install uuidGen 2018-04-18 01:42:04 +02:00
Dennis Klein
b9651437c3
Improve summary and implement find_package2 2018-04-16 20:32:44 +02:00
Dennis Klein
5e39011a5b
Fix build 2018-04-15 18:04:52 +02:00
Dennis Klein
2327fd2115
Move test directory one up 2018-04-12 17:46:18 +02:00
Dennis Klein
ff16a28686 FairMQ: Fix regression bug introduced with PR#728 2018-03-28 09:04:30 +02:00
Dennis Klein
08581e49e1 Add BUILD_OFI_TRANSPORT build switch, default OFF 2018-03-26 13:58:20 +02:00
Dennis Klein
144aa912d7 FairMQ: Implement blocking ofi::Socket Send/Receive with FI_MSG
Completion events are not yet working.
2018-03-26 13:58:20 +02:00
Dennis Klein
5b3a5b9709 FairMQ: Implement ofi address exchange
Control messages are encoded with protobuf.
2018-03-26 13:58:20 +02:00
Dennis Klein
630a1dbbce FairMQ: Setup ofi objects
FI_MSG API
FI_EP_RDM, reliable datagram message (unconnected)
2018-03-26 13:58:20 +02:00
Dennis Klein
eaebfc6933 FairMQ: Partially implement ofi::Message 2018-03-26 13:58:20 +02:00
Dennis Klein
209e521046 FairMQ: Implement ofi::Poller 2018-03-26 13:58:20 +02:00
Dennis Klein
d9595adf45 FairMQ: Implement ofi::Socket 2018-03-26 13:58:20 +02:00
Dennis Klein
9b7841e89e FairMQ: Remove deprecated TransportFactory factory method 2018-03-26 13:58:20 +02:00
Dennis Klein
7cfd93c998 FairMQ: Add ofi transport factory skeleton 2018-03-26 13:58:20 +02:00
Alexey Rybalchenko
778c8e16bb Replace pstreams with Boost.Process 2018-01-23 18:35:09 +01:00
Alexey Rybalchenko
4e942e489b Single Logger implementation for FairLogger & FairMQLogger 2018-01-16 22:53:26 +01:00
laphecet
ac4695b215 Missing explicit reference to lib boost_signals 2017-12-06 13:04:58 +01:00
Dennis Klein
5c0178e882 FairMQ: Remove deprecated header <tools/FairMQTools.h> 2017-11-29 16:40:25 +01:00
Alexey Rybalchenko
0dc4000187 FairMQ: Add uuid generator to tools & let tests use it for session names. 2017-11-29 09:06:17 +01:00
Alexey Rybalchenko
58a312b730 FairMQ: Introduce callbacks for the FairMQUnmanagedRegion.
Callbacks are called when the data buffer of the message assiciated
with the corresponding region is no longer needed by the transport.
Example in examples/advanced/Region/
2017-11-29 09:06:17 +01:00
Alexey Rybalchenko
0bb866ff36 FairMQ: shared memory unmanaged region updates:
- use same mapping for sender/receiver (created/remote) region.
 - rename FairMQRegion -> FairMQUnmanagedRegion.
2017-10-20 14:17:15 +02:00
Alexey Rybalchenko
aa8d16ff9a Move shmem prototype out of MQ examples 2017-10-16 09:49:38 +02:00
Alexey Rybalchenko
e3cb1dd9e5 FairMQ: remove runSimpleMQStateMachine (now via control plugin). 2017-10-05 15:32:12 +02:00
Dennis Klein
4ae2e025c9 FairMQ: Implement DeviceRunner 2017-10-05 15:32:12 +02:00
Dennis Klein
2589ca5ced FairMQ: Rewrite the state machine
* Simplify the code
* Drop Boost.MSM dependency
* Drop threaded execution
* Support deferred state changes
* Monitor state changes/queues via callbacks
2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
179968db1e Remove FairMQConfigurable 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
6d7009b331 Refactor FairMQLogger 2017-10-05 15:32:12 +02:00
Dennis Klein
6ecd0e9085 FairMQ: Rewrite event manager to support multiple subscribers 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
01327426c3 Port DDS plugin to the new plugin system. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
70e46a0b86 Cleanup base/MQ. 2017-10-05 15:32:12 +02:00
Dennis Klein
10f67e4c72 FairMQ: Move static and interactive control modes to plugin
NOT YET FINISHED
2017-10-05 15:32:12 +02:00
Dennis Klein
052ac8487d FairMQ: Add static plugin mechanism
* Add skeleton for Take/ReleaseControl API
* Add skeleton for control_static builtin plugin
2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
221d2567f0 FairMQRegion and examples/advanced/Region 2017-08-08 14:18:10 +02:00
Dennis Klein
a26925cbf5 FairMQ: Fix various errors from CI
* move Plugins::Version to fair::mq::tools
* fix Parser interface
* make device a shared pointer in main
* provide std::hash template specialization
* fix FairMQ.Plugins/FairMQ.PluginsStatic when run with ctest
* fix MQ/serialization example
* add --no-as-needed flag
* GCC 4 does not support member refs, move to pointer types
2017-07-04 07:05:11 +02:00
Dennis Klein
ad0f050c99 FairMQ: Implement PluginServices - Config 2017-07-04 07:05:11 +02:00
Dennis Klein
9b61b924b2 FairMQ: Integration of Plugins through PluginServices 2017-07-04 07:05:11 +02:00
Dennis Klein
60d929b0bd FairMQ: Add plugin mechanism (Plugin and PluginManager classes) 2017-07-04 07:05:11 +02:00
Dennis Klein
d2c78479f0 Refactor FairMQTools
* Split header, add common header
  * Transform to canonical namespace
  * Deprecate old header
  * Adapt to new Tools.h
2017-05-23 13:42:56 +02:00
Alexey Rybalchenko
2a526b8625 shmem: introduce FairMQShmMonitor. 2017-04-28 13:30:51 +02:00
Dennis Klein
bf2b208749 update docs 2017-04-20 11:07:49 +02:00
Alexey Rybalchenko
5aaf27bf02 Refactor the transport interface
- give transport Initialize() method with access to device config.
 - avoid using global context in the transport.
 - simplify shutdown procedure (no need for extra thread).
2017-04-20 11:07:49 +02:00
Dennis Klein
d7eb692951 remove no longer needed special treatment for logger directory 2017-04-20 11:07:49 +02:00
Dennis Klein
9b337ca075 remove redundant include path
The imported ZeroMQ target has a target property
INTERFACE_INCLUDE_DIRECTORIES which carries this
information.
2017-04-20 11:07:49 +02:00
Dennis Klein
d02ae9be2c remove no longer used variable 2017-04-20 11:07:48 +02:00
Giulio Eulisse
1ebbe80539 Add ZeroMQ include path 2017-04-20 11:07:48 +02:00
Dennis Klein
c41cf318bd export FairMQ targets and install headers hierarchically 2017-04-20 11:07:47 +02:00
Dennis Klein
35b3212f7e cmake cleanups and fixes
* fix overlinking of libFairMQ
* import dependencies in find modules
* provide FairMQinstall target
* add gtest helper module
* add cotire helper module
* add many comments
* cleanup fairmq cmake file
* add FindPStreams module
* link logger into libFairMQ
* drop root dictionary generation
* bump min required cmake version to 3.7.2
2017-04-20 11:07:47 +02:00
Matthias Richter
e184610c06 Adding parser implementation for key-value subopt format
A parser implementation for FairMQ channel properties.
The parser handles a comma separated key=value list format by using the
getsubopt function of the standard library.

The option key '--channel-config' can be used with the list of key/value
pairs like e.g.
--channel-config name=output,type=push,method=bind
2017-04-20 11:07:46 +02:00
Dennis Klein
b43d4c142d refactor and integrate with aliBuild/alienv
* accept ZEROMQ_ROOT as env or cmake variable to hint at install
    location (used by aliBuild/alienv)
  * added more docs
  * fixed mismatch between module name ZeroMQ and prefix of
    generated variables ZMQ -> ZeroMQ is now the preferred prefix
    (old variables are still available and marked deprecated)
  * the aliBuild zeromq recipe does not install zmq.hpp header,
    so the module searches now for the zmq.h header
    (which is anyways the only one we use)
  * some cosmetic changes
  * add option ZeroMQ_NO_DEPRECATION to suppress deprecation warning
2017-02-23 19:42:15 +01:00
Alexey Rybalchenko
c66fd6fe91 Adding multiple transports support & other fixes:
- Avoid polling when only one input channel is used.
 - Send only handles for shared memory transport.
 - Avoid waiting in the rate logger thread when nothing to log.
 - Hide warnings from generated files
 - Fix #483
2017-02-23 06:47:09 +01:00
Alexey Rybalchenko
a332d9fc83 First version of the shared memory transport.
Use via `--transport shmem` cmd option. No pub/sub.
2016-12-18 14:50:58 +01:00
Alexey Rybalchenko
b166cedb63 Convert factory methods to return smart ptrs
- Convert factory methods to return smart ptrs.
 - Refactor state machine to use same thread for user states.
 - Remove unused includes and dependencies, use std.
2016-11-18 14:19:16 +01:00
Alexey Rybalchenko
da3010b20c Move config & control DDS functionality into plugins. 2016-10-28 14:14:13 +02:00
Alexey Rybalchenko
16fd63cd5b Enable new callback API
- OnData() channel data handler.
 - ConditionalRun() for devices without incoming data.
 - Header file with common main(), to be extended with getDevice/addCustomOptions.
 - Update examples (MQ/Tutorial3) to use the new API and config.
 - NewSimpleMessage() for simpler creation of small messages (additional copy).
 - Replace SetProperty/GetProperty with fConfig access.
 - Runtime configurable channel names for common devices.
 - Configurable logging interval per channel.
 - FairMQMultiplier for distributing same data to multiple outputs.
 - Cleanup state machine messages.
 - Cmd option to toggle signal handling.
 - Simpler API for send/receive timeouts.
 - Enable --log-to-file.
 - Fix coverity issues, warnings.
 - Various code cleanup and minor tweaks.
2016-09-30 14:36:35 +02:00
winckler
e0a03242ac Simplify logger filtering options, fix log error double printing issue 2016-09-19 11:20:22 +02:00
winckler
64f1fd2a3b - add test for serialization examples 1&2
- rename serialization files and scripts
- remove policy and host header files from fairmq installation
2016-07-01 13:05:37 +02:00
Florian Uhlig
92944fe322 Correct shell scripts to work correctly with dash.
Add more dependencies for fairmq.

Something does not work with rpath when using boost 1.61. The problem is only seen for opensuse.
2016-06-03 15:45:18 +02:00
Radoslaw Karabowicz
fd09321fd9 Modified runDDSCommandUI to accept command line argument.
If no arguments, it behaves like before.
Removed runDDSCommand and replaced calls to it in controlDDS.
2016-05-30 11:28:43 +02:00
Radoslaw Karabowicz
65ca151620 MQ/example9 Adapted to the new running scheme with DDS.
- removed the executables that are running only in DDS (ddsEx9Sampler.cxx,
ddsEx9TaskProcessor.cxx, ddsEx9FileSink.cxx, ddsParameterMQServer.cxx);
- adapted the executables to allow running within the DDS (setting channel names);
- changed the topology file to run the correct executables (ex9-dds-topology.xml);
- ex9-dds.json - changed the parameter channel name back to "data" (to be compatible with other examples);
- added a shell script to controlDDS.sh;

- added fairmq/fairmq-dds-command executable to send single commands to DDS;
- adapted parmq/parmq-server to run within DDS.
2016-05-30 11:28:42 +02:00
Alexey Rybalchenko
253604344f nanomsg: reset maximum receive msg size 2016-05-27 12:58:07 +02:00
Alexey Rybalchenko
a05dc80402 Refactor DDS example and tools to be able to run with/without DDS 2016-05-20 14:09:53 +02:00
Alexey Rybalchenko
b9883d3b13 Configuration and DDS example/tools updates
- Update DDS example command UI and extract it from example.
 - Unify address handling via DDS properties for dynamic deployment.
 - Update DDS docs with the new approach.
 - Allow `--config-key` to be used to access common config in JSON.
 - Allow common channel properties to be specified for all sockets.
 - Update MQ examples and Tuto3 with new config options.
 - Add start scripts to MQ examples for easier use.
2016-04-25 17:35:57 +02:00
Mohammad Al-Turany
9a340fb7f6 Use Variables for Boost libraries instead of explicit names
Use the vairables filled by FindBoost instead explicit names in the cmake files
2016-03-29 15:31:29 +02:00
Alexey Rybalchenko
732373faa2 Update multi-part features (nanomsg) and various fixes
- Implement nanomsg multipart with MessagePack.
 - Use the MessagePack from FairSoft and handle not found case.
 - Update splitter, merger and proxy devices to handle multi-part.
 - Let FairMQParts.At() return pointer reference (can be used for moving).
 - Add missing const specifier in the message interface.
 - Add transmit kernel size setting to channels (ZMQ_SNDBUF).
 - Remove FairMQBuffer device.
 - Remove old multi-part methods from Tutorial3 example (to be replaced with Parts API).
 - Make callback mandatory for newMsg(data, size, callback).
 - Add missing <vector> include in FairMQSocket.
2016-03-24 13:36:47 +01:00
Alexey Rybalchenko
e1fef82657 FairMQ: Extend Multipart and messaging API
- Extend the multipart API to allow sending vectors of messages or helper
   thin wrapper FairMQParts. See example in examples/MQ/8-multipart.
 - NewMessage() can be used in devices instead of
   fTransportFactory->CreateMessage().
   Possible arguments remain unchanged (no args, size or data+size).
 - Send()/Receive() methods can be used in devices instead of
   fChannels.at("chan").at(i).Send()/Receive():
   Send(msg, "chan", i = 0), Receive(msg, "chan", i = 0).
 - Use the new methods in MQ examples and tests.
 - No breaking changes, but FAIRMQ_INTERFACE_VERSION is incremented to 3
   to allow to check for new methods.
2016-02-29 16:25:39 +01:00
Alexey Rybalchenko
f1abb9ecdd Remove compile time transport interface switch
- Remove the compile time check of the transport implementation.
  The transport (zeromq/nanomsg) can be chosen at run time with:
  `device.SetTransport("zeromq"); // possible values are "zeromq" and "nanomsg"`.

  For devices that use FairMQProgOptions, the transport can be configured via cmd option:
  `--transport zeromq` or `--transport nanomsg`. Default values is "zeromq".
  The device receives the configured value with:
  `device.SetTransport(config.GetValue<std::string>("transport"));`

  Old method of setting transport still works. But the NANOMSG constant is not defined.

- Remove old `fairmq/prototest` directory. It was only used as a test for protobuf.
  The protobuf part of Tutorial3 does the same (with different values).

- Fix a bug in FairMQPollerNN, where the `revents` value was not initialized.
  This caused the `poller->CheckOutput()` to trigger when it should not.
2016-01-14 15:19:30 +01:00
NicolasWinckler
8f32579984 add a pointer data type serializer function in the boost serializer add the zmq and nanomsg directories in cmaklists that use the runSimpleMQStateMachine 2015-11-20 09:03:34 +01:00
Florian Uhlig
0894900f52 Add missing link dependencies. 2015-11-19 15:51:44 +01:00
Alexey Rybalchenko
383a220333 Update device configuration
- Move general config files out of example directory to fairmq/run.
 - Use FairMQProgOptions for MQ example 5.
 - Add SendPartAsync() for non-blocking send of a message part.
2015-11-11 11:09:42 +01:00
NicolasWinckler
837490cc38 add FairProgOptionsHelper.h in header list to be installed of fairmq/CMakeLists.txt 2015-11-09 14:17:44 +01:00
Alexey Rybalchenko
d0c20d3729 Rename /example to /examples and move MQ examples in it 2015-11-09 10:36:24 +01:00