Commit Graph

142 Commits

Author SHA1 Message Date
Dennis Klein
44a59f25a7 FairMQ: Move --catch-signals logic to control plugin
* Add StealDeviceControl() API to plugin services
2017-10-05 15:32:12 +02:00
Dennis Klein
7dcd09692c FairMQ: Move static and interactive control modes to plugin (3) 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
01327426c3 Port DDS plugin to the new plugin system. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
334b91785b FairMQ: Move static and interactive control modes to plugin (2) 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
4dd08051cc simplify state change callback handling 2017-08-30 08:38:00 +02:00
Alexey Rybalchenko
eeb994a584 Tests for MQ examples 2017-08-28 10:53:41 +02:00
Alexey Rybalchenko
221d2567f0 FairMQRegion and examples/advanced/Region 2017-08-08 14:18:10 +02:00
Alexey Rybalchenko
65f1b96dc3 Add Version member to FairMQDevice, settable via constructor 2017-07-17 13:25:16 +02:00
Alexey Rybalchenko
6f18cf23da Merge channels from config to device, instead of replace 2017-07-04 07:13:01 +02:00
Dennis Klein
a26925cbf5 FairMQ: Fix various errors from CI
* move Plugins::Version to fair::mq::tools
* fix Parser interface
* make device a shared pointer in main
* provide std::hash template specialization
* fix FairMQ.Plugins/FairMQ.PluginsStatic when run with ctest
* fix MQ/serialization example
* add --no-as-needed flag
* GCC 4 does not support member refs, move to pointer types
2017-07-04 07:05:11 +02:00
Dennis Klein
739460b2fe FairMQ: Implement PluginServices - Control 2017-07-04 07:05:11 +02:00
Dennis Klein
9b61b924b2 FairMQ: Integration of Plugins through PluginServices 2017-07-04 07:05:11 +02:00
Alexey Rybalchenko
b47cc8b29c add callback on state change 2017-07-04 07:05:11 +02:00
Alexey Rybalchenko
035d5fb353 Provide accessors for device members that are part of publiv API 2017-06-23 11:45:46 +02:00
Alexey Rybalchenko
4bc54ad32b add --print-channels to print registered channels of the device 2017-06-23 11:45:46 +02:00
Alexey Rybalchenko
f4e5a74f23 Always have access to FairMQProgOptions from device. 2017-06-04 01:06:12 +02:00
Dennis Klein
8bc21675af Refactor TransportFactory to RAII
* Remove explicit Initialize and Terminate states,
    map them onto ctor/dtor
  * Remove no longer needed Shutdown state
  * Remove deprecated SetTransport()
2017-05-24 16:37:33 +02:00
Dennis Klein
d2c78479f0 Refactor FairMQTools
* Split header, add common header
  * Transform to canonical namespace
  * Deprecate old header
  * Adapt to new Tools.h
2017-05-23 13:42:56 +02:00
Alexey Rybalchenko
6b221d950c Add NewPoller() wrapper. 2017-05-23 08:32:13 +02:00
Dennis Klein
3be2f297f3 generate id if no device id available
* CreateSocket factory no longer accepts id param, but the
    TransportFactory has an id member instead
2017-05-18 20:01:44 +02:00
Dennis Klein
3205e0c378 refactor to more idiomatic RAII
* FairMQTransportFactoryZMQ: move the config invariant
    initialization to ctor
  * FairMQChannel: add new ctor that creates usable channel
  * FairMQSocket*: close sockets in dtor
  * FairMQTransportFactory*: terminate context in dtor
  * FairMQChannel: add Bind/Connect facades (for explicit control, e.g. timing)
2017-05-18 20:01:44 +02:00
Dennis Klein
97ca52aa0e move TransportFactory factory into FairMQTransportFactory namespace 2017-05-18 20:01:44 +02:00
Dennis Klein
ec786dce03 Fix CID 169525 Data race condition 2017-05-17 18:44:51 +02:00
Alexey Rybalchenko
1fa5c836a6 FairMQ: stop device if binding was unsuccessful (not updated dynamically). 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
459b42add2 FairMQ: MakeTransport to create stand-alone transport for tests. 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
541538e3de only reset sockets if needed 2017-04-20 11:07:50 +02:00
Dennis Klein
f963a06e25 make connecting channel wait loop more flexible 2017-04-20 11:07:50 +02:00
Alexey Rybalchenko
5aaf27bf02 Refactor the transport interface
- give transport Initialize() method with access to device config.
 - avoid using global context in the transport.
 - simplify shutdown procedure (no need for extra thread).
2017-04-20 11:07:49 +02:00
Dennis Klein
365a68010f various fixes
* fix include paths
* print on stderr
2017-04-20 11:07:47 +02:00
Alexey Rybalchenko
b237b3f9d7 Remove unused code.
- Remove unused code in FairMQTools.
- Remove obsolete device options (now in device config).
2017-03-14 17:51:26 +01:00
Alexey Rybalchenko
91b7a72ac5 get default network interface from the default route 2017-03-02 13:25:06 +01:00
Alexey Rybalchenko
c66fd6fe91 Adding multiple transports support & other fixes:
- Avoid polling when only one input channel is used.
 - Send only handles for shared memory transport.
 - Avoid waiting in the rate logger thread when nothing to log.
 - Hide warnings from generated files
 - Fix #483
2017-02-23 06:47:09 +01:00
Alexey Rybalchenko
65c19f07bf Fix compatibility with ZeroMQ 4.2.1 2017-02-09 12:19:23 +01:00
Alexey Rybalchenko
a332d9fc83 First version of the shared memory transport.
Use via `--transport shmem` cmd option. No pub/sub.
2016-12-18 14:50:58 +01:00
Alexey Rybalchenko
6c3b01f09c AttachChannel must update the config after it's done, use boost::split 2016-12-14 08:52:05 +01:00
Alexey Rybalchenko
b166cedb63 Convert factory methods to return smart ptrs
- Convert factory methods to return smart ptrs.
 - Refactor state machine to use same thread for user states.
 - Remove unused includes and dependencies, use std.
2016-11-18 14:19:16 +01:00
mkrzewic
94a4d599eb Validate connection method per endpoint, not socket
this allows e.g. to ommit the socket method in config if all endpoints
specify the method modifier.
It still is fully backward compatible.
2016-11-13 14:22:51 +01:00
mkrzewic
c2d7c49cf5 Support multiple endpoints per socket
Sent messages will be scheduled among the endpoints according to socket
type: PUB will send the same data to all endpoints simultaneously, PUSH
will do round robin transfer.
Incoming data is fair queued between endpoints.

This is a feature of at least zeromq and nanomsg.

_____________
To use: in the device configuration, instead of specifying just one address,
specify a comma separated list e.g.

tcp://localhost:123,ipc:///tmp/socket

the connection method (bind/connect) applies to all endpoints in this case.
______________
Mixing binding and connecting endpoints is supported:
prefix "@" means "bind", "+" (or ">") means connect, e.g.

+tcp://localhost:123,@ipc:///tmp/socket,ipc:///tmp/asd

(in case of missing prefix, the default channel method is used for that
endpoint).
2016-11-11 16:58:55 +01:00
Alexey Rybalchenko
da3010b20c Move config & control DDS functionality into plugins. 2016-10-28 14:14:13 +02:00
Alexey Rybalchenko
16fd63cd5b Enable new callback API
- OnData() channel data handler.
 - ConditionalRun() for devices without incoming data.
 - Header file with common main(), to be extended with getDevice/addCustomOptions.
 - Update examples (MQ/Tutorial3) to use the new API and config.
 - NewSimpleMessage() for simpler creation of small messages (additional copy).
 - Replace SetProperty/GetProperty with fConfig access.
 - Runtime configurable channel names for common devices.
 - Configurable logging interval per channel.
 - FairMQMultiplier for distributing same data to multiple outputs.
 - Cleanup state machine messages.
 - Cmd option to toggle signal handling.
 - Simpler API for send/receive timeouts.
 - Enable --log-to-file.
 - Fix coverity issues, warnings.
 - Various code cleanup and minor tweaks.
2016-09-30 14:36:35 +02:00
Alexey Rybalchenko
5e5ddd5b7b Fix further effc++ and reorder warnings. 2016-09-19 11:10:46 +02:00
Alexey Rybalchenko
7c99bd77cc Fix unused variables, init order and effc++ warnings. 2016-09-06 10:19:23 +02:00
Alexey Rybalchenko
4944d7ec41 Meaningful error on wrong channel name 2016-05-20 14:10:14 +02:00
Alexey Rybalchenko
e9f09143b2 Gracefully end the device also in the termination case (instead of abort). 2016-05-17 12:27:18 +02:00
Alexey Rybalchenko
ce42f36126 Remove redundant state change call and revert the termination handler. 2016-05-12 17:28:33 +02:00
Alexey Rybalchenko
3353e214a7 Add shared memory example
- Add shared memory example in examples/MQ/SharedMemory
 - Device/Task termination: try soft first, and abort if it fails
 - Interactive mode: prevent cin from blocking forever (poll)
2016-05-11 14:52:53 +02:00
Alexey Rybalchenko
5421922668 Add a cmd option to control state change mechanism 2016-04-27 15:04:29 +02:00
Alexey Rybalchenko
b9883d3b13 Configuration and DDS example/tools updates
- Update DDS example command UI and extract it from example.
 - Unify address handling via DDS properties for dynamic deployment.
 - Update DDS docs with the new approach.
 - Allow `--config-key` to be used to access common config in JSON.
 - Allow common channel properties to be specified for all sockets.
 - Update MQ examples and Tuto3 with new config options.
 - Add start scripts to MQ examples for easier use.
2016-04-25 17:35:57 +02:00
Florian Uhlig
fa7040fe65 Fix compiler warnings.
Initialize all data members in initializer lists.
Reorder data members in initializer list to have the same order as in the class declaration.
Comment or remove unused parameters and unused variables.
Convert all old style casts to the correct and explicit c++ cast like const_cast, static_cast,
dynamic_cast or reinterpret_cast. In most cases static_cast is used.
2016-03-14 13:37:55 +01:00
Alexey Rybalchenko
c42b6ca4ae Include device ID in the zeromq socket identity.
For request sockets in ZeroMQ the socket identity must be unique, otherwise multiple clients will be rejected.
Update the tests to test this use case.
2016-03-03 13:10:56 +01:00
Alexey Rybalchenko
e1fef82657 FairMQ: Extend Multipart and messaging API
- Extend the multipart API to allow sending vectors of messages or helper
   thin wrapper FairMQParts. See example in examples/MQ/8-multipart.
 - NewMessage() can be used in devices instead of
   fTransportFactory->CreateMessage().
   Possible arguments remain unchanged (no args, size or data+size).
 - Send()/Receive() methods can be used in devices instead of
   fChannels.at("chan").at(i).Send()/Receive():
   Send(msg, "chan", i = 0), Receive(msg, "chan", i = 0).
 - Use the new methods in MQ examples and tests.
 - No breaking changes, but FAIRMQ_INTERFACE_VERSION is incremented to 3
   to allow to check for new methods.
2016-02-29 16:25:39 +01:00
Alexey Rybalchenko
82ab7670a9 Add FlatBuffers & MessagePack examples
- Add FlatBuffers serialization example to Tutorial 3
 - Add MessagePack serialization example to Tutorial 3
 - Performance improvements in Boost serialization example
 - Use `GetEntriesFast()` for FairTestDetectorRecoTask
 - Use `Clear()` instead of `Delete()` in MQ parts of Tutorial 3
 - Fix CMake variables from preventing compilation without nanomsg.
 - create macro/data directories in install directory
 - Get rid of data duplication in fill_parameters.C
 - Various cleanups and fixes
2016-02-22 13:06:37 +01:00
Alexey Rybalchenko
f1abb9ecdd Remove compile time transport interface switch
- Remove the compile time check of the transport implementation.
  The transport (zeromq/nanomsg) can be chosen at run time with:
  `device.SetTransport("zeromq"); // possible values are "zeromq" and "nanomsg"`.

  For devices that use FairMQProgOptions, the transport can be configured via cmd option:
  `--transport zeromq` or `--transport nanomsg`. Default values is "zeromq".
  The device receives the configured value with:
  `device.SetTransport(config.GetValue<std::string>("transport"));`

  Old method of setting transport still works. But the NANOMSG constant is not defined.

- Remove old `fairmq/prototest` directory. It was only used as a test for protobuf.
  The protobuf part of Tutorial3 does the same (with different values).

- Fix a bug in FairMQPollerNN, where the `revents` value was not initialized.
  This caused the `poller->CheckOutput()` to trigger when it should not.
2016-01-14 15:19:30 +01:00
Alexey Rybalchenko
ada23500be Return if an unknown exception has been cought in Channel 2015-11-03 16:07:15 +01:00
Alexey Rybalchenko
8f59db1283 Add methods to set timeout on blocking Send/Receive 2015-10-19 09:52:53 +02:00
Alexey Rybalchenko
91d3f013c0 Fix setting of ZeroMQ I/O Threads (setting was not propagated before) 2015-10-12 13:51:09 +02:00
Alexey Rybalchenko
19afacb504 FairMQ Examples cleanup
- Rename Tutorial3 MQ files for uniform naming.
  - Add search for dylib in FindDDS.cmake (OSX).
  - Add more detail to the DDS example readme.
  - MQ Example 3 (DDS): choose network interface via command line option.
  - Give FairMQ examples their own CMakeLists.txt for clarity.
  - Remove C++11 checks in Tutorial3 from the code (they are now in CMake).
  - Add Serializer for device properties (FairMQDevice::ListProperties()).
2015-10-05 18:06:55 +02:00
Alexey Rybalchenko
5136c88d3a Simplify Tutorial3 structure.
Use same executable for bin/boost/protobuf/root data format,
configured now via `--data-format <binary/boost/protobuf/tmessage>`
command line parameter.
2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
8b71e4d20b Use FairMQDevice::CatchSignals for Tutorial7 2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
fbf7dbf2ba Add orthogonal OK/ERROR states.
Replace state check mutex with atomic.

Update DDS example documentation.
2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
a7ab33a10e Add new Send/Receive methods with smart pointers and no flag checks. 2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
105e734808 Add DDS and Copy+Push examples. 2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
1302e77a16 Several FairMQ fixes and improvements:
- FairMQ: add possibility to poll on multiple channels.
- FairMQ: include command channel when polling on blocking calls (for unblocking without termination).
- FairMQ: move signal handler inside of FairMQDevice class (call FairMQDevice::CatchSignals() in the main function).
- FairMQ: add 'bool CheckCurrentState(statename)' (instead of 'GetCurrentState() == statename' that cannot be thread safe).
- FairMQDevice: add 'InteractiveStateLoop()' method that can be used to change states from the command line.
- FairMQDevice: add automatic transition to IDLE state if Run() exits without an external event.
- FairMQDevice: implement device reset.
- FairMQDevice: use unordered_map for device channels.
- FairMQChannel: improve address validation for channels.
- FairMQChannel: add ExpectsAnotherPart() method to check if another msg part is expected (old approach still works).
- FairMQ: remove invalid transition from the run files.
- FairMQFileSink: disable ROOT termination signal handler.
- Tutorial3: spawn xterm windows from start scripts without overlapping for better visibility.
- FairMQ Examples: update protobuf test and move its files to a common directory.
- FairMQStateMachine: improve feedback on invalid transitions (more readable).
2015-09-28 12:17:22 +02:00
Alexey Rybalchenko
5f20a28b04 Correct mistake in channel validation 2015-06-19 14:49:32 +02:00
Alexey Rybalchenko
252acf41c8 Add WaitForInitialValidation() 2015-06-19 12:33:10 +02:00
Alexey Rybalchenko
d6a413534a Thread safe initialization of the channels 2015-06-19 12:32:56 +02:00
Alexey Rybalchenko
295d9bba57 add WaitForEndOfStateForMs(state, duration) 2015-06-17 16:01:11 +02:00
Alexey Rybalchenko
a3d919b763 Use FairMQProgOptions for Sink and BenchmarkSampler 2015-06-16 12:43:23 +02:00
Alexey Rybalchenko
7fda980710 Update FairMQStateMachine & introduce FairMQChannels
Organize sockets as a map of vectors of FairMQChannels.

Update FairMQStateMachine by removing SETTINGINPUT, SETTINGOUTPUT,
BIND and CONNECT states and by adding INITIALIZING_TASK, RESETTING_TASK
and RESETTING_DEVICE states. Run states functions in their own thread.
2015-06-16 09:29:53 +02:00
Alexey Rybalchenko
26a1033e9d Rename device property to have a common format and some code cleanup.
FairMQDevice: Rename property for socket rate logging to have common format (old name is still available for compatibility).
FairMQ: Avoid using std namespace in class headers (may require adding std namespace to some child devices).
FairMQ: A bit of code cleanup
FairMQConfigurable: Stop with an error if a property assignment failed due to incorrect key.
2015-05-28 09:32:30 +02:00
Alexey Rybalchenko
61f24eb73a Increase maximum number of zeromq sockets and improve output on errors.
FairMQSampler: Modified: comment out an unused loop.
FairMQ: Modified: increase maximum number of ZeroMQ sockets (from default 1024).
FairMQ: Modified: reduce amount of output when finding port from a range.
FairMQ: Modified: stop the Device when socket creation fails and output an error.
2015-05-28 09:32:27 +02:00
Alexey Rybalchenko
8a82afe184 Expose BIND and CONNECT states for use with dynamic configuration
introduce FairMQ interface version
2015-02-11 15:15:23 +01:00
Alexey Rybalchenko
ce58ee2302 Fix warnings produced with -Weffc++ in fairmq, base/MQ and Tutorial3.
Remaining warnings originate from boost::msm and boost::mpl.
Also, warnings with missing initializer list entries for DeviceOptions are false positives, since those are PODs.
2015-01-21 16:19:16 +01:00
Alexey Rybalchenko
e4e88ad1db Allow disabling transfer rate logging per input/output (optional, default is on).
This commit introduces new property to FairMQDevice to control whether rateLogger thread should print rates for a given input/output.
Per default logging is enabled, so existing code works as before and does not need to be updated.
To turn off logging for an initialized input/output, set the `LogInputRate`/`LogOutputRate` property to `0`.

Example:
``` c++
splitter.ChangeState(FairMQSplitter::INIT);
// turn off rate logging for input 1 and output 4
splitter.SetProperty(FairMQSplitter::LogInputRate, 0, 1);
splitter.SetProperty(FairMQSplitter::LogOutputRate, 0, 4);
```
2014-11-13 11:14:57 +01:00
Alexey Rybalchenko
0cef5692b1 Use boost::program_options for managing command line options of the executables.
Existing scripts in example/Tutorial3/macro have been updated to use the new format.
Your own executables are not affected, but your scripts which use FairMQ executables have to be updated to the new format.
Use the `--help` option with any FairMQ executable to find out the available options.
2014-11-04 11:50:57 +01:00
Alexey Rybalchenko
0a610926a1 - Proper process termination:
if interrupted with CTRL+C blocking socket calls will return with -1. Each device should call FairMQDevice::Shutdown() before ending the running state to close open sockets, otherwise the interrupt call itself will block.

- FIX: Update number of received messages for FairMQFileSink.
- Add ability to poll on outputs for FairMQPoller.
2014-08-20 13:48:51 +02:00
Alexey Rybalchenko
281fcc459c Fix CIDs 10587, 10813, 10911, 10912, 10402, 10403, 10577, 10578, 10579, 10848, 10861, 10865, 10868, 10910.
Move classes inheriting from device to a subdirectory.
Make sure only protobuf library installed by fairsoft is used.
Cleanup FairMQDevice and fix some initialization list warnings.
Loop to duplicate input files in Sampler.
Add some documentation to FairMQ.
2014-07-28 14:04:20 +02:00
Alexey Rybalchenko
d14d96ea70 fix CID 10112
fix CIDs 10922, 10921

fix CIDs 10369, 10371, 10372, 10373, 10374

fix CID 10406

fix CID 10414

fix CIDs 10850-52, 10857, 10859-60, 10863, 10869-76

fix CIDs 10014 10080
2014-07-04 13:24:44 +02:00
Florian Uhlig
865c0e010f Add license file LICENSE with LPGL license text copied from https://www.gnu.org/licenses/lgpl.html. Add license text to most of files of the project.
Add license text to all header files

Add license text to all c++ source files

Add license text to all C macro files

Converting file to UNIX format.

Add license text to all build system files

Add license text to all c source files

Rename file LICENCE to LICENSE. Change preamble in all files.
2014-06-06 14:57:56 +02:00
Alexey Rybalchenko
c03d4ad6d5 use clang-format for Tutorial3 2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
08e603911d TMessage transport implemented! 2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
68d51d8ed5 use clang-format for FairMQ 2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
e80e6d4269 add support for I/O threads to the interface (only used with ZeroMQ). 2014-05-27 19:01:49 +02:00
Alexey Rybalchenko
adc7443aa5 Allow easier testing without having to recompile for use of binary/boost/protobuf payload.
to use different data transfer method, just provide a parameter to the script, e.g.:
./startAll bin
./startAll boost
./startAll proto
etc.

if no or incorrect parameters is provided, binary method will be used!
protobuf method works only if the library is available on the system, otherwise it is not compiled.
2014-05-08 12:56:30 +02:00
NicolasWinckler
005321f63c add boost::serialization and template features in tutorial3 and fairmq. Require to modifie base/event/fairtimestamp and fairhit. 2014-05-08 12:56:29 +02:00
Alexey Rybalchenko
9a0ed2ca72 Add (optional) Google Protocol Buffers support (example in Tutorial 3).
To use protobuf, run cmake as follows:
cmake -DUSE_PROTOBUF=1 ..
For this, protobuf library has to be installed on the system.

Further changes:
Clean up splitter/merger: default are N-to-1-merger and 1-to-N-splitter.
Fix bug in nanomsg message deallocation.
Setup proper buffer sizes for nanomsg/zeromq via cmake/bash script.
chmod +x for start scripts.
2014-03-17 12:21:47 +01:00
Alexey Rybalchenko
ac8bbf154a simplify logger for shorter calls and readability 2014-02-13 15:55:25 +01:00
Alexey Rybalchenko
64b9e991c3 add nanomsg implementations + use factory for nanomsg + lots of small stuff 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
88fee245b8 use factory for sockets 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
3803a3d155 a little clean up 2014-01-28 11:56:08 +01:00
Mohammad Al-Turany
5121fe3ae5 merge the trunk with the development of ZeroMQ branch
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@22451 0381ead4-6506-0410-b988-94b70fbc4730
2013-10-25 12:42:48 +00:00
Mohammad Al-Turany
231c7c8f7e Zero MQ implimentation and example (Tutorial3)
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20162 0381ead4-6506-0410-b988-94b70fbc4730
2013-06-07 08:07:48 +00:00