Commit Graph

317 Commits

Author SHA1 Message Date
winckler
445ef356d1 - Rename some function members of Generic MQ Devices and propagate the modifications accordingly.
- Add a generic merger.

add SYSTEM_INCLUDE_DIRECTORIES in project_template CMakeLists.txt so that boost include dirs are found.
2015-05-28 09:32:30 +02:00
Alexey Rybalchenko
26a1033e9d Rename device property to have a common format and some code cleanup.
FairMQDevice: Rename property for socket rate logging to have common format (old name is still available for compatibility).
FairMQ: Avoid using std namespace in class headers (may require adding std namespace to some child devices).
FairMQ: A bit of code cleanup
FairMQConfigurable: Stop with an error if a property assignment failed due to incorrect key.
2015-05-28 09:32:30 +02:00
Alexey Rybalchenko
61f24eb73a Increase maximum number of zeromq sockets and improve output on errors.
FairMQSampler: Modified: comment out an unused loop.
FairMQ: Modified: increase maximum number of ZeroMQ sockets (from default 1024).
FairMQ: Modified: reduce amount of output when finding port from a range.
FairMQ: Modified: stop the Device when socket creation fails and output an error.
2015-05-28 09:32:27 +02:00
Alexey Rybalchenko
8a82afe184 Expose BIND and CONNECT states for use with dynamic configuration
introduce FairMQ interface version
2015-02-11 15:15:23 +01:00
winckler
6d65c4313a Split serializer policies into serializer and deserializer policies, and change serializer method names message() to SerializeMsg() and DeserializeMsg()
Improve boost serializer performance
2015-02-11 11:13:45 +01:00
Alexey Rybalchenko
6518b7cd41 FairMQSocket: add versions of Send/Receive methods with int flags instead of string, which is more flexible and performant.
Int flags are mapped to their ZeroMQ/nanomsg versions behind the transport interface.
Methods with string flags are kept for backwards compatibility.
2015-01-28 20:01:18 +01:00
Alexey Rybalchenko
a9b7e8866c add helper function to find IPs of the node 2015-01-28 19:57:59 +01:00
winckler
bfa26e857d re-organize base/MQ directory, change library name FairMQSampler to BaseMQ, and link tutorial 3,6,7 accordingly.
Functions in has_boostSerialization.h are now in baseMQtools.h. New template header files (.h and .tpl) are copied to proper directory in case of FairRoot external installation
Remove obsolete files in tutorial3.
2015-01-21 16:19:17 +01:00
NicolasWinckler
3e424354e7 add 1) generic MQ-devices (Sampler, Processor, and FileSink) in fairmq, 2) policy classes in base/MQ and 3) a Tutorial7 in example 2015-01-21 16:19:16 +01:00
Alexey Rybalchenko
eb33bc8280 add constructor to DeviceOptions 2015-01-21 16:19:16 +01:00
Alexey Rybalchenko
ce58ee2302 Fix warnings produced with -Weffc++ in fairmq, base/MQ and Tutorial3.
Remaining warnings originate from boost::msm and boost::mpl.
Also, warnings with missing initializer list entries for DeviceOptions are false positives, since those are PODs.
2015-01-21 16:19:16 +01:00
Florian Uhlig
a8854d36ac Get rid of warnings which come from include files which doesn't belong to FairRoot. These are for example Boost or ROOT includes. Define in all CMakeLists.txt files such libraries as system libraries. In this case the compiler will not produce any warning. This change reduces the nightly test time on some machines from more then hour to 1 minute. The reason for this long compile time was the enormous amount of warnings produced before. The output of the warnings sum up to roughly 850 MB. 2015-01-21 16:19:14 +01:00
Alexey Rybalchenko
65d91b8681 bugfix: incorrect buffer size config 2015-01-21 16:19:14 +01:00
Alexey Rybalchenko
e4e88ad1db Allow disabling transfer rate logging per input/output (optional, default is on).
This commit introduces new property to FairMQDevice to control whether rateLogger thread should print rates for a given input/output.
Per default logging is enabled, so existing code works as before and does not need to be updated.
To turn off logging for an initialized input/output, set the `LogInputRate`/`LogOutputRate` property to `0`.

Example:
``` c++
splitter.ChangeState(FairMQSplitter::INIT);
// turn off rate logging for input 1 and output 4
splitter.SetProperty(FairMQSplitter::LogInputRate, 0, 1);
splitter.SetProperty(FairMQSplitter::LogOutputRate, 0, 4);
```
2014-11-13 11:14:57 +01:00
Alexey Rybalchenko
0cef5692b1 Use boost::program_options for managing command line options of the executables.
Existing scripts in example/Tutorial3/macro have been updated to use the new format.
Your own executables are not affected, but your scripts which use FairMQ executables have to be updated to the new format.
Use the `--help` option with any FairMQ executable to find out the available options.
2014-11-04 11:50:57 +01:00
Alexey Rybalchenko
101bc4c9e7 add flag for non-blocking operations 2014-10-21 17:20:29 +02:00
Alexey Rybalchenko
6968f57abc add ROUTER/DEALER/PAIR sockets. 2014-10-15 11:24:41 +02:00
Alexey Rybalchenko
9317f06c10 Add REQ-REP zmq/nn sockets to FairMQ together with a little example program.
Also includes example of using boost::program_options.
2014-10-10 18:41:36 +02:00
Alexey Rybalchenko
bd79420f93 Condition check before ending the main thread.
Make sure the main thread waits for the child thread with the Run() method.

* Add this at the end of your Run() method for each device:
```
boost::lock_guard<boost::mutex> lock(fRunningMutex);
fRunningFinished = true;
fRunningCondition.notify_one();
```
* Then you must replace the `char ch; cin.get(ch);` in your main() function with:
```
boost::unique_lock<boost::mutex> lock(processor.fRunningMutex);
while (!processor.fRunningFinished)
{
    processor.fRunningCondition.wait(lock);
}
```
2014-09-01 15:26:37 +02:00
Alexey Rybalchenko
7d7e1a1084 fix ID in startAll script and a broken link in documentation. 2014-08-26 09:29:55 +02:00
Alexey Rybalchenko
0a610926a1 - Proper process termination:
if interrupted with CTRL+C blocking socket calls will return with -1. Each device should call FairMQDevice::Shutdown() before ending the running state to close open sockets, otherwise the interrupt call itself will block.

- FIX: Update number of received messages for FairMQFileSink.
- Add ability to poll on outputs for FairMQPoller.
2014-08-20 13:48:51 +02:00
Alexey Rybalchenko
8cd120aef4 - Add multipart support to the interface and enable its use out of tasks.
Examples on the use out of tasks are provided in:
  `example/Tutorial3/digitization/TestDetectorDigiLoader.tpl:76-85`: sending a part.
  `example/Tutorial3/reconstruction/FairTestDetectorMQRecoTask.tpl:177-182`: receiving a part.

- This commit also makes structure within processorTask more consistent with samplerTask.

- add macro MQLOG to FairMQLogger.
2014-08-08 14:52:31 +02:00
Alexey Rybalchenko
281fcc459c Fix CIDs 10587, 10813, 10911, 10912, 10402, 10403, 10577, 10578, 10579, 10848, 10861, 10865, 10868, 10910.
Move classes inheriting from device to a subdirectory.
Make sure only protobuf library installed by fairsoft is used.
Cleanup FairMQDevice and fix some initialization list warnings.
Loop to duplicate input files in Sampler.
Add some documentation to FairMQ.
2014-07-28 14:04:20 +02:00
Radoslaw Karabowicz
fe91e5af96 Adding reademe files 2014-07-16 15:41:26 +02:00
Alexey Rybalchenko
d14d96ea70 fix CID 10112
fix CIDs 10922, 10921

fix CIDs 10369, 10371, 10372, 10373, 10374

fix CID 10406

fix CID 10414

fix CIDs 10850-52, 10857, 10859-60, 10863, 10869-76

fix CIDs 10014 10080
2014-07-04 13:24:44 +02:00
Alexey Rybalchenko
699671a0f1 Extend the FairMQ transport interface by allowing the user of the FairMQMessage class to define his own deallocation function.
This function will be called when the transport machanism no longer needs the data.

Use this extension with the Protobuf data format, to enable more efficient transport, avoiding memcpy.
2014-06-12 15:31:43 +02:00
Florian Uhlig
865c0e010f Add license file LICENSE with LPGL license text copied from https://www.gnu.org/licenses/lgpl.html. Add license text to most of files of the project.
Add license text to all header files

Add license text to all c++ source files

Add license text to all C macro files

Converting file to UNIX format.

Add license text to all build system files

Add license text to all c source files

Rename file LICENCE to LICENSE. Change preamble in all files.
2014-06-06 14:57:56 +02:00
Alexey Rybalchenko
c03d4ad6d5 use clang-format for Tutorial3 2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
08e603911d TMessage transport implemented! 2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
622e9e9c09 start experimenting with root transport mechanism.
enable efficient message copying for zeromq (currently used only in benchmarkSampler).
cleanup some code.
2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
68d51d8ed5 use clang-format for FairMQ 2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
e80e6d4269 add support for I/O threads to the interface (only used with ZeroMQ). 2014-05-27 19:01:49 +02:00
Thorsten Kollegger
24d26e802a Using boost::msm inside FairMQStateMachine. 2014-05-08 12:56:30 +02:00
Alexey Rybalchenko
adc7443aa5 Allow easier testing without having to recompile for use of binary/boost/protobuf payload.
to use different data transfer method, just provide a parameter to the script, e.g.:
./startAll bin
./startAll boost
./startAll proto
etc.

if no or incorrect parameters is provided, binary method will be used!
protobuf method works only if the library is available on the system, otherwise it is not compiled.
2014-05-08 12:56:30 +02:00
NicolasWinckler
005321f63c add boost::serialization and template features in tutorial3 and fairmq. Require to modifie base/event/fairtimestamp and fairhit. 2014-05-08 12:56:29 +02:00
Alexey Rybalchenko
9a0ed2ca72 Add (optional) Google Protocol Buffers support (example in Tutorial 3).
To use protobuf, run cmake as follows:
cmake -DUSE_PROTOBUF=1 ..
For this, protobuf library has to be installed on the system.

Further changes:
Clean up splitter/merger: default are N-to-1-merger and 1-to-N-splitter.
Fix bug in nanomsg message deallocation.
Setup proper buffer sizes for nanomsg/zeromq via cmake/bash script.
chmod +x for start scripts.
2014-03-17 12:21:47 +01:00
Alexey Rybalchenko
f31ae77ecb Logger fix: direct constructor call 2014-03-04 15:11:19 +01:00
Alexey Rybalchenko
ac8bbf154a simplify logger for shorter calls and readability 2014-02-13 15:55:25 +01:00
NicolasWinckler
4b8659a654 - Make FairMQSampler generic.
- Remove fairroot dependency from fairmq.
2014-02-13 15:37:47 +01:00
NicolasWinckler
dfc1dd6a06 - Make FairMQSampler generic.
- Remove fairroot dependency from fairmq.
2014-02-07 12:36:01 +01:00
Alexey Rybalchenko
23b88e9e91 cosmetic change 2014-01-31 14:06:34 +01:00
Alexey Rybalchenko
c7b80f3ff9 move zeromq and nanomsg implementation files to separate directories 2014-01-28 13:20:53 +01:00
Alexey Rybalchenko
dde65d3aeb correct include dir in cmake file 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
a97d464d1c typo 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
b45f968c67 change argument passing by value/reference 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
ee70113120 make nanomsg optional 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
64b9e991c3 add nanomsg implementations + use factory for nanomsg + lots of small stuff 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
c041c14373 cmake - use set(INCLUDE_DIRECTORIES) 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
f2fe26e715 rename fTask to fProcessorTask for clarity and to resemble fSamplerTask name 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
2a2b1b5d96 get rid of unused function warning, no need for static. 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
5989845e31 use factory for messages 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
88fee245b8 use factory for sockets 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
a383434c45 add factory classes 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
65cdc33a15 add files for interface implementations 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
3803a3d155 a little clean up 2014-01-28 11:56:08 +01:00
Alexey Rybalchenko
31d10170f7 rename splitter/merger to shorter/generic names 2014-01-28 11:56:07 +01:00
Mohammad Al-Turany
1370259d41 changes from Alexey
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@22782 0381ead4-6506-0410-b988-94b70fbc4730
2013-11-12 08:14:46 +00:00
Mohammad Al-Turany
5bc4746cbc correct struct initialization
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@22608 0381ead4-6506-0410-b988-94b70fbc4730
2013-11-01 13:03:47 +00:00
Mohammad Al-Turany
9c4d64f3b1 New files from Ralf Kliemt
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@22606 0381ead4-6506-0410-b988-94b70fbc4730
2013-11-01 11:24:47 +00:00
Mohammad Al-Turany
5121fe3ae5 merge the trunk with the development of ZeroMQ branch
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@22451 0381ead4-6506-0410-b988-94b70fbc4730
2013-10-25 12:42:48 +00:00
Florian Uhlig
d65d7e490f Add missing dependencies.
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20792 0381ead4-6506-0410-b988-94b70fbc4730
2013-07-22 07:44:58 +00:00
Florian Uhlig
8a8c18b5b4 New macro for the generation of executables. Change all CMakeLists.txt where executables are build accordingly.
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20791 0381ead4-6506-0410-b988-94b70fbc4730
2013-07-22 07:38:23 +00:00
Florian Uhlig
59d1a6415c Correct problem with installation on MacOS. Dont't create a dictionary if there is no LinkDef file defined. Use the GENERATE_LIBRARY macro in fairmq.
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20785 0381ead4-6506-0410-b988-94b70fbc4730
2013-07-19 13:50:02 +00:00
Florian Uhlig
78ee2e3c8e Install more tutorials. Install also the zmq related libraries and executables. Change the templates of the config scripts and the script using the templates to do the installation correct.
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20672 0381ead4-6506-0410-b988-94b70fbc4730
2013-07-11 15:26:01 +00:00
Mohammad Al-Turany
ecebfec9f3 separate process for I/O in zmq example
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20239 0381ead4-6506-0410-b988-94b70fbc4730
2013-06-14 10:11:29 +00:00
Mohammad Al-Turany
5f1984fdbe small corrections
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20194 0381ead4-6506-0410-b988-94b70fbc4730
2013-06-10 12:35:20 +00:00
Mohammad Al-Turany
231c7c8f7e Zero MQ implimentation and example (Tutorial3)
git-svn-id: https://subversion.gsi.de/fairroot/fairbase/trunk@20162 0381ead4-6506-0410-b988-94b70fbc4730
2013-06-07 08:07:48 +00:00