Commit Graph

34 Commits

Author SHA1 Message Date
Alexey Rybalchenko
af971c6ab1 Allow rate limiting in the Shared Memory example 2016-05-18 16:01:21 +02:00
Alexey Rybalchenko
1f09317cd0 Handle both single and multi part payloads in splitter/merger/proxy 2016-05-17 12:27:33 +02:00
Alexey Rybalchenko
28aa5e4caa Let ParseAll exit if unsuccessfull 2016-05-12 17:28:33 +02:00
Alexey Rybalchenko
3353e214a7 Add shared memory example
- Add shared memory example in examples/MQ/SharedMemory
 - Device/Task termination: try soft first, and abort if it fails
 - Interactive mode: prevent cin from blocking forever (poll)
2016-05-11 14:52:53 +02:00
Alexey Rybalchenko
5421922668 Add a cmd option to control state change mechanism 2016-04-27 15:04:29 +02:00
Alexey Rybalchenko
b9883d3b13 Configuration and DDS example/tools updates
- Update DDS example command UI and extract it from example.
 - Unify address handling via DDS properties for dynamic deployment.
 - Update DDS docs with the new approach.
 - Allow `--config-key` to be used to access common config in JSON.
 - Allow common channel properties to be specified for all sockets.
 - Update MQ examples and Tuto3 with new config options.
 - Add start scripts to MQ examples for easier use.
2016-04-25 17:35:57 +02:00
Alexey Rybalchenko
732373faa2 Update multi-part features (nanomsg) and various fixes
- Implement nanomsg multipart with MessagePack.
 - Use the MessagePack from FairSoft and handle not found case.
 - Update splitter, merger and proxy devices to handle multi-part.
 - Let FairMQParts.At() return pointer reference (can be used for moving).
 - Add missing const specifier in the message interface.
 - Add transmit kernel size setting to channels (ZMQ_SNDBUF).
 - Remove FairMQBuffer device.
 - Remove old multi-part methods from Tutorial3 example (to be replaced with Parts API).
 - Make callback mandatory for newMsg(data, size, callback).
 - Add missing <vector> include in FairMQSocket.
2016-03-24 13:36:47 +01:00
Florian Uhlig
fa7040fe65 Fix compiler warnings.
Initialize all data members in initializer lists.
Reorder data members in initializer list to have the same order as in the class declaration.
Comment or remove unused parameters and unused variables.
Convert all old style casts to the correct and explicit c++ cast like const_cast, static_cast,
dynamic_cast or reinterpret_cast. In most cases static_cast is used.
2016-03-14 13:37:55 +01:00
Alexey Rybalchenko
e1fef82657 FairMQ: Extend Multipart and messaging API
- Extend the multipart API to allow sending vectors of messages or helper
   thin wrapper FairMQParts. See example in examples/MQ/8-multipart.
 - NewMessage() can be used in devices instead of
   fTransportFactory->CreateMessage().
   Possible arguments remain unchanged (no args, size or data+size).
 - Send()/Receive() methods can be used in devices instead of
   fChannels.at("chan").at(i).Send()/Receive():
   Send(msg, "chan", i = 0), Receive(msg, "chan", i = 0).
 - Use the new methods in MQ examples and tests.
 - No breaking changes, but FAIRMQ_INTERFACE_VERSION is incremented to 3
   to allow to check for new methods.
2016-02-29 16:25:39 +01:00
Alexey Rybalchenko
82ab7670a9 Add FlatBuffers & MessagePack examples
- Add FlatBuffers serialization example to Tutorial 3
 - Add MessagePack serialization example to Tutorial 3
 - Performance improvements in Boost serialization example
 - Use `GetEntriesFast()` for FairTestDetectorRecoTask
 - Use `Clear()` instead of `Delete()` in MQ parts of Tutorial 3
 - Fix CMake variables from preventing compilation without nanomsg.
 - create macro/data directories in install directory
 - Get rid of data duplication in fill_parameters.C
 - Various cleanups and fixes
2016-02-22 13:06:37 +01:00
Alexey Rybalchenko
f1abb9ecdd Remove compile time transport interface switch
- Remove the compile time check of the transport implementation.
  The transport (zeromq/nanomsg) can be chosen at run time with:
  `device.SetTransport("zeromq"); // possible values are "zeromq" and "nanomsg"`.

  For devices that use FairMQProgOptions, the transport can be configured via cmd option:
  `--transport zeromq` or `--transport nanomsg`. Default values is "zeromq".
  The device receives the configured value with:
  `device.SetTransport(config.GetValue<std::string>("transport"));`

  Old method of setting transport still works. But the NANOMSG constant is not defined.

- Remove old `fairmq/prototest` directory. It was only used as a test for protobuf.
  The protobuf part of Tutorial3 does the same (with different values).

- Fix a bug in FairMQPollerNN, where the `revents` value was not initialized.
  This caused the `poller->CheckOutput()` to trigger when it should not.
2016-01-14 15:19:30 +01:00
Alexey Rybalchenko
0e1a1ad552 Allow to limit number of messages for the Benchmark sampler and sink 2016-01-14 14:19:03 +01:00
Florian Uhlig
d254efb91f Fix compiler warnings. Mostly unused parameters have been removed. 2016-01-14 14:19:02 +01:00
Alexey Rybalchenko
383a220333 Update device configuration
- Move general config files out of example directory to fairmq/run.
 - Use FairMQProgOptions for MQ example 5.
 - Add SendPartAsync() for non-blocking send of a message part.
2015-11-11 11:09:42 +01:00
Alexey Rybalchenko
2e789e4439 Add ParameterMQServer and example of its use 2015-11-06 14:46:15 +01:00
Alexey Rybalchenko
6000e86e74 Remove redundant io-threads cmd option from Sink 2015-11-02 09:27:12 +01:00
Alexey Rybalchenko
91d3f013c0 Fix setting of ZeroMQ I/O Threads (setting was not propagated before) 2015-10-12 13:51:09 +02:00
Alexey Rybalchenko
f13bb5995d Add FairMQ tests (PUB-SUB, PUSH-PULL, REQ-REP). 2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
fbf7dbf2ba Add orthogonal OK/ERROR states.
Replace state check mutex with atomic.

Update DDS example documentation.
2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
a7ab33a10e Add new Send/Receive methods with smart pointers and no flag checks. 2015-09-28 12:17:24 +02:00
Alexey Rybalchenko
1302e77a16 Several FairMQ fixes and improvements:
- FairMQ: add possibility to poll on multiple channels.
- FairMQ: include command channel when polling on blocking calls (for unblocking without termination).
- FairMQ: move signal handler inside of FairMQDevice class (call FairMQDevice::CatchSignals() in the main function).
- FairMQ: add 'bool CheckCurrentState(statename)' (instead of 'GetCurrentState() == statename' that cannot be thread safe).
- FairMQDevice: add 'InteractiveStateLoop()' method that can be used to change states from the command line.
- FairMQDevice: add automatic transition to IDLE state if Run() exits without an external event.
- FairMQDevice: implement device reset.
- FairMQDevice: use unordered_map for device channels.
- FairMQChannel: improve address validation for channels.
- FairMQChannel: add ExpectsAnotherPart() method to check if another msg part is expected (old approach still works).
- FairMQ: remove invalid transition from the run files.
- FairMQFileSink: disable ROOT termination signal handler.
- Tutorial3: spawn xterm windows from start scripts without overlapping for better visibility.
- FairMQ Examples: update protobuf test and move its files to a common directory.
- FairMQStateMachine: improve feedback on invalid transitions (more readable).
2015-09-28 12:17:22 +02:00
Alexey Rybalchenko
2656d8098c Add first two simple FairMQ examples
fairmq/examples/1-sampler-sink
fairmq/examples/2-sampler-processor-sink
2015-06-22 16:35:00 +02:00
Alexey Rybalchenko
d6a413534a Thread safe initialization of the channels 2015-06-19 12:32:56 +02:00
Alexey Rybalchenko
a3d919b763 Use FairMQProgOptions for Sink and BenchmarkSampler 2015-06-16 12:43:23 +02:00
Alexey Rybalchenko
7fda980710 Update FairMQStateMachine & introduce FairMQChannels
Organize sockets as a map of vectors of FairMQChannels.

Update FairMQStateMachine by removing SETTINGINPUT, SETTINGOUTPUT,
BIND and CONNECT states and by adding INITIALIZING_TASK, RESETTING_TASK
and RESETTING_DEVICE states. Run states functions in their own thread.
2015-06-16 09:29:53 +02:00
Alexey Rybalchenko
8a82afe184 Expose BIND and CONNECT states for use with dynamic configuration
introduce FairMQ interface version
2015-02-11 15:15:23 +01:00
Alexey Rybalchenko
eb33bc8280 add constructor to DeviceOptions 2015-01-21 16:19:16 +01:00
Alexey Rybalchenko
ce58ee2302 Fix warnings produced with -Weffc++ in fairmq, base/MQ and Tutorial3.
Remaining warnings originate from boost::msm and boost::mpl.
Also, warnings with missing initializer list entries for DeviceOptions are false positives, since those are PODs.
2015-01-21 16:19:16 +01:00
Alexey Rybalchenko
65d91b8681 bugfix: incorrect buffer size config 2015-01-21 16:19:14 +01:00
Alexey Rybalchenko
0cef5692b1 Use boost::program_options for managing command line options of the executables.
Existing scripts in example/Tutorial3/macro have been updated to use the new format.
Your own executables are not affected, but your scripts which use FairMQ executables have to be updated to the new format.
Use the `--help` option with any FairMQ executable to find out the available options.
2014-11-04 11:50:57 +01:00
Alexey Rybalchenko
bd79420f93 Condition check before ending the main thread.
Make sure the main thread waits for the child thread with the Run() method.

* Add this at the end of your Run() method for each device:
```
boost::lock_guard<boost::mutex> lock(fRunningMutex);
fRunningFinished = true;
fRunningCondition.notify_one();
```
* Then you must replace the `char ch; cin.get(ch);` in your main() function with:
```
boost::unique_lock<boost::mutex> lock(processor.fRunningMutex);
while (!processor.fRunningFinished)
{
    processor.fRunningCondition.wait(lock);
}
```
2014-09-01 15:26:37 +02:00
Florian Uhlig
865c0e010f Add license file LICENSE with LPGL license text copied from https://www.gnu.org/licenses/lgpl.html. Add license text to most of files of the project.
Add license text to all header files

Add license text to all c++ source files

Add license text to all C macro files

Converting file to UNIX format.

Add license text to all build system files

Add license text to all c source files

Rename file LICENCE to LICENSE. Change preamble in all files.
2014-06-06 14:57:56 +02:00
Alexey Rybalchenko
68d51d8ed5 use clang-format for FairMQ 2014-06-06 14:30:41 +02:00
Alexey Rybalchenko
9a0ed2ca72 Add (optional) Google Protocol Buffers support (example in Tutorial 3).
To use protobuf, run cmake as follows:
cmake -DUSE_PROTOBUF=1 ..
For this, protobuf library has to be installed on the system.

Further changes:
Clean up splitter/merger: default are N-to-1-merger and 1-to-N-splitter.
Fix bug in nanomsg message deallocation.
Setup proper buffer sizes for nanomsg/zeromq via cmake/bash script.
chmod +x for start scripts.
2014-03-17 12:21:47 +01:00