Commit Graph

455 Commits

Author SHA1 Message Date
Alexey Rybalchenko
d175a732a6 FairMQ: fix const qualifier for DeviceRunner 2017-10-05 15:32:12 +02:00
Dennis Klein
1bc5771bf3 Fix compilation for GCC 4.9.2 2017-10-05 15:32:12 +02:00
Dennis Klein
4ae2e025c9 FairMQ: Implement DeviceRunner 2017-10-05 15:32:12 +02:00
Dennis Klein
7f23a70670 FairMQ: Implement property change event config API
Replaced the old event manager implementation, which changed the
subscription semantics to bulk event subscriptions.
2017-10-05 15:32:12 +02:00
Dennis Klein
8c8ee45914 FairMQ: Implement GetPropertyKeys config API 2017-10-05 15:32:12 +02:00
Dennis Klein
2589ca5ced FairMQ: Rewrite the state machine
* Simplify the code
* Drop Boost.MSM dependency
* Drop threaded execution
* Support deferred state changes
* Monitor state changes/queues via callbacks
2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
179968db1e Remove FairMQConfigurable 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
26cfe69b41 Revert refactoring that releases lock too early. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
6d7009b331 Refactor FairMQLogger 2017-10-05 15:32:12 +02:00
Dennis Klein
6ecd0e9085 FairMQ: Rewrite event manager to support multiple subscribers 2017-10-05 15:32:12 +02:00
Dennis Klein
88b3b8ef18 FairMQ: Make sure --catch-signals is always handled 2017-10-05 15:32:12 +02:00
Dennis Klein
fd08167a4c FairMQ: Print Entering STATE log lines when user perceived state begins. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
e37e559dce Fix warnings and remove deleted file from installation. 2017-10-05 15:32:12 +02:00
Dennis Klein
44a59f25a7 FairMQ: Move --catch-signals logic to control plugin
* Add StealDeviceControl() API to plugin services
2017-10-05 15:32:12 +02:00
Dennis Klein
7dcd09692c FairMQ: Move static and interactive control modes to plugin (3) 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
01327426c3 Port DDS plugin to the new plugin system. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
2db114bc5c Eliminate warnings in FairLink and FairMQ Plugins. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
f6365d013e Refactor state machine. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
70e46a0b86 Cleanup base/MQ. 2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
334b91785b FairMQ: Move static and interactive control modes to plugin (2) 2017-10-05 15:32:12 +02:00
Dennis Klein
10f67e4c72 FairMQ: Move static and interactive control modes to plugin
NOT YET FINISHED
2017-10-05 15:32:12 +02:00
Dennis Klein
2af3ae99eb FairMQ: Implement Take/ReleaseDeviceControl plugin API
The main function of this API is to make sure only one plugin is
controlling the device state transitions at a time. The
ReleaseDeviceControl() signal is used to implement a function
WaitForPluginsToReleaseDeviceControl() which is used to block the main
thread if a control plugin is used.
2017-10-05 15:32:12 +02:00
Dennis Klein
dfb2bac4bc FairMQ: Add test coverage for PluginServices 2017-10-05 15:32:12 +02:00
Dennis Klein
052ac8487d FairMQ: Add static plugin mechanism
* Add skeleton for Take/ReleaseControl API
* Add skeleton for control_static builtin plugin
2017-10-05 15:32:12 +02:00
Alexey Rybalchenko
17d7cd8ce4 Fix name conflict with FairLogger+FairMQStateMachine 2017-10-04 13:54:44 +02:00
Alexey Rybalchenko
6349438829 Run InitTask in the worker thread (same as all other state handlers). 2017-09-07 14:22:50 +02:00
Alexey Rybalchenko
bfb9feab03 Update shared memory transport documentation. 2017-09-04 16:33:55 +02:00
Alexey Rybalchenko
09e45e27ab Add introspection docs. 2017-09-04 16:33:55 +02:00
Alexey Rybalchenko
7b37eaa507 Revert the parser changes. 2017-09-04 16:33:39 +02:00
Alexey Rybalchenko
5e44273531 Add docs for common devices 2017-08-30 08:38:00 +02:00
Alexey Rybalchenko
9be5a20df6 Add doxygen comment about state handlers running in worker thread 2017-08-30 08:38:00 +02:00
Alexey Rybalchenko
3ab7ee5f02 FairMQParser: Disable deprecated parsing for singular JSON keys 2017-08-30 08:38:00 +02:00
Alexey Rybalchenko
4c7f119ce8 Configuration docs 2017-08-30 08:38:00 +02:00
Alexey Rybalchenko
4dd08051cc simplify state change callback handling 2017-08-30 08:38:00 +02:00
Alexey Rybalchenko
eeb994a584 Tests for MQ examples 2017-08-28 10:53:41 +02:00
Alexey Rybalchenko
221d2567f0 FairMQRegion and examples/advanced/Region 2017-08-08 14:18:10 +02:00
Alexey Rybalchenko
65f1b96dc3 Add Version member to FairMQDevice, settable via constructor 2017-07-17 13:25:16 +02:00
Alexey Rybalchenko
3d8175bfd6 base/MQ and Lmd example cleanup 2017-07-04 15:59:36 +02:00
Alexey Rybalchenko
6f18cf23da Merge channels from config to device, instead of replace 2017-07-04 07:13:01 +02:00
Dennis Klein
a26925cbf5 FairMQ: Fix various errors from CI
* move Plugins::Version to fair::mq::tools
* fix Parser interface
* make device a shared pointer in main
* provide std::hash template specialization
* fix FairMQ.Plugins/FairMQ.PluginsStatic when run with ctest
* fix MQ/serialization example
* add --no-as-needed flag
* GCC 4 does not support member refs, move to pointer types
2017-07-04 07:05:11 +02:00
Dennis Klein
ad0f050c99 FairMQ: Implement PluginServices - Config 2017-07-04 07:05:11 +02:00
Dennis Klein
739460b2fe FairMQ: Implement PluginServices - Control 2017-07-04 07:05:11 +02:00
Dennis Klein
9b61b924b2 FairMQ: Integration of Plugins through PluginServices 2017-07-04 07:05:11 +02:00
Dennis Klein
60d929b0bd FairMQ: Add plugin mechanism (Plugin and PluginManager classes) 2017-07-04 07:05:11 +02:00
Alexey Rybalchenko
ac69607250 make FairProgOptions calls thread safe 2017-07-04 07:05:11 +02:00
Alexey Rybalchenko
b47cc8b29c add callback on state change 2017-07-04 07:05:11 +02:00
Alexey Rybalchenko
b311b0c6da FairMQLogger: undefine DEBUG if it has been previously defined 2017-06-26 16:35:58 +02:00
Alexey Rybalchenko
035d5fb353 Provide accessors for device members that are part of publiv API 2017-06-23 11:45:46 +02:00
Alexey Rybalchenko
92cae6f9a4 FairMQProxy: remove outdated code 2017-06-23 11:45:46 +02:00
Alexey Rybalchenko
a60fac80d9 FairMQProgOptions: add a method to check key existence 2017-06-23 11:45:46 +02:00
Alexey Rybalchenko
4bc54ad32b add --print-channels to print registered channels of the device 2017-06-23 11:45:46 +02:00
Alexey Rybalchenko
1d38a2350f add --print-options to print available options in short format 2017-06-23 11:45:46 +02:00
Alexey Rybalchenko
cae3fd6aa3 Catch any exception type in the main of runFairMQDevice.h 2017-06-23 11:36:20 +02:00
Dennis Klein
25a75c1f82 add channel, poller and testing docs 2017-06-04 01:14:10 +02:00
Alexey Rybalchenko
5e2926e0da FairMQShmMonitor docs 2017-06-04 01:12:30 +02:00
Alexey Rybalchenko
f05ed326a6 Update FairMQShmMonitor 2017-06-04 01:12:30 +02:00
Alexey Rybalchenko
764514a35a Test devices without config & two in a process 2017-06-04 01:06:12 +02:00
Alexey Rybalchenko
f4e5a74f23 Always have access to FairMQProgOptions from device. 2017-06-04 01:06:12 +02:00
Dennis Klein
7bb6a2eed9 add default subchannel index 2017-05-24 16:37:33 +02:00
Dennis Klein
8bc21675af Refactor TransportFactory to RAII
* Remove explicit Initialize and Terminate states,
    map them onto ctor/dtor
  * Remove no longer needed Shutdown state
  * Remove deprecated SetTransport()
2017-05-24 16:37:33 +02:00
Dennis Klein
733794657c update docs 2017-05-23 14:10:34 +02:00
Dennis Klein
89423eb737 enable shmem tests 2017-05-23 14:10:34 +02:00
Dennis Klein
8b93b4f8ed do not call nn_term()
nn_term() will globally lock down the nanosmsg library. The nn_init()
interface is no longer available to reset the global state. Subsequent unit
tests after the first nn_term() call are failing.
2017-05-23 14:10:34 +02:00
Dennis Klein
d2c78479f0 Refactor FairMQTools
* Split header, add common header
  * Transform to canonical namespace
  * Deprecate old header
  * Adapt to new Tools.h
2017-05-23 13:42:56 +02:00
Dennis Klein
9215c81362 Deprecate old TransportFactory factory 2017-05-23 08:37:52 +02:00
Alexey Rybalchenko
9288a2c3d5 Add tests for poller. 2017-05-23 08:32:13 +02:00
Alexey Rybalchenko
6b221d950c Add NewPoller() wrapper. 2017-05-23 08:32:13 +02:00
Dennis Klein
3be2f297f3 generate id if no device id available
* CreateSocket factory no longer accepts id param, but the
    TransportFactory has an id member instead
2017-05-18 20:01:44 +02:00
Dennis Klein
eb614b6005 do not copy strings 2017-05-18 20:01:44 +02:00
Dennis Klein
b5dab60a82 add more single and multithreaded unit tests 2017-05-18 20:01:44 +02:00
Dennis Klein
3205e0c378 refactor to more idiomatic RAII
* FairMQTransportFactoryZMQ: move the config invariant
    initialization to ctor
  * FairMQChannel: add new ctor that creates usable channel
  * FairMQSocket*: close sockets in dtor
  * FairMQTransportFactory*: terminate context in dtor
  * FairMQChannel: add Bind/Connect facades (for explicit control, e.g. timing)
2017-05-18 20:01:44 +02:00
Dennis Klein
87252edbe0 move New*Message apis to TransportFactory
* add facades to Device and Channel
  * add Transport() getter to Device
  * inline Transport() getter from Channel
2017-05-18 20:01:44 +02:00
Dennis Klein
97ca52aa0e move TransportFactory factory into FairMQTransportFactory namespace 2017-05-18 20:01:44 +02:00
Dennis Klein
f522dc1717 add FairMQParts Send/Receive apis 2017-05-18 20:01:44 +02:00
Dennis Klein
8b85dd2191 support inproc addresses 2017-05-18 20:01:44 +02:00
Dennis Klein
3cb80de773 add channel name setter 2017-05-18 20:01:44 +02:00
Dennis Klein
463a49d79c update CONTRIBUTORS 2017-05-18 20:01:44 +02:00
Dennis Klein
2d4282c62d Fix CID 169527 Uncaught Exception 2017-05-17 18:45:44 +02:00
Dennis Klein
a9ac1fa91c Fix CID 169522 Unchecked return value 2017-05-17 18:45:15 +02:00
Dennis Klein
ec786dce03 Fix CID 169525 Data race condition 2017-05-17 18:44:51 +02:00
Dennis Klein
c78541432f add missing link dependency 2017-05-10 06:43:55 +02:00
Dennis Klein
04bc7d4bfd add missing include guard 2017-05-10 06:42:57 +02:00
Dennis Klein
3dc5fd1d6e expose STL iterator interface for FairMQParts 2017-05-09 14:48:34 +02:00
Alexey Rybalchenko
0926a9a764 FairMQ: allow accumulation of parts on receive. 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
1fa5c836a6 FairMQ: stop device if binding was unsuccessful (not updated dynamically). 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
459b42add2 FairMQ: MakeTransport to create stand-alone transport for tests. 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
085de240c2 FairMQ: fix missing includes. 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
2a526b8625 shmem: introduce FairMQShmMonitor. 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
7b4a2ae932 shmem: configurable segment name. 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
14f9084a80 shmem: Heartbeats for the monitor. 2017-04-28 13:30:51 +02:00
Alexey Rybalchenko
a8d40abbd9 FairProgOptions: update ambiguous option name. 2017-04-28 13:30:51 +02:00
Dennis Klein
f39b41d68c tweak startup and linger times 2017-04-21 09:21:57 +02:00
Dennis Klein
bbc8da54b4 clean redundant doxygen comments 2017-04-21 09:21:57 +02:00
Dennis Klein
91831ff701 print debug output immediately
This helps in the cases when get killed by the ctest timeout.
2017-04-21 09:21:57 +02:00
Dennis Klein
a1fe478ce7 fix more -Winconsistent-missing-override warnings 2017-04-20 11:07:50 +02:00
Dennis Klein
132ae69953 add startup and linger times for pubsub 2017-04-20 11:07:50 +02:00
Alexey Rybalchenko
541538e3de only reset sockets if needed 2017-04-20 11:07:50 +02:00
Alexey Rybalchenko
c51575f88c fix clang++ -Winconsistend-missing-override warnings 2017-04-20 11:07:50 +02:00
Dennis Klein
f963a06e25 make connecting channel wait loop more flexible 2017-04-20 11:07:50 +02:00
Alexey Rybalchenko
ee0d2e0c55 add debug output to test devices 2017-04-20 11:07:50 +02:00
Dennis Klein
218af15bbb ensure that FairMQ.Protocols is not run in parallel
The testsuite uses hardcoded system resources (TCP ports)
and is consequently not thread-safe.
2017-04-20 11:07:50 +02:00
Dennis Klein
d866589be1 Two devices can't share device id 2017-04-20 11:07:50 +02:00
Dennis Klein
bf2b208749 update docs 2017-04-20 11:07:49 +02:00
Alexey Rybalchenko
7caf0a6aae Configurable shmem segment size 2017-04-20 11:07:49 +02:00
Alexey Rybalchenko
ac7017deb5 Re-enable rate limiting in benchmarkSampler 2017-04-20 11:07:49 +02:00
Alexey Rybalchenko
5aaf27bf02 Refactor the transport interface
- give transport Initialize() method with access to device config.
 - avoid using global context in the transport.
 - simplify shutdown procedure (no need for extra thread).
2017-04-20 11:07:49 +02:00
Dennis Klein
d7eb692951 remove no longer needed special treatment for logger directory 2017-04-20 11:07:49 +02:00
Dennis Klein
9b337ca075 remove redundant include path
The imported ZeroMQ target has a target property
INTERFACE_INCLUDE_DIRECTORIES which carries this
information.
2017-04-20 11:07:49 +02:00
Dennis Klein
d02ae9be2c remove no longer used variable 2017-04-20 11:07:48 +02:00
Giulio Eulisse
1ebbe80539 Add ZeroMQ include path 2017-04-20 11:07:48 +02:00
Dennis Klein
c41cf318bd export FairMQ targets and install headers hierarchically 2017-04-20 11:07:47 +02:00
Mohammad Al-Turany
608e34f26d Set CMake min. vers. to 3.6.0 and add conrtibutors file 2017-04-20 11:07:47 +02:00
Dennis Klein
365a68010f various fixes
* fix include paths
* print on stderr
2017-04-20 11:07:47 +02:00
Dennis Klein
13c9c09ab9 list fairmq authors centrally instead of in each file
listing authors in source files is inaccurate too. The version
control system documents this information most accurately, so per
file author lists are dropped for simplicity.
2017-04-20 11:07:47 +02:00
Dennis Klein
35b3212f7e cmake cleanups and fixes
* fix overlinking of libFairMQ
* import dependencies in find modules
* provide FairMQinstall target
* add gtest helper module
* add cotire helper module
* add many comments
* cleanup fairmq cmake file
* add FindPStreams module
* link logger into libFairMQ
* drop root dictionary generation
* bump min required cmake version to 3.7.2
2017-04-20 11:07:47 +02:00
Dennis Klein
7e34f7042f refactor tests
* move to gtest
* add shmem tests
* cleanup cmake file (organize tests in suites)
2017-04-20 11:07:47 +02:00
Matthias Richter
55a9d69908 Adding option '--channel-config' to MQProgOptions 2017-04-20 11:07:46 +02:00
Matthias Richter
9943be507e Adding helper function to print the raw boost property tree 2017-04-20 11:07:46 +02:00
Matthias Richter
e184610c06 Adding parser implementation for key-value subopt format
A parser implementation for FairMQ channel properties.
The parser handles a comma separated key=value list format by using the
getsubopt function of the standard library.

The option key '--channel-config' can be used with the list of key/value
pairs like e.g.
--channel-config name=output,type=push,method=bind
2017-04-20 11:07:46 +02:00
Alexey Rybalchenko
341464a793 Compatibility with nanomsg<=0.6 2017-03-21 14:53:07 +01:00
Alexey Rybalchenko
3c73c690f7 Transports: use transportType as static member. 2017-03-14 17:51:26 +01:00
Alexey Rybalchenko
b237b3f9d7 Remove unused code.
- Remove unused code in FairMQTools.
- Remove obsolete device options (now in device config).
2017-03-14 17:51:26 +01:00
Alexey Rybalchenko
2293c5e417 FairMQChannel: API to access socket metrics; make fSocket private. 2017-03-14 17:51:26 +01:00
Giulio Eulisse
2a72d58766 Remove unneeded if when deleting NULL / 0 / nullptr
C++ standard dictates that NULL / 0 / nullptr is a valid argument for
delete which then simply has no effect:

"The value of the first argument supplied to a deallocation function may
be a null pointer value; if so, and if the deallocation function is one
supplied in the standard library, the call has no effect."

I therefore think in these particular case it's safe to remove the ifs.
2017-03-13 15:46:56 +01:00
Florian Uhlig
d1cf852c98 Fox Coverity issues
Fix CID 131712. In case parameter string does not fit in variable return a failure.
Fix CID 162913. Initialize missing data member.
Fix CID 58901.
Fix CID 149137.
Fix CID 149143.
Fix CIDs 149156, 149150, 149147, 149145, 149144, 149137. Don't use command line parameter argv directly.
2017-03-03 15:45:23 +01:00
Alexey Rybalchenko
91b7a72ac5 get default network interface from the default route 2017-03-02 13:25:06 +01:00
Alexey Rybalchenko
eaa220e39e better tests output 2017-03-02 13:25:06 +01:00
Sandro Wenzel
9e477264c8 Better encapsulate Device config: Offer a getter method 2017-03-02 13:24:35 +01:00
Alexey Rybalchenko
c78b7e4cfa Fix nanomsg tests 2017-02-27 17:49:30 +01:00
Alexey Rybalchenko
07b760218f Fix coverity #157658 2017-02-27 13:09:16 +01:00
Alexey Rybalchenko
7c1f7aa3f2 Fix Tuto3 double delete, cleanup test output, undeclared var in NN socket. 2017-02-27 13:09:16 +01:00
Dennis Klein
b43d4c142d refactor and integrate with aliBuild/alienv
* accept ZEROMQ_ROOT as env or cmake variable to hint at install
    location (used by aliBuild/alienv)
  * added more docs
  * fixed mismatch between module name ZeroMQ and prefix of
    generated variables ZMQ -> ZeroMQ is now the preferred prefix
    (old variables are still available and marked deprecated)
  * the aliBuild zeromq recipe does not install zmq.hpp header,
    so the module searches now for the zmq.h header
    (which is anyways the only one we use)
  * some cosmetic changes
  * add option ZeroMQ_NO_DEPRECATION to suppress deprecation warning
2017-02-23 19:42:15 +01:00
Alexey Rybalchenko
c66fd6fe91 Adding multiple transports support & other fixes:
- Avoid polling when only one input channel is used.
 - Send only handles for shared memory transport.
 - Avoid waiting in the rate logger thread when nothing to log.
 - Hide warnings from generated files
 - Fix #483
2017-02-23 06:47:09 +01:00
Alexey Rybalchenko
e53ad151a7 Fix mismatch in FairMQSink parameter type. 2017-02-23 06:47:09 +01:00
Alexey Rybalchenko
3e7cb85816 Fix incorrect calculation of transfer rate for multipart. 2017-02-23 06:47:09 +01:00
Alexey Rybalchenko
8cf1cbb930 Initialize shared memory only if it is used. 2017-02-23 06:47:09 +01:00
Alexey Rybalchenko
ac1da4db66 Require std::move when adding existing message to FairMQParts. 2017-02-23 06:47:09 +01:00
Alexey Rybalchenko
65c19f07bf Fix compatibility with ZeroMQ 4.2.1 2017-02-09 12:19:23 +01:00
Florian Uhlig
617042b3c6 For tests on MacOSX use new MacOSX version 10.11 and FairSoft version may16p1.
Run fairmq tests sequentialy.
2017-01-27 06:16:11 -08:00
Alexey Rybalchenko
62df0b609b disable feature not available on GCC < 5 2016-12-19 12:06:27 +01:00
Alexey Rybalchenko
a332d9fc83 First version of the shared memory transport.
Use via `--transport shmem` cmd option. No pub/sub.
2016-12-18 14:50:58 +01:00
Alexey Rybalchenko
6c3b01f09c AttachChannel must update the config after it's done, use boost::split 2016-12-14 08:52:05 +01:00
Alexey Rybalchenko
0fb49a0986 State machine update
- Fix regression from last commit (preventing static run from proper shutdown).
 - Guard state changes (for the internal transitions) (msm::process_event is not thread safe).
 - Remove unused transition from RUNNING to EXITING.
2016-11-24 22:35:46 +01:00
Alexey Rybalchenko
b166cedb63 Convert factory methods to return smart ptrs
- Convert factory methods to return smart ptrs.
 - Refactor state machine to use same thread for user states.
 - Remove unused includes and dependencies, use std.
2016-11-18 14:19:16 +01:00
Alexey Rybalchenko
12f04c7237 Fix race condition in the state machine & channel. 2016-11-15 16:17:56 +01:00
mkrzewic
94a4d599eb Validate connection method per endpoint, not socket
this allows e.g. to ommit the socket method in config if all endpoints
specify the method modifier.
It still is fully backward compatible.
2016-11-13 14:22:51 +01:00
mkrzewic
892aa8008b Add missing #include <cstring> 2016-11-13 00:05:57 +01:00
mkrzewic
6bec8fc1db Simplify Tokenize(): nicer looking code = better code 2016-11-12 18:04:14 +01:00
mkrzewic
c2d7c49cf5 Support multiple endpoints per socket
Sent messages will be scheduled among the endpoints according to socket
type: PUB will send the same data to all endpoints simultaneously, PUSH
will do round robin transfer.
Incoming data is fair queued between endpoints.

This is a feature of at least zeromq and nanomsg.

_____________
To use: in the device configuration, instead of specifying just one address,
specify a comma separated list e.g.

tcp://localhost:123,ipc:///tmp/socket

the connection method (bind/connect) applies to all endpoints in this case.
______________
Mixing binding and connecting endpoints is supported:
prefix "@" means "bind", "+" (or ">") means connect, e.g.

+tcp://localhost:123,@ipc:///tmp/socket,ipc:///tmp/asd

(in case of missing prefix, the default channel method is used for that
endpoint).
2016-11-11 16:58:55 +01:00
Sebastien Binet
0a3f14c0e3 all: apply fer-json-fmt 2016-11-10 15:20:54 +01:00