Skip to content

Implement streaming json output#2

Open
stevenj wants to merge 206 commits intostevenj:masterfrom
kleinweby:output-json-stream
Open

Implement streaming json output#2
stevenj wants to merge 206 commits intostevenj:masterfrom
kleinweby:output-json-stream

Conversation

@stevenj
Copy link
Owner

@stevenj stevenj commented Oct 19, 2021

Currently when enabling json output, the results are only written once the test
concludes. This is done to output one full json document containing all relevant
informations.

To allow status output during the run while using json as output, this patch
adds a newline-delimited JSON output.

In order to achive this multiple event objects are emitted. These are serialized
as json and printed with a newline seperating them.

Each event contains a event name and its data. The following events have been
introduced: start, interval, end, error, server_output_text and
server_output_json. The data contains the relevant portion of the normal JSON
output.

PLEASE NOTE the following text from the iperf3 license. Submitting a
pull request to the iperf3 repository constitutes "[making]
Enhancements available...publicly":

You are under no obligation whatsoever to provide any bug fixes, patches, or
upgrades to the features, functionality or performance of the source code
("Enhancements") to anyone; however, if you choose to make your Enhancements
available either publicly, or directly to Lawrence Berkeley National
Laboratory, without imposing a separate written license agreement for such
Enhancements, then you hereby grant the following license: a non-exclusive,
royalty-free perpetual license to install, use, modify, prepare derivative
works, incorporate into other computer software, distribute, and sublicense
such enhancements or derivative works thereof, in binary and source code form.

The complete iperf3 license is available in the LICENSE file in the
top directory of the iperf3 source tree.

  • Version of iperf3 (or development branch, such as master or
    3.1-STABLE) to which this pull request applies:

  • Issues fixed (if any):

  • Brief description of code changes (suitable for use as a commit message):

kleinweby and others added 30 commits March 16, 2021 22:34
Currently when enabling json output, the results are only written once the test
concludes. This is done to output one full json document containing all relevant
informations.

To allow status output during the run while using json as output, this patch
adds a newline-delimited JSON output.

In order to achive this multiple event objects are emitted. These are serialized
as json and printed with a newline seperating them.

Each event contains a event name and its data. The following events have been
introduced: start, interval, end, error, server_output_text and
server_output_json. The data contains the relevant portion of the normal JSON
output.
netdial() honors --bind-dev option but iperf_tcp_connect() doesn't,
as a result, only the control socket is bound to the device, but not
the data socket.

Instead of duplicaing code from netdial to iperf_tcp_connect(), this
fix extracts a common util function create_socket() from netdial() and
let iperf_tcp_connect() call create_socket() to create a socket with
optional bindings.

Tested on Raspberry Pi 3 with eth0 and wlan0.
…utput

Fix for issue esnet#952 - different json object names for bidir reverse ch…
…mments.

This provides consistency with the CLI option name --bidir.
Follow-up to esnet#1200: s/bdir/bidir/ in JSON identifiers per review comments
Fix: Avoid spamming stdout (and JSON output) with a warning.
Fix --bind-dev options for TCP streams. (esnet#1099)
TOS = DSCP * 4; so shift DSCP << 2
…ions

Update templates and (some) documentation for discussions
Don't close stdout when using external logfile
Bump version number, update release notes, rename release notes and tweak
markup. Update copyright year.
…port

Use different Up/Downlink ports when cport and bdir options are set
…rupt-sequence-number

Ignore out of order packts during UDP connection in Reverse Mode
Force iperf3 server to exit if it can't open its log file.
bmah888 and others added 30 commits November 8, 2023 10:56
This fixes a problem where every thread would essentially burn
a CPU core busy-waiting, when it didn't need to. It's believed
that this excess CPU usage might contribute to packet loss and
poor performance.

Non-blocking sockets were a necessity with the original single-
thread process model of iperf3, in order to allow for
concurrency between different test streams. With multiple
threads, this is no longer necesary (it's perfectly fine for a
thread to block on I/O, as it only services a single test stream
and won't affect any others).

Problem pointed out by @bltierney. Code review by @swlars.

Fixes IPERF-177.
iperf3 implements a limit intended to allow the receiving side of a
test to abort a test in progress if no data has been received for a
certain length of time. This time limit is configured with the
--rcv-timeout command-line option.

The original implementation didn't work correctly with multi-threading
because the code that implemented the limit had no visibility into the
network I/O activity handled by other threads. The code has been
restructured to make this work correctly, by watching the total number
of blocks transferred in the test and using that to determine progress
(or lack thereof).

A minor change was also made to allow worker threads to be cancelled,
even if they were blocked waiting for network I/O. While necessary for
the testing protocol for this bug, this change might also improve the
correctness of thread handling around the end of tests.

Fixes IPERF-178.
We now handle the case where the worker threads exited on their own
accord before the main thread had a chance to cancel them.

While here, tweaked some of the error messages.

Fixes IPERF-179.
Some architectures without native support for 64-bit atomics need
linking with libatomic.
fix endian type of udp connect / reply messages
Remove Travis CI support as we won't be using it going forward.
Check and link libatomic if needed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.