Framed transport for Kafka message bus

[sibiryakov]

Allows to transfer messages larger than Kafka maximum message size.

[jpbalarini]

Hi @sibiryakov, is there any reason this was not merged yet? I'm dealing with a case where this should fix my problem.

Thanks!

[Prometheus3375]

You have an error here. Look at six sources

if PY3:
    string_types = str,
    integer_types = int,
    class_types = type,
    text_type = str
    binary_type = bytes

    MAXSIZE = sys.maxsize
else:
    string_types = basestring,
    integer_types = (int, long)
    class_types = (type, types.ClassType)
    text_type = unicode
    binary_type = str

    if sys.platform.startswith("java"):
        # Jython always uses 32 bits.
        MAXSIZE = int((1 << 31) - 1)
    else:
        # It's possible to have sizeof(long) != sizeof(Py_ssize_t).
        class X(object):

            def __len__(self):
                return 1 << 31
        try:
            len(X())
        except OverflowError:
            # 32-bit
            MAXSIZE = int((1 << 31) - 1)
        else:
            # 64-bit
            MAXSIZE = int((1 << 63) - 1)
        del X

sys.maxsize is 64 bit in Python 3 (9223372036854775807). As you can see from six, MAXSIZE can be 64 bit both for Python 2 and 3. According to documentation (Python 2 and 3), packing symbol L is used to pack 32 bit uint. You should change it to Q which is used to pack 64 bit uint.

Suggested change

	def random_bytes():
	return pack("L", randint(0, MAXSIZE))
	def random_bytes():
	return pack('Q', randint(0, MAXSIZE))

[Prometheus3375]

Another option is to change MAXSIZE variable to literal 4_294_967_295.