This document describes the current stable version of Kombu (4.5). For development docs, go here.

Python 2 to Python 3 utilities - kombu.five

Python 2/3 compatibility.

Compatibility implementations of features only available in newer Python versions.

class kombu.five.Counter(**kwds)[source]

Dict subclass for counting hashable items. Sometimes called a bag or multiset. Elements are stored as dictionary keys and their counts are stored as dictionary values.

>>> c = Counter('abcdeabcdabcaba')  # count elements from a string
>>> c.most_common(3)                # three most common elements
[('a', 5), ('b', 4), ('c', 3)]
>>> sorted(c)                       # list all unique elements
['a', 'b', 'c', 'd', 'e']
>>> ''.join(sorted(c.elements()))   # list elements with repetitions
'aaaaabbbbcccdde'
>>> sum(c.values())                 # total of all counts
15
>>> c['a']                          # count of letter 'a'
5
>>> for elem in 'shazam':           # update counts from an iterable
...     c[elem] += 1                # by adding 1 to each element's count
>>> c['a']                          # now there are seven 'a'
7
>>> del c['b']                      # remove all 'b'
>>> c['b']                          # now there are zero 'b'
0
>>> d = Counter('simsalabim')       # make another counter
>>> c.update(d)                     # add in the second counter
>>> c['a']                          # now there are nine 'a'
9
>>> c.clear()                       # empty the counter
>>> c
Counter()

Note: If a count is set to zero or reduced to zero, it will remain in the counter until the entry is deleted or the counter is cleared:

>>> c = Counter('aaabbc')
>>> c['b'] -= 2                     # reduce the count of 'b' by two
>>> c.most_common()                 # 'b' is still in, but its count is zero
[('a', 3), ('c', 1), ('b', 0)]
copy()[source]

Return a shallow copy.

elements()[source]

Iterator over elements repeating each as many times as its count.

>>> c = Counter('ABCABC')
>>> sorted(c.elements())
['A', 'A', 'B', 'B', 'C', 'C']

# Knuth’s example for prime factors of 1836: 2**2 * 3**3 * 17**1 >>> prime_factors = Counter({2: 2, 3: 3, 17: 1}) >>> product = 1 >>> for factor in prime_factors.elements(): # loop over factors … product *= factor # and multiply them >>> product 1836

Note, if an element’s count has been set to zero or is a negative number, elements() will ignore it.

classmethod fromkeys(iterable, v=None)[source]

Create a new dictionary with keys from iterable and values set to value.

most_common(n=None)[source]

List the n most common elements and their counts from the most common to the least. If n is None, then list all element counts.

>>> Counter('abcdeabcdabcaba').most_common(3)
[('a', 5), ('b', 4), ('c', 3)]
subtract(**kwds)[source]

Like dict.update() but subtracts counts instead of replacing them. Counts can be reduced below zero. Both the inputs and outputs are allowed to contain zero and negative counts.

Source can be an iterable, a dictionary, or another Counter instance.

>>> c = Counter('which')
>>> c.subtract('witch')             # subtract elements from another iterable
>>> c.subtract(Counter('watch'))    # subtract elements from another counter
>>> c['h']                          # 2 in which, minus 1 in witch, minus 1 in watch
0
>>> c['w']                          # 1 in which, minus 1 in witch, minus 1 in watch
-1
update(**kwds)[source]

Like dict.update() but add counts instead of replacing them.

Source can be an iterable, a dictionary, or another Counter instance.

>>> c = Counter('which')
>>> c.update('witch')           # add elements from another iterable
>>> d = Counter('watch')
>>> c.update(d)                 # add elements from another counter
>>> c['h']                      # four 'h' in which, witch, and watch
4
kombu.five.reload(module)[source]

Reload the module and return it.

The module must have been successfully imported before.

class kombu.five.UserList(initlist=None)[source]

A more or less complete user-defined wrapper around list objects.

append(item)[source]

S.append(value) – append value to the end of the sequence

clear() → None -- remove all items from S[source]
copy()[source]
count(value) → integer -- return number of occurrences of value[source]
extend(other)[source]

S.extend(iterable) – extend sequence by appending elements from the iterable

index(value[, start[, stop]]) → integer -- return first index of value.[source]

Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

insert(i, item)[source]

S.insert(index, value) – insert value before index

pop([index]) → item -- remove and return item at index (default last).[source]

Raise IndexError if list is empty or index is out of range.

remove(item)[source]

S.remove(value) – remove first occurrence of value. Raise ValueError if the value is not present.

reverse()[source]

S.reverse() – reverse IN PLACE

sort(*args, **kwds)[source]
class kombu.five.UserDict(**kwargs)[source]
copy()[source]
classmethod fromkeys(iterable, value=None)[source]
class kombu.five.Callable
class kombu.five.Iterable
class kombu.five.Mapping
get(k[, d]) → D[k] if k in D, else d. d defaults to None.
items() → a set-like object providing a view on D's items
keys() → a set-like object providing a view on D's keys
values() → an object providing a view on D's values
class kombu.five.Queue(maxsize=0)[source]

Create a queue object with a given maximum size.

If maxsize is <= 0, the queue size is infinite.

empty()[source]

Return True if the queue is empty, False otherwise (not reliable!).

This method is likely to be removed at some point. Use qsize() == 0 as a direct substitute, but be aware that either approach risks a race condition where a queue can grow before the result of empty() or qsize() can be used.

To create code that needs to wait for all queued tasks to be completed, the preferred technique is to use the join() method.

full()[source]

Return True if the queue is full, False otherwise (not reliable!).

This method is likely to be removed at some point. Use qsize() >= n as a direct substitute, but be aware that either approach risks a race condition where a queue can shrink before the result of full() or qsize() can be used.

get(block=True, timeout=None)[source]

Remove and return an item from the queue.

If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until an item is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Empty exception if no item was available within that time. Otherwise (‘block’ is false), return an item if one is immediately available, else raise the Empty exception (‘timeout’ is ignored in that case).

get_nowait()[source]

Remove and return an item from the queue without blocking.

Only get an item if one is immediately available. Otherwise raise the Empty exception.

join()[source]

Blocks until all items in the Queue have been gotten and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate the item was retrieved and all work on it is complete.

When the count of unfinished tasks drops to zero, join() unblocks.

put(item, block=True, timeout=None)[source]

Put an item into the queue.

If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until a free slot is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Full exception if no free slot was available within that time. Otherwise (‘block’ is false), put an item on the queue if a free slot is immediately available, else raise the Full exception (‘timeout’ is ignored in that case).

put_nowait(item)[source]

Put an item into the queue without blocking.

Only enqueue the item if a free slot is immediately available. Otherwise raise the Full exception.

qsize()[source]

Return the approximate size of the queue (not reliable!).

task_done()[source]

Indicate that a formerly enqueued task is complete.

Used by Queue consumer threads. For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.

If a join() is currently blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).

Raises a ValueError if called more times than there were items placed in the queue.

exception kombu.five.Empty

Exception raised by Queue.get(block=0)/get_nowait().

exception kombu.five.Full[source]

Exception raised by Queue.put(block=0)/put_nowait().

class kombu.five.LifoQueue(maxsize=0)[source]

Variant of Queue that retrieves most recently added entries first.

class kombu.five.array(typecode[, initializer]) → array

Return a new array whose items are restricted by typecode, and initialized from the optional initializer value, which must be a list, string or iterable over elements of the appropriate type.

Arrays represent basic values and behave very much like lists, except the type of objects stored in them is constrained. The type is specified at object creation time by using a type code, which is a single character. The following type codes are defined:

Type code C Type Minimum size in bytes ‘b’ signed integer 1 ‘B’ unsigned integer 1 ‘u’ Unicode character 2 (see note) ‘h’ signed integer 2 ‘H’ unsigned integer 2 ‘i’ signed integer 2 ‘I’ unsigned integer 2 ‘l’ signed integer 4 ‘L’ unsigned integer 4 ‘q’ signed integer 8 (see note) ‘Q’ unsigned integer 8 (see note) ‘f’ floating point 4 ‘d’ floating point 8

NOTE: The ‘u’ typecode corresponds to Python’s unicode character. On narrow builds this is 2-bytes on wide builds this is 4-bytes.

NOTE: The ‘q’ and ‘Q’ type codes are only available if the platform C compiler used to build Python supports ‘long long’, or, on Windows, ‘__int64’.

Methods:

append() – append a new item to the end of the array buffer_info() – return information giving the current memory info byteswap() – byteswap all the items of the array count() – return number of occurrences of an object extend() – extend array by appending multiple elements from an iterable fromfile() – read items from a file object fromlist() – append items from the list frombytes() – append items from the string index() – return index of first occurrence of an object insert() – insert a new item into the array at a provided position pop() – remove and return item (default last) remove() – remove first occurrence of an object reverse() – reverse the order of the items in the array tofile() – write all items to a file object tolist() – return the array converted to an ordinary list tobytes() – return the array converted to a string

Attributes:

typecode – the typecode character used to create the array itemsize – the length in bytes of one array item

append()

Append new value v to the end of the array.

buffer_info()

Return a tuple (address, length) giving the current memory address and the length in items of the buffer used to hold array’s contents.

The length should be multiplied by the itemsize attribute to calculate the buffer length in bytes.

byteswap()

Byteswap all items of the array.

If the items in the array are not 1, 2, 4, or 8 bytes in size, RuntimeError is raised.

count()

Return number of occurrences of v in the array.

extend()

Append items to the end of the array.

frombytes()

Appends items from the string, interpreting it as an array of machine values, as if it had been read from a file using the fromfile() method).

fromfile()

Read n objects from the file object f and append them to the end of the array.

fromlist()

Append items to array from list.

fromstring()

Appends items from the string, interpreting it as an array of machine values, as if it had been read from a file using the fromfile() method).

This method is deprecated. Use frombytes instead.

fromunicode()

Extends this array with data from the unicode string ustr.

The array must be a unicode type array; otherwise a ValueError is raised. Use array.frombytes(ustr.encode(…)) to append Unicode data to an array of some other type.

index()

Return index of first occurrence of v in the array.

insert()

Insert a new item v into the array before position i.

itemsize

the size, in bytes, of one array item

pop()

Return the i-th element and delete it from the array.

i defaults to -1.

remove()

Remove the first occurrence of v in the array.

reverse()

Reverse the order of the items in the array.

tobytes()

Convert the array to an array of machine values and return the bytes representation.

tofile()

Write all items (as machine values) to the file object f.

tolist()

Convert array to an ordinary list with the same items.

tostring()

Convert the array to an array of machine values and return the bytes representation.

This method is deprecated. Use tobytes instead.

tounicode()

Extends this array with data from the unicode string ustr.

Convert the array to a unicode string. The array must be a unicode type array; otherwise a ValueError is raised. Use array.tobytes().decode() to obtain a unicode string from an array of some other type.

typecode

the typecode character used to create the array

class kombu.five.zip_longest

zip_longest(iter1 [,iter2 […]], [fillvalue=None]) –> zip_longest object

Return a zip_longest object whose .__next__() method returns a tuple where the i-th element comes from the i-th iterable argument. The .__next__() method continues until the longest iterable in the argument sequence is exhausted and then it raises StopIteration. When the shorter iterables are exhausted, the fillvalue is substituted in their place. The fillvalue defaults to None or can be specified by a keyword argument.

class kombu.five.map

map(func, *iterables) –> map object

Make an iterator that computes the function using arguments from each of the iterables. Stops when the shortest iterable is exhausted.

class kombu.five.zip

zip(iter1 [,iter2 […]]) –> zip object

Return a zip object whose .__next__() method returns a tuple where the i-th element comes from the i-th iterable argument. The .__next__() method continues until the shortest iterable in the argument sequence is exhausted and then it raises StopIteration.

kombu.five.string

alias of builtins.str

kombu.five.string_t

alias of builtins.str

kombu.five.bytes_t

alias of builtins.bytes

kombu.five.bytes_if_py2(s)[source]

Convert str to bytes if running under Python 2.

kombu.five.long_t

alias of builtins.int

kombu.five.text_t

alias of builtins.str

kombu.five.module_name_t

alias of builtins.str

class kombu.five.range(stop) → range object

range(start, stop[, step]) -> range object

Return an object that produces a sequence of integers from start (inclusive) to stop (exclusive) by step. range(i, j) produces i, i+1, i+2, …, j-1. start defaults to 0, and stop is omitted! range(4) produces 0, 1, 2, 3. These are exactly the valid indices for a list of 4 elements. When step is given, it specifies the increment (or decrement).

count(value) → integer -- return number of occurrences of value
index(value[, start[, stop]]) → integer -- return index of value.

Raise ValueError if the value is not present.

start
step
stop
kombu.five.items(d)[source]

Get dict items iterator.

kombu.five.keys(d)[source]

Get dict keys iterator.

kombu.five.values(d)[source]

Get dict values iterator.

kombu.five.nextfun(it)[source]

Get iterator next method.

kombu.five.reraise(tp, value, tb=None)[source]

Reraise exception.

class kombu.five.WhateverIO(v=None, *a, **kw)[source]

StringIO that takes bytes or str.

write(data)[source]

Write string to file.

Returns the number of characters written, which is always equal to the length of the string.

kombu.five.with_metaclass(Type, skip_attrs=None)[source]

Class decorator to set metaclass.

Works with both Python 2 and Python 3 and it does not add an extra class in the lookup order like six.with_metaclass does (that is – it copies the original class instead of using inheritance).

class kombu.five.StringIO

Text I/O implementation using an in-memory buffer.

The initial_value argument sets the value of object. The newline argument is like the one of TextIOWrapper’s constructor.

close()

Close the IO object.

Attempting any further operation after the object is closed will raise a ValueError.

This method has no effect if the file is already closed.

closed
getvalue()

Retrieve the entire contents of the object.

line_buffering
newlines

Line endings translated so far.

Only line endings translated during reading are considered.

Subclasses should override.

read()

Read at most size characters, returned as a string.

If the argument is negative or omitted, read until EOF is reached. Return an empty string at EOF.

readable()

Returns True if the IO object can be read.

readline()

Read until newline or EOF.

Returns an empty string if EOF is hit immediately.

seek()

Change stream position.

Seek to character offset pos relative to position indicated by whence:
0 Start of stream (the default). pos should be >= 0; 1 Current position - pos must be 0; 2 End of stream - pos must be 0.

Returns the new absolute position.

seekable()

Returns True if the IO object can be seeked.

tell()

Tell the current file position.

truncate()

Truncate size to pos.

The pos argument defaults to the current file position, as returned by tell(). The current file position is unchanged. Returns the new absolute position.

writable()

Returns True if the IO object can be written.

write()

Write string to file.

Returns the number of characters written, which is always equal to the length of the string.

kombu.five.getfullargspec(func)[source]

Get the names and default values of a callable object’s parameters.

A tuple of seven things is returned: (args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, annotations). ‘args’ is a list of the parameter names. ‘varargs’ and ‘varkw’ are the names of the * and ** parameters or None. ‘defaults’ is an n-tuple of the default values of the last n parameters. ‘kwonlyargs’ is a list of keyword-only parameter names. ‘kwonlydefaults’ is a dictionary mapping names from kwonlyargs to defaults. ‘annotations’ is a dictionary mapping parameter names to annotations.

Notable differences from inspect.signature():
  • the “self” parameter is always reported, even for bound methods
  • wrapper chains defined by __wrapped__ not unwrapped automatically
kombu.five.format_d(i)[source]

Format number.

kombu.five.monotonic() → float

Monotonic clock, cannot go backward.

class kombu.five.buffer_t[source]

Python 3 does not have a buffer type.

kombu.five.python_2_unicode_compatible(cls)[source]

Class decorator to ensure class is compatible with Python 2.