Various utilities

File system

There is a number of methods which are minor expansions of the built-in file utilities:

Network

There is a simple wrapper class utils.net.ClientSocket, which simplifies some operations with the built-in socket module. In addition, it also implements a couple of higher-level ways to send the data: either fixed length (as in the usual socket), with the length prepended (in case the total length is initially unknown at the receiving end), or using a delimiter to mark the end of the message.

In addition, there are several methods for gaining local or remote host information (utils.net.get_local_addr(), utils.net.get_all_local_addr(), utils.net.get_local_hostname(), utils.net.get_all_remote_addr(), utils.net.get_remote_hostname()), receiving JSON-formatted values (utils.net.recv_JSON()), and listening on a given port (utils.net.listen()).

Strings

There are several string manipulation functions present:

  • Powerful to/from string conversion. The main function are utils.string.to_string() and utils.string.from_string(), which can convert a large variety of values: simple scalar values (numbers, strings, booleans, None), containers (lists, tuples, sets, dictionaries), escaped and byte strings (e.g., b"\x00"), complex types such as numpy arrays (represented as, e.g., "array([0, 1, 2, 3, 4])"). The latter version requires setting use_classes=True in utils.string.to_string(), which is not enabled by default to make the results more compatible with other parsers:

    >> pll.to_string(np.arange(5))  # by default, use the standard str method, which makes array look like a list
'[0, 1, 2, 3, 4]'
>> pll.from_string('[0, 1, 2, 3, 4]')  # gets converted back into a list
[0, 1, 2, 3, 4]
>> pll.to_string(np.arange(5), use_classes=True)  # use representation class
'array([0, 1, 2, 3, 4])'
>> pll.from_string('array([0, 1, 2, 3, 4])')  # get converted back into an array
array([0, 1, 2, 3, 4])

    More complex data classes can be added using utils.string.add_conversion_class() and utils.string.add_namedtuple_class():

    >> NamedTuple = collections.namedtuple("NamedTuple", ["field1", "field2"])
>> nt = NamedTuple(1,2)
>> nt
NamedTuple(field1=1, field2=2)
>> pll.to_string(nt, use_classes=True)  # class is not registered, so use the default tuple representation
'(1, 2)'
>> pll.add_namedtuple_class(NamedTuple)
>> pll.to_string(nt, use_classes=True)  # now the name marker is added
'NamedTuple(1, 2)'
>> pll.from_string('NamedTuple(1, 2)')
NamedTuple(field1=1, field2=2)
>> DifferentNamedTuple = collections.namedtuple("DifferentNamedTuple", ["field1", "field2"])
>> pll.from_string('DifferentNamedTuple(1, 2)')  # note that if the class is not registered, it can't be parsed, so the string is returned back
'DifferentNamedTuple(1, 2)'

    Furthermore, there is a couple of auxiliary string functions to parse more complicated situations: utils.string.escape_string() and utils.string.unescape_string() for escaping and unescaping string with potentially confusing or unprintable characters (e.g., quotation marks, spaces, new lines); utils.string.from_string_partial(), utils.string.from_row_string(), utils.string.extract_escaped_string() to determine and extract the first value in a string which potentially has several values.

  • Comparing and searching string: utils.string.string_equal() (compare string using different rules such as case sensitivity), utils.string.find_list_string(), utils.string.find_dict_string() (find string in a list or a dictionary using different comparison rules).

  • Filtering strings: utils.string.get_string_filter(), utils.string.sfglob(), and utils.string.sfregex(). Creates filter functions which may include or exclude certain string patterns; these filter functions can be later used in, e.g., file-related methods such as utils.files.list_dir().

Misc utilities

A variety of small useful methods and classes: