scitools.misc

A collection of Python utilities originally developed for the “Python for Computational Science” book.

scitools.misc.BG

alias of BackgroundCommand

class scitools.misc.BackgroundCommand(result='result', func=None, args=[], kwargs={})

Bases: threading.Thread

Run a function call with assignment in the background. Useful for putting time-consuming calculations/graphics in the background in an interactive Python shell.

>>> b=BG('f', g.gridloop, 'sin(x*y)-exp(-x*y)')
>>> b.start()
running gridloop('sin(x*y)-exp(-x*y)',) in a thread
>>> # continue with other interactive tasks
>>> b.finished
True
>>> b.f  # result of function call

Methods

getName()
isAlive()
isDaemon()
is_alive()
join([timeout])
run()
setDaemon(daemonic)
setName(name)
start()

__class__

alias of type

__delattr__

x.__delattr__(‘name’) <==> del x.name

__dict__ = dict_proxy({'__module__': 'scitools.misc', 'run': <function run at 0x3e42140>, '__doc__': "\n Run a function call with assignment in the background.\n Useful for putting time-consuming calculations/graphics\n in the background in an interactive Python shell.\n\n >>> b=BG('f', g.gridloop, 'sin(x*y)-exp(-x*y)')\n >>> b.start()\n running gridloop('sin(x*y)-exp(-x*y)',) in a thread\n >>> # continue with other interactive tasks\n >>> b.finished\n True\n >>> b.f # result of function call\n ", '__init__': <function __init__ at 0x3e420c8>})

__format__()

default object formatter

__getattribute__

x.__getattribute__(‘name’) <==> x.name

__hash__() <==> hash(x)

__init__(result='result', func=None, args=[], kwargs={})

__module__ = 'scitools.misc'

static __new__(S, ...) → a new object with type S, a subtype of T

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__()

__setattr__

x.__setattr__(‘name’, value) <==> x.name = value

__sizeof__() → int

size of object in memory, in bytes

__str__() <==> str(x)

static __subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

daemon

getName()

ident

isAlive()

isDaemon()

is_alive()

join(timeout=None)

name

run()

setDaemon(daemonic)

setName(name)

start()

class scitools.misc.Command(func, *args, **kwargs)

Alternative to lambda functions.

This class should with Python version 2.5 and later be replaced by functools.partial. However, you cannot simply do a:

Command = functools.partial

to be backward compatible with your old programs that use Command, because Command and functools.partial supply the positional arguments in different manners: Command calls the underlying function with new arguments followed by the originally recorded arguments, while functools.partial does it the other way around (first original arguments, then new positional arguments).

This Command class is kept for backward compatibility. New usage should employ functools.partial instead.

Methods

__call__(*args, **kwargs)

__call__(*args, **kwargs)

__init__(func, *args, **kwargs)

__module__ = 'scitools.misc'

class scitools.misc.DoNothing(*args, **kwargs)

Bases: object

Handy class for making other objects inactive. (DoNothing is a generic dispatcher, accepting anyting and doing nothing.)

Whatever we do, we always get a DoNothing object, with which we can do whatever we want to, but nothing will happen.

For example, say a plot function returns a plot object that is used widely in a code to create windows with visualizations on the screen, and you want to turn off all these visualizations:

>>> from scitools.misc import DoNothing
>>> plot = DoNothing('Plotting turned off')
>>> viz = plot(u, wireframe=True, title='My plot')
>>> type(viz)
<class 'scitools.misc.DoNothing'>
>>> viz.update(T)
trying update but no action (DoNothing object)
>>> q = viz.properties()
trying properties but no action (DoNothing object)
>>> type(q)
<class 'scitools.misc.DoNothing'>

One can turn the messages ‘trying ... but no action’ off by giving the argument silent=True to the constructor:

>>> plot = DoNothing('Plotting turned off', silent=True)
>>> viz = plot(u, wireframe=True, title='My plot')
>>> viz.update(T)
>>> q = viz.properties()

Methods

__call__(*args, **kwargs)
next()

__call__(*args, **kwargs)

__class__

alias of type

__delattr__

x.__delattr__(‘name’) <==> del x.name

__dict__ = dict_proxy({'__module__': 'scitools.misc', '__str__': <function __str__ at 0x3e428c0>, '__getattribute__': <function __getattribute__ at 0x3e42938>, 'next': <function next at 0x3e42a28>, '__dict__': <attribute '__dict__' of 'DoNothing' objects>, '__iter__': <function __iter__ at 0x3e429b0>, '__repr__': <function __repr__ at 0x3e42848>, '__call__': <function __call__ at 0x3e427d0>, '__weakref__': <attribute '__weakref__' of 'DoNothing' objects>, '__doc__': "\n Handy class for making other objects inactive.\n (DoNothing is a generic dispatcher, accepting anyting and\n doing nothing.)\n\n Whatever we do, we always get a DoNothing object, with which\n we can do whatever we want to, but nothing will happen.\n\n For example, say a plot function returns a plot object that\n is used widely in a code to create windows with visualizations\n on the screen, and you want to turn off all these visualizations:\n\n >>> from scitools.misc import DoNothing\n >>> plot = DoNothing('Plotting turned off')\n >>> viz = plot(u, wireframe=True, title='My plot')\n >>> type(viz)\n <class 'scitools.misc.DoNothing'>\n >>> viz.update(T)\n trying update but no action (DoNothing object)\n >>> q = viz.properties()\n trying properties but no action (DoNothing object)\n >>> type(q)\n <class 'scitools.misc.DoNothing'>\n\n One can turn the messages 'trying ... but no action' off by\n giving the argument silent=True to the constructor:\n\n >>> plot = DoNothing('Plotting turned off', silent=True)\n >>> viz = plot(u, wireframe=True, title='My plot')\n >>> viz.update(T)\n >>> q = viz.properties()\n ", '__init__': <function __init__ at 0x3e42758>})

__format__()

default object formatter

__getattribute__(name)

__hash__() <==> hash(x)

__init__(*args, **kwargs)

__iter__()

__module__ = 'scitools.misc'

static __new__(S, ...) → a new object with type S, a subtype of T

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__()

__setattr__

x.__setattr__(‘name’, value) <==> x.name = value

__sizeof__() → int

size of object in memory, in bytes

__str__()

static __subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

next()

class scitools.misc.Download(url, filename)

Bases: threading.Thread

Methods

getName()
isAlive()
isDaemon()
is_alive()
join([timeout])
run()
setDaemon(daemonic)
setName(name)
start()

__class__

alias of type

__delattr__

x.__delattr__(‘name’) <==> del x.name

__dict__ = dict_proxy({'__module__': 'scitools.misc', 'run': <function run at 0x3e42230>, '__doc__': None, '__init__': <function __init__ at 0x3e421b8>})

__format__()

default object formatter

__getattribute__

x.__getattribute__(‘name’) <==> x.name

__hash__() <==> hash(x)

__init__(url, filename)

__module__ = 'scitools.misc'

static __new__(S, ...) → a new object with type S, a subtype of T

__reduce__()

helper for pickle

__reduce_ex__()

helper for pickle

__repr__()

__setattr__

x.__setattr__(‘name’, value) <==> x.name = value

__sizeof__() → int

size of object in memory, in bytes

__str__() <==> str(x)

static __subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object (if defined)

daemon

getName()

ident

isAlive()

isDaemon()

is_alive()

join(timeout=None)

name

run()

setDaemon(daemonic)

setName(name)

start()

class scitools.misc.Recorder(obj)

This class is a wrapper of a module or instance which will record all actions done with the module/instance.

>>> from scitools.misc import Recorder
>>> from scitools.std import plt, plot, linspace
>>> x = linspace(-1,1,10)
>>> y1 = x**2
>>> y2 = x**3
>>> plt._g = Recorder(plt._g)  # make the plot object record itself
>>> plot(x, y1, 'r-',
...      x, y2, 'b-',
...      title='A test')
[<scitools.easyviz.common.Line object at 0x1749c50>, <scitools.easyviz.common.Line object at 0x1749bd0>]
>>> # look at what we have done with the plt._g object
>>> plt._g.replay()
reset()
__call__('unset multiplot',)
__call__('set datafile missing "nan"',)
__call__('set key right top',)
__call__('set title "A test"',)
__call__('unset logscale x',)
__call__('unset logscale y',)
__call__('set autoscale',)
__call__('set xrange[*:*]',)
__call__('set yrange[*:*]',)
__call__('set zrange[*:*]',)
__call__('set size noratio',)
__call__('set size nosquare',)
__call__('set yrange [] noreverse',)
__call__('set hidden3d',)
__call__('unset colorbox',)
__call__('set cbrange [*:*]',)
__call__('set palette model RGB defined (0 "blue", 3 "cyan", 4 "green", 5 "yellow", 8 "red", 10 "black")',)
__call__('unset view',)
__call__('set view map',)
__call__('set xtics',)
__call__('set ytics',)
__call__('set ztics',)
__call__('set border 1+2+4+8+16 linetype -1 linewidth .4',)
__call__('unset xlabel',)
__call__('unset ylabel',)
__call__('unset zlabel',)
__call__('set border 4095 linetype -1 linewidth .4',)
__call__('unset grid',)
plot(<Gnuplot.PlotItems._FIFOFileItem instance at 0x174d998>,)
replot(<Gnuplot.PlotItems._FIFOFileItem instance at 0x174db90>,)

Methods

replay()

__getattr__(name)

__init__(obj)

__module__ = 'scitools.misc'

replay()

scitools.misc.after(string, character)

Return part of string after character.

scitools.misc.before(string, character)

Return part of string before character.

scitools.misc.cmldict(argv, cmlargs=None, validity=0)

The cmldict function takes a dictionary cmlargs with default values for the command-line options and returns a modified form of this dictionary after the options given in the list argv are parsed and inserted. One will typically supply sys.argv[1:] as the argv argument. In case cmlargs is None, the dictionary is built from scratch inside the function. The flag validity is false (0) if any option in argv can be inserted in cmlargs, otherwise the function will issue an error message if an option is not already present in cmlargs with a default value (notice that cmlargs=None and validity=1 is an incompatible setting).

Example: cmlargs = {‘p’ : 0, ‘file’ : None, ‘q’ : 0, ‘v’ : 0} argv = “-p 2 –file out -q 0”.split() p = cmldict(argv, cmlargs)

p equals {‘p’ : 2, ‘file’ : out, ‘q’ : 0}

scitools.misc.f(a, b, max=1.2, min=2.2)

scitools.misc.find(func, rootdir, arg=None)

Traverse the directory tree rootdir and call func for each file. arg is a user-provided argument transferred to func(filename,arg).

scitools.misc.findprograms(programs, searchlibs=[], write_message=False)

Given a list of programs (programs), find the full path of each program and return a dictionary with the program name as key and the full path as value. The value is None if the program is not found.

The program list can either be a list/tuple or a dictionary (in the latter case, the keys are the programs and the values are explanations of the programs). If write_message is true, the function writes a message if a program is not found. In that case, None is returned if not all programs are found.

A single program can also be given as first argument. In that case, findprograms returns True or False according to whether the program is found or not.

Example on usage:

if findprograms('plotmtv'):
    os.system('plotmtv ...')

# write a message if a program is not found:
if findprograms(['plotmtv'], write_message=True):
    os.system('plotmtv ...')

programs = ['gs', 'convert']
path = findprograms(programs)
if path['gs']:
    os.system('gs ...')
if path['convert']:
    os.system('convert ...')

programs = { 'gs' : 'Ghostscript: file format conversions',
             'convert' : 'File format conversion from ImageMagick',
           }
if not findprograms(programs, write_message=True):
    print 'the mentioned programs need to be installed'
    sys.exit(1)

scitools.misc.fix_latex_command_regex(pattern, application='match')

Given a pattern for a regular expression match or substitution,

the function checks for problematic patterns commonly encountered when working with LaTeX texts, namely commands starting with a backslash.

For a pattern to be matched or substituted, and extra backslash is always needed (either a special regex construction like w leads to wrong match, or c leads to wrong substitution since just escapes c so only the c is replaced, leaving an undesired backslash). For the replacement pattern in a substitutions, specified by the application=’replacement’ argument, a backslash before any of the characters abfgnrtv must be preceeded by an additional backslash.

The application variable equals ‘match’ if pattern is used for a match and ‘replacement’ if pattern defines a replacement regex in a re.sub command.

Caveats: let pattern just contain LaTeX commands, not combination of commands and other regular expressions (s, d, etc.) as the latter will end up with an extra undesired backslash.

Here are examples on failures:

>>> re.sub(r'egin\{equation\}', r'\[', r'egin{equation}')
'\begin{equation}'
>>> # match of mbox, not \mbox, and wrong output:
>>> re.sub(r'\mbox\{(.+?)\}', r'

box{g<1>}’, r’mbox{not}’)

‘

box{not}’

Here are examples on using this function:

>>> from scitools.misc import fix_latex_command_regex as fix
>>> pattern = fix(r'egin\{equation\}', application='match')
>>> re.sub(pattern, r'\[', r'egin{equation}')
'\['
>>> pattern = fix(r'\mbox\{(.+?)\}', application='match')
>>> replacement = fix(r'

box{g<1>}’, application=’replacement’)

>>> re.sub(pattern, replacement, r'\mbox{not}')
'\fbox{not}'

Avoid mixing LaTeX commands and ordinary regular expression commands, e.g.:

>>> pattern = fix(r'\mbox\{(\d+)\}', application='match')
>>> pattern
'\\mbox\{(\\d+)\}'
>>> re.sub(pattern, replacement, r'\mbox{987}')
'\mbox{987}'  # no substitution, no match

scitools.misc.flatten(nested_data)

Return a flattened iterator over nested_data.

>>> nested_list = [[1,2],3,[4,5,6,[7,[8,9]]]]
>>> flat = [e for e in flatten(nested_list)]
>>> flat
[1, 2, 3, 4, 5, 6, 7, 8, 9]

(Minor adjustment of code by Goncalo Rodrigues, see http://aspn.activestate.com/ASPN/Mail/Message/python-tutor/2302348)

scitools.misc.fontscheme1(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.fontscheme2(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.fontscheme3(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.fontscheme4(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.fontscheme5(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.fontscheme6(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.fontscheme7(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.fontscheme8(root)

Alternative font scheme for Tkinter-based widgets.

scitools.misc.func_to_method(func, class_, method_name=None)

Add a function to a class class_ as method_name. If method_name is not given, func.__name__ becomes the name of the method. Borrowed from recipe 5.12 in the Python Cookbook.

scitools.misc.function_UI(functions, argv, verbose=True)

User interface for calling a collection of functions from the command line by writing the function name and its arguments. functions is a list of possible functions to be called. argv is sys.argv from the calling code. function_UI returns a command to be evaluated (function call) in the calling code.

This function autogenerates a user interface to a module. Suppose a module has a set of functions:

test_mymethod1(M, q, a, b)
test_method2a(a1, a2, a3=0, doc=None)
test_method2b()

The following code automatically creates a user interface and executes calls to the functions above:

from scitools.misc import function_UI
function_names = [fname for fname in dir() if fname.startswith('test_')]
function_UI(function_names, sys.argv)

On the command line the user can now type:

programname --help

and automatically get a help string for each function, consisting of the function name, all its positional arguments and all its keyword arguments.

Alternatively, writing just the function name:

programname functionname

prints a usage string if this function requires arguments, otherwise the function is just called.

Finally, when arguments are supplied:

programname functionname arg1 arg2 arg3 ...

the function is called with the given arguments. Safe and easy use is ensured by always giving keyword arguments:

programname functionname arg1=value1 arg2=value2 arg3=value3 ...

scitools.misc.hardware_info()

Return a dictionary of various types of hardware info from various modules.

Recommended use:

from scitools.misc import hardware_info
import pprint; pprint.pprint(hardware_info())

scitools.misc.import_module(package, module=None)

scitools.misc.interpret_as_callable_or_StringFunction(s, iv, globals_, **named_parameters)

Return a callable object if s is the name of such an object, otherwise turn s to a StringFunction object with iv as the name of the independent variable. The named_parameters dictionary holds parameters in string expressions. Used by the read_cml function.

scitools.misc.isiterable(data)

Returns true of data is iterable, else False.

scitools.misc.lines2paragraphs(lines)

Return a list of paragraphs from a list of lines (normally holding the lines in a file).

scitools.misc.memusage(_proc_pid_stat='/proc/29916/stat')

Return virtual memory size in bytes of the running python. Copied from the SciPy package (scipy_test.testing.py).

scitools.misc.movefiles(files, destdir, confirm=True, verbose=True, copy=True)

Move a set of files to a a destination directory tree, but let the original complete path be reflected in the destination tree.

files list of filenames destdir root of destination directory tree confirm let the user confirm movement of each file verbose write out the original and new path of each file copy True: copy, False: move

The function is useful for backing up or temporarily moving files; the files are easily restored in their original locations since the original complete path is maintained in the destination directory tree.

scitools.misc.oneline(infile, outfile)

Transform all paragraphs in infile (filename) to one-line strings and write the result to outfile (filename).

scitools.misc.pathsearch(programs=[], modules=[], where=0)

Given a list of programs (programs) and modules (modules), search for these programs and modules in the directories in the PATH and PYTHONPATH environment variables, respectively. Check that each directory has read and write access too. The function is useful for checking that PATH and PYTHONPATH are appropriately set in CGI scripts.

scitools.misc.pow_eff(a, b, powfunc=<built-in function pow>)

Returns a^b. Smart function that happened to be slower than a straight math.pow.

scitools.misc.preprocess_all_files(rootdir, options='')

Run preprocess on all files of the form basename.p.ext in the directory with root rootdir. The output of each preprocess run is directed to basename.ext.

@param rootdir: root of directory tree to be processed. @param options: options (string) to preprocess program. @return: nested list of ((dir, basename.p.ext, basename.p), success)) tuples. success is boolean and indicates if the preprocess command was a success (or not).

scitools.misc.primes(n)

Return the prime numbers <= n. Standard optimized sieve algorithm.

scitools.misc.read_cml(option, default=None, argv=['/usr/bin/sphinx-build', '-b', 'html', '-d', '_build/doctrees', '.', '_build/html'])

Search for option (e.g. ‘-p’, ‘–plotfile’) among the command-line arguments and return the associated value (the proceeding argument). If the option is not found, the default argument is returned as str(default) (to have a unified behavior in that everything returned from read_cml is a string).

The call:

str2obj(read_cml(option, default=...))

will return a Python object (with the right type) corresponding to the value of the object (see the str2obj function).

option	command-line option (str)
default	default value associated with the option
argv	list that is scanned for command-line arguments
return value	the item in argv after the option, or default if option is not found

See the read_cml_func function for reading function expressions or function/instance names on the command line and returning callable objects.

scitools.misc.read_cml_func(option, default_func, iv='t', globals_=None, **named_parameters)

Locate --option on the command line (sys.argv) and find the corresponding value (next sys.argv element). This value is supposed to specify a Python function, an instance with a __call__ method, None, or a string that can be turned into a scitools.StringFunction.StringFunction function with iv as the name of the independent variable(s) (list of strings in case of more than one independent variable). If --option is not found, the argument default_func, a given callable or string, is returned (if string, iv reflects the name of the independent variable(s) in the string).

The globals_ argument is just passed on to the StringFunction object if the value of the option for default function is a string. Similary, the named_parameters dictionary is passed on to the StringFunction object and assumed to hold parameters in the string expressions (variables different from the independent variable).

This function always returns a callable object or None.

Here is an interactive session showing the use of read_cml_func:

>>> from scitools.misc import read_cml_func
>>> import sys
>>> from math import sin, cos, pi
>>>
>>> # fake command-line arguments by filling in sys.argv:
>>> sys.argv[1:] = '--func1 myfunc --func3 sin(x)'.split()
>>>
>>> def myfunc(x):

... return 1 + x ... >>> >>> # –func1 has myfunc as value, must pass on globals() or just >>> # myfunc as name to read_cml_func >>> #f = read_cml_func(‘–func1’, ‘1’, globals_=globals()) >>> f = read_cml_func(‘–func1’, ‘1’, globals_={‘myfunc’: myfunc}) >>> type(f) <type ‘function’> >>> f(10) 11 >>> # –func3 is given as a string expression “sin(x)” on the command line >>> f = read_cml_func(‘–func3’, ‘0’, iv=’x’) >>> type(f) <type ‘instance’> >>> repr(f) “StringFunction(‘sin(x)’, independent_variables=(‘x’,), )” >>> str(f) ‘sin(x)’ >>> f(pi) 1.2246467991473532e-16 >>> >>> # –func2 is not given on the command line, use the default >>> # value “A*cos(w*t)”, which is a string expression. >>> # Pass on a globals_ dict with cos from numpy such that f works >>> # with array argument for t >>> import numpy >>> f = read_cml_func(‘–func2’, ‘A*cos(w*t)’, iv=’t’, A=3, w=pi, globals_={‘cos’: numpy.cos}) >>> # More general case where the string should have all numpy functions: >>> #f = read_cml_func(‘–func2’, ‘A*cos(w*t)’, iv=’t’, A=3, w=pi, globals_=dir(numpy)) >>> type(f) <type ‘instance’> >>> repr(f) “StringFunction(‘A*cos(w*t)’, independent_variables=(‘t’,), A=3, w=3.141592653589793)” >>> str(f) ‘3*cos(3.14159265359*t)’ >>> t = numpy.array([1, 4]) >>> f(t) array([-3., 3.]) >>> >>> # No –func4 on the command line (sys.argv), implying that >>> # f becomes a StringFunction with value 0 >>> f = read_cml_func(‘–func4’, ‘0’) >>> type(f) <type ‘instance’> >>> repr(f) “StringFunction(‘0’, independent_variables=(‘t’,), )” >>> str(f) ‘0’ >>> f(1) 0

scitools.misc.remove_multiple_items(somelist)

Given some list somelist, return a list where identical items are removed.

scitools.misc.sorted_os_path_walk(root, func, arg)

Like os.path.walk, but directories and files are visited and listed in alphabetic order.

scitools.misc.str2bool(s)

Turn a string s, holding some boolean value (‘on’, ‘off’, ‘True’, ‘False’, ‘yes’, ‘no’ - case insensitive) into boolean variable. s can also be a boolean. Example:

>>> str2bool('OFF')
False
>>> str2bool('yes')
True

scitools.misc.str2obj(s, globals_=None, locals_=None, debug=False)

Turn string s into the corresponding object. str2obj is mainly used to take a string from a GUI or the command line and create a Python object. For example:

>>> s = str2obj('0.3')
>>> print s, type(s)
0.3 <type 'float'>
>>> s = str2obj('(1,8)')
>>> print s, type(s)
(1, 8) <type 'tuple'>

Method: eval(s) can normally do the job, but if s is meant to be turned into a string object, eval works only if s has explicit quotes:

>>> eval('some string')
Traceback (most recent call last):
SyntaxError: unexpected EOF while parsing

(eval tries to parse ‘some string’ as Python code.) Similarly, if s is a boolean word, say ‘off’ or ‘yes’, eval will not work.

In this function we first try to see if s is a boolean value, using scitools.misc.str2bool. If this does is not successful, we try eval(s, globals_, locals_), and if it works, we return the resulting object. Otherwise, s is (most probably) a string, so we return s itself. The None value of locals_ and globals_ implies using locals() and globals() in this function.

Examples:

>>> strings = ('0.3', '5', '[-1,2]', '-1+3j', 'dict(a=1,b=0,c=2)',
...            'some string', 'true', 'ON', 'no')
>>> for s in strings:
...     obj = str2obj(s)
...     print '"%s" -> %s %s' % (s, obj, type(obj)
...
"0.3" -> 0.3 <type 'float'>
"5" -> 5 <type 'int'>
"[-1,2]" -> [-1, 2] <type 'list'>
"-1+3j" -> (-1+3j) <type 'complex'>
"dict(a=1,b=0,=2)" ->  {'a': 1, 'c': 2, 'b': 0} <type 'dict'>
"some string" -> some string <type 'str'>
"true" -> True <type 'bool'>
"ON" -> True <type 'bool'>
"no" -> False <type 'bool'>
>>>

If the name of a user defined function, class or instance is sent to str2obj, the calling code must also send locals() and globals() dictionaries as extra arguments. Otherwise, str2obj will not know how to “eval” the string and produce the right object (user-defined types are unknown inside str2obj unless the calling code’s globals and locals are provided). Here is an example:

>>> def myf(x):
...     return 1+x
...
>>> class A:
...     pass
...
>>> a = A()
>>>
>>> s = str2obj('myf')
>>> print s, type(s)   # now s is simply the string 'myf'
myf <type 'str'>
>>> # provide locals and globals such that we get the function myf:
>>> s = str2obj('myf', locals(), globals())
>>> print s, type(s)
<function myf at 0xb70ffe2c> <type 'function'>
>>> s = str2obj('a', locals(), globals())
>>> print s, type(s)
<__main__.A instance at 0xb70f6fcc> <type 'instance'>

With debug=True, the function will print out the exception encountered when doing eval(s, globals_, locals_), and this may point out problems with, e.g., imports in the calling code (insufficient variables in globals_).

Note: if the string argument is the name of a valid Python class (type), that class will be returned. For example, >>> str2obj(‘list’) # returns class list <type ‘list’>

scitools.misc.str2type(value)

Return a function that can take a string and convert it to a Python object of the same type as value.

This function is useful when turning input from GUIs or the command line into Python objects. Given a default value for the input (with the right object type), str2type will return the right conversion function. (str2obj can do the thing, but will often return eval, which turns any string into a Python object - this is less safe than str2type, which never returns eval. That principle helps to detect wrong input.)

Method: If value is bool, we use scitools.misc.str2bool, which is capable of converting strings like “on”, “off”,”yes”, “no”, “true” and “false” to boolean values. Otherwise, we use type(value) as the conversion function. However, there is one problem with type(value) when value is an int while the user intended a general real number - in that case one may get wrong answers because of wrong (int) round off. Another problem concerns user-defined types. For those (which str2type knows nothing about) the str function is returned, implying that the conversion from a string to the right user-defined type cannot be done by the function returned from str2type.

Examples:

>>> f = str2type((1,4,3))
>>> f.__name__
'tuple'

>>> f = str2type(MySpecialClass())
>>> f.__name__
'str'

>>> f = str2type('some string')
>>> f.__name__
'str'

>>> f = str2type(False)
>>> f.__name__
'str2bool'

(Note that we could return eval if value is not a string or a boolean, but eval is never returned from this function to avoid conversion to an unintended type.)

scitools.misc.subst(patterns, replacements, filenames, pattern_matching_modifiers=0)

Replace a set of patterns by a set of replacement strings (regular expressions) in a series of files. The function essentially performs:

for filename in filenames:
    file_string = open(filename, 'r').read()
    for pattern, replacement in zip(patterns, replacements):
        file_string = re.sub(pattern, replacement, file_string)

A copy of the original file is taken, with extension .old~.

patterns	string or list of strings (regex)
replacements	string or list of strings (regex)
filenames	string or list of strings
pattern_matching_modifiers	re.DOTALL, re.MULTILINE, etc., same syntax as for re.compile

scitools.misc.system(command, verbose=True, failure_handling='exit', fake=False)

User-friendly wrapping of the os.system/os.popen commands. Actually, the commands.getstatusoutput function is used on Unix systems, and the output from the system command is fetched.

command	operating system command to be executed
verbose	False: no output, True: print command prior to execution
failure_handling	one of ‘exit’, ‘warning’, ‘exception’, or ‘silent’ (in case of failure, the output from the command is always displayed)
fake	if True, the command is printed but not run (for testing)
return value	the same as commands.getstatusoutput, i.e., a boolean failure variable and the output from the command as a string object

scitools.misc.test_if_module_exists(modulename, msg='', raise_exception=False, abort=True)

Test if modulename can be imported, and if not, write an error message and (optionally) raise an exception, continue or abort with sys.exit(1).

scitools.misc.timer(func, args=[], kwargs={}, repetitions=10, comment='')

Run a function func, with arguments given by the tuple args and keyword arguments given by the dictionary kwargs, a specified number of times (repetitions) and write out the elapsed time and the CPU time together.

scitools.misc.timer_system(command, comment='')

Run an os.system(command) statement and measure the CPU time. With os.system, the CPU time is registered as the user and system time of child processes.

Note: there might be some overhead in the timing compared to running time in the OS instead.

scitools.misc.wrap(infile, outfile, linewidth=70)

Read infile (filename) and format the text such that each line is not longer than linewidth. Write result to outfile (filename).