Base functionality

The base module represents concepts of the bootstrapping process that tasks can interact with and handles the gather, sorting and running of tasks.

• Filesystem handling
  • Volume
  • Partitionmaps
  • Partitions
  • Exceptions
• Package handling
  • Package list
  • Sources list
  • Preferences list
  • Exceptions

Bootstrap information

class bootstrapvz.base.bootstrapinfo.BootstrapInformation(manifest=None, debug=False)

The BootstrapInformation class holds all information about the bootstrapping process. The nature of the attributes of this class are rather diverse. Tasks may set their own attributes on this class for later retrieval by another task. Information that becomes invalid (e.g. a path to a file that has been deleted) must be removed.

_BootstrapInformation__create_manifest_vars(manifest, additional_vars={})

Creates the manifest variables dictionary, based on the manifest contents and additional data.

Parameters:

• manifest (Manifest) – The Manifest
• additional_vars (dict) – Additional values (they will take precedence and overwrite anything else)

Returns:

The manifest_vars dictionary

Return type:

dict

class bootstrapvz.base.bootstrapinfo.DictClass

Tiny extension of dict to allow setting and getting keys via attributes

Manifest

The Manifest module contains the manifest that providers and plugins use to determine which tasks should be added to the tasklist, what arguments various invocations should have etc..

class bootstrapvz.base.manifest.Manifest(path=None, data=None)

This class holds all the information that providers and plugins need to perform the bootstrapping process. All actions that are taken originate from here. The manifest shall not be modified after it has been loaded. Currently, immutability is not enforced and it would require a fair amount of code to enforce it, instead we just rely on tasks behaving properly.

load_data(data=None)

Loads the manifest and performs a basic validation. This function reads the manifest and performs some basic validation of the manifest itself to ensure that the properties required for initalization are accessible (otherwise the user would be presented with some cryptic error messages).

load_modules()

Loads the provider and the plugins.

parse()

Parses the manifest. Well… “parsing” is a big word. The function really just sets up some convenient attributes so that tasks don’t have to access information with info.manifest.data[‘section’] but can do it with info.manifest.section.

schema_validator(data, schema_path)

This convenience function is passed around to all the validation functions so that they may run a json-schema validation by giving it the data and a path to the schema.

Parameters:

• data (dict) – Data to validate (normally the manifest data)
• schema_path (str) – Path to the json-schema to use for validation

validate()

Validates the manifest using the provider and plugin validation functions. Plugins are not required to have a validate_manifest function

validation_error(message, data_path=None)

This function is passed to all validation functions so that they may raise a validation error because a custom validation of the manifest failed.

Parameters:

• message (str) – Message to user about the error
• data_path (list) – A path to the location in the manifest where the error occurred

Raises:

ManifestError – With absolute certainty

Tasklist

The tasklist module contains the TaskList class.

class bootstrapvz.base.tasklist.TaskList(tasks)

The tasklist class aggregates all tasks that should be run and orders them according to their dependencies.

run(info, dry_run=False)

Converts the taskgraph into a list and runs all tasks in that list

Parameters:

• info (dict) – The bootstrap information object
• dry_run (bool) – Whether to actually run the tasks or simply step through them

bootstrapvz.base.tasklist.check_ordering(task)

Checks the ordering of a task in relation to other tasks and their phases.

This function checks for a subset of what the strongly connected components algorithm does, but can deliver a more precise error message, namely that there is a conflict between what a task has specified as its predecessors or successors and in which phase it is placed.

Parameters:task (Task) – The task to check the ordering for Raises:TaskListError – If there is a conflict between task precedence and phase precedence

bootstrapvz.base.tasklist.create_list(taskset, all_tasks)

Creates a list of all the tasks that should be run.

bootstrapvz.base.tasklist.get_all_classes(path=None, prefix='', excludes=[])

Given a path to a package, this function retrieves all the classes in it

Parameters:

• path (str) – Path to the package
• prefix (str) – Name of the package followed by a dot
• excludes (list) – List of str matching module names that should be ignored

Returns:

A generator that yields classes

Return type:

generator

Raises:

Exception – If a module cannot be inspected.

bootstrapvz.base.tasklist.get_all_tasks(loaded_modules)

Gets a list of all task classes in the package

Returns:A list of all tasks in the package Return type:list

bootstrapvz.base.tasklist.load_tasks(function, manifest, *args)

Calls function on the provider and all plugins that have been loaded by the manifest. Any additional arguments are passed directly to function. The function that is called shall accept the taskset as its first argument and the manifest as its second argument.

Parameters:

• function (str) – Name of the function to call
• manifest (Manifest) – The manifest
• args (list) – Additional arguments that should be passed to the function that is called

bootstrapvz.base.tasklist.strongly_connected_components(graph)

Find the strongly connected components in a graph using Tarjan’s algorithm.

Source: http://www.logarithmic.net/pfh-files/blog/01208083168/sort.py

Parameters:graph (dict) – mapping of tasks to lists of successor tasks Returns:List of tuples that are strongly connected comoponents Return type:list

bootstrapvz.base.tasklist.topological_sort(graph)

Runs a topological sort on a graph.

Source: http://www.logarithmic.net/pfh-files/blog/01208083168/sort.py

Parameters:graph (dict) – mapping of tasks to lists of successor tasks Returns:A list of all tasks in the graph sorted according to ther dependencies Return type:list

Logging

This module holds functions and classes responsible for formatting the log output both to a file and to the console.

class bootstrapvz.base.log.ColorFormatter(fmt=None, datefmt=None)

Colorizes log messages depending on the loglevel

format(record)

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

class bootstrapvz.base.log.FileFormatter(fmt=None, datefmt=None)

Formats log statements for output to file Currently this is just a stub

class bootstrapvz.base.log.SourceFormatter(fmt=None, datefmt=None)

Adds a [source] tag to the log message if it exists The python docs suggest using a LoggingAdapter, but that would mean we’d have to use it everywhere we log something (and only when called remotely), which is not feasible.

format(record)

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

bootstrapvz.base.log.get_console_handler(debug, colorize)

Returns a log handler for the console The handler color codes the different log levels

Params bool debug:  Whether to set the log level to DEBUG (otherwise INFO) Params bool colorize:  Whether to colorize console output Returns:The console logging handler

bootstrapvz.base.log.get_file_handler(path, debug)

Returns a log handler for the given path If the parent directory of the logpath does not exist it will be created The handler outputs relative timestamps (to when it was created)

Params str path:  The full path to the logfile Params bool debug:  Whether to set the log level to DEBUG (otherwise INFO) Returns:The file logging handler

bootstrapvz.base.log.get_log_filename(manifest_path)

Returns the path to a logfile given a manifest The logfile name is constructed from the current timestamp and the basename of the manifest

Parameters:manifest_path (str) – The path to the manifest Returns:The path to the logfile Return type:str

Task

class bootstrapvz.base.task.Task

The task class represents a task that can be run. It is merely a wrapper for the run function and should never be instantiated.

classmethod run(info)

The run function, all work is done inside this function

Parameters:info (BootstrapInformation) – The bootstrap info object.

Phase

class bootstrapvz.base.phase.Phase(name, description)

The Phase class represents a phase a task may be in. It has no function other than to act as an anchor in the task graph. All phases are instantiated in common.phases

pos()

Gets the position of the phase

Returns:The positional index of the phase in relation to the other phases Return type:int