datafs.config package

Submodules

datafs.config.config_file module

class datafs.config.config_file.ConfigFile(config_file=None, default_profile='default')

Bases: object

edit_config_file()

get_config_from_api(api, profile=None)

get_profile_config(profile=None)

parse_configfile_contents(config)

read_config()

write_config(fp=None)

write_config_from_api(api, config_file=None, profile=None)

Create/update the config file from a DataAPI object

Parameters:

• api (object) – The datafs.DataAPI object from which to create the config profile
• profile (str) – Name of the profile to use in the config file (default “default-profile”)
• config_file (str or file) – Path or file in which to write config (default is your OS’s default datafs application directory)

Examples

Create a simple API and then write the config to a buffer:

>>> from datafs import DataAPI
>>> from datafs.managers.manager_mongo import MongoDBManager
>>> from fs.osfs import OSFS
>>> from fs.tempfs import TempFS
>>> import os
>>> import tempfile
>>> import shutil
>>>
>>> api = DataAPI(
...      username='My Name',
...      contact = 'me@demo.com')
>>>
>>> manager = MongoDBManager(
...     database_name = 'MyDatabase',
...     table_name = 'DataFiles')
>>>
>>> manager.create_archive_table(
...     'DataFiles',
...     raise_on_err=False)
>>>
>>> api.attach_manager(manager)
>>>
>>> tmpdir = tempfile.mkdtemp()
>>> local = OSFS(tmpdir)
>>>
>>> api.attach_authority('local', local)
>>>
>>> # Create a StringIO object for the config file
...
>>> try:
...   from StringIO import StringIO
... except ImportError:
...   from io import StringIO
...
>>> conf = StringIO()
>>>
>>> config_file = ConfigFile(default_profile='my-api')
>>> config_file.write_config_from_api(
...     api,
...     profile='my-api',
...     config_file=conf)
>>>
>>> print(conf.getvalue())   
default-profile: my-api
profiles:
  my-api:
    api:
      user_config: {contact: me@demo.com, username: My Name}
    authorities:
      local:
        args: [...]
        service: OSFS
        kwargs: {}
    manager:
      args: []
      class: MongoDBManager
      kwargs:
        client_kwargs: {}
        database_name: MyDatabase
        table_name: DataFiles

>>> conf.close()
>>> local.close()
>>> shutil.rmtree(tmpdir)

At this point, we can retrieve the api object from the configuration file:

>>> try:
...   from StringIO import StringIO
... except ImportError:
...   from io import StringIO
...
>>> conf = StringIO("""
... default-profile: my-api
... profiles:
...   my-api:
...     api:
...       user_config: {contact: me@demo.com, username: My Name}
...     authorities:
...       local:
...         args: []
...         service: TempFS
...         kwargs: {}
...     manager:
...       args: []
...       class: MongoDBManager
...       kwargs:
...         client_kwargs: {}
...         database_name: MyDatabase
...         table_name: DataFiles
... """)
>>>
>>> import datafs
>>> from fs.tempfs import TempFS
>>> api = datafs.get_api(profile='my-api', config_file=conf)
>>>
>>> cache = TempFS()
>>> api.attach_cache(cache)
>>>
>>> conf2 = StringIO()
>>>
>>> config_file = ConfigFile(default_profile='my-api')
>>> config_file.write_config_from_api(
...     api,
...     profile='my-api',
...     config_file=conf2)
>>>
>>> print(conf2.getvalue())   
default-profile: my-api
profiles:
  my-api:
    api:
      user_config: {contact: me@demo.com, username: My Name}
    authorities:
      local:
        args: []
        service: TempFS
        kwargs: {}
    cache:
        args: []
        service: TempFS
        kwargs: {}
    manager:
      args: []
      class: MongoDBManager
      kwargs:
        client_kwargs: {}
        database_name: MyDatabase
        table_name: DataFiles

exception datafs.config.config_file.ProfileNotFoundError

Bases: exceptions.KeyError

datafs.config.constructor module

class datafs.config.constructor.APIConstructor

Bases: object

classmethod attach_cache_from_config(api, config)

classmethod attach_manager_from_config(api, config)

classmethod attach_services_from_config(api, config)

static generate_api_from_config(config)

datafs.config.helpers module

datafs.config.helpers.check_requirements(to_populate, prompts, helper=False)

Iterates through required values, checking to_populate for required values

If a key in prompts is missing in to_populate and helper==True, prompts the user using the values in to_populate. Otherwise, raises an error.

Parameters:

• to_populate (dict) – Data dictionary to fill. Prompts given to the user are taken from this dictionary.
• prompts (dict) – Keys and prompts to use when filling to_populate

datafs.config.helpers.get_api(profile=None, config_file=None, requirements=None)

Generate a datafs.DataAPI object from a config profile

get_api generates a DataAPI object based on a pre-configured datafs profile specified in your datafs config file.

To create a datafs config file, use the command line tool datafs configure --helper or export an existing DataAPI object with datafs.ConfigFile.write_config_from_api()

Parameters:

• profile (str) – (optional) name of a profile in your datafs config file. If profile is not provided, the default profile specified in the file will be used.
• config_file (str or file) – (optional) path to your datafs configuration file. By default, get_api uses your OS’s default datafs application directory.

Examples

The following specifies a simple API with a MongoDB manager and a temporary storage service:

>>> try:
...   from StringIO import StringIO
... except ImportError:
...   from io import StringIO
...
>>> import tempfile
>>> tempdir = tempfile.mkdtemp()
>>>
>>> config_file = StringIO("""
... default-profile: my-data
... profiles:
...     my-data:
...         manager:
...             class: MongoDBManager
...             kwargs:
...                 database_name: 'MyDatabase'
...                 table_name: 'DataFiles'
...
...         authorities:
...             local:
...                 service: OSFS
...                 args: ['{}']
... """.format(tempdir))
>>>
>>> # This file can be read in using the datafs.get_api helper function
...
>>>
>>> api = get_api(profile='my-data', config_file=config_file)
>>> api.manager.create_archive_table(
...     'DataFiles',
...     raise_on_err=False)
>>>
>>> archive = api.create(
...     'my_first_archive',
...     metadata = dict(description = 'My test data archive'),
...     raise_on_err=False)
>>>
>>> with archive.open('w+') as f:
...     res = f.write(u'hello!')
...
>>> with archive.open('r') as f:
...     print(f.read())
...
hello!
>>>
>>> # clean up
...
>>> archive.delete()
>>> import shutil
>>> shutil.rmtree(tempdir)

datafs.config.helpers.to_config_file(api, config_file=None, profile='default')

Module contents