Markdown docs: | Shell | Language Reference | Using Directories

Mash | My Automation Shell¶

A shell that can be used to for automation and (REST) resource discovery. It can be used as a cli, repl. It exposes a complete programming language with variables, functions, conditions and pipes. The language can be tailored towards domain-specific applications and it has interoperability with Bash. In addition, this repository contains a library of utilities.

Features:

A DSL that can interpret user-defined commands: shell.py.
A file-browser. Query both static datastructures and REST APIs: examples/filesystem.py,
A REST client to browse APIs with a programmatic yet intuitive interface: examples/discoverable_api.py.
An object parser which converts JSON data to Python classes: object_parser.py.
An OAS-generator for Python classes: oas.py.
A subshell wrapper, to redirect the output of shell scripts: subshell.py.
A parallelization framework for load testing: parallel.py.

Links

Mash language reference
PyPI
github
docs

Shell¶

Example of a shell with a dropdown completion menu

pip install mash-shell
python -m mash

See src/examples for advances usage examples.

Documentation

| Implementation: Shell

Language: Reference.

Table of Contents

Usage
Library & Tools
- Parallelization Utilities
- Object Parser

Usage¶

See reference.

CLI¶

▶ python -m mash -h
usage: shell.py [-hvsr][-f FILE] [--session SESSION] [cmd [cmd ...]]

If no positional arguments are given then an interactive subshell is started.

positional arguments:
  cmd                   A comma- or newline-separated list of commands

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose
  -s, --safe            Safe-mode. Ask for confirmation before executing commands.
  -f FILE, --file FILE  Read and run FILE as a commands
  -r, --reload          Reload last session
  --session SESSION     Use session SESSION

Filesystem (CRUD Operations)¶

See examples/filesystem.py and examples/discoverable.py.

Example	Description
`cd [DIR]`	Change the current working directory. Alias: `use`
`list [DIR]`	List the items in a directory. Use the current working directory by default. Alias: `l`
`foreach [DIR]`	Iterate over a directory structure. E.g. a resource `users/{id}/email`.
`get NAME`	Retrieve a file.
`set NAME VALUE`	Modify a file.
`new NAME [NAME..]`	Create new directories.
`show [NAME]`	Display detailed information about a directory.
`cp`, `mv`, `rm`	Modify files. I.e. copy, move, rename or remove files.

Usage Examples¶

For real-world examples, see lib.

Example 1¶

See src/examples/shell_example.py. It shows how to use a user-definnable mapping of custom functions. It uses the library quo to create a user-friendly subshell with autocompletion prompts.

# python3 src/examples/shell_example.py echo 'hello world'
hello world

Example 2: REST Client¶

Browse through a REST API as if it is mounted as a filesystem.

# python3 src/examples/rest_client_implicit.py

$ cd users # browse some.api.com/v1/users
$ get 2
|       | values             |
|:------|:-------------------|
| email | name.2@company.com |
| name  | name_2             |

$ flatten 1 2 >>= get users $ email
name.1@company.com
name.2@company.com

$ cd -
$ tree
{ 'document': 'https://dummy-api.com/v1',
  'users': { 0: {'email': 'name.0@company.com', 'name': 'name_0'},
             1: {'email': 'name.1@company.com', 'name': 'name_1'},
             2: {'email': 'name.2@company.com', 'name': 'name_2'},
...

Example 3: Commands¶

Run commands from a file with python src/shell -f FILE or python -m src.shell -f FILE.

# write to file
print A sentence. > out.txt 
!cat out.txt |> export x # save text from file
print $x

y <- shell expr 2 + 2 # store result in variable $y
print "result:" $y # prints "result: 4

shell expr 2 + 2 -> z # store result in variable $z
print "result:" $z # prints "result: 4

Example 4: File System Simulation¶

Support both static and dynamic data.

With Static Data¶

See src/examples/filesystem_example.py. This simulates a REST resources with a directory hierarchy. In addition, it provides fuzzy name completion.

$ python src/examples/shell_example.py tree
# example data with dicts and lists
repository = {'worlds': [
    {'name': 'earth',
     'animals': [
         {'name': 'terrestrial',
          'snakes': [{'name': 'python'},
                     {'name': 'cobra'}]},
         {'name': 'aquatic',
          'penquins': [{'name': 'tux'}]}
     ]}]}

# note the autocompletion
$ python src/examples/filesystem.py 'cd world; cd a; cd t; cd snakes; ll'
python
cobra

With Dynamic Data¶

See examples/discoverable.py.

# list remote/auto-generated data
$ python src/examples/discoverable.py 'ls'
department_805, department_399

# refresh data, then save
$ py src/examples/discoverable.py 'ls ; save'
department_750, department_14

# reload data
$ python src/examples/discoverable.py 'ls'
department_750, department_14

Library & Tools¶

Repository structure

src
├── examples
├── lib # mash shell scripts
└── mash
    ├── filesystem # Directory-based datastructures: FileSystem, View
    ├── object_parser # Parse JSON objects
    └── shell # Shell, ShellWithFileSystem
test

Setup¶

Using a Makefile for convenience.

make install
make test

Parallelization Utilities¶

Some experiments with parallelization, concurrency and asyncio in Python.

Test¶

Start a dummy server.

python3 src/server.py

Do a simple load test

python3 src/parallel.py -v

Object Parser¶

src/object_parser/object_parser.py parses JSON data and instantiate Python objects.
src/object_parser/oas.py converts domain-models to OAS.

Example¶

python src/object_parser_example.py

REST API¶

Server

python src/object_parser_server.py

Client

curl -X 'POST' 'http://localhost:5000/v1/organizations' \
  -H 'Content-Type: application/json' \
  -d '{ "board": [ "string" ], "ceo": "string", "departments": [ { "manager": "string", "teams": [ { "manager": "string", "members": [ "string" ], "team_type": "A", "active": true, "capacity": 0, "value": 0 } ] } ] }'