Appcelerator Microservice Platform (AMP)

AMP's goal is to provide a platform and simple tools to help developers manage and monitor complex, orchestrated service applications. AMP is based entirely on the swarm mode features introduced with Docker 1.12, providing developers a simple path to exploit these features in a hosted environment (that runs locally during development) without straying from the core Docker ecosystem and the need to adopt complex third party frameworks.

Prerequisites

All of the infrastructure services and almost all tooling is completely containerized:

Docker the main prerequisite that is required.

• Docker
• make (almost all the Makefile rules execute in containers now)

Currently, to run tests and a few tools like gofmt, Go still needs to be installed on the system.

• Go

Directories

• api/rpc - contains the directories for specifying our rpc services and messages. They should be segregated by service and only contain *.proto files for definitions and *.pb.go files that are generated by the rpc rule in the Makefile (make rpc). Running make install or make build will automatically invoke the rpc rule.
• api/server - contains the implementation files for the api/rpc/* service interfaces, and also contains server.go, which registers each service with a server on port 50501.
• api/cmd - contains directories corresponding to the two generated executable binaries: amp (the CLI) and amplifier (the server daemon).
• data - backend datastores for configuration, state, stats, and logs.
• vendor - along with glide.yaml and glide.yaml, used for dependency management.

Makefile

• make - (no arguments) will print version/build info, run a check on your code, then install amp and amplifier in $GOPATH/bin.
• make check - runs a series of formatting and lint checks on the source. Make sure to run before submitting a PR.
• make fmt - will format the source according to go conventions.
• make clean - cleans up auto-generated code and deletes the amp and amplifier executables from $GOPATH/src.
• make rpc - run this if you want to update generated files without doing a full build.
• make install - will (re)build the project and install the executable binaries (amp and amplifier) in $GOPATH/src.
• make build - will build a docker image (appcelerator/amp:latest) that contains the binaries.
• make run - will run amplifier in a container.
• make test - will run automated tests.
• make proto - will regenerate all protocol buffer files using protoc.
• make install-deps - will reinstall all required packages under the vendor directory.
• make update-deps - will update all package dependencies.

Package Management

The project uses Glide for vendoring, which is an approach to locking down packages that you have tested for a specific version. Glide isn't the only tool that supports vendoring, but it is one of the more popular ones. The Go community generally commits all the vendor artifacts (the vendor directory and lock file (glide.lock), along with the dependency specification (glide.yaml) file).

When you want to add new dependencies to the project, run glide get <package>.

When you want to reinstall dependencies, run make install-deps.

When you want to update to the latest versions, run make update-deps. Run make test to ensure tests continue to pass after updates before submitting a PR.

Swarm

Use the swarm shell script to launch amp cluster services. Available commands are:

• pull
• start
• ls
• restart
• stop
• monitor

The usual workflow looks like this:

$ ./swarm pull
$ sudo ./swarm start --min
$ ./swarm monitor
$ sudo ./swarm restart --min (equivalent to stop, pull, start)
$ sudo ./swarm stop

CLI

AMP currently uses GitHub authentication for logging in. The CLI command amp login will use your GitHub credentials to authenticate you via GitHub. You need to be a member of the AMP team in the Appcelerator organization to be authorized. For the GitHub OAuth flow to work, you need to start amplifier with a client ID and secret before you can run the amp login command. Follow these instructions for now:

• Create an OAuth application (to simulate the amp backend)
• Note the Client ID and Client Secret
• Start amplifier with the following flags: $ amplifier --clientid --clientsecret &
• Use the CLI to login $ amp login
• When prompted, enter username, password, and two-factor auth code

Logs

The amp logs command is used to query or stream logs. It provides useful filtering options to manage what is presented.

$ amp logs --help

Search through all the logs of the system and fetch entries matching provided criteria.

Usage:
  amp logs [flags]

Flags:
      --container string      Filter by the given container id
  -f, --follow                Follow log output
      --from string           Fetch from the given index (default "-1")
      --message string        Filter the message content by the given pattern
  -m, --meta                  Display entry metadata
      --node string           Filter by the given node id
  -n, --number string         Number of results (default "100")
      --service-id string     Filter by the given service id
      --service-name string   Filter by the given service name

Global Flags:
      --Config string   Config file (default is $HOME/.amp.yaml)
      --server string   Server address (default "localhost:50101")
      --target string   target environment ("local"|"virtualbox"|"aws") (default "local")
  -v, --verbose         verbose output

A few useful examples:

• To fetch and follow all the logs from the platform:

  $ amp logs -f

• To fetch and follow the logs for a specific service, with the message content only:

  $ amp logs -f --service_name etcd

Stats

The amp stats command provides useful information about resource consumption. There is a comprensive set of options to query and monitor specfic metrics that complements and extends what is visible in the web dashboard (http://localhost:6001).

$ amp stats --help

Get statistics on containers, services, nodes about cpu, memory, io, net.

Usage:
  amp stats [flags]

Flags:
      --container               display stats on containers
      --container-id string     filter on container id
      --container-name string   filter on container name
      --cpu                     display cpu stats
      --datacenter string       filter on datacenter
  -f, --follow                  Follow stat output
      --host string             filter on host
      --image string            filter on container image name
      --io                      display disk io stats
      --mem                     display memory stats
      --net                     display net rx/tx stats
      --node                    display stats on nodes
      --node-id string          filter on node id
      --period string           historic period of metrics extraction, duration + time-group as 1m, 10m, 4h, see time-group
      --service                 displat stats on services
      --service-id string       filter on service id
      --service-name string     filter on service name
      --since string            date defining when begin the historic metrics extraction, format: YYYY-MM-DD HH:MM:SS.mmm
      --task                    display stats on tasks
      --task-id string          filter on task id
      --task-name string        filter on task name
      --time-group string       historic extraction group can be: s:seconds, m:minutes, h:hours, d:days, w:weeks
      --until string            date defining when stop the historic metrics extraction, format: YYYY-MM-DD HH:MM:SS.mmm

Global Flags:
      --Config string   Config file (default is $HOME/.amp.yaml)
      --target string   target environment ("local"|"virtualbox"|"aws") (default "local")
  -v, --verbose         verbose output

A few useful examples:

• To display list of services with cpu, mem, io, net metrics and follow them

  $ amp stats --service -f

• To display last 10 minutes of historic of the containers of service kafka with cpu, mem, io, net metrics and follow them:

  $ amp stats --container --service-name=kafka --period=10m  -f

• To display list of tasks with only cpu and mem metrics

  $ amp stats --task --cpu --mem