WIP: Add boundary comm inner loop specializations for performance and allow setting vector length

[lroberts36]

PR Summary

This PR adds various inner loop specializations to avoid calling reductions, coordinate transformations, and masks in the inner boundary buffer setting loops where possible. I found this made a little bit of a difference for the multigrid communication, not sure that it really matters elsewhere.

It also exposes a flag parthenon/mesh/comm_vector_length which sets the vector length of the outer team policy. We have always used the Kokkos default value here (which is one). I am not sure if changing this value has any impact, but I at least wanted to be able to play around with it.

As an aside, once #1142 is in, we may want to switch these loops to par_fors instead of raw Kokkos loops.

PR Checklist

• Code passes cpplint
• New features are documented.
• Adds a test for any bugs fixed. Adds tests for new features.
• Code is formatted
• Changes are summarized in CHANGELOG.md
• Change is breaking (API, behavior, ...)
  • Change is additionally added to CHANGELOG.md in the breaking section
  • PR is marked as breaking
  • Short summary API changes at the top of the PR (plus optionally with an automated update/fix script)
• CI has been triggered on Darwin for performance regression tests.
• Docs build
• (@lanl.gov employees) Update copyright on changed files

[Yurlungur]

LGTM! This is a good change that should hopefully help with performance.

[lroberts36]

This PR is out of date with @bprather changes in #1196, as well as some other changes. It probably is still worth including at some point, but the required changes are pretty simple to just put in a new PR.