Skip to content

Meeting Notes May 2020

Jump to bottom
Patricia Grubel edited this page May 11, 2020 · 12 revisions

Monday, May 11, 2020

Attendees -

Wednesday, May 06, 2020

Attendees - Rusty Davis (rstyd), Qiang Guan (guanxyz), Al McPherson (mcpherson), Pat Grubel (pagrubel), Tim Randles (trandles-lanl), Jake Tronge (jtronge)

PR Review

  1. NONE

Issue Review

  1. 155(rstyd) - Come up with an id scheme for the WFM
    • Al will look how that impacts the database.
  2. 156(guanxyz) - discussed roles of the components need to add resource monitor and arrow back to WFM from TM

Discussion (ToDo?)

  • We need to write up a data model.
  • When we have the database refactor #119, Wiki page: Graph Database Entities.
    • Where's the slurm job id - should be in the task meta data.
    • There is no schema it is freeform. We can write a utility to spit our the metadata at completion of the workflow.
  • Qiang - Do we have a big picture with all the components?

Around the room

  • Rusty
    • for pexpect test, I need to build it into pytest
    • working on WSGI Tim you should be able to have a user run it
    • can we containerize it which should be cleaner
  • Qiang
    • Working on the interface design, looking at Kubernetes design
  • Al
    • Still working on parsing looking at how to get scatter, gather and globbing in the database
    • Also, working on stuff for Steven
  • Pat
    • Separate out the container runtime and maybe the resource manager at the same time
  • Jake
    • Working on BEEstart, debug is working for graph database but not working for TM and WFM
      • Tim - look at stderr and stdout in subprocess attaching
  • Tim
    • Working on FY21 planning
    • Big Picture will be working on it try to get that out by noon Friday

Monday, May 04, 2020

Attendees - Rusty Davis (rstyd), Qiang Guan (guanxyz), Al McPherson (mcpherson), Pat Grubel (pagrubel), Tim Randles (trandles-lanl), Jake Tronge (jtronge)

PR Review

  1. 152(pagrubel) - Image extension fix 146
    • APPROVED
  2. 154(pagrubel) - Parser passes all step hints to the Workflow-153.

Issue Review

  1. 156(guanxyz) - Interface Design and Spec (TM to Resource Manager)
  2. 155(rstyd) - Come up with an id scheme for the WFM

Discussion (ToDo?)

Around the room

  • Rusty
    • pytest
    • had trouble accessing cypher-shell in neo4j
    • doco on failure possibilies WfM-TM
    • WSGI stuff
  • Qiang
    • interface and PySlurm
  • Al
    • parsing difficulties
    • scatter/gather is going to be difficult
      • file globbing in CWL is complicated
      • can't know apriori
    • work with Steven to combine parsing and database redesign
    • maybe more complex communication between WfM/TM
  • Pat
    • looking at CWL requirements
    • abstracting out container runtimes and workload managers
  • Jake
    • added logdir logging ability to BEEStart
    • work with Rusty to talk about how logging works with WSGI
Clone this wiki locally