Related Work

[chicoqueiroz]

WILSON, Greg et al. Best Practices for Scientific Computing. PLoS Biology, v. 12, n. 1, p. e1001745, 2014. http://dx.plos.org/10.1371/journal.pbio.1001745

WILSON, Greg et al. Good Enough Practices in Scientific Computing.
arXiv preprint arXiv:1609.00037, 2016. http://arxiv.org/abs/1609.00037

[sandragesing]

This is a great start. Let's collect related work here so that we can define the scope of the whole paper again to create a white paper that's complementary to ones, which just got published.

[kyleniemeyer]

I've added these to wssspe.bib, and will try to draft some text around them.

We can also consider (some from past WSSSPE workshops):

• Blanton B, Lenhardt C. A Scientist’s Perspective on Sustainable Scientific Software. Journal of Open Research Software 2014;2:e17–4. https://doi.org/10.5334/jors.ba
• Löffler F, Brandt SR, Allen GD, Schnetter E. Cactus: Issues for Sustainable Simulation Software. Journal of Open Research Software 2014;2:e12. https://doi.org/doi:10.5334/jors.au
• Stodden V, Miguez S. Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research. Journal of Open Research Software 2014;2:e21. https://doi.org/10.5334/jors.ay

[chicoqueiroz]

Also consider:

• BAXTER, Susan M et al. Scientific software development is not an oxymoron. PLoS Computational Biology, v. 2, n. 9, p. e87, 2006.
• KELLY, Diane; SMITH, Spencer. 3rd CASCON Workshop on Software Engineering for Science. 2010, [S.l.]: IBM Corp., 2010. p. 420-422.

[iliant]

Hello all,

The references below are a collection due to my interests in software best practices pre 2007. Despite the age of these the findings and recommendations remain valid. I updated the first reference below a few years ago when I supplied a report to EC RI with respect to Materials Modelling software best practices and quality measures.

1. I Sommerville, Software Engineering. 9th ed. Addison-Wesley Publishing Co, Reading, MA, 3 March 2010
2. ASCI Software Quality Engineering: Goals, Principles and Guidelines, Sandia National Laboratories, DOE/DP/ASC-SQE-2000-FDRIT-VERS2, February 2001
3. LLNL Site Specific ASCI Software Quality Engineering Recommended Practices: Overview, Lawrence Livermore National Laboratory, UCRL-ID-147309, February 2002
4. LLNL Site-Specific ASCI Software Quality Engineering Recommended Practices, Version 1.0, Lawrence Livermore National Laboratory, UCRL-ID-143698, May 2001
5. LANL ASCI Software Engineering Requirements, Los Alamos National Laboratory, LA-UR-02-888, February 2002
6. LANL ASCI Software Engineering Requirements: Version 2, Los Alamos National Laboratory, LA-UR-04-6048, July 2004
7. SANDIA National Laboratories Advanced Simulation and Computing (ASC) Software Quality Plan – Part 1: ASC Software Quality Engineering Practices Version 1.0, Sandia Report SAND2004-6602, January 2005
8. SANDIA National Laboratories Advanced Simulation and Computing (ASC) Software Quality Plan – Part 2: Mapping for the ASC Software Quality Engineering Practices Version 1.0, Sandia Report SAND2004-6601, January 2005
9. SSD Space Product Assurance Procedure, CLRC Rutherford Appleton Laboratory,
10. Document IS09/SPAP/SOFT/0001, Issue 1.1, 1991
11. D Post and R Kendall, “Software Project Management and Quality Engineering Practices for Complex, Coupled Multi-Physics, Massively Parallel Computational Simulations: Lessons Learned from ASCI”, Los Alamos National Laboratory, La-UR-03-1274, March 2003
12. M van de Vanter, DE Post and ME Zosel, “HPC needs a Tool Strategy”, Los Alamos National Laboratories, LA-UR-05-1592
13. M Paulk, The Capability Maturity Model, New York, Addison-Wesley, 1994
14. The Software Engineering Institute (SEI): http://www.sei.cmu.edu/
15. C Greenough, DJ Worth, “Transforming legacy Fortran software”, Rutherford
16. Appleton Laboratory Technical Report
17. Polyhedron Ltd http://www.polyhedron.com under Fortran Compiler Comparison
18. Institute of Electrical and Electronics Engineers. IEEE Standard Glossary of Software Engineering Terminology. IEEE Standard 610.12-1990. New York, NY, 1990
19. Institute of Electrical and Electronics Engineers. IEEE Standard for Software Test Documentation. IEEE Standard 829-1998. New York, NY, 1998
20. Institute of Electrical and Electronics Engineers. IEEE Standard for Software Verification and Validation Plans. IEEE Standard 1012-1998. New York, NY, 1998
21. Institute of Electrical and Electronics Engineers. Supplement to IEEE Standard for Software Verification and Validation Plans: Content Map to IEEE/EIA 12207.1-1997. IEEE Standard 1012a-1998. New York, NY, 1998
22. Institute of Electrical and Electronics Engineers. IEEE Standard for Software Reviews. IEEE Standard 1028-1997. New York, NY, 1997
23. Institute of Electrical and Electronics Engineers. IEEE Guide for Software Verification and Validation Plans. IEEE Standard 1059-1993. New York, NY, 1993
24. Institute of Electrical and Electronics Engineers. IEEE Standard for Application and Management of the Systems Engineering Process. IEEE Standard 1220-1998. New York, NY, 1998.
25. Institute of Electrical and Electronics Engineers. IEEE/EIA Software Life Cycle Processes. IEEE Standard 12207.0-1996. New York, NY, 1996.
26. Institute of Electrical and Electronics Engineers. IEEE/EIA Software Life Cycle Processes – Life Cycle Data.IEEE Standard 12207.1-1997. New York, NY, 1997
27. Institute of Electrical and Electronics Engineers. IEEE/EIA Software Life Cycle Processes – Implementation Considerations. IEEE Standard 12207.2-1997. New York, NY, 1997
28. Department of Energy, ASCI Program Plan. DOE/DP-99-000010592. Sandia National Laboratories, Albuquerque, NM, January 2000
29. DOE Notice 203.1, Software Quality Assurance, U.S. Department of Energy, Washington, DC, October 2, 2000
30. Guidelines for Software Measurement, SQAS97-001. April 1997
31. NWC Glossary of Preferred Software Engineering Terminology, SQAS90-001, October 1990
32. QC-1 – DOE/AL, Quality Criteria (QC-1). Revision 9. February 5, 1998.
33. DOE O 414.1A – Quality Assurance, United States Department of Energy, September 19, 1999
34. TBP-306 – NWC Technical Business Practice, Software Product Processes. Issue B. July 15, 1999
35. B Beizer, Software Testing Techniques. 2d ed. Van Nostrand Reinhold, New York, NY, 1990
36. R Black, Managing the Testing Process. Microsoft Press, Redmond, WA, 1999
37. C Kaner et al., Testing Computer Software. 2d ed. Van Nostrand Reinhold, New York, NY, 1993
38. G Myers, The Art of Software Testing. John Wiley and Sons, New York, NY, 1979
39. Workflows: The Exploratory Process Model (ExP). LA-UR-00-3697. Los Alamos National Laboratory Los Alamos, NM, 2000.
40. BW Boehm, “A Spiral Model of Software Development and Enhancement.” Computer, Volume 21, Issue 5, May 1988: 61-72
41. M Fowler, et al., UML Distilled. Addison-Wesley Publishing Co, Reading, MA, 1997
42. T Gilb and S Finzi (eds.), Principles of Software Engineering Management. Wokingham, England: Addison-Wesley Publishing Co, Reading, MA, 1988.
43. A Hodges, G Froehlich, M Pilch, and D Peercy, Sandia National Laboratories ASCI
44. Applications V&V Software Quality Engineering Guidelines: Practices and Work Products, Revision 1.10. Albuquerque, NM, Sandia National Laboratories, September 2000
45. WS Humphrey, Introduction to the Personal Software Process. Addison-Wesley
46. Publishing Co, Reading, MA, 1997
47. WS Humphrey, Managing the Software Process. Software Engineering Institute Series in Software Engineering. Addison-Wesley Publishing Co, Reading, MA, 1989
48. S McConnell, Code Complete. Microsoft Press, Redmond, WA, 1993
49. S McConnell, Rapid Development. Microsoft Press, Redmond, WA, 1996
50. P Naur and B Randell (eds.), Software Engineering: A Report on a Conference sponsored by the NATO Science Committee. NATO, Garmisch, Germany, 1969
51. MC Paulk, et al., The Capability Maturity Model: Guidelines for Improving the Software Process. SEI Series in Software Engineering, Carnegie Mellon University, Pittsburgh, PA, 1994
52. RS Pressman, Software Engineering: A Practitioner's Approach. 3d ed. New York: McGraw-Hill, 1992
53. Strategic Computing and Simulation Validation and Verification Program – Program Plan, Version 2.5, April1998
54. K Wiegers, Software Process Improvement: Eight Traps to Avoid. Crosstalk, September 1998

[chicoqueiroz]

Hi @iliant ,
I am in the process of including references into the bib file. Would you perhaps have those references in bibtex format already?

Thank you very much.

Best,

[fangohr]

Collecting references as bibtex sounds like a good idea to me. I have started this for another working group, see https://github.com/WSSSPE/testing-in-science/blob/master/literature/papers.bib

We also included (trivial) tests that attempt to create a latex file with the bibtex citations. By running them automatically, we get some confidence that the bibtex file is syntactically correct. (I suppose we could have looked for something like BibTeXlint). Output of the continuous integration is at https://travis-ci.org/WSSSPE/testing-in-science

[iliant]

Hi @chicoqueiroz,

No, I don't have the references. I am not sure that some of the national lab reports have references. I sifted the titles and checked their existence on the web in 2014 as part of a EU RI working group on software quality and metrics.

Ilian

[chicoqueiroz]

@fangohr : given the number of references and authors, continuous integration could be a good idea. Incidentally, the last article I've added to the list has you as an author.

@iliant : no problem, I'll try to organize and add them to the .bib file later.

Thank you both