Rest Framework Tests: Improve speed and repeatability #10503

Maffooch · 2024-07-03T01:30:46Z

ViewSets: Support get_queryset function for more flexible queryset manipulation
Authorization helpers: Specify order by id for most queries. For queries that already had a specified ordering, it was left alone if the field was unique on the model, such as Products. Otherwise, the ordering was changed to id
Viewsets: enagegement checklist viewset was not used, so removed
Tests: Accomodate the enforced ordering
Tests: Reduce logs for CICD tests to speed up testing overall by eliminating write operations
Tests: Enable FailFast mode ("Tells Django to stop running the test suite after first failed test.")
Tests: Enable shuffle tests to identify areas of tests that are dependent on each other
Tests: Enable parallel tests to drastically reduce testing time
- In local testing, I went from ~240 seconds to ~45 seconds

[sc-6766]

dryrunsecurity · 2024-07-03T01:31:42Z

Hi there 👋, @DryRunSecurity here, below is a summary of our analysis and findings.

DryRun Security	Status	Findings
Server-Side Request Forgery Analyzer	✅	0 findings
Configured Codepaths Analyzer	✅	0 findings
IDOR Analyzer	✅	0 findings
Sensitive Files Analyzer	❕	1 finding
SQL Injection Analyzer	✅	0 findings
Authn/Authz Analyzer	❕	52 findings
Secrets Analyzer	✅	0 findings

Note

🟢 Risk threshold not exceeded.

Change Summary (click to expand)

The following is a summary of changes in this pull request made by me, your security buddy 🤖. Note that this summary is auto-generated and not meant to be a definitive list of security issues but rather a helpful summary from a security perspective.

Summary:

The changes in this pull request focus on improving the reliability, efficiency, and security of the DefectDojo application's unit tests and API endpoints. The key changes include:

Optimizing the unit test execution by adding parallel test running, excluding non-parallel tests, and using the --failfast option.
Enhancing the authorization and access control mechanisms for various entities, such as engagements, findings, JIRA projects and issues, and product-related data.
Introducing new API endpoints for various features, including notifications, surveys, announcements, and more, while ensuring proper authorization and schema validation.
Improving the overall test coverage and robustness by adding new test cases for various API endpoints and functionality.

These changes demonstrate a strong commitment to application security, as they focus on ensuring that the application's core functionality and data access are properly secured and tested. The changes address potential security concerns, such as unauthorized access, data leakage, and input validation, which are crucial for maintaining the overall security posture of the DefectDojo application.

Files Changed:

docker/entrypoint-unit-tests.sh: Optimized the unit test execution by adding parallel test running, excluding non-parallel tests, and using the --failfast option.
docker-compose.override.unit_tests_cicd.yml: Improved the test environment configuration by setting environment variables for logging, database, and Celery.
docker/entrypoint-unit-tests-devDocker.sh: Implemented similar optimizations for the unit test execution as in the entrypoint-unit-tests.sh file.
dojo/engagement/queries.py: Improved the authorization logic for retrieving authorized engagements.
dojo/cred/queries.py: Optimized the retrieval of authorized credential mappings.
dojo/api_v2/views.py: Introduced new API endpoints and enhanced the authorization and access control mechanisms.
dojo/endpoint/queries.py: Improved the authorization logic for retrieving authorized endpoints and endpoint statuses.
dojo/group/queries.py: Optimized the retrieval of authorized group members.
dojo/finding/queries.py: Enhanced the authorization logic for retrieving authorized findings and stub findings.
dojo/product_type/queries.py: Optimized the retrieval of authorized product type members and groups.
dojo/risk_acceptance/queries.py: Optimized the retrieval of authorized risk acceptances.
dojo/product/queries.py: Improved the authorization logic for various product-related entities.
dojo/jira_link/queries.py: Enhanced the authorization logic for JIRA projects and issues.
dojo/tool_product/queries.py: Optimized the retrieval of authorized tool product settings.
dojo/test/queries.py: Improved the authorization logic for tests and test imports.
dojo/urls.py: Updated the Django URL configuration to include new API endpoints.
unittests/dojo_test_case.py: Minor changes to the test utility methods.
unittests/test_apiv2_notifications.py: Added unit tests for the Notifications API.
unittests/test_apiv2_scan_import_options.py: Added unit tests for the scan import API endpoint.
unittests/test_import_reimport.py: Added unit tests for the import and reimport functionality.
unittests/test_jira_config_engagement.py: Added unit tests for the Jira integration with engagements.
unittests/tools/test_deepfence_threatmapper_parser.py: Added unit tests for the Deepfence Threatmapper parser.
unittests/test_rest_framework.py: Added new test classes and enhanced the existing tests for various API endpoints.

Powered by DryRun Security

mtesauro

Approved

Maffooch · 2024-07-03T15:00:07Z

@kiblik I was not planning on figuring out the shuffle issue quite yet, but addressing inconsistent ordering in API responses largely corrected those problems. I did not intended to steal your thunder, and I hope you do not feel that way 🙏🏼

I also did not plan on the parallel flag either, but I ran it just for kicks and encountered a different error this time around. It keyed me into to looking at how files were passed as class variables. Files, stack traces, and a few other things are not able to be serialized. Anyway, I thought you'd like to know what the underlying issue was 😄

cneill

Just a couple spots I wasn't sure about - otherwise this looks good! Very excited for faster, more stable test runs 😄

cneill · 2024-07-03T16:42:28Z

unittests/test_jira_config_engagement.py

-        engagement = self.add_engagement_without_jira_project(expected_delta_jira_project_db=0)
-        self.empty_jira_project_for_engagement(engagement, expected_delta_jira_project_db=0)
-        self.assertEqual(jira_mock.call_count, 0)
+        with contextlib.suppress(ValueError):


Could you add a comment about why suppressing ValueErrors is necessary here?

cneill · 2024-07-03T16:42:44Z

unittests/test_jira_config_engagement.py

-        # expecting ValueError as we can't delete existing JIRA Projects
-        self.empty_jira_project_for_engagement(engagement, expected_delta_jira_project_db=0, expect_error=True)
-        self.assertEqual(jira_mock.call_count, 1)
+        with contextlib.suppress(ValueError):


Same as above - could you add a quick comment explaining this line?

kiblik · 2024-07-03T20:51:00Z

@kiblik I was not planning on figuring out the shuffle issue quite yet, but addressing inconsistent ordering in API responses largely corrected those problems. I did not intended to steal your thunder, and I hope you do not feel that way 🙏🏼

I also did not plan on the parallel flag either, but I ran it just for kicks and encountered a different error this time around. It keyed me into to looking at how files were passed as class variables. Files, stack traces, and a few other things are not able to be serialized. Anyway, I thought you'd like to know what the underlying issue was 😄

No hard feelings 😄 I will be happy when it will be solved

Rest Framework Tests: Improve speed and repeatability

d6a282b

Maffooch added the performance performance improvement or bug report label Jul 3, 2024

github-actions bot added docker apiv2 unittests labels Jul 3, 2024

Maffooch added 2 commits July 2, 2024 20:32

Fix ruff

ef3ae7d

Fix flake8

ebe9f12

mtesauro approved these changes Jul 3, 2024

View reviewed changes

Maffooch mentioned this pull request Jul 3, 2024

Bump djangorestframework from 3.14.0 to 3.15.2 #10431

Merged

cneill reviewed Jul 3, 2024

View reviewed changes

Add comments

1e47024

cneill approved these changes Jul 3, 2024

View reviewed changes

hblankenship approved these changes Jul 8, 2024

View reviewed changes

blakeaowens approved these changes Jul 8, 2024

View reviewed changes

Maffooch merged commit cf864b1 into DefectDojo:bugfix Jul 8, 2024
125 checks passed

Maffooch mentioned this pull request Jul 8, 2024

Add extra path in Engagement endpoint for checklists #7771

Merged

kiblik mentioned this pull request Jul 8, 2024

feat(tests): Run unittests with --parallel #10408

Closed

Maffooch mentioned this pull request Jul 9, 2024

API: Convert get_filterset calls to get_queryset #10543

Merged

Maffooch deleted the order branch July 16, 2024 22:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rest Framework Tests: Improve speed and repeatability #10503

Rest Framework Tests: Improve speed and repeatability #10503

Maffooch commented Jul 3, 2024 •

edited

Loading

dryrunsecurity bot commented Jul 3, 2024 •

edited

Loading

mtesauro left a comment

Maffooch commented Jul 3, 2024

cneill left a comment

cneill Jul 3, 2024

cneill Jul 3, 2024

kiblik commented Jul 3, 2024

Rest Framework Tests: Improve speed and repeatability #10503

Rest Framework Tests: Improve speed and repeatability #10503

Conversation

Maffooch commented Jul 3, 2024 • edited Loading

dryrunsecurity bot commented Jul 3, 2024 • edited Loading

mtesauro left a comment

Choose a reason for hiding this comment

Maffooch commented Jul 3, 2024

cneill left a comment

Choose a reason for hiding this comment

cneill Jul 3, 2024

Choose a reason for hiding this comment

cneill Jul 3, 2024

Choose a reason for hiding this comment

kiblik commented Jul 3, 2024

Maffooch commented Jul 3, 2024 •

edited

Loading

dryrunsecurity bot commented Jul 3, 2024 •

edited

Loading