Reading the security report
What is this about?
The security report shows what we checked and what came out of it. We produce it monthly; for now we start the run by hand. It is a self-assessment and not an independent test.
Protection goal
You should be able to read what we claim instead of having to believe it. For that, the report has to be understandable without security knowledge.
How it works
The report consists of fixed parts that return every month:
- In numbers: number of automated tests, attack scenarios checked, result of the mutation testing, attempts to access other users’ data in a test environment and the result of the scan of the beta instance.
- Attack scenarios: a table with the 14 scenarios and their result: passed, passed with documented limits or known remaining measure.
- Found and fixed: what the checks uncovered and how it was corrected.
- Open points: what we know about and have not fixed yet.
- Limits: what the tests do not prove.
Measures
We follow three rules so that the report does not sugarcoat anything:
- The report is produced from a real test run, not from a list maintained by hand.
- A scenario only counts as passed if its tests appear in the current run and pass. An earlier result is not carried over silently, and a regression is reported.
- The section on limits belongs to every report and is not shortened.
If you find a vulnerability, the reporting route is in the security policy at bluegreen-solutions.de/sicherheitsrichtlinie.
What we do not promise
- The report is a snapshot. It describes defined properties under documented conditions and says nothing about future vulnerabilities.
- It does not replace an independent penetration test. There has not been one so far.
- Load tests, social engineering and attackers with root access to the server are not covered.
How we check this
The assignment of scenarios to test classes is fixed in the source code. A test makes sure that every test class named there exists, so that the report does not claim anything that is not checked. This month’s figures are in the report itself and are not repeated here.
Evidence tests
| Test | What the test proves |
|---|---|
RedTeamValidationCatalogCoverageTest |
Every test class the report assigns to an attack scenario really exists. |
Change history
- : First version.