# Secret Scanning — DevSecOps

Source: https://www.skillbyai.com/en/devsecops/dso-secret-scanning

> Describe how secret scanners detect credentials using patterns and entropy, and where to run them in the workflow.

## Patterns, entropy and verification

**Secret scanners** such as Gitleaks, TruffleHog and the scanning built into code hosts look for credentials in code, history and sometimes logs. They use three techniques. **Patterns**: many providers use recognisable prefixes, so a regular expression like `AKIA[0-9A-Z]{16}` finds AWS access key IDs. **Entropy**: random-looking strings have high Shannon entropy, which catches generic tokens without a known prefix, at the cost of more false positives. **Verification**: some tools try the found credential against the provider's API to tell you whether it is live, which helps you prioritise. Run scanning in three places: a **pre-commit hook** to stop the mistake locally, a **CI job** on every pull request because hooks can be skipped, and **push protection** on the code host, which blocks a push containing a known secret format. Also scan full history once when adopting the tool, because old leaks still matter.

## A toy scanner: pattern plus entropy

Real scanners are far more complete, but the mechanics look like this. The key below is the documented AWS example value, not a real credential.

```python
import math, re
from collections import Counter

def entropy(s):
    n = len(s)
    return -sum(c / n * math.log2(c / n) for c in Counter(s).values())

AWS_ID = re.compile(r"AKIA[0-9A-Z]{16}")
lines = ['region = "ap-south-1"',
         'aws_key = "AKIAIOSFODNN7EXAMPLE"',
         'token = "q8Zr2LmX9vTn4KpW7sYb3HdF"']
for line in lines:
    value = line.split('"')[1]
    hit = "pattern" if AWS_ID.search(line) else ("entropy" if entropy(value) > 4.0 else "-")
    print(f"{entropy(value):.2f}  {hit:<8} {line}")
```

Output:

```
3.12  -        region = "ap-south-1"
3.68  pattern  aws_key = "AKIAIOSFODNN7EXAMPLE"
4.58  entropy  token = "q8Zr2LmX9vTn4KpW7sYb3HdF"
```

## Allow-list false positives explicitly

Test fixtures and example keys will trigger findings. Use the scanner's allow-list or inline ignore comment with a reason, rather than turning the rule off for the whole repository.

**Quiz:** Why run a secret scan in CI if developers already have a pre-commit hook?

- [ ] CI scanners are always more accurate
- [ ] Pre-commit hooks cannot read files
- [x] Hooks are local and can be skipped or not installed, so CI is the enforced check
- [ ] CI scans are required by Git

*Answer:* Hooks are local and can be skipped or not installed, so CI is the enforced check. A pre-commit hook gives fast feedback, but only a server-side check guarantees every change was scanned.