CIS Benchmark Tools Suck.
There, I said it.
Tell me:
- Why some checks have to be answered manually? (where some = too many)
- Why the “cis value” and the observed system state(s) are NOT displayed in the report? (you have to dig through an XML file for them, pah!)
- Why is said data not presented in detail, i.e. showing partial failures as opposed to simple traffic lights? (I understand clarity and a violation is a failure, but detail could be supplied IN THE DETAILS SECTION!)
- Why referencing to external sources is rubbish? (it’s nice to cross check, even if the NSA did write it)
- Why are they written in Java? (don’t get me wrong, I’m a Java fan boy as much as the next, but sometimes a JVM is not available and installing it is just not gonna fly!)
- Why are they NOT maintained? (e.g. rule checks relating to pre-service-pack “original” state; OK, this might be forgiven as an update standard is probably required before tool updates can occur, but the staleness is merely moved up the chain to something more important: it’s origin!)
- And WHY SOME RULES ARE ACTUALLY WRONG? (e.g. SafeDllSearchMode is default ON in Windows 2003, yet the tool reports this as failed as it can’t find the registry key *sigh*)
Unfortunately, the CIS tools detract from the overall high quality of the standards themselves. However, they’re better than nothing.
I would like to see more maintenance. I would also like to see the rule file made more manageable (e.g. provide a UI for value tweaking, rule editing). Clearly, somewhere right now a wheel is being reinvented.