If you really want to vibe code or otherwise treat AI-generated code as a black box, your tests have to be immaculate and extensive. For example, are you testing for all of the following (not an exhaustive list and in no particular order)?
All Functionality, including all corner cases.
No unnecessary functionality or complexity introduced by the AI.
Abuse cases for all data entry points, including the complete UI and any APIs
Security vulnerabilities, privacy guarantees.
Legal and regulatory compliance.
Usability, frustration-free UX.
Updatability.
Scalability.
Fault tolerance.
Reliable backup and recovery.
Performant under load, elasticity
DoS vulnerabilities and rejection.
Data integrity tested against real data.
Toxic-data containment.
Efficient and cost-effective use of external services like AWS.
Efficient use of the network and database (if you have one).
Unnecessary complexity (e.g., cyclomatic complexity).
Maximized Efficiency/execution speed in heavily executed code.
UI responsiveness.
Minimal size (essential for context management).
Unused features and dead code.
If your app contains AI: guardrail/sandbox effectiveness and coverage
Testing the tests (coverage, mutation testing, etc.).
Code quality in general (e.g., readability and good structure if manual inspection is required).
Refactorability (can you change the code without tests changing)?
Unnecessary coupling.
Unnecessary dependencies, including "god classes" or equivalent.
Back doors.
Don’t get me wrong. Vibe coding has its place (e.g., throwaway prototypes), but mindlessly accepting AI-generated code in a production app without adequate testing is irresponsible. Your customers will bear the brunt.
Not all projects need to check all the boxes. It depends on what the system does, how many customers you have, etc. The real issue is that I've seen a lot of programs, particularly the ones that are vibe coded, that check NONE of the boxes.
To be fair, I havn‘t seen a single project that was coded manually and did check all your boxes.