PENTESTY · EDITORIAL ANALYSIS · · 4 min read
AI code rewrites: how to evaluate performance and security controls
A fast benchmark does not demonstrate that an application preserved its access rules. Learn how to assess a migration developed with AI assistance.
By Pentesty
What a language comparison can establish
CyberSec Brazil reported a comparison of Campfire implementations in different languages. Basecampâs public repositories expose distinct implementations and verification procedures. This article does not repeat the reported performance figures: measurements depend on the tested revision and configuration. The case raises an engineering question: what evidence supports replacing an application in production?
Define parity before measuring speed
Start with observable behaviors the existing application must preserve. Include account creation, access recovery, permissions, attachments, search and administrative operations relevant to the product. For each behavior, record inputs, expected outcomes and persistent effects. A correct HTTP response may conceal an incorrect write or a missing authorization check, so the response alone is insufficient evidence.
Choose business examples that exercise boundaries. Can a person read another groupâs conversation? Does a suspended account retain a valid session? Does receiving the same event twice duplicate an operation? Convert these questions into acceptance criteria agreed with the product maintainers. This is a proposed migration assessment, rather than a claim that the cited projects contain those defects.
Compare controls using different identities
Run equivalent scenarios against both implementations using test accounts with different roles. Use recognizable synthetic records to identify an inappropriate response. Capture the initial state, authorized request and observed outcome. The assessment should establish both that an allowed operation succeeds and that an operation outside the accountâs scope is denied.
- Authorization: preserve boundaries between organizations, groups and roles.
- Sessions: check expiration, logout and revocation across agreed workflows.
- Data: compare validation, persistence and exposure through responses and logs.
- Integrations: verify identity, permissions and failure handling for each connected service.
Make the benchmark reproducible
Pin the code revision, runtime version, available resources and storage configuration. Document the volume and shape of synthetic data. If one implementation uses a warm cache while another starts empty, record the difference and run comparable scenarios before attributing the result to a programming language. Preserve the configuration alongside the measurement so another maintainer can repeat it.
Describe load as a set of user journeys, with a mix of reads, writes and persistent connections appropriate to the hypothesis being tested. Observe latency, errors and resource use together. A run returning more responses per second might reject work, omit checks or measure a different operation. Investigate these possibilities before accepting the improvement as evidence for migration.
Approve migration with evidence and a recovery path
The NIST SSDF provides practices for integrating security into development. In the approach proposed here, the team maintains a record connecting requirements, tests and results to the candidate revision. That record should let another accountable person repeat the scenario and understand acceptance, without relying on an explanation produced by the AI tool that helped write the implementation.
Separate technical acceptance from the production release decision. Confirm data compatibility and the feasibility of recovery before changing traffic. A schema migration can prevent recovery by simply restoring the previous application version. Rehearse the return path with an authorized, protected copy of data, defining who can interrupt the migration and how the restored service will be validated.
Pentesty can evaluate access paths and controls within an agreed scope. Combine that assessment with functional and performance tests maintained by the product team. The decision needs complementary evidence: business behavior remains valid, access boundaries hold and the implementation meets the operational objective that justified the rewrite. Record unresolved differences explicitly rather than allowing a headline throughput figure to stand in for acceptance.
Frequently asked questions
Does a benchmark prove Rust is always faster?
No. Its result describes an implementation under particular conditions. Compare revisions, configurations, completed operations, errors and resource use before drawing conclusions for your own application.
Does AI-generated code require different tests?
It must satisfy the same product requirements. Migration additionally needs evidence of parity with the previous implementation, including authorization boundaries and persistent effects, rather than relying on similar-looking output.
Which control should be assessed first?
Select the access boundaries with the greatest impact on the product. Use identities with different permissions and verify both authorized outcomes and the denial of operations outside each identityâs scope.
When can a rewrite be accepted?
When it meets agreed functional, security and operational criteria with reproducible results and a viable recovery path. A speed improvement alone does not establish that the other conditions have been met.
Sources and references
- Cyber Security Brazil: editorial news reference
- Basecamp: Campfire implementation in Rust
- Basecamp: Campfire implementation and verification in Elixir
- NIST SP 800-218: secure software development
Original Pentesty analysis, published October 15, 2026. The proposed scenarios are editorial guidance, with scope and validation to be defined for each environment.