HNNewShowAskJobs
Built with Tanstack Start
Branching with or Without PII: The Future of Environments(neon.com)
19 points by emschwartz 10 days ago | 3 comments
  • vedhant9 days ago

    This looks amazing! I have a question. When we create a branch, are we cloning the entire production data? From what I understood was that neon runs a separate compute layer but the storage is the same.

    • gsalomao9 days ago |parent

      Neon's engineer here: If you're asking about how branches work in general, they use Copy-on-Write (CoW), which means that no data is copied when they are created. However, when writes occur in a branch, only the modified specific Postgres pages are copied (working like deltas in your data), which is why they are created very quickly and don't increase storage usage when created, increasing only in the modified portion.

      If you're asking about how the anonymized branches work, they currently use static masking, which means that all PII data selected for masking will be overridden in the anonymized branch, resulting in increased storage usage based on the amount of selected masked data. However, the team is also working on dynamic masking, which doesn't change the stored data but applies the masking rules when querying it.

      • amw-zero2 days ago |parent

        And to clarify - only writes on the _branch_ lead to a copy being created right? Writes on the original don’t get propagated to the branch?