Is click-fraud protection worth it? Test the value

scenario,arm,currency,closed_customers,contribution_before_acquisition,gross_ad_charges,verified_credits,tool_fee,review_cost,flagged_spend_claim
observed_gain,control,USD,10,1600.00,1020.00,20.00,0.00,50.00,0.00
observed_gain,protected,USD,12,1800.00,1030.00,30.00,40.00,80.00,250.00
unchanged_outcome,control,USD,10,1600.00,1020.00,20.00,0.00,50.00,0.00
unchanged_outcome,protected,USD,10,1600.00,1030.00,30.00,40.00,80.00,250.00
no_closed_customers,control,USD,0,0.00,400.00,0.00,0.00,20.00,0.00
no_closed_customers,protected,USD,0,0.00,400.00,0.00,40.00,20.00,250.00

#!/usr/bin/env python3
"""Local arithmetic example. No network requests, account access or output files.

Usage: python3 protection-value-demo.py protection-value-example.csv
Inputs must be allocated to a comparable scope/period before this program runs.
The script does not establish treatment assignment or causal savings.
"""
import csv
import json
import io
import re
import sys
from decimal import Decimal, ROUND_HALF_UP
from pathlib import Path

FIELDS = ['scenario', 'arm', 'currency', 'closed_customers',
          'contribution_before_acquisition', 'gross_ad_charges',
          'verified_credits', 'tool_fee', 'review_cost', 'flagged_spend_claim']
CENT = Decimal('0.01')
LIMIT = Decimal('1000000000000')


def money(value, field):
    if not re.fullmatch(r'-?[0-9]+(?:\.[0-9]{1,2})?', value or ''):
        raise ValueError(field + ': use an unformatted decimal with at most two places')
    number = Decimal(value)
    if abs(number) > LIMIT:
        raise ValueError(field + ': exceeds this example\'s amount limit')
    if field != 'contribution_before_acquisition' and number < 0:
        raise ValueError(field + ': use a non-negative amount')
    return number


def show(value):
    return str(value.quantize(CENT, rounding=ROUND_HALF_UP))


def evaluate(text):
    reader = csv.DictReader(io.StringIO(text), strict=True)
    if reader.fieldnames != FIELDS:
        raise ValueError('CSV headers/order do not match the example')
    groups = {}
    for line, row in enumerate(reader, 2):
        if None in row or any(value is None for value in row.values()):
            raise ValueError('row %s: missing or extra fields' % line)
        row = {key: value.strip() for key, value in row.items()}
        scenario, arm, currency = row['scenario'], row['arm'], row['currency']
        if not re.fullmatch(r'[a-z0-9_]{1,64}', scenario):
            raise ValueError('row %s: invalid scenario name' % line)
        if arm not in ('control', 'protected') or not re.fullmatch(r'[A-Z]{3}', currency):
            raise ValueError('row %s: invalid arm or currency' % line)
        if not re.fullmatch(r'[0-9]+', row['closed_customers']):
            raise ValueError('row %s: closed_customers must be a non-negative integer' % line)
        customers = int(row['closed_customers'])
        amounts = {field: money(row[field], field) for field in FIELDS[4:]}
        if amounts['verified_credits'] > amounts['gross_ad_charges']:
            raise ValueError('row %s: credits must belong to the selected gross charges; do not use a carried balance' % line)
        group = groups.setdefault(scenario, {})
        if arm in group:
            raise ValueError('row %s: duplicate scenario/arm' % line)
        net_ad = amounts['gross_ad_charges'] - amounts['verified_credits']
        acquisition = net_ad + amounts['tool_fee'] + amounts['review_cost']
        net = amounts['contribution_before_acquisition'] - acquisition
        # flagged_spend_claim is validated but never counted as savings.
        group[arm] = {'currency': currency, 'net': net,
                      'report': {'net_ad_cost': show(net_ad),
                                 'all_in_acquisition_cost': show(acquisition),
                                 'net_contribution': show(net),
                                 'cost_per_closed_customer': show(acquisition / customers) if customers else None}}
    if not groups:
        raise ValueError('CSV has no comparison rows')
    output = []
    for scenario, group in sorted(groups.items()):
        if set(group) != {'control', 'protected'}:
            raise ValueError(scenario + ': include exactly one control and one protected row')
        if group['control']['currency'] != group['protected']['currency']:
            raise ValueError(scenario + ': comparison currencies must match')
        output.append({'scenario': scenario, 'currency': group['control']['currency'],
                       'control': group['control']['report'],
                       'protected': group['protected']['report'],
                       'observed_net_difference': show(group['protected']['net'] - group['control']['net'])})
    return output


def main(argv):
    if len(argv) != 1:
        raise ValueError('usage: python3 protection-value-demo.py protection-value-example.csv')
    # Parse and validate every row before emitting any report.
    output = evaluate(Path(argv[0]).read_text(encoding='utf-8'))
    print(json.dumps(output, indent=2))


if __name__ == '__main__':
    try:
        main(sys.argv[1:])
    except (ValueError, OSError, csv.Error) as error:
        print('protection-value-demo: ' + str(error), file=sys.stderr)
        sys.exit(1)

CLICK-FRAUD PROTECTION — VALUE EVALUATION

One problem the protection must solve:
Primary business outcome and definition:
Measurement, lead acceptance and qualification already verified:

SCOPE AND ASSIGNMENT
Accounts / campaigns / sites / channels in scope:
Eligibility and assignment unit:
Control condition and protected condition:
How treatment stays out of the control:
Account-level exclusion lists affecting both arms:
Site-wide rules, shared audiences and cross-campaign identity effects:
If isolation is unavailable, comparison limitations to report:
Other changes held steady / unavoidable changes recorded:
Start / end / account timezone:
Conversion and business-qualification follow-up cutoff:
Predefined sample/duration, stop conditions and decision rule:
Owner, legitimate-visitor exception path and rollback:

EVIDENCE (keep each stage separate)
Observed visit and review reason:
Protection decision:
Website action, if applicable:
Platform exclusion request / synchronization status / scope:
Native invalid-traffic reporting:
Verified invoice adjustment mapped to the original charge period:
Accepted / qualified / closed business outcomes:
Legitimate customers affected and resolution:

COST INPUTS PER ARM, FOR THE SAME ALLOCATED SCOPE
Currency and FX policy if needed (the demo does not convert currency):
Contribution from realized customers after product/fulfilment/refund costs,
but before ads, tool fees and review costs:
Gross ad charges:
Verified credits belonging to those selected charges:
Net ad cost (gross minus verified credits; do not subtract credits again):
Tool/license fee, allocated under a rule chosen before the comparison:
Implementation and review time cost, allocated consistently:
Unresolved refunds / sales / credits and sensitivity range:
Flagged-spend estimate (record separately; not booked savings):

DECISION
Observed net contribution for control / protected:
Observed difference, with uncertainty and contamination limits:
Evidence supporting causal interpretation, if any:
Enough evidence to continue / revise / stop / remain inconclusive:
Next review date and owner:

Do not paste customer records into shared notes.
A positive arithmetic difference alone is not an experiment result.