# Growth 3: A/B Testlash va Statistik Rigor — MDE, Sample Size, p-value va Peeking Xatosi

> **Holat:** 2026-yil sentabr. "Menga yangi tugma yoqyapti, keling uni chiqaramiz" — degan yondashuv muhandislik emas. Growth Engineering qat'iy **statistik tekshirilgan tajribalar (Controlled Experiments)** ustiga quriladi. Agar A/B test noto'g'ri tashkil qilinsa, siz mahsulotni rivojlantirish o'rniga tasodifiy shovqinni (random noise) muvaffaqiyat deb o'ylab, xatoga yo'l qo'yasiz. Ushbu darsda statistik xavfsiz testlash qoidalari tushuntiriladi.

## Mundarija

1. [Gipoteza tuzishning ilmiy standarti](#gipoteza)
2. [Statistik parametrlar: Alfa, Beta, Power va MDE](#statistika)
3. [Namuna hajmini (Sample Size) hisoblash](#sample-size)
4. [Eng katta xato: "Peeking Problem" (Erta to'xtatish)](#peeking)
5. [Swift'da deterministik Feature Flag va foydalanuvchini guruhlash](#swift-flags)
6. [Amaliy Python: A/B test davomiyligini hisoblagich](#python-ab)
7. [Xulosa](#xulosa)
8. [Manbalar](#manbalar)

---

## 1. Gipoteza tuzishning ilmiy standarti {#gipoteza}

Har bir eksperiment aniq shablon bo'yicha yoziladi:

> **"Agar biz [o'zgarish X] qilsak, unda [metrika Y] [Z%] ga oshadi, chunki [sabab R]."**

Misol:
*"Agar biz ro'yxatdan o'tishni to'liq ekranni ko'rishdan oldin emas, birinchi dars tugagandan keyin so'rasak (X), 1-kunlik faollik konversiyasi (Y) 15% ga oshadi (Z), chunki foydalanuvchi hisob ochishdan oldin ilovaning haqiqiy foydasini his qiladi (R)."*

---

## 2. Statistik parametrlar: Alfa, Beta, Power va MDE {#statistika}

- **Nol gipoteza ($H_0$):** Boshqaruv (Control) va Sinov (Variant) guruhlari o'rtasida hech qanday farq yo'q.
- **I turdagi xato ($\alpha$ — False Positive):** Farq yo'q bo'lsa-da, "farq bor" deb xulosa chiqarish. Sanoat standarti: $\alpha = 0.05$ (95% ishonchlilik darajasi).
- **II turdagi xato ($\beta$ — False Negative):** Haqiqiy farq bor bo'lsa-da, uni sezmay qolish. Standart: $\beta = 0.20$ (Statistik quvvat $1 - \beta = 80\%$).
- **MDE (Minimum Detectable Effect):** Siz aniqlamoqchi bo'lgan minimal o'zgarish darajasi (masalan, konversiyani 5% dan 5.5% ga oshirish, MDE = 10%).

---

## 3. Namuna hajmini (Sample Size) hisoblash {#sample-size}

Testni boshlashdan **oldin** har bir guruhga nechtadan foydalanuvchi kerakligini hisoblash shart. Binomial metrikalar (masalan, Click yoki Purchase) uchun Evan Miller formulasi:

$$n \approx \frac{16 \cdot p \cdot (1 - p)}{\Delta^2}$$

Bu yerda $p$ — boshlang'ich konversiya, $\Delta$ — mutlaq kutilayotgan o'zgarish ($p \times \text{MDE}$).

---

## 4. Eng katta xato: "Peeking Problem" (Erta to'xtatish) {#peeking}

Eng keng tarqalgan xato — har kuni ertalab dashboard'ga qarab, $p < 0.05$ bo'lishi bilan testni darhol g'olib deb to'xtatish.

- **Nega bu xato?** Vaqtinchalik tebranishlar tufayli p-value test davomida bir necha bor 0.05 dan pastga tushib chiqadi.
- Agar siz har kuni "mo'ralab" qarasangiz, sizning haqiqiy xato qilish ehtimolingiz (False Positive Rate) 5% dan **30-40% gacha oshib ketadi!**
- **Qoida:** Test belgilangan namuna hajmiga yetgunga qadar yoki to'liq 1-2 haftalik sikl (hafta kunlari tsikli) tugamaguncha xulosa chiqarilmaydi.

---

## 5. Swift'da deterministik Feature Flag va foydalanuvchini guruhlash {#swift-flags}

Foydalanuvchini A yoki B guruhiga tasodifiy (`Double.random()`) ajratish mumkin emas — har safar app ochilganda foydalanuvchi bir xil guruhda qolishi shart. Buning uchun deterministik heshlash qo'llaniladi:

```swift
import Foundation

public enum ExperimentGroup: String, Sendable {
  case control = "Control"
  case variant = "Variant"
}

public struct ExperimentManager: Sendable {
  public static func assignGroup(userId: String, experimentKey: String) -> ExperimentGroup {
    let combined = "\(userId):\(experimentKey)"
    // Oddiy va tezkor deterministik hesh:
    let hash = abs(combined.hashValue)
    let bucket = hash % 100

    // 50/50 taqsimot:
    return bucket < 50 ? .control : .variant
  }
}
```

---

## 6. Amaliy Python: A/B test davomiyligini hisoblagich {#python-ab}

Quyidagi skript ilovangizning kunlik trafigiga qarab, eksperimentni necha kun yurgizish kerakligini hisoblab beradi:

```python
import math

def calculate_ab_sample_size(baseline_conv, mde_relative, daily_traffic):
    # Standart koeffitsient (alpha=0.05, power=0.80 uchun taxminan 16):
    delta = baseline_conv * mde_relative
    p = baseline_conv
    variance = p * (1 - p)
    
    # Har bir guruh uchun talab qilinadigan foydalanuvchilar:
    n_per_variant = math.ceil((16 * variance) / (delta ** 2))
    total_required = n_per_variant * 2
    required_days = math.ceil(total_required / daily_traffic)

    print(f"Boshlang'ich konversiya: {baseline_conv * 100:.1f}%")
    print(f"Kutilayotgan nisbiy o'sish (MDE): {mde_relative * 100:.1f}%")
    print(f"Har bir variant uchun zarur namuna: {n_per_variant:,} kishi")
    print(f"Jami zarur auditoriya: {total_required:,} kishi")
    print(f"Tavsiya etilgan test davomiyligi: {required_days} kun")

# Misol: Kuniga 1000 ta tashrif, 5% lik checkout konversiyasini 10% ga oshirish:
calculate_ab_sample_size(baseline_conv=0.05, mde_relative=0.10, daily_traffic=1000)
```

---

## 7. Xulosa {#xulosa}

- A/B testlar shaxsiy hissiyotlar va taxminlarni ob'ektiv ma'lumotlar bilan almashtiradi.
- Kichik auditoriyada juda kichik farqlarni (masalan 1% lik MDE) o'lchashga urinmang — buning uchun oylab vaqt ketishi mumkin.
- Peeking muammosidan qoching: g'olibni faqat rejalashtirilgan namuna hajmi to'plangach e'lon qiling.

---

## 8. Manbalar {#manbalar}

1. [Kohavi et al. — Trustworthy Online Controlled Experiments (Cambridge University Press)](https://experimentguide.com/)
2. [Evan Miller — Sample Size Calculator Formula](https://www.evanmiller.org/ab-testing/sample-size.html)
3. [Booking.com — 150 Successful and Unsuccessful Experiments](https://blog.booking.com/)