QA 21: CI/CD'da testlar — GitHub Actions, hisobotlar, sharding, flaky testlar va quality gate

🧪 QA Engineer — Zero to Hero

QA 21: CI/CD'da testlar — GitHub Actions, hisobotlar, sharding, flaky testlar va quality gate

Holat: 2026-yil oktabr. Dars GitHub Actions'ning shu oydagi eng so'nggi major versiyalari (actions/checkout@v7, actions/setup-node@v7, actions/upload-artifact@v7, actions/download-artifact@v8), Playwright 1.64 va mcr.microsoft.com/playwright:v1.64.0-noble Docker image'i asosida yozilgan. Barcha YAML fayllar parser bilan tekshirilgan; sharding, blob hisobotlarni birlashtirish va flaky aniqlash lokal ishga tushirib sinalgan.

Bu darsdan keyin siz: testlaringizni "mening kompyuterimda o'tadi" holatidan har commit'da avtomatik ishlaydigan, natijasi hammaga ko'rinadigan va sifatsiz kodni main'ga o'tkazmaydigan tizimga aylantirasiz.

• CI/CD tushunchalarini va testlar pipeline'ning qaysi bosqichida ishlashini tushuntirasiz;
• Playwright uchun GitHub Actions workflow'larini yozasiz: PR smoke, nightly regression, qo'lda ishga tushirish;
• testlarni sharding bilan 4 mashinaga bo'lib, hisobotlarni bitta HTML'ga birlashtirasiz;
• HTML, JUnit, Allure hisobotlari va Telegram/Slack xabarlarini sozlaysiz;
• flaky testlarni aniqlash, karantin va tuzatish jarayonini qurasiz;
• quality gate'lar bilan "qizil" kodning main'ga tushishini to'xtatasiz.

Mundarija

1. CI/CD: asosiy tushunchalar
2. Testlar pipeline'ning qayerida ishlaydi
3. Birinchi workflow: GitHub Actions
4. Secrets, vars va muhitlar
5. Sharding: testlarni 4 mashinaga bo'lish
6. Trigger'lar: PR smoke, nightly, qo'lda ishga tushirish
7. Docker konteynerda ishga tushirish
8. Hisobotlar va xabarnomalar
9. Flaky testlar: aniqlash, karantin, tuzatish
10. Quality gate'lar
11. GitLab CI, Jenkins va mobil CI
12. Amaliy mashqlar
13. Xulosa
14. Manbalar

1. CI/CD: asosiy tushunchalar {#tushunchalar}

Continuous Integration (CI) — dasturchilar o'zgarishlarni tez-tez umumiy branch'ga birlashtiradi va har bir o'zgarish avtomatik build va testlardan o'tadi. Maqsad — xatoni kiritilgan daqiqada topish, bir hafta keyin emas.

Continuous Delivery (CD) — testlardan o'tgan har bir versiya istalgan payt reliz qilishga tayyor turadi (reliz tugmasini odam bosadi). Continuous Deployment — bir qadam oldinga: testlardan o'tgan o'zgarish avtomatik production'ga chiqadi.

QA uchun asosiy xulosa: avtomatik ishga tushmaydigan test — unutilgan test. Siz yozgan 300 ta Playwright testi faqat sizning noutbukingizda ishlasa, ular jamoa uchun deyarli foydasiz.

🧠 Esda tuting: CI — shift-left'ning amaliy ko'rinishi (QA 3): bug qancha erta topilsa, uni tuzatish shuncha arzon. PR'da 5 daqiqada topilgan bug — reliz kuni topilgan bug'dan bir necha barobar arzon.

2. Testlar pipeline'ning qayerida ishlaydi {#qayerda}

Hamma testni har commit'da ishga tushirish — sekin va qimmat. Shuning uchun testlar qatlamlarga bo'linadi:

CI pipeline: trigger, tayyorlash, tezkor tekshiruv, sharded regression, hisobot va quality gate

Bu darsda QA 19 va QA 20da qurgan qa-demo-tests loyihasini GitHub Actions'ga ulaymiz.

3. Birinchi workflow: GitHub Actions {#birinchi-workflow}

3.1. Config: CI'ni tanib olish

GitHub Actions har bir job'da CI=true muhit o'zgaruvchisini o'zi o'rnatadi. Config'ning CI'ga xos qatorlari shunga tayanadi:

// playwright.config.ts (qismi)
export default defineConfig({
  forbidOnly: !!process.env.CI, // CI'da test.only qolib ketsa — xato
  retries: process.env.CI ? 2 : 0, // CI'da 2 marta qayta urinish
  workers: process.env.CI ? 1 : undefined,
  reporter: process.env.CI
    ? [["github"], ["html", { open: "never" }]]
    : [["list"], ["html", { open: "never" }]],
  // ...
});

• forbidOnly — kimdir lokal debug uchun test.only qoldirib commit qilsa, CI faqat o'sha bitta testni ishlatib "yashil" bo'lib qolardi. Bu flag buning oldini oladi.
• workers: 1 — Playwright hujjatlari CI'da barqarorlik va qayta ishlab chiqarish uchun 1 ta worker tavsiya qiladi; parallellikni esa sharding orqali olamiz (5-bo'lim).
• github reporter — xatolarni GitHub UI'da kodning to'g'ri qatoriga annotatsiya qilib ko'rsatadi.

3.2. Workflow fayli

# .github/workflows/playwright.yml
name: Playwright Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    timeout-minutes: 30
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7

      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run Playwright tests
        run: npx playwright test
        env:
          SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }}
          SAUCE_PASSWORD: ${{ secrets.SAUCE_PASSWORD }}

      - uses: actions/upload-artifact@v7
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

Qatorma-qator:

💡 Maslahat: Brauzerlarni actions/cache bilan keshlash shart emas: Playwright hujjatlariga ko'ra keshni tiklash vaqti brauzerni yuklab olish vaqti bilan deyarli bir xil. --with-deps esa baribir har safar kerak.

⚠️ Diqqat: Playwright hujjatlaridagi namunalarda ba'zan eskiroq major versiyalar turadi (masalan, checkout@v6, upload-artifact@v4). 2026-yil oktabr holatiga eng so'nggilari: checkout@v7, setup-node@v7, upload-artifact@v7, download-artifact@v8. Ularni qo'lda kuzatish o'rniga Dependabot'ni yoqing (pastda).

3.3. Natijani ko'rish

1. GitHub → Actions tab → kerakli run.
2. Yiqilgan testlar github reporter tufayli xulosada va PR'ning "Files changed" qismida annotatsiya bo'lib ko'rinadi.
3. Pastdagi Artifacts bo'limidan playwright-report arxivini yuklab, ochasiz:

npx playwright show-report ~/Downloads/playwright-report

HTML hisobotda har bir yiqilgan test uchun skrinshot, xato matni va (retry bo'lgan bo'lsa) trace bor — trace viewer'da qadam-baqadam DOM, tarmoq va konsolni ko'rasiz (QA 18).

3.4. Versiyalarni avtomatik yangilash: Dependabot

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: github-actions
    directory: /
    schedule:
      interval: weekly
  - package-ecosystem: npm
    directory: /
    schedule:
      interval: weekly

Dependabot har hafta action'lar va npm paketlari (shu jumladan @playwright/test) uchun yangilash PR'larini ochadi — va bu PR'lar ham sizning testlaringizdan o'tadi. QA uchun ideal: yangilanish testlarni buzsa, buni merge'dan oldin ko'rasiz.

4. Secrets, vars va muhitlar {#secrets}

Login, parol, token — hech qachon kodda yoki YAML'da ochiq yozilmaydi.

Muhim xavfsizlik qoidalari:

• GitHub log'da chiqib qolgan secret qiymatlarini avtomatik yashiradi (***), lekin hujjatlar secret'larni ataylab chop etmaslikni so'raydi. echo $PASSWORD — taqiqlangan odat.
• Fork'dan kelgan PR'larda GITHUB_TOKENdan boshqa secret'lar runner'ga berilmaydi. Ochiq repoda tashqi hissa qo'shuvchining PR'i secret talab qiladigan testlarni ishga tushira olmaydi — buni pipeline dizaynida hisobga oling.
• Lokal .env fayl .gitignoreda qoladi, CI esa o'sha nomdagi qiymatlarni secrets'dan oladi. Kod ikkala holatda ham bir xil: process.env.SAUCE_PASSWORD.

🎯 Intervyuda: "Test parollarini CI'da qanday saqlaysiz?" — CI platformasining secret store'ida (GitHub Secrets, GitLab CI/CD variables "masked/protected"), muhit o'zgaruvchisi sifatida testga beriladi; kodda va log'da yo'q. Test akkauntlari production akkauntlaridan alohida va minimal huquqli bo'ladi.

5. Sharding: testlarni 4 mashinaga bo'lish {#sharding}

Testlar soni o'sgan sari bitta runner'dagi vaqt ham o'sadi. Sharding test to'plamini N bo'lakka bo'lib, har birini alohida mashinada parallel ishga tushiradi:

npx playwright test --shard=1/4   # 1-chorak
npx playwright test --shard=2/4   # 2-chorak ...

Har bir shard o'z hisobotini yozadi. Ularni birlashtirish uchun CI'da blob reporter ishlatamiz:

// playwright.config.ts
reporter: process.env.CI
  ? [['blob'], ['github']]
  : [['list'], ['html', { open: 'never' }]],

5.1. Sharded workflow

# .github/workflows/regression.yml
name: Regression (sharded)

on:
  push:
    branches: [main]
  schedule:
    - cron: "0 1 * * 1-5" # dushanba–juma 01:00 UTC = 06:00 Toshkent
  workflow_dispatch:
    inputs:
      grep:
        description: "Faqat shu teglar (masalan @smoke). Bo'sh = hammasi"
        required: false
        default: ""
      base_url:
        description: "Qaysi muhitda (URL)"
        required: false
        default: "https://www.saucedemo.com"

concurrency:
  group: regression-${{ github.ref }}
  cancel-in-progress: true

jobs:
  playwright-tests:
    timeout-minutes: 30
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium

      - name: Run tests (shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
        run: |
          if [ -n "$GREP" ]; then
            npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --grep "$GREP"
          else
            npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
          fi
        env:
          GREP: ${{ inputs.grep }}
          BASE_URL: ${{ inputs.base_url || vars.BASE_URL || 'https://www.saucedemo.com' }}
          SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }}
          SAUCE_PASSWORD: ${{ secrets.SAUCE_PASSWORD }}

      - uses: actions/upload-artifact@v7
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report
          retention-days: 1

  merge-reports:
    if: ${{ !cancelled() }}
    needs: [playwright-tests]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci

      - uses: actions/download-artifact@v8
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true

      - name: Merge into HTML report
        run: npx playwright merge-reports --reporter html ./all-blob-reports

      - uses: actions/upload-artifact@v7
        with:
          name: html-report--attempt-${{ github.run_attempt }}
          path: playwright-report
          retention-days: 14

  notify:
    if: ${{ failure() }}
    needs: [playwright-tests]
    runs-on: ubuntu-latest
    steps:
      - name: Telegram'ga xabar
        env:
          TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
          TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }}
          RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
        run: |
          curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
            --data-urlencode "chat_id=${TELEGRAM_CHAT_ID}" \
            --data-urlencode "text=❌ ${GITHUB_REPOSITORY}: regression yiqildi. ${RUN_URL}"

Asosiy g'oyalar:

• **matrix** bitta job'ni 4 nusxada ishga tushiradi: shardIndex 1, 2, 3, 4.
• **fail-fast: false** — bitta shard yiqilsa, qolganlari to'xtatilmaydi. Aks holda boshqa shard'lardagi xatolarni ko'rmay qolasiz.
• Har shard o'z blob-report papkasini alohida artifact qilib yuklaydi (blob-report-1 ... -4).
• **merge-reports** job'i needs bilan shard'larni kutadi, pattern: blob-report-* + merge-multiple: true bilan hammasini bitta papkaga yuklab oladi va bitta HTML hisobot yasaydi. if: ${{ !cancelled() }} — shard'lar yiqilgan bo'lsa ham hisobot yasalsin.
• **concurrency** — bir branch'ga ketma-ket push qilinsa, eskirgan run bekor qilinadi va daqiqalar tejaladi.

5.2. Lokal sinov: biz nimani ko'rdik

Workflow'ni push qilishdan oldin xuddi shu mantiqni lokal tekshirdik — har bir shard CI rejimida, alohida blob papkasi bilan:

CI=1 PLAYWRIGHT_BLOB_OUTPUT_DIR=blob-shard-1 npx playwright test --shard=1/4
# ... 2/4, 3/4, 4/4
mkdir all-blob-reports && cp blob-shard-*/*.zip all-blob-reports/
npx playwright merge-reports --reporter html ./all-blob-reports

Natija (o'sha paytdagi 35 ta test bilan): shard'lar 11, 10, 10 va 8 tadan test bajardi, blob fayllar report-1.zip ... report-4.zip nomlandi va birlashtirilgan hisobotda hammasi bitta joyda ko'rindi. setup project'i (QA 19'dagi login) UI testlari tushgan har bir shard'da qaytadan ishladi — bu normal: har shard alohida mashina, unga o'z sessiya fayli kerak. github reporter esa log'ga ::notice title=🎭 Playwright Run Summary:: 11 passed kabi qatorlarni chiqardi — GitHub ularni annotatsiyaga aylantiradi.

💡 Maslahat: Nechta shard? Taxminiy hisob: testlar soni × o'rtacha vaqt ÷ maqsadli vaqt, ustiga har shard uchun 1–2 daqiqalik tayyorlash (checkout, npm ci, brauzer o'rnatish). Juda ko'p shard — tayyorlash vaqti foydani yeb qo'yadi va runner daqiqalari ko'payadi.

6. Trigger'lar: PR smoke, nightly, qo'lda ishga tushirish {#triggerlar}

6.1. PR uchun tezkor smoke

PR muallifi 5 daqiqada javob olishi kerak, 40 daqiqada emas. Alohida, yengil workflow:

# .github/workflows/pr-smoke.yml
name: PR smoke

on:
  pull_request:
    branches: [main]

jobs:
  smoke:
    timeout-minutes: 15
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium

      - name: Smoke testlar (karantindagilarsiz)
        run: npx playwright test --grep @smoke --grep-invert @quarantine --reporter=github,html
        env:
          SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }}
          SAUCE_PASSWORD: ${{ secrets.SAUCE_PASSWORD }}

      - uses: actions/upload-artifact@v7
        if: ${{ !cancelled() }}
        with:
          name: smoke-report
          path: playwright-report/
          retention-days: 7

--reporter=github,html config'dagi CI reporter'ini (blob) shu run uchun almashtiradi — smoke'da sharding yo'q, bizga to'g'ridan-to'g'ri HTML kerak. Lokal CI=1 bilan sinaganimizda bu buyruq 6 ta testni ishlatdi: 2 ta setup (login) + 4 ta @smoke (login, savatcha — desktop va mobile-chrome project'larida, API'da bron yaratish). Setup testlari --grepga mos kelmasa ham ishladi — chunki ular tanlangan testlarning dependency'si.

⚠️ Diqqat: --grep qiymati katta-kichik harfni farqlamaydigan regex sifatida ishlaydi (Playwright uni gi flag'lari bilan yaratadi). --grep HAR yozsak, nomida "har" so'zi bor testlar ham tanlanib ketganini o'z ko'zimiz bilan ko'rdik. Shuning uchun teglar @ bilan boshlanadi: @smoke, @quarantine — oddiy so'z bilan tasodifan mos kelmaydi.

Yana bir foydali flag: --only-changed=origin/main — faqat o'zgargan test fayllarini ishga tushiradi (git tarixi kerak: checkout'da fetch-depth: 0). Katta suite'da PR feedback'ini tezlashtiradi, lekin to'liq regression o'rnini bosmaydi.

6.2. Nightly: schedule

on:
  schedule:
    - cron: "0 1 * * 1-5" # dushanba–juma 01:00 UTC = 06:00 Toshkent

• Cron vaqti UTCda. Toshkent UTC+5 (yozgi vaqt yo'q) — 06:00 Toshkent = 01:00 UTC. Jamoa ishga kelganda natija tayyor.
• Scheduled workflow default branch'ning oxirgi commit'ida ishlaydi.
• GitHub hujjatlariga ko'ra yuklama yuqori paytlarda schedule kechikishi, hatto ba'zi navbatdagi job'lar tashlab yuborilishi mumkin — soat aniqligiga tayanmang.
• Eng qisqa interval — 5 daqiqa.

6.3. Qo'lda: workflow_dispatch

Regression workflow'idagi workflow_dispatch.inputs GitHub UI'da Run workflow tugmasi va forma paydo qiladi: QA reliz oldidan staging URL'ini kiritib, faqat @checkout testlarini ishga tushira oladi. inputs.grep bo'sh bo'lsa (push yoki schedule'da), shell'dagi if hammasini ishlatadi.

7. Docker konteynerda ishga tushirish {#docker}

Playwright rasmiy Docker image'ini chiqaradi: unda Node, brauzerlar va barcha tizim kutubxonalari tayyor.

# .github/workflows/playwright-container.yml
name: Playwright in Docker

on: workflow_dispatch

jobs:
  playwright:
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/playwright:v1.64.0-noble
      options: --user 1001
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
      - run: npm ci
      - run: npx playwright test

Teglar: v1.64.0-noble (Ubuntu 24.04), v1.64.0-jammy (22.04), v1.64.0-resolute (26.04). Alpine qo'llab-quvvatlanmaydi.

⚠️ Diqqat: Image versiyasi package.jsondagi @playwright/test versiyasi bilan aynan bir xil bo'lishi shart. Aks holda Playwright brauzer fayllarini topa olmaydi. Dependabot @playwright/testni 1.65'ga ko'targanda image tegini ham bir PR'da yangilang.

7.1. Vizual etalonlarni Linux'da yaratish

QA 19da ko'rdik: macOS'da yaratilgan inventory-chromium-darwin.png Linux CI'da ishlatilmaydi — u inventory-chromium-linux.pngni qidiradi va shriftlar ham boshqacha chiziladi. Yechim — etalonlarni CI bilan bir xil image'da yaratish:

# .github/workflows/update-snapshots.yml
name: Update visual snapshots

on: workflow_dispatch

jobs:
  update:
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/playwright:v1.64.0-noble
      options: --user 1001
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
      - run: npm ci
      - run: npx playwright test --grep @visual --update-snapshots
      - uses: actions/upload-artifact@v7
        with:
          name: linux-snapshots
          path: tests/**/*-snapshots/

QA workflow'ni qo'lda ishga tushiradi, linux-snapshots artifact'ini yuklab, ko'zdan kechiradi (!) va repo'ga commit qiladi. Etalonni ko'rmasdan commit qilish — bug'ni "yangi norma" deb qabul qilish xavfi.

8. Hisobotlar va xabarnomalar {#hisobotlar}

Telegram xabarini yuqoridagi notify job'ida ko'rdingiz: bot @BotFather orqali yaratiladi, TELEGRAM_BOT_TOKEN va TELEGRAM_CHAT_ID secrets'ga yoziladi. if: ${{ failure() }} va needs birga ishlaydi: bog'liq job'lardan biri yiqilsa — xabar ketadi.

Slack incoming webhook bilan xuddi shu g'oya:

steps:
  - name: Slack'ga xabar
    if: ${{ failure() }}
    env:
      SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
    run: |
      curl -s -X POST -H 'Content-type: application/json' \
        --data "{\"text\":\"❌ ${GITHUB_REPOSITORY}: testlar yiqildi\"}" \
        "$SLACK_WEBHOOK_URL"

🇺🇿 Mahalliy kontekst: O'zbekistondagi ko'plab jamoalarda ish muloqoti Telegram'da. Nightly natijasi uchun alohida "QA alerts" guruhi oching: faqat yiqilgan run'lar, havolasi bilan. Har bir yashil run haqida xabar yuborish — kanalni shovqinga aylantiradi va odamlar uni o'qimay qo'yadi.

9. Flaky testlar: aniqlash, karantin, tuzatish {#flaky}

Flaky (beqaror) test — kod o'zgarmagan holda goh o'tadigan, goh yiqiladigan test. U eng xavfli "qarz": jamoa qizil natijaga ishonmay qo'yadi ("e, yana o'sha flaky") va haqiqiy bug ham e'tiborsiz qoladi.

9.1. Playwright flaky'ni qanday ko'rsatadi

Retry yoqilganda natija uch toifaga bo'linadi: passed, flaky (avval yiqilib, retry'da o'tgan) va failed. Buni ataylab tekshirdik:

import { test, expect } from "@playwright/test";

test("flaky namuna", async ({}, testInfo) => {
  // 1-urinishda yiqiladi, retry'da o'tadi
  expect(testInfo.retry).toBeGreaterThan(0);
});

Ya'ni sukut bo'yicha retry flaky testni yashirib qo'yadi. --fail-on-flaky-tests flag'i (yoki config'da failOnFlakyTests: !!process.env.CI) uni qizil qiladi — jiddiy jamoalarda shu qoida.

9.2. Sabablar va yechimlar

9.3. Flaky'ni qayta hosil qilish

npx playwright test tests/ui/cart.spec.ts --repeat-each=20   # bir testni 20 marta
npx playwright test --shuffle                                 # tartibni aralashtirish
npx playwright test --last-failed                              # faqat oxirgi yiqilganlar

Retry'da yozilgan trace (trace: 'on-first-retry') — eng kuchli vosita: yiqilgan va o'tgan urinishni yonma-yon ochib, farqni ko'rasiz.

9.4. Karantin siyosati

Flaky test topildi, lekin hozir tuzatishga vaqt yo'q. Uni o'chirib yubormaymiz va PR'larni ham bloklatmaymiz:

1. Testga @quarantine tegi qo'shiladi va unga ticket ochiladi (egasi va muddati bilan).
2. PR workflow'i uni chiqarib tashlaydi: --grep-invert @quarantine (6.1-bo'lim).
3. Nightly'da karantindagi testlar alohida ishlaydi va natijasi kuzatiladi — ko'rinmay qolib ketmaydi.
4. Kelishilgan muddatda (masalan, ikki sprint) tuzatilmasa — test qayta yoziladi yoki o'chiriladi. Abadiy karantin — o'chirilgan test bilan bir xil.

Kuzatiladigan metrika: flaky rate = flaky natijalar / jami ishga tushirishlar. Uning o'sishi — framework sog'lig'i yomonlashayotganining birinchi belgisi (QA 27).

🎯 Intervyuda: "Flaky test bilan nima qilasiz?" — Avval qayta hosil qilaman (--repeat-each, trace), sababini toifalayman (timing, ma'lumot, muhit, tashqi servis), tuzataman. Darhol tuzatib bo'lmasa — ticket bilan karantinga olaman, PR'larni bloklamaydi, lekin nightly'da kuzatiladi. Retry'ni oshirib, muammoni yashirmayman.

10. Quality gate'lar {#quality-gate}

Quality gate — "bu shart bajarilmasa, kod keyingi bosqichga o'tmaydi" degan avtomatik qoida. QA'ning ishi — bu darvozalarni loyiha xavflariga mos loyihalash.

10.1. Matrix bilan required check: gate job

Matrix'li workflow'da har bir shard alohida check bo'ladi (playwright-tests (1, 4) va hokazo). merge-reports job'ini required qilish xato: u !cancelled() sharti bilan testlar yiqilganda ham ishlaydi va yashil bo'ladi. Keng tarqalgan yechim — bitta "gate" job:

jobs:
  gate:
    if: ${{ always() }}
    needs: [playwright-tests]
    runs-on: ubuntu-latest
    steps:
      - name: Barcha shard'lar yashilmi?
        run: test "${{ needs.playwright-tests.result }}" = "success"

Ruleset'da faqat gateni required qilasiz — shard'lar soni o'zgarsa ham qoidani tahrirlash shart emas. Bu pattern amaliyotda keng qo'llanadi (tayyor variant: re-actors/alls-green action). Yangi repoda uni bir marta ataylab yiqiladigan test bilan sinab ko'ring: gate qizil bo'lishi va merge tugmasi bloklanishi kerak.

🧠 Esda tuting: Quality gate — jazolash vositasi emas, kelishuv. Jamoa bilan birga belgilanadi, yozib qo'yiladi va asoslanmagan "bu safar o'tkazib yuboraylik" istisnolari hujjatlashtiriladi.

11. GitLab CI, Jenkins va mobil CI {#boshqa-ci}

Kompaniyangiz GitHub ishlatmasa ham g'oyalar o'zgarmaydi: tayyorlash → test → artifact → gate.

GitLab CI (.gitlab-ci.yml) — Playwright image'i bilan, parallel orqali sharding:

stages:
  - test

playwright:
  stage: test
  image: mcr.microsoft.com/playwright:v1.64.0-noble
  parallel: 4
  script:
    - npm ci
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  artifacts:
    when: always
    paths:
      - blob-report/
    expire_in: 1 day

CI_NODE_INDEX/CI_NODE_TOTAL — GitLab parallel uchun o'zi beradigan o'zgaruvchilar. Birlashtirish keyingi stage'da xuddi GitHub'dagidek merge-reports bilan qilinadi; JUnit XML'ni artifacts:reports:junit ga bersangiz, natijalar merge request sahifasida ko'rinadi.

Jenkins — Declarative Pipeline'da agent sifatida o'sha image:

agent { docker { image 'mcr.microsoft.com/playwright:v1.64.0-noble' } }

Mobil CI (Xcode Cloud, Fastlane, GitHub Actions macOS runner'lari, simulator/emulator'lar) — kursning DevOps qismida batafsil: iOS CI/CD 2026 va CI/CD — Fastlane. Mobil UI testlarning o'zi — QA 22.

12. Amaliy mashqlar {#mashqlar}

Mashq 1: Buzilgan workflow

Hamkasbingiz shu workflow'ni yozdi. Kamida 6 ta muammoni toping va tuzatilgan variantini yozing.

name: tests
on: [push]
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v7
      - run: npm install
      - run: npx playwright install --with-deps
      - run: echo "Parol: ${{ secrets.SAUCE_PASSWORD }}"
      - run: npx playwright test
      - uses: actions/upload-artifact@v7
        with:
          name: report
          path: playwright-report/

<details> <summary>💡 Javobni ko'rish</summary>

1. Fayl YAML sifatida ham yaroqsiz: run: echo "Parol: ..." — qo'shtirnoqsiz qiymat ichidagi : YAML'da yangi kalit deb o'qiladi. Biz uni parser'dan o'tkazganimizda xato chiqdi: Nested mappings are not allowed in compact mappings. GitHub bunday workflow'ni umuman ishga tushirmaydi.
2. Secret'ni chop etish — qatorni butunlay olib tashlash kerak; secret testga env: orqali beriladi.
3. **checkout@v2** — juda eski major versiya; v7.
4. **setup-nodeda node-version va cache yo'q** — runner'dagi tasodifiy Node versiyasi ishlatiladi.
5. **npm install** o'rniga npm ci — lock fayldagi aniq versiyalar.
6. Barcha brauzerlar o'rnatiladi — faqat ishlatiladiganini o'rnating (chromium), vaqt tejaladi.
7. **if: ${{ !cancelled() }} yo'q** — testlar yiqilsa hisobot yuklanmaydi, ya'ni eng kerakli paytda hisobot yo'q.
8. **timeout-minutes yo'q** va faqat push — PR'larda tekshiruv ishlamaydi.

name: tests
on:
  pull_request:
    branches: [main]
jobs:
  e2e:
    timeout-minutes: 30
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test
        env:
          SAUCE_PASSWORD: ${{ secrets.SAUCE_PASSWORD }}
      - uses: actions/upload-artifact@v7
        if: ${{ !cancelled() }}
        with:
          name: report
          path: playwright-report/
          retention-days: 14

</details>

Mashq 2: Nechta shard kerak?

DemoBozor'da 400 ta UI test bor, bitta testning o'rtacha vaqti 15 soniya, har shard'da 1 worker. Regression 15 daqiqadan oshmasligi kerak, har shard'ning tayyorlash vaqti taxminan 2 daqiqa. Nechta shard kerak?

<details> <summary>💡 Javobni ko'rish</summary>

• Jami test vaqti: 400 × 15 s = 6000 s = 100 daqiqa.
• Har shard'da testlar uchun qolgan vaqt: 15 − 2 = 13 daqiqa.
• 100 ÷ 13 ≈ 7,7 → 8 ta shard.

Lekin bu "ideal" hisob: testlar shard'lar orasida hech qachon mutlaqo teng bo'linmaydi, eng sekin shard butun run'ni belgilaydi. Shuning uchun bir oz zaxira (9–10 shard) yoki sekin testlarni optimallashtirish (login'ni storageStatega, tayyorgarlikni API'ga ko'chirish — QA 19–20) kerak. Ikkinchi yo'l ko'pincha arzonroq.

</details>

Mashq 3: Cron

"Har shanba Toshkent vaqti bilan 03:00 da to'liq cross-browser regression" uchun cron yozing.

<details> <summary>💡 Javobni ko'rish</summary>

Toshkent UTC+5, ya'ni shanba 03:00 Toshkent = juma 22:00 UTC:

on:
  schedule:
    - cron: "0 22 * * 5"

Cron maydonlari: daqiqa, soat, oy kuni, oy, hafta kuni (0 = yakshanba, 5 = juma). Kun almashinuvini unutish — eng ko'p uchraydigan xato: 0 22 * * 6 yozilsa, test Toshkent bo'yicha yakshanba tongida ishlaydi.

</details>

Mashq 4: Flaky tashxisi

Savatcha testi lokal har doim o'tadi, CI'da esa taxminan har 10 run'dan birida expected "2", received "1" bilan yiqiladi. Kod:

await inventoryPage.addToCart("Sauce Labs Backpack");
await inventoryPage.addToCart("Sauce Labs Bike Light");
const badge = await inventoryPage.header.cartBadge.textContent();
expect(badge).toBe("2");

Sabab nima va qanday tuzatasiz? Qanday isbotlaysiz?

<details> <summary>💡 Javobni ko'rish</summary>

Sabab: textContent() qiymatni bir marta o'qiydi va kutmaydi. CI sekinroq bo'lgani uchun ba'zan ikkinchi bosishdan keyingi UI yangilanishidan oldin o'qiladi va "1"ni oladi. expect(badge).toBe('2') — oddiy (retry qilmaydigan) assertion.

Tuzatish: web-first assertion — u kerakli qiymat paydo bo'lguncha (timeout ichida) qayta tekshiradi:

await inventoryPage.addToCart("Sauce Labs Backpack");
await inventoryPage.addToCart("Sauce Labs Bike Light");
await expect(inventoryPage.header.cartBadge).toHaveText("2");

Isbot: tuzatishdan oldin va keyin npx playwright test tests/ui/cart.spec.ts --repeat-each=50. CI'ga yaqin sharoit uchun CPU'ni sekinlatish yoki --workers sonini oshirib yuklamani ko'paytirish ham mumkin. ESLint'ning playwright/prefer-web-first-assertions qoidasi bunday kodni review'dan oldin ushlaydi.

</details>

Mashq 5: DemoBozor uchun gate'lar

DemoBozor web jamoasi uchun quality gate'larni loyihalang: qaysilari PR'ni bloklaydi, qaysilari faqat nightly'da ishlaydi va nega?

<details> <summary>💡 Javobni ko'rish</summary>

Asosiy qoida: PR gate'lari tez va ishonchli bo'lishi kerak — aks holda jamoa ularni chetlab o'tish yo'lini topadi.

</details>

Mashq 6: Ma'lumotli xabar

notify job'idagi Telegram xabariga branch nomi va commit'ning qisqa SHA'sini (7 belgi) qo'shing.

<details> <summary>💡 Javobni ko'rish</summary>

steps:
  - name: Telegram'ga xabar
    env:
      TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }}
      TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }}
      BRANCH: ${{ github.ref_name }}
      SHA: ${{ github.sha }}
      RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
    run: |
      curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
        --data-urlencode "chat_id=${TELEGRAM_CHAT_ID}" \
        --data-urlencode "text=❌ ${GITHUB_REPOSITORY} (${BRANCH}, ${SHA:0:7}) yiqildi: ${RUN_URL}"

${SHA:0:7} — bash'ning substring sintaksisi (runner'da run: sukut bo'yicha bash'da ishlaydi). Qiymatlarni to'g'ridan-to'g'ri ${{ }} bilan buyruq ichiga qo'yish o'rniga env: orqali berish — xavfsizroq odat: branch nomi kabi foydalanuvchi boshqaradigan matn shell buyrug'iga aylanib ketmaydi.

</details>

O'zingizni tekshiring {#checklist}

• [ ] CI, Continuous Delivery va Continuous Deployment farqini tushuntira olaman
• [ ] Playwright uchun GitHub Actions workflow yoza olaman va hisobotni artifact'dan ocha olaman
• [ ] Secrets, vars va environments farqini bilaman
• [ ] Sharding + blob + merge-reports zanjirini sozlay olaman
• [ ] PR smoke, nightly (UTC cron) va qo'lda ishga tushirishni ajrata olaman
• [ ] Flaky testni qayta hosil qilish, karantinga olish va tuzatish jarayonini bilaman
• [ ] Matrix'li workflow uchun to'g'ri required check (gate) quraman

Xulosa {#xulosa}

• CI har bir o'zgarishni avtomatik tekshiradi; avtomatik ishlamaydigan test — unutilgan test. Testlar qatlamlarga bo'linadi: PR'da tez smoke, main'da to'liq regression, nightly'da og'ir va cross-browser testlar.
• GitHub Actions workflow: checkout@v7 → setup-node@v7 (+npm kesh) → npm ci → playwright install --with-deps → test → upload-artifact@v7 (!cancelled() bilan). Versiyalarni Dependabot kuzatadi.
• Maxfiy qiymatlar — secrets'da, oddiy sozlamalar — vars'da; fork PR'larida secret'lar berilmaydi.
• Sharding: matrix + --shard=N/4 + blob reporter + merge-reports → bitta HTML hisobot. Biz bu zanjirni lokal 4 shard bilan sinadik.
• Docker image versiyasi @playwright/test bilan aynan bir xil bo'lsin; vizual etalonlar CI bilan bir xil image'da yaratiladi.
• Flaky testlar retry ortida yashiringan bo'ladi: --fail-on-flaky-tests, --repeat-each, trace va karantin siyosati bilan boshqariladi.
• Quality gate — kelishilgan avtomatik qoidalar; matrix'li workflow'da bitta gate job'ini required qiling, merge-reportsni emas.
• Keyingi dars: QA 22: Mobil avtomatlashtirish — Maestro, Appium 3, XCUITest va Espresso

Manbalar {#manbalar}

1. Playwright — Setting up CI
2. Playwright — Continuous Integration (GitLab, Jenkins, konteynerlar)
3. Playwright — Sharding
4. Playwright — Docker
5. Playwright — Command line (CLI flag'lari)
6. Playwright — Retries va flaky testlar
7. GitHub Docs — Events that trigger workflows (schedule, fork PR)
8. GitHub Docs — Contexts (needs, vars, secrets, inputs)
9. GitHub Docs — Variables reference (CI=true)
10. GitHub Docs — Using secrets in GitHub Actions
11. GitHub Docs — About rulesets
12. GitHub Docs — Keeping your actions up to date with Dependabot
13. actions/checkout
14. actions/upload-artifact
15. re-actors/alls-green — gate job uchun tayyor action
16. GitLab CI/CD YAML syntax reference
17. Telegram Bot API (sendMessage)
18. Slack — Sending messages using incoming webhooks
19. Martin Fowler — Continuous Integration