DevOps 4: Observability va server asoslari — crash, MetricKit, SLO, VPS

🛠️ DevOps Engineering

DevOps 4: Observability va server asoslari — crash, MetricKit, SLO, VPS

Holat: 2026-yil sentabr. Ilova App Store'ga chiqqandan keyin ish tugamaydi: u foydalanuvchida qanday ishlayotganini ko'rish, muammoni tez topish va backend'ni barqaror ushlash kerak. Bu dars shu haqida.

Mundarija

1. Observability nima?
2. [iOS'da strukturali log: os.Logger](#logger)
3. Crash reporting va dSYM
4. MetricKit — Apple'ning bepul telemetriyasi
5. Xcode Organizer metrikalari
6. SLI, SLO va error budget
7. Incident va blameless postmortem
8. Kichik backend uchun VPS: real konfiguratsiya
9. Secrets va CI
10. Xulosa
11. Manbalar

1. Observability nima? {#observability}

Observability — tizimning ichki holatini uning tashqi signallari orqali tushunish qobiliyati. Uch asosiy signal bor:

Backend'da bu uchalasini bitta standart — OpenTelemetry — orqali yig'ish odat bo'lgan. iOS'da esa asosiy manbalar boshqacha: tizim logi (os.Logger), crash reporter (Crashlytics yoki Sentry), MetricKit va Xcode Organizer.

2. iOS'da strukturali log: os.Logger {#logger}

print() production'da yaramaydi: filtrlab bo'lmaydi, darajasi yo'q, maxfiy ma'lumotni ochiq yozadi. Apple'ning unified logging tizimi (os.Logger) esa log'ni subsystem/category bo'yicha ajratadi, darajalar beradi va maxfiylikni boshqaradi:

import Foundation
import os

enum Log {
    static let network = Logger(subsystem: "uz.azimjondev.app", category: "network")
    static let purchase = Logger(subsystem: "uz.azimjondev.app", category: "purchase")
}

func fetchProfile(userID: String) async {
    // Dinamik qiymatlar standart holatda maxfiy (<private>) yoziladi.
    Log.network.info("fetching profile for \(userID, privacy: .private)")
    do {
        let url = URL(string: "https://api.example.uz/profile")!
        let (_, response) = try await URLSession.shared.data(from: url)
        let status = (response as? HTTPURLResponse)?.statusCode ?? -1
        Log.network.debug("profile status \(status, privacy: .public)")
    } catch {
        Log.network.error("profile failed: \(error.localizedDescription, privacy: .public)")
    }
}

• Darajalar: debug → info → notice → error → fault. debug odatda faqat qurilmaga ulanganda ko'rinadi.
• Log'larni Console.appda subsystem bo'yicha filtrlab ko'rasiz.
• privacy: .public faqat maxfiy bo'lmagan qiymatlarga qo'yiladi (status kodi, xato matni). Email, ID, token — hech qachon .public emas.

3. Crash reporting va dSYM {#crash}

Crash bo'lganda qurilma xotira manzillarini yozadi (0x0000000104a3c2f8). Ularni funksiya nomi va qator raqamiga aylantirish — symbolication — uchun dSYM fayli kerak. U archive paytida yaratiladi va crash xizmatiga yuklanishi shart.

Sentry uchun CI qadami (SENTRY_AUTH_TOKEN secret sifatida):

#!/bin/bash
# Archive'dan keyin dSYM'larni Sentry'ga yuklash
set -euo pipefail
ARCHIVE_PATH="$RUNNER_TEMP/MyApp.xcarchive"

sentry-cli debug-files upload \
  --org my-org --project my-ios-app \
  --include-sources \
  "$ARCHIVE_PATH/dSYMs"

Crashlytics'ni Swift Package Manager orqali qo'shgan bo'lsangiz, Firebase hujjatidagi Run Script build phase'ini qo'shasiz. Uning aniq yo'li va "Input Files" ro'yxati Firebase — Get readable crash reportsda.

Qoida: har bir Release build'ning dSYM'i yuklanganini CI'da tekshiring. Aks holda crash'lar "o'qib bo'lmaydigan" bo'lib qoladi.

4. MetricKit — Apple'ning bepul telemetriyasi {#metrickit}

MetricKit (iOS 13+) — tizim o'zi yig'adigan performance va diagnostika ma'lumotlari. Ilova ularni MXMetricManager orqali oladi, qo'shimcha SDK kerak emas:

• **MXMetricPayload** — odatda kuniga bir marta keladigan yig'ma metrikalar: ishga tushish vaqti, CPU, xotira, disk yozuvlari, hang vaqti, animatsiya "hitch"lari.
• **MXDiagnosticPayload** — alohida hodisalar: crash, hang, CPU exception, disk write exception. Har biri call stack bilan.

import MetricKit
import os

final class MetricsReporter: NSObject, MXMetricManagerSubscriber {
    private let logger = Logger(subsystem: "uz.azimjondev.app", category: "metrics")

    /// AppDelegate'da ilova ishga tushganda chaqiring.
    func start() {
        MXMetricManager.shared.add(self)
    }

    nonisolated func didReceive(_ payloads: [MXMetricPayload]) {
        for payload in payloads {
            // jsonRepresentation() — serveringizga yuborish uchun tayyor JSON
            let json = payload.jsonRepresentation()
            logger.info("metric payload: \(json.count) bytes")
        }
    }

    nonisolated func didReceive(_ payloads: [MXDiagnosticPayload]) {
        for payload in payloads {
            let crashes = payload.crashDiagnostics?.count ?? 0
            let hangs = payload.hangDiagnostics?.count ?? 0
            logger.error("diagnostics: crashes=\(crashes) hangs=\(hangs)")
        }
    }
}

Muhim: bu callback'lar main thread'da emas. Shu sababli metodlar nonisolated deb belgilangan (Swift 6 strict concurrency). UI'ni yangilashingiz kerak bo'lsa, MainActorga o'ting.

Sentry ishlatsangiz, MetricKit'ni o'zingiz ulashingiz shart emas: Sentry Cocoa SDK 8.14.0+ (iOS 15+) da options.enableMetricKit = true sozlamasi bor. Manba: Sentry — MetricKit.

5. Xcode Organizer metrikalari {#organizer}

Xcode → Window → Organizer → Metrics bo'limi ma'lumot ulashishga rozi bo'lgan real foydalanuvchilardan yig'iladi. U yerda versiyalar bo'yicha quyidagilar ko'rinadi:

• Launch Time — ilovaning ishga tushish vaqti
• Hang Rate — UI "qotib qolish" chastotasi
• Scrolling — "hitch" (kadr tushishi)
• Memory, Disk Writes, Battery Usage
• Terminations — tizim ilovani nima sababdan yopgani

Yangi reliz chiqqach (ayniqsa phased release paytida — DevOps 2) shu grafiklarni oldingi versiya bilan solishtiring.

6. SLI, SLO va error budget {#slo}

• SLI (Service Level Indicator) — o'lchov. Masalan: crash-free sessions ulushi.
• SLO (Service Level Objective) — maqsad. Masalan: oylik crash-free sessions ≥ 99.9%.
• Error budget — 1 − SLO: shuncha "xato qilishga ruxsat" bor.

Hisoblaymiz: ilovada oyiga 1 200 000 sessiya bor, SLO = 99.9%.

sessions_per_month = 1_200_000
slo = 0.999

budget_sessions = sessions_per_month * (1 - slo)
print(f"Oyiga ruxsat etilgan crash'li sessiyalar: {budget_sessions:.0f}")

# Xuddi shu 99.9% backend uptime uchun (30 kunlik oy):
minutes_per_month = 30 * 24 * 60
print(f"Ruxsat etilgan downtime: {minutes_per_month * (1 - slo):.1f} daqiqa")

Natija: oyiga 1 200 ta crash'li sessiya yoki backend uchun 43.2 daqiqa downtime.

Byudjet tugasa nima bo'ladi? Jamoa yangi funksiyalarni to'xtatib, barqarorlikka o'tadi. Bu qoidani oldindan kelishib qo'yish kerak — aks holda SLO shunchaki "chiroyli raqam" bo'lib qoladi.

Mo'ljal uchun: vendor hisobotlariga ko'ra, iste'molchi ilovalar uchun crash-free sessions ≥ 99.5%, fintech va sog'liq ilovalari uchun ≥ 99.9% yaxshi daraja hisoblanadi (Luciq 2026 benchmarks). Bu Apple'ning rasmiy standarti emas — o'z ilovangiz tarixiga qarab maqsad qo'ying.

7. Incident va blameless postmortem {#incident}

Severity (og'irlik darajasi) — muammoning ta'sirini tasvirlaydi, aybni emas:

Mobil ilovadagi SEV1 uchun odatiy harakatlar ketma-ketligi: kill switch bilan funksiyani o'chirish → phased release'ni pauza qilish → hotfix → expedited review.

Blameless postmortem (Google SRE madaniyati) — "kim aybdor?" emas, "tizim nega buni o'tkazib yubordi?" degan savol. Hodisadan keyingi bir necha ish kuni ichida yoziladi. Shablon:

Sarlavha: 2.4.0 — iOS 17'da ishga tushishda crash (SEV1)
Xulosa: 2 jumlada nima bo'ldi va ta'siri.
Ta'sir: qancha foydalanuvchi, qancha vaqt, qaysi versiyalar.
Timeline: 14:02 phased release 5% → 14:40 crash-free 97% → 14:55 pauza → ...
Sabablar: tizim darajasida (test matritsasida iOS 17 yo'q edi, alert chegarasi yo'q edi).
Nima yaxshi ishladi: kill switch 10 daqiqada ishga tushdi.
Action items: [egasi, muddat, bajarilganlik mezoni] har biri uchun.

Manba: Google SRE Book — Postmortem Culture.

8. Kichik backend uchun VPS: real konfiguratsiya {#vps}

Ko'p iOS dasturchilar ilova uchun kichik API (push, obuna tekshiruvi, admin panel) ham yuritadi. Quyidagi konfiguratsiyalar o'ylab topilmagan — ular shu sayt (azimjondev.uz) serverida ishlab turgan fayllarning soddalashtirilgan ko'rinishi: Node.js ilova + systemd + nginx + Let's Encrypt.

SSH: faqat kalit bilan kirish

/etc/ssh/sshd_configdagi asosiy sozlamalar:

PasswordAuthentication no
PermitRootLogin no
PubkeyAuthentication yes

Ogohlantirish: parolni o'chirishdan oldin boshqa terminal oynasida kalit bilan kirish ishlashini tekshiring. Aks holda serverdan qulflanib qolasiz. O'zgartirishdan keyin: sudo systemctl reload ssh.

Firewall (ufw)

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

systemd service — ilovani doim ishlab turadigan qilish

/etc/systemd/system/myapi.service:

[Unit]
Description=My API (Node/Express)
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/myapi
EnvironmentFile=/var/www/myapi/.env
Environment=NODE_ENV=production
Environment=PORT=3000
ExecStart=/usr/bin/node server.js
Restart=always
RestartSec=5

# Hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/var/www/myapi/data

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapi
journalctl -u myapi -f   # log'larni jonli kuzatish

• Restart=always — jarayon yiqilsa, 5 soniyadan keyin qayta ko'tariladi.
• EnvironmentFile — secret'lar kod bilan birga emas, serverdagi alohida faylda turadi (chmod 600).
• Hardening qatorlari jarayonga faqat kerakli papkaga yozishga ruxsat beradi.

nginx — reverse proxy

/etc/nginx/sites-available/myapi:

server {
    listen 80;
    listen [::]:80;
    server_name api.example.uz;

    client_max_body_size 2m;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 30s;
    }

    # yashirin fayllar (.env, .git) hech qachon berilmaydi
    location ~ /\. {
        deny all;
    }
}

sudo ln -s /etc/nginx/sites-available/myapi /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

nginx -t konfiguratsiyani tekshiradi. Xato bo'lsa, reload qilinmaydi va ishlab turgan sayt buzilmaydi.

HTTPS — Let's Encrypt

sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d api.example.uz

Certbot 443-port blokini va 80→443 redirect'ni o'zi qo'shadi, sertifikatni esa avtomatik yangilaydi.

Docker qachon kerak?

Bitta server va bitta ilova uchun systemd + nginx yetarli va sodda. Docker muhitni har joyda bir xil qilish kerak bo'lganda foydali: bir nechta servis, jamoada turli OS, yoki ilovani boshqa serverga tez ko'chirish. Minimal multi-stage Dockerfile — birinchi bosqich build qiladi, ikkinchisiga faqat ishlash uchun kerakli fayllar o'tadi:

FROM node:20-alpine AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY --from=build /app/dist ./dist
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

package*.json kod'dan oldin ko'chiriladi. Shunda dependency'lar o'zgarmasa, Docker npm ci qatlamini cache'dan oladi va build tezlashadi.

9. Secrets va CI {#secrets}

• Parol, token, .p8 kalit — hech qachon repo'da emas (.gitignoreda bo'lsa ham). Ular CI secret yoki serverdagi EnvironmentFileda turadi.
• Kalit sizib chiqqan bo'lishi mumkin bo'lsa — darhol rotate qiling: yangisini yarating, eskisini bekor qiling.
• Bulutga (AWS/GCP) deploy qilishda uzoq muddatli access key o'rniga GitHub Actions OIDC'dan foydalaning (permissions: id-token: write). Bunda har bir ish uchun qisqa muddatli token beriladi.

10. Xulosa {#xulosa}

• iOS'da log uchun os.Logger: darajalar, kategoriyalar, privacy. print() production'da emas.
• dSYM yuklanmasa, crash'lar o'qib bo'lmaydigan bo'ladi. Buni CI'da tekshiring.
• MetricKit va Xcode Organizer — bepul va rasmiy performance ma'lumoti.
• SLO + error budget "barqarorlikmi, yangi funksiyami?" degan bahsni raqam bilan hal qiladi.
• Kichik backend uchun systemd + nginx + certbot yetarli. nginx -t va journalctl — eng yaqin do'stlaringiz.

Manbalar {#manbalar}

1. Apple — Logging (os.Logger)
2. Apple — MetricKit
3. Sentry — MetricKit configuration
4. Firebase — Get readable crash reports (dSYM)
5. Google SRE Book — Postmortem Culture
6. Google SRE Book — Service Level Objectives
7. Luciq — Mobile App Performance Benchmarks 2026
8. Certbot — nginx on Ubuntu