AI events and alerts — people / vehicle counting, restricted-zone intrusion, face recognition

Besides plate reading, TingPush cameras have an AI event framework: the camera analyses the image and detects events, the TingPush server stores them and notifies phones immediately through TingPush's notification service — a push notification, or a phone call (full-screen ringing like an incoming call, even on a locked phone) when someone intrudes.

The TingPush apps are in Vietnamese; screen labels are quoted as they appear, with a translation in parentheses.

Feature	What the camera does	Phone alert (default)
Restricted-zone intrusion	A person (or vehicle) whose feet are inside the drawn polygon for ≥ N seconds	Phone call (+ notification when calls are off)
Face recognition	Compares faces with a library of known people; reports known people / strangers	Push notification
Face learning (familiar faces)	An indoor camera remembers strangers who keep appearing and counts the days; seen N days in a row (default 7) → added to the household automatically — section 1	Alert when someone is added automatically
Time attendance	Uses the same face library (employees); records arrival / departure times, sent like plates — see Face time attendance — a TingPush camera as a time clock	No alert (see the Attendance page)
People / vehicle counting	Counts each crossing of a line by direction in / out (in then back out = 1 in + 1 out) and sends counts every period	No alert; optional "alert on many entries" threshold
Number inside	The camera adds entries and subtracts exits over all lines, resets to 0 daily at the reset time, survives restarts	Capacity: notification as soon as it passes the threshold (optional call)
Plate reading (built in)	As before	Notification when a plate on the watch list appears (on automatically)

AI models: the framework is complete (object tracking, lines, zones, faces, schedules, alerts). The person / vehicle and face recognition models are loaded separately (section 7). Without a model you can still test alerts with the Thử (Test) buttons in the app (section 8).

Camera ──(WebSocket, instant, no image)──► TingPush server ──► TingPush (POST /notifications) ──► APNs / FCM ──► App
       └─(HTTP /api/device/events, with images, queued while offline)──┘   │
                                                                         └─► stores events and counts (web, app, API /api/v1)

1. Using it in the TingPush Camera app

Camera → pick the camera → ⚙ Cài đặt camera (Camera settings) → Tính năng AI & cảnh báo (AI features & alerts).
Turn on a feature, then:
- Intrusion: tap the preview to draw restricted zones on the live image (drag the round handles, Thêm điểm to add points; up to 4 zones × 12 points), choose Người / Xe (People / Vehicles), how long they must stay in the zone, Không báo lặp trong (Don't repeat within, seconds), active hours and days.
- Counting: draw a line across the walkway; the arrow points in (Đảo chiều vào / ra swaps direction). Choose how often counts are sent. Charts are under Sự kiện (Events) → 📈 Thống kê đếm (Count statistics): today by hour, 7 / 30 days by day, per line.
- Number inside (in − out): see the current number, Sửa (Edit: reset to 0 or enter the real number, e.g. at the start of a shift), Giờ đặt lại về 0 mỗi ngày (Daily reset time), Báo khi bên trong từ N (Alert from N inside; 0 = off), Không báo lại trong (Don't alert again within, seconds — so a number hovering around the threshold doesn't keep alerting). Several doors: draw one line per door, all arrows pointing inside; the camera adds them up. Every entrance must have a line within the camera's view for the number to be right.
- Faces: add known people in Thư viện khuôn mặt (Face library; a clear portrait photo, choose which cameras recognise them), set the minimum similarity, Báo cả người lạ (Also alert on strangers), and an optional recognition zone.
Choose how to be alerted per feature: Thông báo đẩy (Push notification), Gọi điện (Phone call). Each person (the owner and everyone it's shared with) chooses for themselves; shared users can only change how they're alerted, not turn features on or off.
Tap Lưu (Save; settings are stored on the camera). Tap Thử cảnh báo xâm nhập (Test intrusion alert) to check the phone rings.

The Sự kiện (Events) tab lists every event (filter by type / camera) and updates as they happen; tap one for the scene image, the object close-up, Xem trực tiếp camera (Live view) and Xem lại thẻ nhớ (Memory card playback). Web: https://webrtc.tingpush.com/events.

Alert calls

Everyone with Gọi điện turned on rings at the same time. The first person to tap "Xem ngay" (View now) opens the camera's live view, and the server tells the other phones to stop ringing (with a notification that someone has answered).
If nobody answers within 45 seconds, ringing stops and a missed alert notification is sent.
Android 14+: the first time, go to Tài khoản → Cảnh báo AI (Account → AI alerts) → allow showing calls on the lock screen. Xiaomi / Oppo / Vivo / Realme / Huawei phones: turn on Autostart and turn off Battery optimisation for the app (as for Zalo) so ringing is reliable.

Face learning — familiar faces (indoor cameras)

No need to photograph every household member: turn on Face learning (AI features → Face recognition; web: the Người hay gặp / Familiar faces menu) and the camera does this by itself:

Every unknown face (not in the library) seen clearly enough is grouped into a cluster by its face features. Each cluster records: how many days it was seen, the run of consecutive days, visits (≥ 1 minute apart), the times of day, and the clearest face image.
A cluster seen N days in a row (1–60, default 7; 0 = never add automatically) is added to the household (face library, temporary name "Người nhà mới XXXX" — "new household member"); the server stores the photo and notifies the camera owner (always, even with face alerts turned off) — if it isn't a household member, delete it on the Employees / household page. Renaming it there updates the camera immediately.
Before that, the owner reviews Familiar faces (app: AI features → Familiar faces, or Employees & faces → ⋮; web /face-learn; Windows: Employees → Familiar faces…): Household member (give a name → added to the library at once), Merge (two tiles are the same person — e.g. daylight vs. infrared at night), Delete, Delete all.
Stranger events include how many days the person has been seen: "Phát hiện người lạ (đã thấy 3 ngày)".
Unconfirmed clusters not seen for N days (default 30) are deleted; clusters seen only once are deleted after 3 days (passers-by); at most 300 clusters. The camera periodically merges clusters that are too similar.

Notes:

Cluster data (features + face images) stays on the camera (/userdata/lpr/face_learn.db, face_learn/<id>.jpg); the photo goes to the server only when the person becomes a household member. Turning learning off keeps the list until Delete all.
In a row means no missed day: one day away restarts the count. A housekeeper / guard who comes every day will also become a "household member" — rename or delete them.
Risk: a stranger who deliberately shows up N days in a row is added too. That's why the camera always notifies when it adds someone; homes that need tighter security should set 0 days (suggestions only, the owner confirms).
Clustering uses a stricter threshold than recognition (better to split one person into two tiles you can merge than to merge two people).
Tell people in view if the camera covers outsiders (guests, housekeepers) — face data is sensitive personal data.

2. Server configuration (administrators)

camera_server/appsettings.json:

"TingPush": {
  "ServerUrl": "https://tingpush.com",
  "AppId": "15712fca-623e-410b-be30-f3b278a12eb9",
  "ApiKey": "",
  "ExternalIdPrefix": "u",
  "CallTimeoutS": 45
}

The psk_… API key is a secret — don't put it in appsettings.json or in git. Set it as an environment variable of the service:

# systemd: /etc/systemd/system/cameraserver.service → [Service]
Environment=TingPush__ApiKey=psk_xxxxxxxxxxxxxxxx
# Docker: -e TingPush__ApiKey=psk_xxx      Development: dotnet user-secrets set "TingPush:ApiKey" "psk_xxx"

Without an ApiKey the server still stores events; it only logs instead of sending notifications. If the key leaks, create a new one in the TingPush admin site and delete the old one.

Recipients: the app signs in to the SDK with external_id = ExternalIdPrefix + account ID (e.g. u12), which the server returns from GET /api/me/push. The camera server doesn't need to store device tokens.

Plans (web → Quản trị → Gói dịch vụ, Admin → Plans): Sự kiện AI (AI events: storage + notifications), Cảnh báo bằng cuộc gọi (Call alerts), Sự kiện AI / tháng (AI events per month; 0 = unlimited; past the limit events stop being accepted until next month), Số khuôn mặt tối đa (Max faces). Notifications / calls sent are shown on the AI events page and the usage table.

In the TingPush admin site (tingpush.com)

Android: upload the service account JSON of the Firebase project the app uses.
iOS: upload Apple's .p8 key (APNs), Team ID, Key ID, bundle com.tingpush.camera. Calls on iOS go through VoIP push (PushKit) → CallKit.

3. Building the app with push notifications (app developers)

The app uses the Flutter package tingpush (+ firebase_core). Firebase must be configured for Android / iOS — pick one way:

Native config files: android/app/google-services.json (plus the Gradle plugin com.google.gms.google-services) and ios/Runner/GoogleService-Info.plist; or
dart-define (no files):

flutter build apk --release \
  --dart-define=FIREBASE_API_KEY=AIza... \
  --dart-define=FIREBASE_ANDROID_APP_ID=1:1234567890:android:abc... \
  --dart-define=FIREBASE_IOS_APP_ID=1:1234567890:ios:def... \
  --dart-define=FIREBASE_SENDER_ID=1234567890 \
  --dart-define=FIREBASE_PROJECT_ID=tingpush-camera

The TingPush App ID is preset in lib/core/config.dart (override with --dart-define=TINGPUSH_APP_ID=…). The API key never goes in the app.

iOS (Xcode → Signing & Capabilities): add Push Notifications; Background Modes (Remote notifications, Voice over IP) are already declared in Info.plist. Android: AndroidManifest.xml already has showWhenLocked, turnScreenOn, USE_FULL_SCREEN_INTENT. Without Firebase the app still runs; the AI alerts screen says they can't be enabled.

The web (and desktop) versions of the app have no push notifications; events still appear in real time while open.

4. Camera configuration (`/userdata/lpr/ai.conf`)

The app writes it with the ai_set command (PUT /api/cameras/{id}/ai, body = the keys below). Coordinates are normalised 0..1 to the frame.

Key	Default	Meaning
`plate.enable`	1	Plate reading
`<feature>.enable` / `.schedule` / `.days` / `.cooldown_s`	0 / empty / 127 / 0·60·300	On, active hours (`06:00-18:00,22:00-02:00`, empty = all day), days (bit 0 = Sunday … bit 6 = Saturday), repeat suppression. Features: `count`, `intrusion`, `face`
`count.lines`		`Name\|x1,y1,x2,y2;…` (up to 4). Crossing from the left to the right side of the line (looking from A to B) = in
`count.classes` / `count.interval_s`	`person,vehicle` / 60	Classes counted / how often counts are sent (10–3600 s)
`count.occupancy_max` / `count.reset_time` / `count.cooldown_s`	0 / `00:00` / 300	Capacity alert when the number inside ≥ threshold (0 = off) / daily reset time / don't alert again within (seconds)
`intrusion.zones`		`Name\|x1,y1,x2,y2,x3,y3,…;…` (≥ 3 points, up to 4 zones × 12 points)
`intrusion.classes` / `intrusion.dwell_s`	`person` / 2	Classes to alert on / seconds inside the zone
`face.zone` / `face.threshold` / `face.unknown`	empty / 42 / 0	Recognition zone / % similarity (cosine × 100) / alert on strangers
`face.learn` / `face.learn_days` / `face.learn_keep`	0 / 7 / 30	Face learning / add to household after N consecutive days (0 = never, max 60) / forget unconfirmed clusters not seen for N days (7–365)
`object.conf` / `object.ratio`	45 / 1	Minimum % score of the person / vehicle model / run the model on 1 of every N AI frames

Decisions use the object's foot point (centre of the box's bottom edge). Each person / vehicle gets a tracking ID (IoU across frames). Line crossing: the camera remembers which side of the line an object is on; each full move to the other side is 1 crossing (left → right looking from A to B = in, the opposite = out). A 2% dead band on either side of the line means swaying on the line isn't counted, and walking around the end of a line isn't counted. Inside a restricted zone, an intrusion alerts only once. The number inside is stored in /userdata/lpr/ai_state (written every period) and never goes negative.

5. Event format sent by the camera

The camera sends over 2 paths with the same event_id (the server drops duplicates; the second copy only adds images):

Instant (intrusion, faces): WebSocket {"type":"ai_event","event":{…no images…}}.
Complete: POST https://apiwebrtc.tingpush.com/api/device/events, header Authorization: Device <camera_id>:<key>, with a queue on the camera (resent after outages, up to 300 records).

{
  "event_id": "LPR-9MHK902C8J5Z-1792170903-000031",
  "type": "intrusion",
  "device_id": "CONG-KHO",
  "camera_id": "LPR-9MHK902C8J5Z",
  "time": "2026-10-16T22:15:03+07:00",
  "timestamp": 1792170903,
  "summary": "Có người xâm nhập vùng Kho",
  "data": { "zone": "Kho", "zone_index": 0, "class": "person", "track_id": 17, "box": [0.67, 0.5, 0.83, 0.8], "dwell_s": 2 },
  "image_type": "jpeg", "image": "<scene image base64>",
  "crop_image_type": "jpeg", "crop_image": "<object close-up base64>"
}

summary is in Vietnamese (here: "Someone entered zone Kho").

`type`	`data`
`intrusion`	`zone`, `zone_index`, `class` (`person` / `vehicle`), `track_id`, `box` [x1,y1,x2,y2] 0..1, `dwell_s`
`face`	`known` (bool), `person_id`, `name` (when known), `similarity` 0..1, `box`. Strangers with face learning on: `cluster_id`, `seen_days`, `seen_visits`, `first_seen`. Camera added a household member: `known` = true, `auto_member` = true, `person_id`, `name`, `streak_days` (with `crop_image` = face image)
`count`	`start` (UNIX seconds at the start of the period), `period_s`, `total_in`, `total_out`, `occupancy`, `lines`: `[{"name","in":{"person","vehicle"},"out":{…}}]`
`io` (peripherals)	`output`, `output_name`, `action` (`pulse` / `on` / `off`), `trigger` (the rule, or `manual`), `by` (reason / who pressed), `plate` (when caused by a plate), `ok`, `error` — see Peripherals — opening barriers, roller doors, sirens and lights from a TingPush camera. Not counted against the quota
`occupancy` (capacity)	`occupancy`: `{"person","vehicle","total","max","day","reset_time"}` — `total` = sum of the counted classes, `day` = counting day yyyymmdd

data.test = true marks a test event (the Test button): still stored and notified, not counted against limits; test counts don't go into statistics.

Pulling the data into your software: GET /api/v1/events, /api/v1/counts — TingPush public API — fetching stored plate data.


`GET /api/events?camera=&type=intrusion,face&before=&limit=`	Event list (newest first)
`GET /api/events/{id}/image/scene` · `/crop`	Images
`GET /api/counts?camera=&from=&to=&bucket=hour\\|day\\|month&by_line=`	Counts
`GET /api/occupancy?camera=`	Number inside per camera `[{"camera","person","vehicle","total","max","updatedAt"}]` (0 on a new counting day before the camera reports)
`POST /api/cameras/{id}/occupancy` `{"person":0,"vehicle":0}`	The owner resets / corrects the number inside
`GET` / `PUT /api/cameras/{id}/ai`	AI configuration + status on the camera (owner)
`POST /api/cameras/{id}/ai/test` `{"type":"intrusion"}`	The camera creates a test event (`intrusion`, `occupancy`, `face`, `count`; custom models: `{"type":"custom","model":"oil"}`)
`GET` / `PUT /api/cameras/{id}/alerts`	How the signed-in user is alerted: `[{"feature","push","call","filter"}]` — `filter`: faces `all`/`known`/`unknown`; counting = entry threshold; plates = a list like `30A12345, 51F*`. Custom models: `feature` = `c:<id>` (GET with `?custom=oil,smoke` = the models on the camera)
`GET /api/cameras/{id}/models` · `POST` `{"package":"oil","version":null}` · `GET …/models/status` · `DELETE …/models/{id}`	AI models on the camera + the server's package store; install a package (the camera downloads it, checks SHA-256 and reloads without restarting); progress; remove a custom model
`GET /api/models` · admin `GET/POST/DELETE /api/admin/models`	Model package store (admins upload a `.tar.gz`, or `.rknn` + `.json`)
`GET /api/faces` · `POST /api/faces` (multipart `name`, `note`, `cameras`, `image`) · `PUT /api/faces/{id}/cameras` · `DELETE /api/faces/{id}`	Face library
`GET /api/cameras/{id}/face-learn` · `GET …/face-learn/{cluster}/image` · `POST …/face-learn/{cluster}/promote` `{"name"}` · `POST …/face-learn/{cluster}/merge` `{"other"}` · `DELETE …/face-learn/{cluster}` · `DELETE …/face-learn`	Familiar faces (camera owner): list `{"enabled","autoDays","keepDays","autoAdded","items":[{"id","firstSeen","lastSeen","visits","days","streak","hours","hasImage"}]}`, face image, confirm a household member (returns the library person), merge cluster `other` into the one in the path, delete one / all
`POST /api/calls/{callId}/answer`	Alert call answered → other phones stop ringing
`GET /api/me/push`	`appId`, `externalId` for the TingPush SDK
WebSocket `/ws/view`	Adds `{"type":"ai_event","event":{…}}` messages for new events / when images arrive

Data attached to notifications (data): screen = event (opens the detail, event_id), live (call, camera, call_id), plates (watched plates).

7. Loading models (firmware developers)

Source: lpr_camera/src/app/ai_logic.c (tracking, lines, zones, face matching — tested on a PC), ai_engine.c (schedules, repeat suppression, building events, images), ai_config.c, src/lpr/lpr_ai.cpp (running models on the NPU).

People / vehicles: place models/object.rknn — YOLO COCO 80 classes, INT8 for rv1126b, box coordinates normalised 0..1: build it with tools/make_object_rknn.py (Ultralytics' newer yolo export format=rknn outputs pixel coordinates → not usable). lpr_ai.cpp scales the NV12 frame to BGR 640 pixels wide; class 0 → person; 1, 2, 3, 5, 7 (bicycle, car, motorcycle, bus, truck) → vehicle; it only runs while a feature is on within its active hours. Another model (e.g. 2 classes) → change OBJECT_NC and coco_to_ai().

Faces: needs 2 parts using the same recognition model (128-dimension vectors, AI_FACE_DIM):

Every frame: detect faces + compute vectors → call ai_engine_on_face(vector, box_0_1, &frame, lpr_now_mono(), time(NULL)) (the engine normalises the vector, does cosine matching against the library, and applies the threshold / zone / repeat suppression before sending events).
Adding people: implement lpr_platform_t.face_embed(jpeg_path, out[128], err, cap) — compute the vector from the portrait the server sends (the face_enroll command; the image link expires after 10 minutes). The library is stored in /userdata/lpr/faces.db.

Report model status to the app: ai_engine_set_model(0 | 1, loaded, "error").

Custom models (oil spills, smoke…, no code changes): models/custom/<id>.rknn + <id>.json, events of type custom (data.model, model_name, class, box, duration_s, level, zone). See Adding a new AI model to the camera (no code changes).

8. Testing without a model / without hardware

App: Tính năng AI & cảnh báo → Thử cảnh báo xâm nhập / quá đông / khuôn mặt (Test intrusion / capacity / face alert; the camera creates a sample event with a snapshot).
PC simulator (make sim, see lpr_camera/README.md), type into stdin:

obj person 0.9 0.42 0.5 0.58 0.8                 # 1 frame: a person (class score x1 y1 x2 y2); repeat lines to make it walk
obj person 0.9 0.45 0.5 0.61 0.8 vehicle 0.8 0.1 0.4 0.3 0.7
obj                                              # a frame with nobody
face 111 0.4 0.2 0.5 0.4                         # a face with the fake vector "dimension 111"

When you add a person in the app, the simulator prints that photo's vector dimension so you can type face <k> … to recognise them.