run llm on 6 TOPS Soc

• testing ai and llm on rockchip rk3588

使用 GPU

• 香橙派5部署chatglm2
• 香橙派5部署chatglm2-6b模型
• 香橙派5部署chatglm2

一般，使用TVM

• A卡跑大模型，性能达到4090的80%（价格只有一半）：陈天奇TVM团队出品
• Talk | 卡内基梅隆大博士生侯博涵：MLC-LLM-机器学习编译与在大语言模型上的应用
• 利用GPU加速，在Orange Pi上跑LLMs

msys2 pacman -S failed, key not verified

原來用得好好的，某一次後要安裝 package 就 fail，而且一直發生 db lock，
猜想可能是做過 upgrade : pacman -Syuu 的關係。
所以去找 release 的各個版本來試。
發現在 20231026 之後(包含)，用 -S 安裝package 就發生 key 的問題了。
所以所有的版本要是做過 -Syuu 更新到最新版本，當然就會出問題。
-- 是過最新 daily build 版本也依樣。

所以只能拿 20230718 的版本來用，並且記得絕對不能 升級。

repo: no module named 'formatter'

ref:

• help: switch from formatter module to textwrap

因為repo 使用的 python 在 python 3.10 之後，已經把 formatter 這個 module 移除了。
所以 google 建議修改成:

diff --git a/subcmds/help.py b/subcmds/help.py
index 1e16019..9c4756b 100644
--- a/subcmds/help.py
+++ b/subcmds/help.py
@@ -17,7 +17,7 @@
 from __future__ import print_function
 import re
 import sys
-from formatter import AbstractFormatter, DumbWriter
+import textwrap

 from subcmds import all_commands
 from color import Coloring
@@ -88,7 +88,7 @@ Displays detailed usage information about a command.
         Coloring.__init__(self, gc, 'help')
         self.heading = self.printer('heading', attr='bold')

-        self.wrap = AbstractFormatter(DumbWriter())
+        self._first = True

       def _PrintSection(self, heading, bodyAttr):
         try:
@@ -123,9 +123,12 @@ Displays detailed usage information about a command.
             self.nl()
             continue

-          self.wrap.add_flowing_data(para)
-          self.wrap.end_paragraph(1)
-        self.wrap.end_paragraph(0)
+        lines = textwrap.warp(para.replace(' ',' '), width=80, break_long_word=False, break_on_hyphens=False)
+
+        for line in lines:
+            self.write('%s',line)
+            self.nl()
+        self.nl()

     out = _Out(self.manifest.globalConfig)
     out._PrintSection('Summary', 'helpSummary')

---

其實可以去 .repo/repo 下面，加入 git-repo 做remote source，然後 checkout 一個新版也可以

git remote add gitrepo https://github.com/GerritCodeReview/git-repo
git fetch gitrepo
git checkout v2.29

.. 因為這個 repo mirror 最小的 tag 好像只有 v2.29，沒有更早的。

Windows Terminal

ref:

• Adding MSYS to windows terminal
• msys2 - Terminals

Windows 提供一個新的 terminal 程式，或者說是 front-end，背後可以是 PowerShell, Cmd, 或是自己install 的 msys。
以 msys2 的 shell 為例，自己增加一個 profile。

選 Setting，最下面的 "開啟 json 檔"
json 檔中的 "profiles" 就是一堆 shell 的command.

Android Platform API, packages in api level

有關 Android 各版本提供的 api，可以查詢:

• Android Platform Packages

網頁左邊的panel，有一個 API level 的下拉選單，可以選 Api level
依照你選的 level 變更，不支援的 package 會變成灰色。
另外在每一個 package 的說明頁都會有 "Added in API level " 的標示。

api level 對應 Android OS version，android 10 之後:

Platform Version    API level
     14             34
     13             33
     12             32
                    31
     11             30
     10             29

另外，這個: Android OS core topics，左邊panel 的每個 item 都有列出每個feature 在 哪個 Android 版本開始實作。
太多了，每個都有細項可以看。


大概列一下每個版本新增的package api.

Android 14(level 34):

• adservice : 不清楚，好想是open advertising id 供 ap 使用
• documentid :
• sdk sandbox : application sandbox 的 user api

有關 application 的 security, isolation，goole 有一篇說明從 android 5 到 android 10 的改進 -- app-sandbox:

• In Android 5.0, SELinux provided mandatory access control (MAC) separation between the system and apps. However, all third-party apps ran within the same SELinux context so inter-app isolation was primarily enforced by UID DAC.
• In Android 6.0, the SELinux sandbox was extended to isolate apps across the per-physical-user boundary. In addition, Android also set safer defaults for application data: For apps with targetSdkVersion >= 24, default DAC permissions on an app's home dir changed from 751 to 700. This provided safer default for private app data (although apps may override these defaults).
• In Android 8.0, all apps were set to run with a seccomp-bpf filter that limited the syscalls that apps were allowed to use, thus strengthening the app/kernel boundary.
• In Android 9 all non-privileged apps with targetSdkVersion >= 28 must run in individual SELinux sandboxes, providing MAC on a per-app basis. This protection improves app separation, prevents overriding safe defaults, and (most significantly) prevents apps from making their data world accessible.
• In Android 10 apps have a limited raw view of the filesystem, with no direct access to paths like /sdcard/DCIM. However, apps retain full raw access to their package-specific paths, as returned by any applicable methods, such as Context.getExternalFilesDir().

其他的新功能還有 apk cached (freezer): cached-apps-freezer，加速 app 啟動時間。

大的功能是硬體的檢查: Android Health，一樣在每個android 版本有不同的實作。
一開始的brief:

Android 9 includes android.hardware.health HAL 2.0, a major version upgrade from health@1.0 HAL. This new HAL has the following advantages:

• Cleaner separation between framework and vendor code.
• Deprecates the unnecessary healthd daemon.
• Greater degrees of freedom for vendor customization in health information reports.
• More device health information than just battery.

Android 11 includes android.hardware.health HAL 2.1, a minor version upgrade from health@2.0 HAL. This new HAL has the following advantages:

• Easier to implement
• Better conformance with existing 2.0 HAL APIs
• Better Treble separation in off-mode charging code
• Better support for the framework to indicate the battery health of the device

Android 13 includes android.hardware.health AIDL HAL, a conversion from health@2.1 HAL. This new HAL has the following advantages:

• Remove unused charger-related APIs
• Remove unused StorageAttribute and related fields
• Support dock charging.

Android 12 後:
新增 App Hibernation 功能。一段時間沒用的 app，會自動 release 他用的 memory 和 cache，把她放回 storage。
新增一個 memory monitor 的機制: MM Event -Historical Memory Statics

Android 14 新增一個 Power Mode : GAME.

test build android 14 for raspberry pi 4

因為要試試看 Android 12, 13, 14 的差異，所以用 raspberry pi 來測試。
順便紀錄一下很久沒做android (上次到 android 9) 之後，覺得是新的東西。
先照:

• raspberry-vanilla

果然 prequreirement package 有新的:

sudo apt-get install bc coreutils dosfstools e2fsprogs fdisk kpartx mtools ninja-build pkg-config python3-pip
sudo pip3 install meson mako jinja2 ply pyyaml dataclasses

以 ubunutu 22.04，已經 build 過 android12 的系統。
結果缺 mtools (mcopy command). 和 pip install 的 meson, mako, ply.
結果用 pip3 install 還是 build fail，最後是用 apt (deb) 的 package 才 build OK

bionic/apex

結果是一種file format.從Android 10 就有了。
這種file 是用來更新用(?)
以往的 apk 都是用在 application 更新，所以都是由 package manager 負責。
但是如果是系統 component (例如native service)，不是由 vm run 的程式，就沒版辦法了。
這個 apex 系統就是為了這個目的。
apex 檔案裡面主要是一個 ext4 的 img file，裡面包含需要的 program. so 和一些config file。
android 另外新增一個 apexd 服務，boot 時會檢查 /apex/ 目錄下有沒有 *.apex 檔案。
有的話就 把img loop mount，然後叫起裡面的服務。

另外，他也支援更新。因為 /apex 目錄是 ro，所以安裝都放在其他位置(/data ?)。
apexd 會去見查 /data 下有沒有 /apex/ 內檔案的對應新版本，有就用，沒有就用 /apex/ 下的。

結果在 android 14 上，幾乎所有東西都 apex 化了。
framework 裡的 service，甚至一些 apk 也都包裝成 apex.
out 目錄中，有一個 apex 目錄。裡面很多 componments folder.


raspberry-vanilla 開機失敗。
試試看： android-rpi。

update: 開機成功。
失敗是因為使用 pi4, 2GB 版本，logcat 一直出現 oom, 最後 watchdog 出現 wait too long，然後 goodbye 重新啟動 dalvik ? activitymanager.
改用 4GB 版本直接開啟 OK

---

lunch aosp_rpi4_tv 的話，build fail:

ERROR: files are incompatible: android.hardware.camera.provider@2.5::ICameraProvider/legacy/0 is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.gatekeeper@1.0::IGatekeeper/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.graphics.allocator@4.0::IAllocator/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.graphics.composer@2.4::IComposer/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.keymaster@4.1::IKeymasterDevice/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.tv.cec@1.0::IHdmiCec/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.usb.gadget@1.2::IUsbGadget/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.camera.provider@2.5::ICameraProvider/legacy/0 is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.gatekeeper@1.0::IGatekeeper/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.graphics.allocator@4.0::IAllocator/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.graphics.composer@2.4::IComposer/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.keymaster@4.1::IKeymasterDevice/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.tv.cec@1.0::IHdmiCec/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.usb.gadget@1.2::IUsbGadget/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.camera.provider@2.5::ICameraProvider/legacy/0 is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.gatekeeper@1.0::IGatekeeper/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.graphics.composer@2.4::IComposer/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.keymaster@4.1::IKeymasterDevice/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.tv.cec@1.0::IHdmiCec/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.usb.gadget@1.2::IUsbGadget/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.camera.provider@2.5::ICameraProvider/legacy/0 is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.gatekeeper@1.0::IGatekeeper/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.graphics.allocator@4.0::IAllocator/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.graphics.composer@2.4::IComposer/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.keymaster@4.1::IKeymasterDevice/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.tv.cec@1.0::IHdmiCec/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
android.hardware.usb.gadget@1.2::IUsbGadget/default is deprecated in compatibility matrix at FCM Version 8; it should not be served.
The following instances are in the device manifest but not specified in framework compatibility matrix:
    android.hardware.camera.provider@2.5::ICameraProvider/legacy/0
    android.hardware.gatekeeper@1.0::IGatekeeper/default
    android.hardware.graphics.allocator@4.0::IAllocator/default
    android.hardware.graphics.composer@2.4::IComposer/default
    android.hardware.keymaster@4.1::IKeymasterDevice/default
    android.hardware.tv.cec@1.0::IHdmiCec/default
    android.hardware.usb.gadget@1.2::IUsbGadget/default
    android.hardware.wifi.supplicant.ISupplicant/default (@1)
Suggested fix:
1. Update deprecated HALs to the latest version.
2. Check for any typos in device manifest or framework compatibility matrices with FCM version >= 8.
3. For new platform HALs, add them to any framework compatibility matrix with FCM version >= 8 where applicable.
4. For device-specific HALs, add to DEVICE_FRAMEWORK_COMPATIBILITY_MATRIX_FILE or DEVICE_PRODUCT_COMPATIBILITY_MATRIX_FILE.: Success
INCOMPATIBLE
16:15:08 ninja failed with: exit status 1                                                                                                                                                 
#### failed to build some targets (43:52 (mm:ss)) ####   

update: 錯了，上面的error 是因為用 "m" 來build 導致的。
如果 follow 原 po 的說明，用

make bootimage systemimage vendorimage -j$(nproc)

就沒有 build fail 了。

---

android14 的話，還需要 texlive-full ，領外 meson 也要 0.6以上，所以 ubuntu 20.04 的話，apt 的 meson 版本不夠 (0.52)，要用 pip 的版本才夠。
另外，要用 python3，所以乾脆整個系統都用 python3 了： python-is-python3

LVGL set font and filesystem

ref:

• LVGL sample code to change font
• lvgl doc : fonts

會遭遇兩個問題：

• 如何讀入字型檔
• 如何設定字型

讀入就跟 filesystem 有關。
filesystem 就跟 platform 有關，是用 windows, linux 還是 mcu，filesystem 的 api 不一樣。
lvgl 用 lv_conf.h 來設定用哪個 api:

/*File system interfaces for common APIs */

/*API for fopen, fread, etc*/
#define LV_USE_FS_STDIO 1
#if LV_USE_FS_STDIO
    #define LV_FS_STDIO_LETTER 'A'     /*Set an upper cased letter on which the drive will accessible (e.g. 'A')*/
    #define LV_FS_STDIO_PATH ""         /*Set the working directory. File/directory paths will be appended to it.*/
    #define LV_FS_STDIO_CACHE_SIZE 0    /*>0 to cache this number of bytes in lv_fs_read()*/
#endif

/*API for open, read, etc*/
#define LV_USE_FS_POSIX 0
#if LV_USE_FS_POSIX
    #define LV_FS_POSIX_LETTER 'A'     /*Set an upper cased letter on which the drive will accessible (e.g. 'A')*/
    #define LV_FS_POSIX_PATH ""         /*Set the working directory. File/directory paths will be appended to it.*/
    #define LV_FS_POSIX_CACHE_SIZE 0    /*>0 to cache this number of bytes in lv_fs_read()*/
#endif

/*API for CreateFile, ReadFile, etc*/
#define LV_USE_FS_WIN32 0
#if LV_USE_FS_WIN32
    #define LV_FS_WIN32_LETTER '\0'     /*Set an upper cased letter on which the drive will accessible (e.g. 'A')*/
    #define LV_FS_WIN32_PATH ""         /*Set the working directory. File/directory paths will be appended to it.*/
    #define LV_FS_WIN32_CACHE_SIZE 0    /*>0 to cache this number of bytes in lv_fs_read()*/
#endif

那個 LETTER 不是代表 windows 的 A 槽，B槽。而是用來代表 device.
像我用 ubuntu，試過用 "A:~/font.bin" 不行，一定要寫full path: "A:\home\charles\font.bin"，然後用的是 FS_STDIO (會用 fopen())

load font 之後放進style，再把 style 設定到 object。
設定 font 跟 style 的 code:

     lv_font_t *myfont = lv_font_load("A:/home/charles/font.bin");
     static lv_style_t style;
     lv_style_init(&style);
     lv_style_set_text_font(&style,myfont);

要注意 stype 要是 static，因為之後系統會 periodically render 畫面，那時候會 access style 變數。如果是 local 變數，已經 free 調的話，就會出現 segment fault

下面用 example 的 label 為例子，設定 style:

    lv_obj_t * label1 = lv_label_create(lv_scr_act());
    lv_label_set_long_mode(label1, LV_LABEL_LONG_WRAP);     /*Break the long lines*/
    lv_label_set_recolor(label1, true);                      /*Enable re-coloring by commands in the text*/
    lv_label_set_text(label1, "TEST MY FONT");
    lv_obj_set_width(label1, 150);  /*Set smaller width to make the lines wrap*/
    lv_obj_set_style_text_align(label1, LV_TEXT_ALIGN_CENTER, 0);
    lv_obj_align(label1, LV_ALIGN_CENTER, 0, -40);
    
    lv_obj_add_style(label1,&style,0);

HUB 8735 -- NN

ref:

• NPU 开发工具安装
• HUB 8735 國產IC應用實作開發系列活動
• ambpro2 sdk : YOLOv4 Object Detection Demo Quick Start
• HUB 8735 開發板【離線AI模型轉換】_20230317
• AMB82 AI 模型轉換