High Altitude Oolong: 3月 2021

2021/3/30

bookmark: read file in C++, sleep in C++

2021/3/24

curl download file, get progress and ...

ref:

extract download progress from curl output

他的command 是..

$ curl --progress-bar http://127.0.0.1/test.tar.bz2 -o test.tar.bz2 2>&1 | tr $'\r' $'\n' | sed -r 's/[# ]+/#/g;'

配合 exit code 檢查，可以知道 curl 是否下載完成, 0 是完成，其他都失敗。

$echo $?
0

要續傳的化，加上 "-C -"

$ curl -C - --progress-bar http://127.0.0.1/test.tar.bz2 -o test.tar.bz2 2>&1 | tr $'\r' $'\n' | sed -r 's/[# ]+/#/g;'

ref:

get exit code of program invoked by system call

要是用 system( ) 來呼叫 curl 下載檔案，curl 的 exit code 可以用上面ref 說的...

#include <stdio.h>
#include <sys/wait.h>
int main()
{
    int ret = system("./a.out");
    if (WEXITSTATUS(ret) == 0x10)
      return 0;
    else
      return 1;
}

來檢查。

2021/3/23

follow ..build static linked ffmpeg.

ref:

build static ffmpeg

就 download libx264 跟 ffmpeg 的 source..
然後 config 要讓 ffmpeg 找到

pi@ubuntu:~/ffempg-static/libx264-git$ ./configure --prefix=/home/pi/ffmpeg-static/workspace --enable-static --enable-pic
platform:      AARCH64
byte order:    little-endian
system:        LINUX
cli:           yes
libx264:       internal
shared:        no
static:        yes
asm:           yes
interlaced:    yes
avs:           avxsynth
lavf:          no
ffms:          no
mp4:           no
gpl:           yes
thread:        posix
opencl:        yes
filters:       crop select_every 
debug:         no
gprof:         no
strip:         no
PIC:           yes
bit depth:     8
chroma format: all

You can run 'make' or 'make fprofiled' now.

make 完後..install

pi@ubuntu:~/ffempg-static/libx264-git$ make install
install -d /home/pi/ffmpeg-static/workspace/bin
install x264 /home/pi/ffmpeg-static/workspace/bin
install -d /home/pi/ffmpeg-static/workspace/include
install -d /home/pi/ffmpeg-static/workspace/lib
install -d /home/pi/ffmpeg-static/workspace/lib/pkgconfig
install -m 644 ./x264.h /home/pi/ffmpeg-static/workspace/include
install -m 644 x264_config.h /home/pi/ffmpeg-static/workspace/include
install -m 644 x264.pc /home/pi/ffmpeg-static/workspace/lib/pkgconfig
install -m 644 libx264.a /home/pi/ffmpeg-static/workspace/lib
ranlib /home/pi/ffmpeg-static/workspace/lib/libx264.a

然後是 ffmpeg..

./configure --prefix=/home/pi/ffmpeg-static/workspace \
--extra-version=patrickz --disable-debug --disable-shared --enable-static \
--disable-doc --enable-gpl  --enable-nonfree --enable-version3 \
--pkg-config-flags="--static" \
--enable-libx264 \
--enable-filters \
--extra-cflags="-I--prefix=/home/pi/ffmpeg-static/workspace/include -static" \
--extra-ldflags="-L--prefix=/home/pi/ffmpeg-static/workspace/lib -static"

...
接著來 enable neon 看看...

2021/3/19

do .. while(0) not in macro

do {
  ...
  ...
} while(0);

一般的說明都是在 MACRO 定義，因為後面那個 ';' 的關係。

如果不是在 macro 。這樣寫有什麼好處呢？
這一篇有說..
因為在 while() 中，所以 break 有用，就可以把 break 當作 goto 來用..
以上面那個 link 中的 example:

int main()
{
    do{
        if(testOK())
            break;

        if(func2())
            break;

        if(func3())
            break;

        return 1;
    }while(0);

    do_complete();
    
    return 0;
}

2021/3/14

iOS 的 Documents
很方便，可以取代Files 的功能，同時還內建 http file server，
和 sendanywhere 很像，藉由https://docstransfer.com/ 的一個 4 位數的數字，讓 desktop browser 連到iphone Document app 的 http server。
是 local 直接連，不經過外界的 server

如果是用 usb 連接得話，甚至可以用 iphone 分享，然後就用 usb net 傳輸。不用受 wifi 頻寬限制。

配合 Document, PC 端可以用 vsftpd。
vsftpd default config 是 write disabled，要修改 /etc/vsftpd.conf，把 PUT writable 打開。

2021/3/5

log .. translate stat to mmcblk

列出 root 的 maj/min id:

stat -C "%D" /
b309

取出字串的最後兩個 characters:

str=abcd
echo "${str: -2}"
cd

合起來...

majmin=`stat -c "%D" /`
root_dev=${majmin: -1}

.. 要注意，這個懶惰的 script 只 support 1-9...

go StdinPipe and shell script

因為要做一個翻譯script，連接 go 程式和系統程式，所以要做這個測試。
大概是原 go 程式 (客戶的)，exec.Command ("a_program").
然後我們是 embedded system，沒有 a_program，而是自己寫的 b_program。

go 的程式...

 cmd := exec.Command("a_program")
 pipe, err := cmd.StdinPipe()
 err = cmd.Start()
 for k, v := range vars {
    _, err := fmt.Fprint(pipe, "%s%s%s\n",k," ", v)
 }
 pipe.Close()
 ..

其中 vars 是 [string]string 的 map type

所以仿照，測試 go program:

package main
import (
    "fmt"
    "os"
    "os/exec"
)

func main() {
    cmd := exec.Command("./ecc.sh")
    cmdin, err := cmd.StdinPipe()
    if err != nil {
        fmt.Println(err)
    }

    _, err = cmdin.Write([]byte("abc\n"))
    if err != nil {
        fmt.Println(err)
    }

    _, err = cmdin.Write([]byte("Def\n"))
    if err != nil {
        fmt.Println(err)
    }
    cmdin.Close()

    cmd.Stdout = os.Stdout
    cmd.Start()
}

然後 shell script (ecc.sh) 是...

#!/bin/bash
while read LINE; do
	echo ${LINE}
	case ${LINE} in 
		"abc") echo is ABC
			;;
		"def") echo is DEF
			;;
		*) echo unknown
	esac
done

echo AAA

exit 0

測試結果：

$ go run tstdin.go 
abc
is ABC
Def
unknown
AAA

所以.. stdin close 之後， shell script 會自己 exit while read

2021/3/4

go , cross compile for arm

ref:

golang environment

原來 go toolchain 已經 build in 一堆的 target，所以只要 define 適當的變數，就可以 cross build...
在 amd64 linux, build for arm64 linx..

export GOOS=linux
export GOARCH=arm64

然後照一般 build 的方式 go build .. 就可以 build 出 arm64 的執行檔。

.. 其實還是很麻煩，如果是要用自己的 cross-toolchain，就要宣告 CC

export CC=aarch64-linux-gnu-gcc

然後一堆 include path. libraru 問題。
要是跟 c module 共用，使用 CGO 的話，問題更多。

... 所以還是到 pi 上面 build 好了 ......

紀錄：ubuntu 20.04 server LTS on pi4

因為沒有要 gui，所以裝 server。
開機 login

username: ubuntu
password: ubuntu

第一次 login 就強制變更password (raspberry)。
開機進入發現 ssh 好像已經開好。

為了怕忘記，adduser: pi/raspberry, add to sudo group
因為....所以修改 /etc/systemd/resolv.conf，才能 apt update..
purge unattended-upgrade

2021/3/3

find /dev/root partition

有些系統 mount :

/dev/root on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=96476k,nr_inodes=24119,mode=755)
...

但是到 /dev 下，沒有 root。

這樣怎麼知道 root partition 是那一個 device / partition ?

有幾種方法：

# rdev
/dev/mmcblk0p9 /

如果沒有 rdev 這個 command，可以用 stat

# stat /
  File: /
  Size: 1024      	Blocks: 2          IO Block: 1024   directory
Device: b309h/45833d	Inode: 2           Links: 23
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-03-03 05:09:31.000000000
Modify: 2021-02-26 08:39:45.000000000
Change: 2021-02-26 08:39:45.000000000

其中 Device: b309h 就是 device 的 maj/min number

# ls -l /dev/mmcblk*
brw-rw----    1 root     disk      179,   0 Apr 11  2019 /dev/mmcblk0
brw-rw----    1 root     disk      179,  32 Apr 11  2019 /dev/mmcblk0boot0
brw-rw----    1 root     disk      179,  64 Apr 11  2019 /dev/mmcblk0boot1
brw-rw----    1 root     disk      179,   1 Apr 11  2019 /dev/mmcblk0p1
brw-rw----    1 root     disk      179,  10 Apr 11  2019 /dev/mmcblk0p10
brw-rw----    1 root     disk      179,   2 Apr 11  2019 /dev/mmcblk0p2
brw-rw----    1 root     disk      179,   3 Apr 11  2019 /dev/mmcblk0p3
brw-rw----    1 root     disk      179,   4 Apr 11  2019 /dev/mmcblk0p4
brw-rw----    1 root     disk      179,   5 Apr 11  2019 /dev/mmcblk0p5
brw-rw----    1 root     disk      179,   6 Apr 11  2019 /dev/mmcblk0p6
brw-rw----    1 root     disk      179,   7 Apr 11  2019 /dev/mmcblk0p7
brw-rw----    1 root     disk      179,   8 Apr 11  2019 /dev/mmcblk0p8
brw-rw----    1 root     disk      179,   9 Apr 11  2019 /dev/mmcblk0p9
crw-------    1 root     root      250,   0 Apr 11  2019 /dev/mmcblk0rpmb
brw-rw----    1 root     disk      179,  96 Apr 11  2019 /dev/mmcblk1
brw-rw----    1 root     disk      179,  97 Apr 11  2019 /dev/mmcblk1p1

b3h = 179d，所以知道是 mmcblk0
09h = 09d，所以是 mmcblk0p9

ubuntu 現在好像沒有 rdev 這個 command，redhat 在 2000 年也拿掉這個command，不知道為什麼..

2021/3/2

mender.io 製作系統更新檔

document source:

deploy a system update

以 pi 為範例。
系統檔由 pi 上取得，用 mender-artifact 這個 tool 來做。
mender-artifact 執行檔在 meder server 下載，參考上面的 link

首先 pi 3 要跟 PC 有網路連接，ssh login OK
然後在 PC 端下：

mender-artifact write rootfs-image -f ssh://"pi@172.16.200.12" -t "raspberrypi3" -n system-v1 -o system-v1.mender -S "-p 22"
pi@172.16.200.12's password: 
..................................................Writing Artifact...
Version             	✓
Manifest            	✓
Manifest signature  	✓
Header              	✓
Payload
........................................................................ - 100 %

輸出就是 system-v1.mender

看source code，payloads = rootfs-image，這種 type，install module 是寫死的，也不容許有 custom script

換一個好了...

在 target board 上，使用這個 command，出現 Error:

mender-artifact write rootfs-image -f ssh://"root@10.0.1.2" -t "dm1" -n system-d1 -o system-d1.mender -S "-p 22"
time="2021-03-03T08:23:43Z" level=error msg="Device-local data is stored on the partition being snapshotted: /var/lib/mender. 
The recommended approach is to have a separate data-partition mounted on \"/data\" and add a symbolic link (/var/lib/mender -> /data). 
Consult either https://docs.mender.io/system-updates-debian-family/overview#partition-layoutfor Debian family 
or https://docs.mender.io/system-updates-yocto-project/overview#partition-layoutin case you use Yocto"
time="2021-03-03T08:23:43Z" level=error msg="data store (/var/lib/mender) is located on filesystem /"
imgFilesystemType error: imgFilesystemType: blkid command failed: exit status 2

看一下 example 的 raspberrypi3 的 /var/lib:

lrwxrwxrwx 1 root root   12 Jan 21 03:06 mender -> /data/mender

把 /var/lib/mender copy 到 /data 後，再

ln -s /data/mender /var/lib/mender

這個 folder 存的好像是 device identity，所以要是copy 這個 folder 到令一個 device。這兩格 device 在 mender.io 會被認為是同一個。
所以不能 copy
第一次 run mender daemon 時，會產生這個 folder 的資料。

之後再 mender-artifact write 就正確download 到，並且產生 system-d1.mender

upload 到 server, make release (create 一個 dm1group.把 dm1 家入), deploy 時 fail...server 端 log:

2021-03-03 08:35:47 +0000 UTC info: Running Mender client version: 7cb96ca
2021-03-03 08:35:47 +0000 UTC debug: Handling update fetch state
2021-03-03 08:35:47 +0000 UTC debug: received the GET_AUTH_TOKEN action
2021-03-03 08:35:47 +0000 UTC debug: Connecting to server docker.mender.io
2021-03-03 08:35:47 +0000 UTC debug: Status reported, response 204 No Content
2021-03-03 08:35:47 +0000 UTC debug: Received fetch update response &{200 OK 200 HTTP/1.1 1 1 
map[Accept-Ranges:[bytes] Connection:[keep-alive] Content-Length:[144589824] Content-Security-Policy:[block-all-mixed-content] 
Content-Type:[application/vnd.mender-artifact] Date:[Wed, 03 Mar 2021 08:35:48 GMT] Etag:["c8154---"] 
Expires:[Wed, 03 Mar 2021 08:43:55 GMT] Last-Modified:[Wed, 03 Mar 2021 08:33:58 GMT] 
Server:[openresty/1.13.6.2] Strict-Transport-Security:[max-age=0; includeSubdomains; preload] 
Vary:[Origin] X-Amz-Request-Id:[1668C829E54429D3] X-Content-Type-Options:[nosniff] X-Frame-Options:[DENY] 
X-Xss-Protection:[1; mode=block]] 0x400045abc0 144589824 [] false false map[] 0x400051e800 }+
2021-03-03 08:35:47 +0000 UTC info: State transition: update-fetch [Download_Enter] -> update-store [Download_Enter]
2021-03-03 08:35:47 +0000 UTC debug: received the GET_AUTH_TOKEN action
2021-03-03 08:35:47 +0000 UTC debug: Handling update install state
2021-03-03 08:35:47 +0000 UTC debug: received the GET_AUTH_TOKEN action
2021-03-03 08:35:47 +0000 UTC debug: Connecting to server docker.mender.io
2021-03-03 08:35:47 +0000 UTC debug: Status reported, response 204 No Content
2021-03-03 08:35:47 +0000 UTC debug: Reading data from the device manifest file: /var/lib/mender/device_type
2021-03-03 08:35:47 +0000 UTC debug: device_type=dm1
2021-03-03 08:35:47 +0000 UTC debug: Current manifest data: dm1
2021-03-03 08:35:47 +0000 UTC info: No public key was provided for authenticating the artifact
2021-03-03 08:35:47 +0000 UTC debug: Checking if device [dm1] is on compatible device list: [dm1]
2021-03-03 08:35:47 +0000 UTC debug: Installer: Successfully read artifact [name: system-d1; version: 3; compatible devices: [dm1]]
2021-03-03 08:35:47 +0000 UTC error: Artifact install failed: Payload: can not install Payload: rootfs.tmp214618134: No match between boot and root partitions.: exec: "fw_printenv": executable file not found in $PATH
2021-03-03 08:35:47 +0000 UTC info: State transition: update-store [Download_Enter] -> cleanup [Error]
2021-03-03 08:35:47 +0000 UTC debug: Transitioning to error state
2021-03-03 08:35:47 +0000 UTC debug: statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds
2021-03-03 08:35:47 +0000 UTC debug: statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds
2021-03-03 08:35:47 +0000 UTC debug: Handling Cleanup state
2021-03-03 08:35:47 +0000 UTC info: State transition: cleanup [Error] -> update-status-report [none]
2021-03-03 08:35:47 +0000 UTC debug: statescript: The timeout for executing scripts is not defined; using default of 1h0m0s seconds
2021-03-03 08:35:47 +0000 UTC debug: Handling update status report state
2021-03-03 08:35:47 +0000 UTC debug: received the GET_AUTH_TOKEN action
2021-03-03 08:35:47 +0000 UTC debug: Connecting to server docker.mender.io
2021-03-03 08:35:47 +0000 UTC debug: Status reported, response 204 No Content
2021-03-03 08:35:47 +0000 UTC debug: Attempting to upload deployment logs for failed update

因為 mender client hard coding env tool : fw_printenv

Custom Update Module

除了寫死，固定的 dual-rootfs 更新，其他的更新都是在這裡。
讓你自由的(?)做出更新任何東西(bootloader, deb install, files, directory ..)

document source:

Create A Custom Update Moule

回到 rootfs 更新..
device 端手動執行 mender daemon，看 log..
deploy,, 進行 download ...

INFO[0934] Correct request for getting image from: 
https://s3.docker.mender.io:9000/mender-artifact-storage/127a8a88-ada3-4a7a-9212-c16a376db1cc?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20210303%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210303T033619Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&response-content-type=application%2Fvnd.mender-artifact&X-Amz-Signature=88d79-- [name: system-v2; devices: [raspberrypi3]] 
INFO[0934] State transition: update-check [Sync] -> update-fetch [Download_Enter] 
INFO[0934] Running Mender client version: 2.5.0         
INFO[0934] State transition: update-fetch [Download_Enter] -> update-store [Download_Enter] 
INFO[0934] No public key was provided for authenticating the artifact 
INFO[0934] Opening device "/dev/mmcblk0p2" for writing  
INFO[0934] Native sector size of block device /dev/mmcblk0p2 is 512 bytes. Mender will write in chunks of 1048576 bytes

訂閱：文章 (Atom)