昨天，特斯拉前 AI 总监、OpenAI 联合创始人 Andrej Karpathy 开源了 llama2.c 。 只用 500 行纯 C 语言就能训练和推理 llama 2 模型的结构，没有任何繁杂的 python 依赖。这个项目一推出就受到咱们的追捧，24 小时内 GitHub 收获 4000 颗星！
但是，C 编译的原生机器码不能跨渠道，不安全，也不可被调度。这些问题使得它的使用场景非常有限。这时，一个大胆的想法油然而生！把 llama2.c 编译成 Wasm 在 WasmEdge 里运转！
这么做的好处是：
- 轻量级：一个 Wasm 文件只有几十 KB 大小，相比于 Python 镜像动辄几百上千 MB，差了一万倍。
- 安全：沙箱机制，供给隔离性，合适多租户的云布置。
- 可移植：Wasm 文件无需任何改变，可以在 x86, ARM, Apple, RISC-V 机器上运转
- 功能：没有冷启动，且运转速度挨近本机速度
- 能够被 Docker 和 kuberbetes 等容器东西进行管理
下面，咱们来详细看看是怎么完成的。
先决条件
请参考 WasmEdge 的官方文档安装 WasmEdge runtime。
curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | sudo bash -s — -p /usr/local
预备 wasi-sdk
export WASI_VERSION=20
export WASI_VERSION_FULL=${WASI_VERSION}.0
wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_VERSION}/wasi-sdk-${WASI_VERSION_FULL}-linux.tar.gz
tar xvf wasi-sdk-${WASI_VERSION_FULL}-linux.tar.gz
export WASI_SDK_PATH=`pwd`/wasi-sdk-${WASI_VERSION_FULL}
CC="${WASI_SDK_PATH}/bin/clang --sysroot=${WASI_SDK_PATH}/share/wasi-sysroot"
把 llama2.c 编译成 Wasm
git clone https://github.com/karpathy/llama2.c.git
cd llama2.c
$CC run.c -D_WASI_EMULATED_PROCESS_CLOCKS -lwasi-emulated-process-clocks -o run.wasm
优化 wasm file 而且运转
这里咱们将使用 WasmEdge 的 AOT 编译器对编译好的 Wasm 文件进行优化，以提升 Wasm 的功能。
$ wget https://karpathy.ai/llama2c/model.bin -P out
$ wasmedgec run.wasm run-aot.wasm
[2023-07-24 16:39:52.851] [info] compile start
[2023-07-24 16:39:52.858] [info] verify start
[2023-07-24 16:39:52.862] [info] optimize start
[2023-07-24 16:39:53.251] [info] codegen start
[2023-07-24 16:39:53.608] [info] output start
[2023-07-24 16:39:53.611] [info] compile done
[2023-07-24 16:39:53.611] [info] output start
运转这个 wasm 文件
$ wasmedge --dir .:. run-aot.wasm out/model.bin
输出如下：
Once upon a time, there was a wealthy man. He lived in a big house with many things. The wealthy man liked to play in the fog.
One day, the wealthy man saw that the fog was increasing. The fog was getting stronger and the weight on the man's body made it hard to walk. The man said, "Oh no, I need to find a place to stop."
The wealthy man walked and walked, looking for a safe place. Soon, he found a small house. To his surprise, the house was full of toys and candy! The man said, "I found this house of good value. I can keep all the toys and candy in it." And from that day on, the wealthy man never played in the fog again.
<s>
Once upon a time, there was a little girl named Lily. She loved to play with her toys and sing songs. One day, Lily's friend Timmy came over to play.
"Hi Lily, do you want to play with my new toy car?" asked Timmy.
"Yay, thank you!" replied Lily.
But after a while, Lily started to feel sleep
achieved tok/s: 30.738912
便是这样啦。 WasmEdge 也将逐步支撑 Llama2 7B 及更大的 model。
最后。 假如你有爱好使用 Wasm 作为 Python 的高功能替代品在生产环境中进行 AI 推理，请检查咱们根据 Rust 的库 mediapipe-rs。 这是 Google 的 mediapipe 模型。而且同时支撑 TF Lite 和 Pytorch！
