“本文正在参加「金石计划 . 分割6万现金大奖」/post/716209…“
元国际(Metaverse),是人类运用数字技能构建的,由实践国际映射或超越实践国际,可与实践国际交互的虚拟国际,具有新式社会体系的数字生活空间。
可见元国际第一步是创立专属虚拟形象,但创立3D虚拟形象需求3D根底知识。关于大部分android
开发者(包含我本人)来说没有这方面的积累。莫非因而咱们就难以进入元国际的国际吗?不,今日咱们借助即构渠道供给的Avatar SDK,只需有Android
根底即可进入最火的元国际国际!先看效果:
上面gif被紧缩的比较狠,这儿放一张截图:
1 免费注册即构开发者
前往即构操控台网站:console.zego.im/注册开发者账户。注册成功后,创立项目:
操控台中能够得到AppID和AppSign两个数据,这两个数据是重要凭据,后边会用到。
因为咱们用到了即构的Avatar
功用,但目前官方没有供给线上主动敞开方式,需求主动找客服请求(当然,这是免费的),只需供给自己项目的包名,即可注册Avatar权限。翻开doc-zh.zego.im/article/152…右下角有“联络咱们”
,点击即可跟客服请求免费注册权限。
留意,假如不向客服请求Avatar
权限,调用AvatarSDK
会失利!
2 准备开发环境
前往即构官方元国际开发SDK网站doc-zh.zego.im/article/153…下载SDK,得到如下文件列表:
接下来过程如下:
- 翻开
SDK
目录,将里边的ZegoAvatar.aar
复制至app/libs
目录下。 - 增加
SDK
引用。翻开app/build.gradle
文件,在dependencies
节点引进libs
下一切的jar
和aar
:
dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar', "*.aar"]) //通配引进
//其他略
}
- 设置权限。根据实践运用需求,设置运用所需权限。进入
app/src/main/AndroidManifest.xml
文件,增加权限。
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-feature
android:glEsVersion="0x00020000"
android:required="true" />
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />
因为
Android 6.0
在一些比较重要的权限上要求有必要请求动态权限,不能只经过AndroidMainfest.xml
文件请求静态权限。详细动态请求权限代码可看附件源码。
3 导入资源
上一小节下载的zip
文件中,有个assets
目录。里边包含了Avatar
形象相关资源,如:衣服、眉毛、鞋子等。这是即构官方免费供给的资源,能够满意一般性需求了。当然了,假如想要自己定制资源也是能够的。assets
文件内容如下:
资源名称 | 阐明 |
---|---|
AIModel.bundle | Avatar 的 AI 模型资源。当运用表情随动、声响随动、AI 捏脸等才能时,有必要先将该资源的绝对路径设置给 Avatar SDK。 |
base.bundle | 美术资源,包含根底 3D 人物模型资源、资源映射表、人物模型默许外形等。 |
Packages | 美妆、挂件、装饰等资源。 每个资源 200 KB ~ 1 MB 不等,跟资源复杂度相关。 |
留意:因为资源文件很大,上面下载的美术资源只包含少量有必要资源。假如需求悉数资源,能够去官网找客服索要:doc-zh.zego.im/article/148…
上面的文件需求存放到Android
本地SDCard
上,这儿有2个计划可供参阅:
- 计划一: 将资源先放入到
app/src/assets
目录内,然后在app发动时,主动将assets的内容复制到SDcard
中。 - 计划二: 将资源放入到服务器端,运行时主动从服务器端下载。
为了简单起见,咱们这儿选用计划一。参阅代码如下, 详细代码能够看附件:
AssetsFileTransfer.copyAssetsDir2Phone(this.getApplication(),
"AIModel.bundle", "assets");
AssetsFileTransfer.copyAssetsDir2Phone(this.getApplication(),
"base.bundle", "assets");
AssetsFileTransfer.copyAssetsDir2Phone(this.getApplication(),
"Packages", "assets");
4 创立虚拟形象
创立虚拟形象本质上来说就是调用即构的Avatar SDK
,其大致流程如下:
接下来咱们逐渐完结上面流程。
需求留意的是,上面暗示图中选用的是AvatarView,能够十分方便的直接展现Avatar形象,但是不方便后期将画面经过RTC实时传递, 因而,咱们后边的详细完结视经过TextureView替代AvatarView。
4.1 请求权鉴
这儿再次强调一下,一定要翻开doc-zh.zego.im/article/152…点击右下角有“联络咱们”
,向客服请求免费注册Avatar权限。否则无法运用Avatar SDK
。
请求权鉴代码如下:
public class KeyCenter {
// 操控台地址: https://console.zego.im/dashboard
public static long APP_ID = 这儿值能够在操控台查询,参阅第一节; //这儿填写APPID
public static String APP_SIGN = 这儿值能够在操控台查询,参阅第一节;
// 鉴权服务器的地址
public final static String BACKEND_API_URL = "https://aieffects-api.zego.im?Action=DescribeAvatarLicense";
public static String avatarLicense = null;
public static String getURL(String authInfo) {
Uri.Builder builder = Uri.parse(BACKEND_API_URL).buildUpon();
builder.appendQueryParameter("AppId", String.valueOf(APP_ID));
builder.appendQueryParameter("AuthInfo", authInfo);
return builder.build().toString();
}
public interface IGetLicenseCallback {
void onGetLicense(int code, String message, ZegoLicense license);
}
/**
* 在线拉取 license
* @param context
* @param callback
*/
public static void getLicense(Context context, final IGetLicenseCallback callback) {
requestLicense(ZegoAvatarService.getAuthInfo(APP_SIGN, context), callback);
}
/**
* 获取license
* */
public static void requestLicense(String authInfo, final IGetLicenseCallback callback) {
String url = getURL(authInfo);
HttpRequest.asyncGet(url, ZegoLicense.class, (code, message, responseJsonBean) -> {
if (callback != null) {
callback.onGetLicense(code, message, responseJsonBean);
}
});
}
public class ZegoLicense {
@SerializedName("License")
private String license;
public String getLicense() {
return license;
}
public void setLicense(String license) {
this.license = license;
}
}
}
在获取Lincense时,只需调用getLicense
函数,例如在Activity
类中只需如下调用:
KeyCenter.getLicense(this, (code, message, response) -> {
if (code == 0) {
KeyCenter.avatarLicense = response.getLicense();
showLoading("正在初始化...");
avatarMngr = AvatarMngr.getInstance(getApplication());
avatarMngr.setLicense(KeyCenter.avatarLicense, this);
} else {
toast("License 获取失利, code: " + code);
}
});
4.2 初始化AvatarService
初始化AvatarService过程比较绵长(可能要几秒),经过敞开worker线程后台加载以防止主线程阻塞。因而咱们界说一个回调函数,待完结初始化后回调通知:
public interface OnAvatarServiceInitSucced {
void onInitSucced();
}
public void setLicense(String license, OnAvatarServiceInitSucced listener) {
this.listener = listener;
ZegoAvatarService.addServiceObserver(this);
String aiPath = FileUtils.getPhonePath(mApp, "AIModel.bundle", "assets"); // AI 模型的绝对路径
ZegoServiceConfig config = new ZegoServiceConfig(license, aiPath);
ZegoAvatarService.init(mApp, config);
}
@Override
public void onStateChange(ZegoAvatarServiceState state) {
if (state == ZegoAvatarServiceState.InitSucceed) {
Log.i("ZegoAvatar", "Init success");
// 要记得及时移除通知
ZegoAvatarService.removeServiceObserver(this);
if (listener != null) listener.onInitSucced();
}
}
这儿setLicense
函数内完结初始化AvatarService
,初始化完结后会回调onStateChange函数。但是要留意,在初始化之前有必要把资源文件复制到本地SDCard,即完结资源导入:
private void initRes(Application app) {
// 先把资源复制到SD卡,留意:线上运用时,需求做一下判别,防止屡次复制。资源也能够做成从网络下载。
if (!FileUtils.checkFile(app, "AIModel.bundle", "assets"))
FileUtils.copyAssetsDir2Phone(app, "AIModel.bundle", "assets");
if (!FileUtils.checkFile(app, "base.bundle", "assets"))
FileUtils.copyAssetsDir2Phone(app, "base.bundle", "assets");
if (!FileUtils.checkFile(app, "human.bundle", "assets"))
FileUtils.copyAssetsDir2Phone(app, "human.bundle", "assets");
if (!FileUtils.checkFile(app, "Packages", "assets"))
FileUtils.copyAssetsDir2Phone(app, "Packages", "assets");
}
4.3 创立虚拟形象
前面在doc-zh.zego.im/article/153…下载SDK
包含了helper
目录,这个目录里边有十分重要的两个文件:
其间ZegoCharacterHelper
文件是个接口界说类,即虽然是个类,但详细的完结悉数在ZegoCharacterHelperImpl
中。咱们先一睹为快,看看ZegoCharacterHelper
包含了哪些可处理的特点:
public class ZegoCharacterHelper {
public static final String MODEL_ID_MALE = "male";
public static final String MODEL_ID_FEMALE = "female";
//****************************** 捏脸维度的 key 值 ******************************/
public static final String FACESHAPE_BROW_SIZE_Y = "faceshape_brow_size_y";// 眉毛厚度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_BROW_SIZE_X = "faceshape_brow_size_x";// 眉毛长度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_BROW_ALL_Y = "faceshape_brow_all_y";// 眉毛高度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_BROW_ALL_ROLL_Z = "faceshape_brow_all_roll_z";// 眉毛旋转, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_EYE_SIZE = "faceshape_eye_size"; // 眼睛大小, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_EYE_SIZE_Y = "faceshape_eye_size_y";
public static final String FACESHAPE_EYE_ROLL_Y = "faceshape_eye_roll_y";// 眼睛高度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_EYE_ROLL_Z = "faceshape_eye_roll_z";// 眼睛旋转, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_EYE_X = "faceshape_eye_x";// 双眼眼距, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_NOSE_ALL_X = "faceshape_nose_all_x";// 鼻子宽度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_NOSE_ALL_Y = "faceshape_nose_all_y";// 鼻子高度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_NOSE_SIZE_Z = "faceshape_nose_size_z";
public static final String FACESHAPE_NOSE_ALL_ROLL_Y = "faceshape_nose_all_roll_y";// 鼻头旋转, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_NOSTRIL_ROLL_Y = "faceshape_nostril_roll_y";// 鼻翼旋转, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_NOSTRIL_X = "faceshape_nostril_x";
public static final String FACESHAPE_MOUTH_ALL_Y = "faceshape_mouth_all_y";// 嘴巴上下, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_LIP_ALL_SIZE_Y = "faceshape_lip_all_size_y";// 嘴唇厚度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_LIPCORNER_Y = "faceshape_lipcorner_y";// 嘴角旋转, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_LIP_UPPER_SIZE_X = "faceshape_lip_upper_size_x"; // 上唇宽度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_LIP_LOWER_SIZE_X = "faceshape_lip_lower_size_x"; // 下唇宽度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_JAW_ALL_SIZE_X = "faceshape_jaw_all_size_x";// 下巴宽度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_JAW_Y = "faceshape_jaw_y";// 下巴高度, 取值规模0.0-1.0,默许值0.5
public static final String FACESHAPE_CHEEK_ALL_SIZE_X = "faceshape_cheek_all_size_x";// 脸颊宽度, 取值规模0.0-1.0,默许值0.5
//其他函数略
}
能够看到,上面数据根本包含一切人脸特点了,根本具有了捏脸才能,篇幅原因,咱们这儿不详细去完结。有这方面需求的读者,能够经过在界面上调整上面相关特点来完结。
接下来咱们开端创立虚拟形象,首要创立一个User实体类:
public class User {
public String userName; //用户名
public String userId; //用户ID
public boolean isMan; //性别
public int width; //预览宽度
public int height; //预览高度
public int bgColor; //布景色彩
public int shirtIdx = 0; // T-shirt资源id
public int browIdx = 0; //眉毛资源id
public User(String userName, String userId, int width, int height) {
this.userName = userName;
this.userId = userId;
this.width = width;
this.height = height;
this.isMan = true;
bgColor = Color.argb(255, 33, 66, 99);
}
}
示例作用,为了简单起见,咱们这儿只针对眉毛和衣服资源做选取。接下来创立一个Activity:
public class AvatarActivity extends BaseActivity {
private int vWidth = 720;
private int vHeight = 1080;
private User user = new User("C_0001", "C_0001", vWidth, vHeight);
private TextureView mTextureView; //用于显示Avatar形象
private AvatarMngr mAvatarMngr; // 用于保护管理Avatar
private ColorPickerDialog colorPickerDialog; //用于布景色选取
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_avatar);
// ....
// 其他初始化界面相关代码略...
// ....
user.isMan = true;
mTextureView = findViewById(R.id.avatar_view);
mZegoMngr = ZegoMngr.getInstance(getApplication());
// 敞开虚拟形象预览
mAvatarMngr.start(mTextureView, user);
}
最终一行代码中敞开了虚拟形象预览,那么这儿敞开虚拟形象预览详细做了哪些作业呢?show me the code:
public class AvatarMngr implements ZegoAvatarServiceDelegate, RTCMngr.CaptureListener {
private static final String TAG = "AvatarMngr";
private static AvatarMngr mInstance;
private boolean mIsStop = false;
private User mUser = null;
private TextureBgRender mBgRender = null;
private OnAvatarServiceInitSucced listener;
private ZegoCharacterHelper mCharacterHelper;
private Application mApp;
public interface OnAvatarServiceInitSucced {
void onInitSucced();
}
public void setLicense(String license, OnAvatarServiceInitSucced listener) {
this.listener = listener;
ZegoAvatarService.addServiceObserver(this);
String aiPath = FileUtils.getPhonePath(mApp, "AIModel.bundle", "assets"); // AI 模型的绝对路径
ZegoServiceConfig config = new ZegoServiceConfig(license, aiPath);
ZegoAvatarService.init(mApp, config);
}
public void stop() {
mIsStop = true;
mUser = null;
stopExpression();
}
public void updateUser(User user) {
mUser = user;
if (user.shirtIdx == 0) {
mCharacterHelper.setPackage("m-shirt01");
} else {
mCharacterHelper.setPackage("m-shirt02");
}
if (user.browIdx == 0) {
mCharacterHelper.setPackage("brows_1");
} else {
mCharacterHelper.setPackage("brows_2");
}
}
/**
* 发动Avatar,调用此函数之前,请确保现已调用过setLicense
*
* @param avatarView
*/
public void start(TextureView avatarView, User user) {
mUser = user;
mIsStop = false;
initAvatar(avatarView, user);
startExpression();
}
private void initAvatar(TextureView avatarView, User user) {
String sex = ZegoCharacterHelper.MODEL_ID_MALE;
if (!user.isMan) sex = ZegoCharacterHelper.MODEL_ID_FEMALE;
// 创立 helper 简化调用
// base.bundle 是头模, human.bundle 是全身人模
mCharacterHelper = new ZegoCharacterHelper(FileUtils.getPhonePath(mApp, "human.bundle", "assets"));
mCharacterHelper.setExtendPackagePath(FileUtils.getPhonePath(mApp, "Packages", "assets"));
// 设置形象装备
mCharacterHelper.setDefaultAvatar(sex);
updateUser(user);
// 获取当前妆容数据, 能够保存到用户资料中
String json = mCharacterHelper.getAvatarJson();
}
// 发动表情检测
private void startExpression() {
// 发动表情检测前要请求摄像头权限, 这儿是在 MainActivity 现已请求过了
ZegoAvatarService.getInteractEngine().startDetectExpression(ZegoExpressionDetectMode.Camera, expression -> {
// 表情直接塞给 avatar 驱动
mCharacterHelper.setExpression(expression);
});
}
// 停止表情检测
private void stopExpression() {
// 不必的时候记得停止
ZegoAvatarService.getInteractEngine().stopDetectExpression();
}
// 获取到 avatar 纹路后的处理
public void onCaptureAvatar(int textureId, int width, int height) {
if (mIsStop || mUser == null) { // rtc 的 onStop 是异步的, 可能activity现已运行到onStop了, rtc还没
return;
}
boolean useFBO = true;
if (mBgRender == null) {
mBgRender = new TextureBgRender(textureId, useFBO, width, height, Texture2dProgram.ProgramType.TEXTURE_2D_BG);
}
mBgRender.setInputTexture(textureId);
float r = Color.red(mUser.bgColor) / 255f;
float g = Color.green(mUser.bgColor) / 255f;
float b = Color.blue(mUser.bgColor) / 255f;
float a = Color.alpha(mUser.bgColor) / 255f;
mBgRender.setBgColor(r, g, b, a);
mBgRender.draw(useFBO); // 画到 fbo 上需求反向的
ZegoExpressEngine.getEngine().sendCustomVideoCaptureTextureData(mBgRender.getOutputTextureID(), width, height, System.currentTimeMillis());
}
@Override
public void onStartCapture() {
if (mUser == null) return;
// // 收到回调后,开发者需求执行发动视频收集相关的事务逻辑,例如敞开摄像头号
AvatarCaptureConfig config = new AvatarCaptureConfig(mUser.width, mUser.height);
// // 开端捕获纹路
mCharacterHelper.startCaptureAvatar(config, this::onCaptureAvatar);
}
@Override
public void onStopCapture() {
mCharacterHelper.stopCaptureAvatar();
stopExpression();
}
private void initRes(Application app) {
// 先把资源复制到SD卡,留意:线上运用时,需求做一下判别,防止屡次复制。资源也能够做成从网络下载。
if (!FileUtils.checkFile(app, "AIModel.bundle", "assets"))
FileUtils.copyAssetsDir2Phone(app, "AIModel.bundle", "assets");
if (!FileUtils.checkFile(app, "base.bundle", "assets"))
FileUtils.copyAssetsDir2Phone(app, "base.bundle", "assets");
if (!FileUtils.checkFile(app, "human.bundle", "assets"))
FileUtils.copyAssetsDir2Phone(app, "human.bundle", "assets");
if (!FileUtils.checkFile(app, "Packages", "assets"))
FileUtils.copyAssetsDir2Phone(app, "Packages", "assets");
}
@Override
public void onError(ZegoAvatarErrorCode code, String desc) {
Log.e(TAG, "errorcode : " + code.getErrorCode() + ",desc : " + desc);
}
@Override
public void onStateChange(ZegoAvatarServiceState state) {
if (state == ZegoAvatarServiceState.InitSucceed) {
Log.i("ZegoAvatar", "Init success");
// 要记得及时移除通知
ZegoAvatarService.removeServiceObserver(this);
if (listener != null) listener.onInitSucced();
}
}
private AvatarMngr(Application app) {
mApp = app;
initRes(app);
}
public static AvatarMngr getInstance(Application app) {
if (null == mInstance) {
synchronized (AvatarMngr.class) {
if (null == mInstance) {
mInstance = new AvatarMngr(app);
}
}
}
return mInstance;
}
}
以上代码完结了整个虚拟形象的创立,关键代码悉数展现,假如还需求详细悉数代码,直接从附件中下载即可。
5 附件
- 源码:github.com/RTCWang/Met…
- 即构元国际官网:doc-zh.zego.im/article/153…