Android使用CameraX采集视频数据

ShaoweiChang Aug 03, 2020

上篇文章提到 Android 视频数据的采集大致有以下三种并介绍了最简单的第一种方法，

使用系统预装、或者已经安装的相机应用（例如美图等）采集视频数据；
使用 Jetpack CameraX 库采集视频数据；
使用 android 系统提供的 Camera API 开发自己的相机应用。

下面一起学习第二种方法：使用 Jetpack CameraX 库采集视频数据。

Jetpack 是 Google 18年推出的一套库、工具和指南，帮助开发者快速高效开发优质应用，包括满足后台调度需求的 WorkManager, 实现数据存储持久性的 Room, 管理应用导航流程的 Nabigation 等。

CameraX 是谷歌为了帮助开发者快速开发相机类应用，在19年 I/O 大会上推出的包含在 Jetpack 里的一个库。Android camera platform team 的产品经理在会上对 CameraX 做了一个简单的介绍，CameraX 之前，要开发自己的相机应用是一项繁琐的工作，开发者需要面对繁琐的 CameraAPI，Android 系统的碎片化和设备的多样化，要想开发出来的应用兼容不同的设备，需要在各种设备上做大量测试。谷歌工程师说他们在中国的 Google Developer Day(GDD)上听说有些开发团队需要在上百台设别上测试应用，于是谷歌打造了一个 CameraX 自动化测试实验室，可以7x24小时，在不同的设备上测试，根据测试结果，谷歌工程师发现和修复了以下问题，包括：

1. 前置和后置摄像头转换的崩溃；

2. 闪光灯不工作；

3. 优化摄像头预览跟随屏幕旋转，摄像头的关闭等。

截止到目前（2020.8）CameraX 包含了以下几个库：

androidx.camera.camera2
androidx.camera.core
androidx.camera.extensions
androidx.camera.lifecycle
androidx.camera.view

需要注意的是核心库 androidx.camera.core 目前处于测试阶段。Beta 版功能稳定，并且具有功能完整的 API Surface。它们可以投入实际使用，但可能包含错误。

那么使用CameraX 有哪些优点呢？听一听谷歌工程师的介绍：

1. 使用简单，CameraX 基于 Camera2 API 做的开发和封装，跟camera2 API 相比，使用CameraX 可以减少70%的代码量；

例如 camera 360 这款相机应用，使用 CameraX 后，减少了很多特定设备的测试，减少了75%的代码，缩小 APK 包大小。

2. 向后兼容到 AndroidL；

3. 提供一致的开发API，如果要使用底层 Camera API, 不用考虑选择使用 Camera 还是 Camera2 API；

4. 感知生命周期，不需要开发者自己处理；

5. 不用自己创建线程操作相机。

如果你之前使用 CameraAPI 开发相机应用，能体会到上面的几点绝对是开发者的福音，AndroidDeveloper 提供了比较详细的文档介绍了 CameraX 库，并给出了两个示例工程，一个入门应用，实现基本的拍照功能，另一个CameraXBasic 实现了拍照，存储及存储照片的预览，有这些资料，足够我们对 CameraX 有一个全面的认识。

下面参考这两个示例工程，结合代码学习 CameraX 的使用。

一个相机应用的基本功能大致包括：图像预览，图像拍摄和图像分析，CameraX 把实现这些功能的类封装成了用例 (UseCase) 方便开发者理解和使用，这些用例包括 Preview, ImageAnalysis, ImageCapture, VideoCapture。实现拍照功能，需要用到的用例和类有：

PreviewView

用于预览图像的自定义 View, 继承自FrameLayout，管理 Surface 生命周期以及预览图像的长宽比和方向，内部使用 TextureView 或者 SurfaceView 渲染图像；

Preview

提供在屏幕上显示相机预览流的用例。预览流已经连接到了显示的 Surface, Preview 决定了 Surface 的显示方式，并负责管理 Surface 的生命周期。

ImageCapture

用于拍照的用例，提供 takePicture() 函数拍摄图片，保存图片到内存或者文件并提供图片元数据，相机聚焦以后会以自动模式拍摄照片。ImageCapture 可以通过 Preview 控制对焦和曝光测光区域，当拍摄的照片保存在内存时，通过 ImageProxy 获取和使用图像。

ImageAnalysis

为应用提供 CPU 可访问的图像，并对其进行分析的用例。ImageAnalysis 通过 ImageReader 从相机获取图像，每个图像都通过 ImageProxy 提供给 ImageAnalysis.Analyzer 函数进行数据分析。

ProcessCameraProvider

用于将相机的生命周期绑定到应用程序进程中的任何 LifecycleOwner的单例对象。

下面我们尝试把这几个主要的类串联起来，完成拍照的功能。

首先创建一个Fragment 的子类 CameraXFragment, 在它的布局文件中放置一个 PreviewView 用来预览相机画面:

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:id="@+id/camera_container"
    android:background="@android:color/black"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <androidx.camera.view.PreviewView
        android:id="@+id/view_finder"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

接下来要做的是在 CameraXFragment 的生命周期函数里，执行以下一系列操作设置和绑定拍照有关的操作，为了方便理解，这里省略与主流程关系不大的细节。

获取屏幕的宽高比 screenAspectRatio 和屏幕方向 rotation

1	// Get screen metrics used to setup camera for full screen resolution
2	val metrics = DisplayMetrics().also { viewFinder.display.getRealMetrics(it) }
3	val screenAspectRatio = aspectRatio(metrics.widthPixels, metrics.heightPixels)
4	val rotation = viewFinder.display.rotation

设置 ProcessCameraProvider 和当前使用的摄像头:

val cameraProviderFuture = ProcessCameraProvider.getInstance(requireContext())
cameraProviderFuture.addListener(Runnable{
  cameraProvider = cameraProviderFuture.get()

  val lensFacing =  when{
    hasBackCamera() -> CameraSelector.LENS_FACING_BACK
    hasFrontCamera() -> CameraSelector.LENS_FACING_FRONT
    else -> throw IllegalStateException("Back and front camera are unavailable")
  }
}, ContextCompat.getMainExecutor(requireContext()

构建 Preview :

preview = Preview.Builder()
    // We request aspect ratio but no resolution
    .setTargetAspectRatio(screenAspectRatio)
    // Set initial target rotation
    .setTargetRotation(rotation)
    .build()

设置camera 和 preview

private lateinit var viewFinder: PreviewView
//...

// A variable number of use-cases can be passed here -
// camera provides access to CameraControl & CameraInfo
camera = cameraProvider.bindToLifecycle(
  this, cameraSelector, preview, imageCapture, imageAnalyzer
)

// Attach the viewfinder's surface provider to preview use case
preview?.setSurfaceProvider(viewFinder.createSurfaceProvider())

构建 ImageCapture：

imageCapture = ImageCapture.Builder()
    .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
    // We request aspect ratio but no resolution to match preview config, but letting
    // CameraX optimize for whatever specific resolution best fits our use cases
    .setTargetAspectRatio(screenAspectRatio)
    // Set initial target rotation, we will have to call this again if rotation changes
    // during the lifecycle of this use case
    .setTargetRotation(rotation)
    .build()

构建 ImageAnalysis，这里简单的分析图像亮度的变化：

imageAnalyzer = ImageAnalysis.Builder()
    // We request aspect ratio but no resolution
    .setTargetAspectRatio(screenAspectRatio)
    // Set initial target rotation, we will have to call this again if rotation changes
    // during the lifecycle of this use case
    .setTargetRotation(rotation)
    .build()
    // The analyzer can then be assigned to the instance
    .also {
        it.setAnalyzer(cameraExecutor, LuminosityAnalyzer { luma ->
            // Values returned from our analyzer are passed to the attached listener
            // We log image analysis results here - you should do something useful
            // instead!
            Log.d(TAG, "Average luminosity: $luma")
        })
    }

处理手机屏幕的旋转

需要获取 DisplayManager，并向其注册一个 DisplayListener，屏幕方向变化后及时更新 imageCapture 的 targetRotation：

private val displayManager by lazy {
        requireContext().getSystemService(Context.DISPLAY_SERVICE) as DisplayManager
}

/**
   * We need a display listener for orientation changes that do not trigger a configuration
   * change, for example if we choose to override config change in manifest or for 180-degree
   * orientation changes.
   */
private val displayListener = object : DisplayManager.DisplayListener {
  override fun onDisplayChanged(displayId: Int) =
  view?.let {
    if (displayId == this@CameraxFragment.displayId) {
      imageCapture?.targetRotation = it.display.rotation
    }
  } ?: Unit

  override fun onDisplayAdded(displayId: Int) = Unit
  override fun onDisplayRemoved(displayId: Int) = Unit
}

override fun onViewCreated(view: View, savedInstanceState: Bundle?){
  // Every time the orientation of the device changes, update the rotation for use case
  displayManager.registerDisplayListener(displayListener, null)
  //...
}

响应拍照按钮的点击事件，保存为本地文件：

controllerView.findViewById<ImageButton>(R.id.camera_capture_button).setOnClickListener {
						// Get a stable reference of the modifiable image capture use case
            imageCapture?.let { imageCapture ->
                // Create output file to hold the image
                val photoFile = createFile(outputDirectory, FILENAME, PHOTO_EXTENSION)

                // Setup image capture metadata
                val metadata = Metadata().apply {
                    // Mirror image when using the front camera
                    isReversedHorizontal = lensFacing == CameraSelector.LENS_FACING_FRONT
                }

                // Create output options object which contains file + metadata
                val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile)
                    .setMetadata(metadata)
                    .build()

                // Setup image capture listener which is triggered after photo has been taken
                imageCapture.takePicture(outputOptions, cameraExecutor, object : ImageCapture.OnImageSavedCallback {
                    override fun onImageSaved(outputFileResults: ImageCapture.OutputFileResults) {
                        val savedUri = outputFileResults.savedUri ?: Uri.fromFile(photoFile)
            
                        // We can only change the foreground Drawable using API level 23+ API
                        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
                            // Update the gallery thumbnail with latest picture taken
                            setGalleryThumbnail(savedUri)
                        }                     
                    }

                    override fun onError(exception: ImageCaptureException) {
                        Log.e(TAG, "Photo capture failed:${exception.message}", exception)
                    }
                })              
            }
      }

到这里就完成了使用 CameraX 拍摄照片的基本功能，要想把项目做的丰富些，还应该添加查看已拍摄照片，删除已拍摄的照片，分享照片到其它应用的功能，这些已经在完整的示例工程中实现。要获取完成的代码细节，请移步官方示例工程 CameraXBasic 或者 StreamingTour。

1	<?xml version="1.0" encoding="utf-8"?>
2	<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
3	xmlns:app="http://schemas.android.com/apk/res-auto"
4	android:id="@+id/camera_container"
5	android:background="@android:color/black"
6	android:layout_width="match_parent"
7	android:layout_height="match_parent">
8
9	<androidx.camera.view.PreviewView
10	android:id="@+id/view_finder"
11	android:layout_width="match_parent"
12	android:layout_height="match_parent" />
13
14	</androidx.constraintlayout.widget.ConstraintLayout>

1	val cameraProviderFuture = ProcessCameraProvider.getInstance(requireContext())
2	cameraProviderFuture.addListener(Runnable{
3	cameraProvider = cameraProviderFuture.get()
4
5	val lensFacing = when{
6	hasBackCamera() -> CameraSelector.LENS_FACING_BACK
7	hasFrontCamera() -> CameraSelector.LENS_FACING_FRONT
8	else -> throw IllegalStateException("Back and front camera are unavailable")
9	}
10	}, ContextCompat.getMainExecutor(requireContext()

1	preview = Preview.Builder()
2	// We request aspect ratio but no resolution
3	.setTargetAspectRatio(screenAspectRatio)
4	// Set initial target rotation
5	.setTargetRotation(rotation)
6	.build()

1	private lateinit var viewFinder: PreviewView
2	//...
3
4	// A variable number of use-cases can be passed here -
5	// camera provides access to CameraControl & CameraInfo
6	camera = cameraProvider.bindToLifecycle(
7	this, cameraSelector, preview, imageCapture, imageAnalyzer
8	)
9
10	// Attach the viewfinder's surface provider to preview use case
11	preview?.setSurfaceProvider(viewFinder.createSurfaceProvider())

1	imageCapture = ImageCapture.Builder()
2	.setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
3	// We request aspect ratio but no resolution to match preview config, but letting
4	// CameraX optimize for whatever specific resolution best fits our use cases
5	.setTargetAspectRatio(screenAspectRatio)
6	// Set initial target rotation, we will have to call this again if rotation changes
7	// during the lifecycle of this use case
8	.setTargetRotation(rotation)
9	.build()

1	imageAnalyzer = ImageAnalysis.Builder()
2	// We request aspect ratio but no resolution
3	.setTargetAspectRatio(screenAspectRatio)
4	// Set initial target rotation, we will have to call this again if rotation changes
5	// during the lifecycle of this use case
6	.setTargetRotation(rotation)
7	.build()
8	// The analyzer can then be assigned to the instance
9	.also {
10	it.setAnalyzer(cameraExecutor, LuminosityAnalyzer { luma ->
11	// Values returned from our analyzer are passed to the attached listener
12	// We log image analysis results here - you should do something useful
13	// instead!
14	Log.d(TAG, "Average luminosity: $luma")
15	})
16	}

1	private val displayManager by lazy {
2	requireContext().getSystemService(Context.DISPLAY_SERVICE) as DisplayManager
3	}
4
5	/**
6	* We need a display listener for orientation changes that do not trigger a configuration
7	* change, for example if we choose to override config change in manifest or for 180-degree
8	* orientation changes.
9	*/
10	private val displayListener = object : DisplayManager.DisplayListener {
11	override fun onDisplayChanged(displayId: Int) =
12	view?.let {
13	if (displayId == this@CameraxFragment.displayId) {
14	imageCapture?.targetRotation = it.display.rotation
15	}
16	} ?: Unit
17
18	override fun onDisplayAdded(displayId: Int) = Unit
19	override fun onDisplayRemoved(displayId: Int) = Unit
20	}
21
22	override fun onViewCreated(view: View, savedInstanceState: Bundle?){
23	// Every time the orientation of the device changes, update the rotation for use case
24	displayManager.registerDisplayListener(displayListener, null)
25	//...
26	}

1	controllerView.findViewById<ImageButton>(R.id.camera_capture_button).setOnClickListener {
2	// Get a stable reference of the modifiable image capture use case
3	imageCapture?.let { imageCapture ->
4	// Create output file to hold the image
5	val photoFile = createFile(outputDirectory, FILENAME, PHOTO_EXTENSION)
6
7	// Setup image capture metadata
8	val metadata = Metadata().apply {
9	// Mirror image when using the front camera
10	isReversedHorizontal = lensFacing == CameraSelector.LENS_FACING_FRONT
11	}
12
13	// Create output options object which contains file + metadata
14	val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile)
15	.setMetadata(metadata)
16	.build()
17
18	// Setup image capture listener which is triggered after photo has been taken
19	imageCapture.takePicture(outputOptions, cameraExecutor, object : ImageCapture.OnImageSavedCallback {
20	override fun onImageSaved(outputFileResults: ImageCapture.OutputFileResults) {
21	val savedUri = outputFileResults.savedUri ?: Uri.fromFile(photoFile)
22
23	// We can only change the foreground Drawable using API level 23+ API
24	if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
25	// Update the gallery thumbnail with latest picture taken
26	setGalleryThumbnail(savedUri)
27	}
28	}
29
30	override fun onError(exception: ImageCaptureException) {
31	Log.e(TAG, "Photo capture failed:${exception.message}", exception)
32	}
33	})
34	}
35	}