当前位置 博文首页 > wanggao的专栏：使用ffmpeg以v4l2输入打开相机进行h264编码
网上有很多关于ffmpeg打开输入流的博客，雷神的博客仅在windows下介绍了如何通过dshow打开摄像头，有关linux使用video4linux2（v4l2）方式的较少，也不全面。
本文以树莓派linux为例，详细介绍ffmpeg在树莓派使用v4l2方式打开的完整设置，并且详细配置硬编码h264_omx的选项设置。
本节前置知识：
使用的api介绍如下。根据指定参数打开输入流，并返回输入封装的上下文AVFormatContext。
int avformat_open_input(AVFormatContext **ps,
const char *url, // 打开的地址，文件或设备等
AVInputFormat *fmt, // 指定输入格式，若为空则自动检测
AVDictionary **options); // 指定demuxer的private options
通常，打开一个文件或者直播流，可以使用r如下代码
AVFormatContext *input_fmt_ctx = NULL;
const char* file_path = "test.mp4";
//const char* file_path = "rtmp://192.168.3.100:1935/live/test"; // 亦可
avformat_open_input(&input_fmt_ctx, file_path, NULL, NULL);
那么在linux下打开摄像头，应该如何操作呢？我们逐步介绍。
直观的想法，直接更换 file_path 为 /dev/video0，得到输出结果
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 331776 kb/s, 30 fps, 30 tbr, 1000k tbn
可以打开成功，默认打开流的分辨率是1280x720，帧率30fps，比特率331Mbps。
使用ffmpeg -hide_banner -h demuxer=v4l2查看v4l2 demuxer的选项如下。
pi@raspberrypi:~ $ ffmpeg -hide_banner -h demuxer=v4l2
Demuxer video4linux2,v4l2 [Video4Linux2 device grab]:
V4L2 indev AVOptions:
-standard <string> .D....... set TV standard, used only by analog frame grabber
-channel <int> .D....... set TV channel, used only by frame grabber (from -1 to INT_MAX) (default -1)
-video_size <image_size> .D....... set frame size
-pixel_format <string> .D....... set preferred pixel format
-input_format <string> .D....... set preferred pixel format (for raw video) or codec name
-framerate <string> .D....... set frame rate
-list_formats <int> .D....... list available formats and exit (from 0 to INT_MAX) (default 0)
all .D....... show all available formats
raw .D....... show only non-compressed formats
compressed .D....... show only compressed formats
-list_standards <int> .D....... list supported standards and exit (from 0 to 1) (default 0)
all .D....... show all supported standards
-timestamps <int> .D....... set type of timestamps for grabbed frames (from 0 to 2) (default default)
default .D....... use timestamps from the kernel
abs .D....... use absolute timestamps (wall clock)
mono2abs .D....... force conversion from monotonic to absolute timestamps
-ts <int> .D....... set type of timestamps for grabbed frames (from 0 to 2) (default default)
default .D....... use timestamps from the kernel
abs .D....... use absolute timestamps (wall clock)
mono2abs .D....... force conversion from monotonic to absolute timestamps
-use_libv4l2 <boolean> .D....... use libv4l2 (v4l-utils) conversion functions (default false)
使用命令ffmpeg -hide_banner -f v4l2 -list_formats all -i /dev/video0查看当前相机设备支持输出编码格式
pi@raspberrypi:~ $ ffmpeg -hide_banner -f v4l2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x11ef1c0] Raw : yuv420p : Planar YUV 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : yuyv422 : YUYV 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : rgb24 : 24-bit RGB 8-8-8 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Compressed: mjpeg : JFIF JPEG : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Compressed: h264 : H.264 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Compressed: mjpeg : Motion-JPEG : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : Unsupported : YVYU 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : Unsupported : VYUY 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : uyvy422 : UYVY 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : nv12 : Y/CbCr 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : bgr24 : 24-bit BGR 8-8-8 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : yuv420p : Planar YVU 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : Unsupported : Y/CrCb 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x11ef1c0] Raw : bgr0 : 32-bit BGRA/X 8-8-8-8 : {32-3280, 2}x{32-2464, 2}
当前相机支持输出，非压缩raw编码格式yuyv422、yuv420p、rgb24uyvy422、nv12\bgr24、gbr0等，以及压缩的编码格式mjpeg、h264等。分辨率也是可调节的。
（1）设置分辨率、帧率
如果后续应用使用的分辨率和帧率不是默认值，还需进行额外的处理。因此这里我们直接设置参数，简化后续流程。使用选项video_size 和 framerate 例如，设置分辨率 1024*768，帧率15。代码如下：
int ret;
const char* input_file = "/dev/video0";
AVDictionary *options = NULL;
//av_dict_set(&options, "f", "v4l2", 0); // 加快探测流的速度，
//av_dict_set(&options, "input_format", "yuv420p", 0); // 指定格式
av_dict_set(&options, "video_size", "1024*768", 0);
av_dict_set(&options, "framerate", "15", 0);
// 指定打开输入的demux参数
if ((ret = avformat_open_input(&s_raspiEncCtx.input_fmt_ctx, input_file, NULL, &options)) < 0) {
av_log(NULL, AV_LOG_ERROR, "Cannot open input file\n");
return ret;
}
av_dict_free(&options);
运行得到了期望结果，码率降低为131Mbps。
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1024x768, 141557 kb/s, 15 fps, 15 tbr, 1000k tbn
另外，使用 选项av_dict_set(&options, "f", "v4l2", 0);和 指定参数ifmt效果相同。通常linux可以不用设置，默认是支持v4l2且自动识别，而在windows下vfwp、dshow需要明确指定。
AVInputFormat *ifmt = av_find_input_format("v4l2"); // 加快探测流的速度，
avformat_open_input(&s_raspiEncCtx.input_fmt_ctx, input_file, ifmt, NULL);
等效与
AVDictionary *options = NULL;
av_dict_set(&options, "f", "v4l2", 0);
avformat_open_input(&s_raspiEncCtx.input_fmt_ctx, input_file, NULL, &options);
（2）设置编码格式
后面应用需要使用非压缩的raw编码格式为rgb格式，如果不对输入编码格式指定，还需要进行图像的变换。还比如后续仅需要保存h264格式，应该直接要求输入编码为压缩的H264流，而不应该进行编码操作。以上两种选项，只能通过AVOption设置，并且参数不一样。
-pixel_format <string> .D....... set preferred pixel format
-input_format <string> .D....... set preferred pixel format (for raw video) or codec name
av_dict_set(&options, "pixel_format", "rgb24", 0);
av_dict_set(&options, "input_format", "rgb24", 0); // 同上
av_dict_set(&options, "input_format", "h264", 0);
av_dict_set(&options, "input_format", "mjpeg", 0);
这里就给出实际的运行结果了。
不同编解码器支持的选项不同。例如，先给出libx264支持的选项。使用命令行ffmpeg -hide_banner -h encoder=libx264如下，也可以在 libavcodec\libx264.c文件中查看。
pi@raspberrypi:~ $ ffmpeg -hide_banner -h encoder=libx264
Encoder libx264 [libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10]:
General capabilities: delay threads
Threading capabilities: auto
Supported pixel formats: yuv420p yuvj420p yuv422p yuvj422p yuv444p yuvj444p nv12 nv16 nv21 yuv420p10le yuv422p10le yuv444p10le nv20le
libx264 AVOptions:
-preset <string> E..V..... Set the encoding preset (cf. x264 --fullhelp) (default "medium")
-tune <string> E..V..... Tune the encoding params (cf. x264 --fullhelp)
-profile <string> E..V..... Set profile restrictions (cf. x264 --fullhelp)
-fastfirstpass <boolean> E..V..... Use fast settings when encoding first pass (default true)
-level <string> E..V..... Specify level (as defined by Annex A)
-passlogfile <string> E..V..... Filename for 2 pass stats
-wpredp <string> E..V..... Weighted prediction for P-frames
-a53cc <boolean> E..V..... Use A53 Closed Captions (if available) (default true)
-x264opts <string> E..V..... x264 options
-crf <float> E..V..... Select the quality for constant quality mode (from -1 to FLT_MAX) (default -1)
-crf_max <float> E..V..... In CRF mode, prevents VBV from lowering quality beyond this point. (from -1 to FLT_MAX) (default -1)
-qp <int> E..V..... Constant quantization parameter rate control method (from -1 to INT_MAX) (default -1)
-aq-mode <int> E..V..... AQ method (from -1 to INT_MAX) (default -1)
none E..V.....
variance E..V..... Variance AQ (complexity mask)
autovariance E..V..... Auto-variance AQ
autovariance-biased E..V..... Auto-variance AQ with bias to dark scenes
-aq-strength <float> E..V..... AQ strength. Reduces blocking and blurring in flat and textured areas. (from -1 to FLT_MAX) (default -1)
-psy <boolean> E..V..... Use psychovisual optimizations. (default auto)
-psy-rd <string> E..V..... Strength of psychovisual optimization, in <psy-rd>:<psy-trellis> format.
-rc-lookahead <int> E..V..... Number of frames to look ahead for frametype and ratecontrol (from -1 to INT_MAX) (default -1)
-weightb <boolean> E..V..... Weighted prediction for B-frames. (default auto)
-weightp <int> E..V..... Weighted prediction analysis method. (from -1 to INT_MAX) (default -1)
none E..V.....
simple E..V.....
smart E..V.....
-ssim <boolean> E..V..... Calculate and print SSIM stats. (default auto)
-intra-refresh <boolean> E..V..... Use Periodic Intra Refresh instead of IDR frames. (default auto)
-bluray-compat <boolean> E..V..... Bluray compatibility workarounds. (default auto)
-b-bias <int> E..V..... Influences how often B-frames are used (from INT_MIN to INT_MAX) (default INT_MIN)
-b-pyramid <int> E..V..... Keep some B-frames as references. (from -1 to INT_MAX) (default -1)
none E..V.....
strict E..V..... Strictly hierarchical pyramid
normal E..V..... Non-strict (not Blu-ray compatible)
-mixed-refs <boolean> E..V..... One reference per partition, as opposed to one reference per macroblock (default auto)
-8x8dct <boolean> E..V..... High profile 8x8 transform. (default auto)
-fast-pskip <boolean> E..V..... (default auto)
-aud <boolean> E..V..... Use access unit delimiters. (default auto)
-mbtree <boolean> E..V..... Use macroblock tree ratecontrol. (default auto)
-deblock <string> E..V..... Loop filter parameters, in <alpha:beta> form.
-cplxblur <float> E..V..... Reduce fluctuations in QP (before curve compression) (from -1 to FLT_MAX) (default -1)
-partitions <string> E..V..... A comma-separated list of partitions to consider. Possible values: p8x8, p4x4, b8x8, i8x8, i4x4, none, all
-direct-pred <int> E..V..... Direct MV prediction mode (from -1 to INT_MAX) (default -1)
none E..V.....
spatial E..V.....
temporal E..V.....
auto E..V.....
-slice-max-size <int> E..V..... Limit the size of each slice in bytes (from -1 to INT_MAX) (default -1)
-stats <string> E..V..... Filename for 2 pass stats
-nal-hrd <int> E..V..... Signal HRD information (requires vbv-bufsize; cbr not allowed in .mp4) (from -1 to INT_MAX) (default -1)
none E..V.....
vbr E..V.....
cbr E..V.....
-avcintra-class <int> E..V..... AVC-Intra class 50/100/200 (from -1 to 200) (default -1)
-me_method <int> E..V..... Set motion estimation method (from -1 to 4) (default -1)
dia E..V.....
hex E..V.....
umh E..V.....
esa E..V.....
tesa E..V.....
-motion-est <int> E..V..... Set motion estimation method (from -1 to 4) (default -1)
dia E..V.....
hex E..V.....
umh E..V.....
esa E..V.....
tesa E..V.....
-forced-idr <boolean> E..V..... If forcing keyframes, force them as IDR frames. (default false)
-coder <int> E..V..... Coder type (from -1 to 1) (default default)
default E..V.....
cavlc E..V.....
cabac E..V.....
vlc E..V.....
ac E..V.....
-b_strategy <int> E..V..... Strategy to choose between I/P/B-frames (from -1 to 2) (default -1)
-chromaoffset <int> E..V..... QP difference between chroma and luma (from INT_MIN to INT_MAX) (default -1)
-sc_threshold <int> E..V..... Scene change threshold (from INT_MIN to INT_MAX) (default -1)
-noise_reduction <int> E..V..... Noise reduction (from INT_MIN to INT_MAX) (default -1)
-x264-params <string> E..V..... Override the x264 configuration using a :-separated list of key=value parameters
而树莓派支持的硬编码器h264_omx支持的选项就很少。
pi@raspberrypi:~ $ ffmpeg -hide_banner -h encoder=h264_omx
Encoder h264_omx [OpenMAX IL H.264 video encoder]:
General capabilities: delay
Threading capabilities: none
Supported pixel formats: yuv420p
h264_omx AVOptions:
-omx_libname <string> ED.V..... OpenMAX library name
-omx_libprefix <string> ED.V..... OpenMAX library prefix
-zerocopy <int> E..V..... Try to avoid copying input frames if possible (from 0 to 1) (default 0)
-profile <int> E..V..... Set the encoding profile (from -99 to 100) (default -99)
baseline E..V.....
main E..V.....
high E..V.....
树莓派硬编码支持选项虽然少，但AVCodecContext可以设置其他参数。以h264_omx编码器的profile选项设置说明。
AVCodec *enc = avcodec_find_encoder_by_name("h264_omx");
enc_ctx = avcodec_alloc_context3(enc);
enc_ctx->pix_fmt = AV_PIX_FMT_YUV420P;
enc_ctx->width = w;
enc_ctx->height = h;
// enc_ctx->framerate = {fps,1};
// enc_ctx->time_base = {1,fps};
enc_ctx->framerate.num = fps;
enc_ctx->framerate.den = 1;
enc_ctx->time_base.num = 1;
enc_ctx->time_base.den = fps;
enc_ctx->gop_size = fps;
enc_ctx->bit_rate = bitrate;
avcodec_open2(enc_ctx, enc, NULL);
（1）直接对成员变量访问设置
enc_ctx->profile = FF_PROFILE_H264_HIGH; // 设置profile
avcodec_open2(enc_ctx, enc, NULL);
（2）使用av_opt_set系列函数方法
av_opt_set(enc_ctx->priv_data,"profile","high", 0);
avcodec_open2(enc_ctx, enc, NULL);
（3）使用AVDictiony系列函数方法
AVDictionary *dict = NULL;
av_dict_set(&dict, "profile", "high", 0);
//av_dict_set_int(&dict, "profile", FF_PROFILE_H264_HIGH, 0); // 同上
avcodec_open2(enc_ctx, enc, &dict);
当前硬编码的视频流编码I帧前进第一帧是有SPS\PPS，做额外的处理。参见博客【ffmpeg学习 编码为h264裸流添加sps、pps】。
cs