前言:
之前有很多朋友问在rk平台上怎么适配mipi摄像头,或者说在设备树里面去修改,今天分享一篇不错的文章给大家!
MIPI-CSI:
RK3588s MIPI-CSI资源介绍:

注意:D/C-PHY 仅用作DPHY-V2.0。
VICAP:
RK3588/RK3588S VICAP支持输入输出规格:

ISP:
RK3588/RK3588s 的ISP属于RK ISP v3.0版本,拥有2个ISP:

注:CIS全称CMOS Image Sensor
RK3588 Camera相关 DTS配置介绍:
这里以RK3588-EVB开发板的camera配置来进行说明;RK3588 的camera通路,下图是RK3588 camera连接链路示意图,可以支持7路camera。

框图说明:
-
rk3588支持两个dcphy,节点名称分别为csi2_dcphy0/csi2_dcphy1。每个dcphy硬件支持RX/TX同时使用,对于camera输入使用的是RX。支持DPHY/CPHY协议复用;需要注意的是同一个dcphy的TX/RX只能同时使用DPHY或同时使用CPHY。其他dcphy参数请查阅rk3588数据手册。
-
rk3588支持2个dphy硬件,这里我们称之为dphy0_hw/dphy1_hw,两个dphy硬件都可以工作在full mode 和split mode两种模式下。
dphy0_hw:
-
full mode:节点名称使用csi2_dphy0,最多支持4 lane。
-
split mode: 拆分成2个phy使用,分别为csi2_dphy1(使用0/1 lane)、csi2_dphy2(使用2/3 lane),每个phy最多支持2 lane。
-
当dphy0_hw使用full mode时,链路需要按照csi2_dphy1这条链路来配置,但是节点名称csi2_dphy1需要修改为csi2_dphy0,软件上是通过phy的序号来区分phy使用的模式。
dphy1_hw:
-
full mode:节点名称使用csi2_dphy3,最多支持4 lane。
-
split mode: 拆分成2个phy使用,分别为csi2_dphy4(使用0/1 lane)、csi2_dphy5(使用2/3 lane),每个phy最多支持2 lane。
-
当dphy1_hw使用full mode时,链路需要按照csi2_dphy4这条链路来配置,但是节点名称csi2_dphy4需要修改为csi2_dphy3,软件上是通过phy的序号来区分phy使用的模式。
使用上述mipi phy节点,需要把对应的物理节点配置上。
(csi2_dcphy0_hw/csi2_dcphy1_hw/csi2_dphy0_hw/csi2_dphy1_hw) 每个mipi phy都需要一个csi2模块来解析mipi协议,节点名称分别为mipi0_csi2~mipi5_csi2。
rk3588所有camera数据都需要通过vicap,再链接到isp。rk3588仅支持一个vicap硬件,这个vicap支持同时输入6路mipi phy,及一路dvp数据,所以我们将vicap分化成rkcif_mipi_lvds~rkcif_mipi_lvds5、rkcif_dvp等7个节点,各个节点的绑定关系需要严格按照框图的节点序号配置。
每个vicap节点与isp的链接关系,通过对应虚拟出的XXX_sditf来指明链接关系。
rk3588支持2个isp硬件,每个isp设备可虚拟出多个虚拟节点,软件上通过回读的方式,依次从ddr读取每一路的图像数据进isp处理。对于多摄方案,建议将数据流平均分配到两个isp上。
直通与回读模式:
-
直通:指数据经过vicap采集,直接发送给isp处理,不存储到ddr。需要注意的是hdr直通时,只有短帧是真正的直通,长帧需要存在ddr,isp再从ddr读取。
-
回读:指数据经过vicap采集到ddr,应用获取到数据后,将buffer地址推送给isp,isp再从ddr获取图像数据。
再dts配置时,一个isp硬件,如果只配置一个虚拟节点,默认使用直通模式,如果配置了多个虚拟节点默认使用回读模式。
单路Camera的dts配置说明:
RK3588 EVB开发板默认只有1路camera,其链接方式如上图的mipi camera0。DTS配置在kernel-5.10\arch\arm64\boot\dts\rockchip\rk3588-evb1-imx415.dtsi,可以结合上的图来理解这个dts的配置,如下:
// SPDX-License-Identifier: (GPL-2.0+ OR MIT) /* * Copyright (c) 2021 Rockchip Electronics Co., Ltd. * */ / { 红外滤光片(IRCUT)由两根线控制,对这两根线施加3.5v~6v的电源,通过对IRCUT供电电源的正负极对调,且满足通电时间100ms±10%,能够实现IRCUT的切换。驱动通过两个gpio控制电机驱动器的电流输出方向,gpio命令为open(红线)、close(黑线)。电流由open流向close,为红外截止滤光片,白天工作状态;电流由close流向open,为白玻璃片,夜晚工作状态。其配置如下: cam_ircut0: cam_ircut { status = "okay"; compatible = "rockchip,ircut"; ircut-open-gpios = <&gpio3 RK_PC1 GPIO_ACTIVE_HIGH>; ircut-close-gpios = <&gpio3 RK_PC0 GPIO_ACTIVE_HIGH>; rockchip,camera-module-index =; rockchip,camera-module-facing = "back"; }; }; //这里使用的是csi2_dcphy0 &csi2_dcphy0 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi_in_ucam0: endpoint@1 { reg =; //sensor端的port名 remote-endpoint = <&imx415_out0>; //mipi lane数,1lane为,4lane为,这里配置的是4lane,这个必须要配置否则会无法识别mipi类型 data-lanes =; }; }; port@1 { reg =; #address-cells =; #size-cells =; csidcphy0_out: endpoint@0 { reg =; //mipi0_csi2_input是对应的isp端的port名 remote-endpoint = <&mipi0_csi2_input>; }; }; }; }; &i2c5 { status = "okay"; imx415: imx415@1a { compatible = "sony,imx415";//需要与驱动定义的字符串一致 reg =;//sensor的I2C设备地址,7位 clocks = <&cru CLK_MIPI_CAMARAOUT_M1>;//sensor clockin配置 clock-names = "xvclk"; pinctrl-names = "default"; pinctrl-0 = <&mipim0_camera1_clk>; //电源配置 power-domains = <&power RK3588_PD_VI>; //power管脚配置 pwdn-gpios = <&gpio1 RK_PA0 GPIO_ACTIVE_HIGH>; avdd-supply = <&vcc_mipidcphy0>; //模组编号,该编号不要重复 rockchip,camera-module-index =; //模组朝向,有前摄“front”、后摄“back” rockchip,camera-module-facing = "back"; //模组名称 rockchip,camera-module-name = "CMK-OT2022-PX1"; //IR CUT设备 rockchip,camera-module-lens-name = "IR0147-50IRC-8M-F20"; lens-focus = <&cam_ircut0>; port { imx415_out0: endpoint { //mipi dphy端的port名 remote-endpoint = <&mipi_in_ucam0>; //mipi lane数,1lane为,4lane为,这里配置的是4lane,这个必须要配置否则会无法识别mipi类型 data-lanes =; }; }; }; }; &mipi_dcphy0 { status = "okay"; }; &mipi0_csi2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi0_csi2_input: endpoint@1 { reg =; //csi2 dphy端的port名 remote-endpoint = <&csidcphy0_out>; }; }; port@1 { reg =; #address-cells =; #size-cells =; mipi0_csi2_output: endpoint@0 { reg =; //vicap端的port名 remote-endpoint = <&cif_mipi_in0>; }; }; }; }; //vicap的配置,这里用rkcif节点代表vicap &rkcif { status = "okay"; }; &rkcif_mipi_lvds { status = "okay"; port { cif_mipi_in0: endpoint { //csi2 host端的port名 remote-endpoint = <&mipi0_csi2_output>; }; }; }; &rkcif_mipi_lvds_sditf { status = "okay"; port { mipi_lvds_sditf: endpoint { //isp虚拟设备端port名 remote-endpoint = <&isp0_vir0>; }; }; }; &rkcif_mmu { status = "okay"; }; &rkisp0 { status = "okay"; }; &isp0_mmu { status = "okay"; }; &rkisp0_vir0 { status = "okay"; port { #address-cells =; #size-cells =; isp0_vir0: endpoint@0 { reg =; //vicap mipi sditf的端点名 remote-endpoint = <&mipi_lvds_sditf>; }; }; };
6路camera的DTS配置说明:
基于RK3588-EVB开发板加6路camera转接板可以实现6路camera,其DTS配置在kernel-5.10\arch\arm64\boot\dts\rockchip\rk3588-evb1-cam-6x.dtsi。6路camera的链路可以参考上面的图,并根据图来理解dts配置,如下:
// SPDX-License-Identifier: (GPL-2.0+ OR MIT) /* * Copyright (c) 2021 Rockchip Electronics Co., Ltd. * */ //下面是phy的配置一共6个phy &csi2_dcphy0 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi_in_ucam0: endpoint@1 { reg =; remote-endpoint = <&imx464_out0>; data-lanes =; }; }; port@1 { reg =; #address-cells =; #size-cells =; csidcphy0_out: endpoint@0 { reg =; remote-endpoint = <&mipi0_csi2_input>; }; }; }; }; &csi2_dcphy1 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi_in_ucam1: endpoint@1 { reg =; remote-endpoint = <&imx464_out1>; data-lanes =; }; }; port@1 { reg =; #address-cells =; #size-cells =; csidcphy1_out: endpoint@0 { reg =; remote-endpoint = <&mipi1_csi2_input>; }; }; }; }; &csi2_dphy0_hw { status = "okay"; }; &csi2_dphy1_hw { status = "okay"; }; &csi2_dphy1 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi_in_ucam2: endpoint@1 { reg =; remote-endpoint = <&imx464_out2>; data-lanes =; }; }; port@1 { reg =; #address-cells =; #size-cells =; csidphy1_out: endpoint@0 { reg =; remote-endpoint = <&mipi2_csi2_input>; }; }; }; }; &csi2_dphy2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi_in_ucam3: endpoint@1 { reg =; remote-endpoint = <&imx464_out3>; data-lanes =; }; }; port@1 { reg =; #address-cells =; #size-cells =; csidphy2_out: endpoint@0 { reg =; remote-endpoint = <&mipi3_csi2_input>; }; }; }; }; &csi2_dphy4 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi_in_ucam4: endpoint@1 { reg =; remote-endpoint = <&imx464_out4>; data-lanes =; }; }; port@1 { reg =; #address-cells =; #size-cells =; csidphy4_out: endpoint@0 { reg =; remote-endpoint = <&mipi4_csi2_input>; }; }; }; }; &csi2_dphy5 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi_in_ucam5: endpoint@1 { reg =; remote-endpoint = <&imx464_out5>; data-lanes =; }; }; port@1 { reg =; #address-cells =; #size-cells =; csidphy5_out: endpoint@0 { reg =; remote-endpoint = <&mipi5_csi2_input>; }; }; }; }; //下面是sensor的配置一共配置imx464_0到imx464_5 六个sensor &i2c3 { status = "okay"; /* module 77/79 0x1a 78/80 0x36 */ imx464_2: imx464-2@1a { compatible = "sony,imx464"; status = "okay"; reg =; clocks = <&cru CLK_MIPI_CAMARAOUT_M3>; clock-names = "xvclk"; power-domains = <&power RK3588_PD_VI>; pinctrl-names = "default"; pinctrl-0 = <&mipim0_camera3_clk>; avdd-supply = <&vcc_mipicsi0>; pwdn-gpios = <&gpio1 RK_PB3 GPIO_ACTIVE_HIGH>; rockchip,camera-module-index =; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "CMK-OT1980-PX1"; rockchip,camera-module-lens-name = "SHG102"; port { imx464_out2: endpoint { remote-endpoint = <&mipi_in_ucam2>; data-lanes =; }; }; }; imx464_3: imx464-3@36 { compatible = "sony,imx464"; status = "okay"; reg =; clocks = <&cru CLK_MIPI_CAMARAOUT_M3>; clock-names = "xvclk"; power-domains = <&power RK3588_PD_VI>; avdd-supply = <&vcc_mipicsi0>; pwdn-gpios = <&gpio1 RK_PA7 GPIO_ACTIVE_HIGH>; rockchip,camera-module-index =; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "CMK-OT1980-PX1"; rockchip,camera-module-lens-name = "SHG102"; port { imx464_out3: endpoint { remote-endpoint = <&mipi_in_ucam3>; data-lanes =; }; }; }; }; &i2c4 { status = "okay"; pinctrl-0 = <&i2c4m3_xfer>; /* 77/79 0x1a 78/80 0x36 */ imx464_4: imx464-4@1a { compatible = "sony,imx464"; status = "okay"; reg =; clocks = <&cru CLK_MIPI_CAMARAOUT_M4>; clock-names = "xvclk"; power-domains = <&power RK3588_PD_VI>; pinctrl-names = "default"; pinctrl-0 = <&mipim0_camera4_clk>; avdd-supply = <&vcc_mipicsi1>; pwdn-gpios = <&gpio1 RK_PB4 GPIO_ACTIVE_HIGH>; rockchip,camera-module-index =; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "CMK-OT1980-PX1"; rockchip,camera-module-lens-name = "SHG102"; port { imx464_out4: endpoint { remote-endpoint = <&mipi_in_ucam4>; data-lanes =; }; }; }; imx464_5: imx464-5@36 { compatible = "sony,imx464"; status = "okay"; reg =; clocks = <&cru CLK_MIPI_CAMARAOUT_M4>; clock-names = "xvclk"; power-domains = <&power RK3588_PD_VI>; avdd-supply = <&vcc_mipicsi1>; pwdn-gpios = <&gpio1 RK_PB0 GPIO_ACTIVE_HIGH>; rockchip,camera-module-index =; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "CMK-OT1980-PX1"; rockchip,camera-module-lens-name = "SHG102"; port { imx464_out5: endpoint { remote-endpoint = <&mipi_in_ucam5>; data-lanes =; }; }; }; }; &i2c5 { status = "okay"; /* 77/79 0x1a 78/80 0x36 */ imx464_0: imx464-0@1a { compatible = "sony,imx464"; status = "okay"; reg =; clocks = <&cru CLK_MIPI_CAMARAOUT_M1>; clock-names = "xvclk"; power-domains = <&power RK3588_PD_VI>; pwdn-gpios = <&gpio1 RK_PA0 GPIO_ACTIVE_HIGH>; pinctrl-names = "default"; pinctrl-0 = <&mipim0_camera1_clk>; avdd-supply = <&vcc_mipidcphy0>; rockchip,camera-module-index =; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "CMK-OT1980-PX1"; rockchip,camera-module-lens-name = "SHG102"; port { imx464_out0: endpoint { remote-endpoint = <&mipi_in_ucam0>; data-lanes =; }; }; }; imx464_1: imx464-1@36 { compatible = "sony,imx464"; status = "okay"; reg =; clocks = <&cru CLK_MIPI_CAMARAOUT_M2>; clock-names = "xvclk"; power-domains = <&power RK3588_PD_VI>; pinctrl-names = "default"; pinctrl-0 = <&mipim0_camera2_clk>; pwdn-gpios = <&gpio1 RK_PA1 GPIO_ACTIVE_HIGH>; avdd-supply = <&vcc_mipidcphy0>; rockchip,camera-module-index =; rockchip,camera-module-facing = "back"; rockchip,camera-module-name = "CMK-OT1980-PX1"; rockchip,camera-module-lens-name = "SHG102"; port { imx464_out1: endpoint { remote-endpoint = <&mipi_in_ucam1>; data-lanes =; }; }; }; }; &mipi_dcphy0 { status = "okay"; }; &mipi_dcphy1 { status = "okay"; }; //下面是mipi 控制器的配置,共6个控制器 &mipi0_csi2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi0_csi2_input: endpoint@1 { reg =; remote-endpoint = <&csidcphy0_out>; }; }; port@1 { reg =; #address-cells =; #size-cells =; mipi0_csi2_output: endpoint@0 { reg =; remote-endpoint = <&cif_mipi_in0>; }; }; }; }; &mipi1_csi2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi1_csi2_input: endpoint@1 { reg =; remote-endpoint = <&csidcphy1_out>; }; }; port@1 { reg =; #address-cells =; #size-cells =; mipi1_csi2_output: endpoint@0 { reg =; remote-endpoint = <&cif_mipi_in1>; }; }; }; }; &mipi2_csi2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi2_csi2_input: endpoint@1 { reg =; remote-endpoint = <&csidphy1_out>; }; }; port@1 { reg =; #address-cells =; #size-cells =; mipi2_csi2_output: endpoint@0 { reg =; remote-endpoint = <&cif_mipi_in2>; }; }; }; }; &mipi3_csi2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi3_csi2_input: endpoint@1 { reg =; remote-endpoint = <&csidphy2_out>; }; }; port@1 { reg =; #address-cells =; #size-cells =; mipi3_csi2_output: endpoint@0 { reg =; remote-endpoint = <&cif_mipi_in3>; }; }; }; }; &mipi4_csi2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi4_csi2_input: endpoint@1 { reg =; remote-endpoint = <&csidphy4_out>; }; }; port@1 { reg =; #address-cells =; #size-cells =; mipi4_csi2_output: endpoint@0 { reg =; remote-endpoint = <&cif_mipi_in4>; }; }; }; }; &mipi5_csi2 { status = "okay"; ports { #address-cells =; #size-cells =; port@0 { reg =; #address-cells =; #size-cells =; mipi5_csi2_input: endpoint@1 { reg =; remote-endpoint = <&csidphy5_out>; }; }; port@1 { reg =; #address-cells =; #size-cells =; mipi5_csi2_output: endpoint@0 { reg =; remote-endpoint = <&cif_mipi_in5>; }; }; }; }; //下面是vicap配置,分为6个vicap节点及对应的6个sditf虚拟链接节点 &rkcif { status = "okay"; }; &rkcif_mipi_lvds { status = "okay"; port { cif_mipi_in0: endpoint { remote-endpoint = <&mipi0_csi2_output>; }; }; }; &rkcif_mipi_lvds_sditf { status = "okay"; port { mipi_lvds_sditf: endpoint { remote-endpoint = <&isp0_vir0>; }; }; }; &rkcif_mipi_lvds1 { status = "okay"; port { cif_mipi_in1: endpoint { remote-endpoint = <&mipi1_csi2_output>; }; }; }; &rkcif_mipi_lvds1_sditf { status = "okay"; port { mipi1_lvds_sditf: endpoint { remote-endpoint = <&isp1_vir0>; }; }; }; &rkcif_mipi_lvds2 { status = "okay"; port { cif_mipi_in2: endpoint { remote-endpoint = <&mipi2_csi2_output>; }; }; }; &rkcif_mipi_lvds2_sditf { status = "okay"; port { mipi2_lvds_sditf: endpoint { remote-endpoint = <&isp0_vir1>; }; }; }; &rkcif_mipi_lvds3 { status = "okay"; port { cif_mipi_in3: endpoint { remote-endpoint = <&mipi3_csi2_output>; }; }; }; &rkcif_mipi_lvds3_sditf { status = "okay"; port { mipi3_lvds_sditf: endpoint { remote-endpoint = <&isp1_vir1>; }; }; }; &rkcif_mipi_lvds4 { status = "okay"; port { cif_mipi_in4: endpoint { remote-endpoint = <&mipi4_csi2_output>; }; }; }; &rkcif_mipi_lvds4_sditf { status = "okay"; port { mipi4_lvds_sditf: endpoint { remote-endpoint = <&isp0_vir2>; }; }; }; &rkcif_mipi_lvds5 { status = "okay"; port { cif_mipi_in5: endpoint { remote-endpoint = <&mipi5_csi2_output>; }; }; }; &rkcif_mipi_lvds5_sditf { status = "okay"; port { mipi5_lvds_sditf: endpoint { remote-endpoint = <&isp1_vir2>; }; }; }; &rkcif_mmu { status = "okay"; }; //下面是isp的配置,将2个isp分为6个虚拟节点,每个isp虚拟出3个节点 &rkisp0 { status = "okay"; }; &isp0_mmu { status = "okay"; }; &rkisp0_vir0 { status = "okay"; port { #address-cells =; #size-cells =; isp0_vir0: endpoint@0 { reg =; remote-endpoint = <&mipi_lvds_sditf>; }; }; }; &rkisp0_vir1 { status = "okay"; port { #address-cells =; #size-cells =; isp0_vir1: endpoint@0 { reg =; remote-endpoint = <&mipi2_lvds_sditf>; }; }; }; &rkisp0_vir2 { status = "okay"; port { #address-cells =; #size-cells =; isp0_vir2: endpoint@0 { reg =; remote-endpoint = <&mipi4_lvds_sditf>; }; }; }; &rkisp1 { status = "okay"; }; &isp1_mmu { status = "okay"; }; &rkisp1_vir0 { status = "okay"; port { #address-cells =; #size-cells =; isp1_vir0: endpoint@0 { reg =; remote-endpoint = <&mipi1_lvds_sditf>; }; }; }; &rkisp1_vir1 { status = "okay"; port { #address-cells =; #size-cells =; isp1_vir1: endpoint@0 { reg =; remote-endpoint = <&mipi3_lvds_sditf>; }; }; }; &rkisp1_vir2 { status = "okay"; port { #address-cells =; #size-cells =; isp1_vir2: endpoint@0 { reg =; remote-endpoint = <&mipi5_lvds_sditf>; }; }; };