The i.MX95 processor provides:

  • Two Camera Sensor Interface (CSI) modules (one of them shares the PHY with DSI)

  • Image Signal Processing (ISP) on the camera stream

On the ConnectCore 95 Development Kit:

  • A MIPI CSI-2 connector is available.

The BSP includes support for Digilent Pcam 5C MIPI camera (with Omnivision OV5640 sensor).

Make sure you use the correct cable to connect the MIPI camera to the ConnectCore 95 Development Kit. Using the wrong cable may damage the camera.

For ConnectCore 95 Development Kit, use a 15 pin, 1mm pitch FFC cable with contacts on bottom on both sides. Pin 1 of the ConnectCore 95 Development Kit connector aligns with pin 1 of the camera connector.

ConnectCore 95 Development Kit MIPI camera connection

Kernel configuration

You can manage the CSI driver support and Video4Linux (V4L2) capture driver through the following kernel configuration options:

  • NXP i.MX95 CSI Pixel Formatter Driver (CONFIG_VIDEO_IMX_CSI_FORMATTER)

  • NXP NEO ISP v4l2 hardware driver (CONFIG_VIDEO_NXP_NEOISP)

  • Freescale i.MX9 DPHY Rx (CONFIG_PHY_FSL_IMX9_DPHY_RX`)

  • DesignWare Cores MIPI CSI-2 receiver found on i.MX93 (CONFIG_VIDEO_DWC_MIPI_CSIS)

  • OmniVision OV5640 sensor support (CONFIG_VIDEO_OV5640)

These options are enabled as built-in on the default ConnectCore 95 kernel configuration file.

Kernel driver

The drivers for CSI capture are located at:

File Description

drivers/media/platform/nxp/imx-csi-formatter.c

NXP i.MX95 CSI Pixel Formatter Driver

drivers/media/platform/nxp/neoisp/

NXP NEO ISP v4l2 hardware driver

drivers/phy/freescale/phy-fsl-imx9-dphy-rx.c

Freescale i.MX9 DPHY Rx

drivers/media/platform/nxp/dwc-mipi-csi2.c

DesignWare Cores MIPI CSI-2 receiver found on i.MX93

drivers/media/platform/nxp/imx8-isi/

Image Sensing Interface driver

drivers/media/i2c/ov5640.c

OmniVision OV5640 sensor support

Device tree bindings and customization

Common bindings for video receiver and transmitter interfaces are described at Documentation/devicetree/bindings/media/video-interfaces.yaml.

The device tree must contain entries for:

  • The V4L2 capture interfaces and processing elements

  • The camera sensor

  • The IOMUX

The i.MX95 capture and processing elements are defined on the i.MX95 device tree include file.

The camera, video routing, and IOMUX are defined in a device tree overlay arch/arm64/boot/dts/digi/ccimx95-dvk_pcam5c.dtso.

V4L2 capture interfaces (CSI) and processing elements

i.MX95 device tree
	cameramix_csr: syscon@4ac10000 {
		compatible = "fsl,imx95-cameramix-csr", "syscon";
		[...]

		csi_pixel_formatter_0: formatter@20 {
			compatible = "fsl,imx95-csi-formatter";
			[...]
			status = "disabled";

			ports {
				port@0 {
					reg = <0>;
				};

				port@1 {
					reg = <1>;
				};
			};
		};

		csi_pixel_formatter_1: formatter@120 {
			compatible = "fsl,imx95-csi-formatter";
			[...]
			status = "disabled";

			ports {
				port@0 {
					reg = <0>;
				};

				port@1 {
					reg = <1>;
				};
			};
		};
	};

	neoisp0: isp@4ae00000 {
		compatible = "nxp,imx95-a0-neoisp";
		[...]
		status = "disabled";
	};

	mipi_csi0: csi@4ad30000 {
		compatible = "snps,dw-mipi-csi2", "syscon";
		reg = <0x0 0x4ad30000 0x0 0x10000>;
		[...]
		status = "disabled";

		ports {
			port@0 {
				reg = <0>;
			};

			port@1 {
				reg = <1>;
			};
		};
	};

	mipi_csi1: csi@4ad40000 {
		compatible = "snps,dw-mipi-csi2", "syscon";
		reg = <0x0 0x4ad40000 0x0 0x10000>;
		[...]
		status = "disabled";

		ports {
			port@0 {
				reg = <0>;
			};

			port@1 {
				reg = <1>;
			};
		};
	};

	combo_phy_csr: syscon@4ad20000 {
		compatible = "syscon", "simple-mfd";
		reg = <0x0 0x4ad20000 0x0 0x114>;
		clocks = <&scmi_clk IMX95_CLK_CAMAPB>;

		dphy_rx: dphy-rx {
			compatible = "fsl,imx95-dphy-rx";
			[...]
			status = "disabled";
		};

		combo_rx: combo-rx {
			compatible = "fsl,imx95-combo-rx";
			[...]
			status = "disabled";
		};
	};

	isi: isi@4ad50000 {
		compatible = "nxp,imx95-isi";
		reg = <0x0 0x4ad50000 0x0 0x80000>;
		[...]
		status = "disabled";

		ports {
			port@0 {
				reg = <0>;
			};

			port@1 {
				reg = <1>;
			};
		};
	}

Camera sensor (LPI2C3 slave)

Camera device tree overlay
&lpi2c3 {
	ov5640_mipi: ov5640_mipi@3c {
		compatible = "ovti,ov5640";
		reg = <0x3c>;
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_mipi>;
		clocks = <&mipi_csi_xtal24m>;
		clock-names = "xclk";
		powerdown-gpios = <&gpio5 17 GPIO_ACTIVE_LOW>;
		digi,use-pm-ops;
		status = "okay";

		port {
			ov5640_mipi_ep: endpoint {
				remote-endpoint = <&mipi_csi0_ep>;
				data-lanes = <1 2>;
				clock-lanes = <0>;
			};
		};
	};
};

IOMUX configuration

Camera device tree overlay
&scmi_iomuxc {
	/* MIPI Camera */
	pinctrl_mipi: mipi {
		fsl,pins = <
			/* Camera PWUP */
			IMX95_PAD_GPIO_IO37__GPIO5_IO_BIT17		0x31e
		>;
	};
};

Use the camera

Identify the camera capture devices

Camera configuration and operation is divided between several interconnected subdevices, called entities, sharing video data.

You can use the media-ctl tool to obtain the full camera subsystem topology, a list of subdevices, the connections between them, and some additional postprocessing operations that can be performed on the fly such as cropping or downscaling:

# media-ctl -p
Media controller API version 6.6.52

Media device information
------------------------
driver          mxc-isi
model           FSL Capture Media Device
serial
bus info        platform:4ad50000.isi
hw revision     0x0
driver version  6.6.52

Device topology
- entity 1: crossbar (13 pads, 10 links, 8 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        routes:
                2/0 -> 5/0 [ACTIVE]
                2/0 -> 6/0 [ACTIVE]
                2/0 -> 7/0 [ACTIVE]
                2/0 -> 8/0 [ACTIVE]
                2/0 -> 9/0 [ACTIVE]
                2/0 -> 10/0 [ACTIVE]
                2/0 -> 11/0 [ACTIVE]
                2/0 -> 12/0 [ACTIVE]
        pad0: Sink
        pad1: Sink
        pad2: Sink
                [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                <- "4ac10000.syscon:formatter@20":1 [ENABLED,IMMUTABLE]
        pad3: Sink
        pad4: Sink
                <- "mxc_isi.output":0 [ENABLED,IMMUTABLE]
        pad5: Source
                [stream:0 fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "mxc_isi.0":0 [ENABLED,IMMUTABLE]

	[...]

- entity 120: ov5640 2-003c (1 pad, 1 link, 0 routes)
              type V4L2 subdev subtype Sensor flags 0
              device node name /dev/v4l-subdev11
        pad0: Source
                [stream:0 fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
                 crop.bounds:(0,0)/2624x1964
                 crop:(16,14)/2592x1944]
                -> "csidev-4ad30000.csi":0 []

Alternatively, you can create a dot graph using the same tool to have a visual representation of the whole camera subsystem:

# media-ctl --print-dot > graph.dot

Use your host PC to convert the graph.dot file to a PNG image:

$ dot -Tpng -Nfontname=Roboto -Nfontsize=10 -Efontname=Roboto -Efontsize=10 graph.dot > graph.png
ConnectCore 95 camera topology

This graph, generated by the media-ctl tool, shows the different video pipes the data can flow through, from the camera sensor on the top (source) to the Linux video devices on the bottom (sink).

This step is required before you run any capture with the camera.

Configure the video path that will flow from the camera sensor to the Linux video devices with the desired resolution and format.

  1. Reset all media links

    # media-ctl -r
  2. Set a local variable FORMAT with format and resolution (in this example, raw video at 1280x720):

    # FORMAT=UYVY8_1X16/1280x720
  3. Establish links between the media entities and configure the format and frame size they will operate at:

    # media-ctl -d platform:4ad50000.isi -l '"ov5640 2-003c":0->"csidev-4ad30000.csi":0 [1]'
    # media-ctl -d platform:4ad50000.isi -l '"csidev-4ad30000.csi":1->"4ac10000.syscon:formatter@20":0 [1]'
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'ov5640 2-003c':0[fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'csidev-4ad30000.csi':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'4ac10000.syscon:formatter@20':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'crossbar':2 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.0':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.1':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.2':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.3':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.4':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.5':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.6':0 [fmt:${FORMAT} field:none]"
    # media-ctl -d platform:4ad50000.isi --set-v4l2 "'mxc_isi.7':0 [fmt:${FORMAT} field:none]"

Preview a camera image using gstreamer

To get a camera preview:

  1. Configure the media links (see Configure the media links).

  2. Capture the camera image and preview it using gstreamer and Wayland sink:

    # gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,format=YUY2,width=1280,height=720,framerate=30/1 ! videoconvert ! autovideosink

Take a picture with the camera using gstreamer

Capture a still image with the camera and save it using gstreamer:

# gst-launch-1.0 v4l2src device=/dev/video2 num-buffers=5 ! video/x-raw,width=1280,height=720 ! jpegenc ! multifilesink location="sample.jpg" max-files=1
The command takes five frames but only saves the last. Discarding the first four eliminates undesired frames on cameras that have auto-exposure.

To show the captured image using gstreamer:

# gst-launch-1.0 filesrc location=sample.jpg ! jpegdec ! imagefreeze ! waylandsink

Record a video with the camera using gstreamer

Capture a moving image with the camera and save it using gstreamer:

# gst-launch-1.0 v4l2src device=/dev/video2 num-buffers=300 ! video/x-raw,format=YUY2,width=1280,height=720,framerate=30/1 ! queue ! v4l2h264enc ! h264parse ! avimux ! filesink location=output.avi

To play the recorded video:

# gplay-1.0 output.avi