Kornia is a differentiable computer vision library built on PyTorch offering over 1,200 operators spanning image processing, geometric vision, and deep learning layers. It differentiates itself through per-sample batched parameters — such as (B, 2) sigma for gaussian_blur2d and (B, 3, 3) matrices for warp_perspective — eliminating host synchronization for CUDA and MPS (PyTorch ≥2.13) calls since v0.9.1. Core geometry includes batched minimal solvers (7-point fundamental, 5-point essential, DLT homography, PnP) returning all real solutions in padded tensors, and RANSAC with compile=True (PyTorch ≥2.14) executing the full estimation graph on-device. Differentiable geometry covers depth-to-3D conversions, Brown–Conrady and Kannala–Brandt camera models, and Lie groups (So3, Se3) with gradients through depth, pose, and intrinsics. AugmentationSequential jointly transforms images, masks, boxes, and keypoints with invertible 2D pipelines and GPU-resident sampled parameters guarded by sync tests. Local features run on GPU: SIFT, HardNet, DISK, ALIKED, XFeat, DeDoDe, LoFTR, LightGlue with mutual-NN and ratio-test matchers. Morphology supports arbitrary structuring elements on batched tensors. Nearly all probed operators run under torch.compile matching eager mode, and over 1,000 export to ONNX. Benchmarks on RTX 4090 (batch 32) show 10×–70× speedups over per-image OpenCV/scikit-image CPU in eager mode; torch.compile pushes several past 100×. Compiled Kornia beats torchvision on hue, color jitter, and Gaussian blur but remains slower on flips, crops, and brightness. CPU half-precision manifests (PyTorch 2.15.0) record 243 float16 and 256 bfloat16 parametrized failures across modules like augmentation (59/48), geometry.transform (31/31), and geometry.conversions (65/65); some operators compute internally in float32/float64 and return input dtype. CUDA CI is absent; historical local runs (RTX 4090, PyTorch 2.14) showed 451 float16 and 500 bfloat16 failures. The model zoo is frozen; new integrations require a named maintainer sponsor. Licensed Apache 2.0.
Read original
github-trending/python