ncnn
Tencent's zero-dependency C++ inference library (腾讯 ncnn), BSD 3-Clause. Runs inside WeChat and QQ, with prebuilt libraries for HarmonyOS, WebAssembly, RISC-V and LoongArch.
Overview
BSD 3-Clause, no protobuf, no OpenCV, no vendor SDK. That description has held since Tencent opened ncnn in 2017 and it explains the project's staying power. The library runs inside WeChat and QQ, which means its ARM NEON paths have been stress-tested at a scale very little open source can claim, and the attribution-only licence lets it into closed firmware that a custom community licence would never clear.
Releases are date-stamped rather than semantically versioned, so 20260526 means 26 May 2026 and nothing more. That build ships prebuilt libraries for Android, iOS, macOS, Linux, Windows, HarmonyOS, WebAssembly, watchOS, tvOS and visionOS, in both Vulkan and CPU-only forms. Hand-written kernels cover ARM NEON, x86 up to AVX512, RISC-V vector, MIPS and LoongArch, which is why ncnn is often the only inference engine that will build for a Loongson 2K1000 or an AllWinner D1 board. Model conversion goes through PNNX, which reads PyTorch directly and preserves the original graph structure rather than flattening it through ONNX.
The honest limitation is scope. This is a vision engine in a year when most on-device AI work is transformer decode, and although the recent notes mention large model and startup time work, MNN and llama.cpp are further along there. More restrictive still, there is no NPU backend: CPU and Vulkan only, so any dedicated neural accelerator on your board goes unused. Judged as what it is, a tiny dependency-free CPU and Vulkan inference library for awkward hardware, nothing else in open source comes close.
Key Features
- ✓ BSD 3-Clause licence, permissive enough for closed firmware with only an attribution notice, no copyleft and no field-of-use terms
- ✓ No third-party runtime dependencies at all: no protobuf, no OpenCV, no vendor SDK, which keeps binary size and integration cost low on embedded targets
- ✓ Hand-written CPU kernels for ARM NEON, x86 through AVX512, RISC-V vector, MIPS and LoongArch
- ✓ Vulkan compute path for GPU acceleration on Mali, Adreno and desktop GPUs without proprietary drivers
- ✓ Release 20260526 ships prebuilt libraries for Android, iOS, macOS, Linux, Windows, HarmonyOS, WebAssembly, watchOS, tvOS and visionOS, in Vulkan and CPU-only variants
- ✓ PNNX converter takes PyTorch models directly and keeps graph structure intact instead of routing through ONNX
Where it holds
- • Around 23.8k GitHub stars and close to a decade in production inside WeChat and QQ, so the ARM paths are exercised on hundreds of millions of devices
- • Date-stamped releases such as 20260526 make pinning and reproducing a build years later trivial
- • Runs where nothing else will: Loongson 2K1000, AllWinner D1 RISC-V boards, Raspberry Pi, Jetson and HarmonyOS handsets
- • Attribution-only licensing avoids the legal review that custom Chinese community licences trigger in export-sensitive products
Where it breaks
- • Vision-first by design. On-device transformer decode has moved on and MNN or llama.cpp are further ahead for language models
- • No NPU backend of any kind. CPU and Vulkan only, so a Hexagon or Kirin block on the same die stays idle
- • Documentation is a GitHub wiki split between English and Chinese, and issue threads run mostly in Chinese
- • Unsupported operators mean writing C++ kernels by hand, and doing it again per architecture if you care about NEON and AVX512 both
My Take
BSD 3-Clause with zero runtime dependencies is a rare combination anywhere, and it is why this library ends up inside firmware where a compliance review kills everything else. The 20260526 release ships prebuilt binaries for HarmonyOS, visionOS and RISC-V boards, which tells you exactly who it is for: engineers stuck with whatever silicon procurement actually delivered. Scope is the weakness. With no NPU backend at all, a phone carrying a capable Hexagon or Kirin block burns CPU cores instead, and for on-device language models the lead has passed to MNN.
Quick Info
- Pricing:
- open-source
- Openness:
- Open source
- Licence:
- BSD 3-Clause
- Starting at:
- Free under BSD 3-Clause, attribution only. No commercial tier and no CLA required to use it. The real cost is writing custom C++ kernels per architecture when your model uses an operator ncnn does not implement.
- Added:
- Aug 2026
- Updated:
- Aug 2026
Use Cases
Judge it on your own work
The notes above say where ncnn holds and where it breaks. The fastest check is your own workload.
Visit website ↗Alternatives to ncnn
MNN
open-sourceAlibaba's on-device inference engine (阿里 MNN), Apache 2.0. Runs Qwen3, Qwen3-VL and DeepSeek distills on phones across CPU, OpenCL, Metal, Vulkan, CUDA and Qualcomm Hexagon.
ONNX Runtime
open-sourceMicrosoft's cross-platform inference engine for ONNX models, with execution providers for CUDA, TensorRT, OpenVINO, CoreML, NNAPI, QNN, DirectML and XNNPACK.
NVIDIA Jetson and TensorRT
paidNVIDIA's edge inference stack: Jetson modules from Orin Nano to Thor T5000, with TensorRT compiling models into hardware-specific FP4, FP8, INT8 and INT4 engines.