NumPy

Last updated: Oct 28, 2026

---

Rationale

NumPy is the fundamental package for scientific and numerical computing in Python. In matches, Fluid Attacks uses it to do an exact in-memory cosine similarity search. This search compares unstructured threat descriptions with the vulnerability criteria definitions of Fluid Attacks.

These are the primary reasons why Fluid Attacks selected it and not a dedicated vector database:

• It is open source.
• It is already a core, pinned dependency in the Python stack of Fluid Attacks. Thus, it does not add packages, native wheel compilations, or foreign runtime bridges.
• The criteria dataset has approximately 147 parent definitions and approximately 600 KB of float32 embeddings. For this dataset, exact matrix multiplication (np.dot) runs in tens of microseconds with hardware-accelerated BLAS and SIMD vectorization.
• It gives 100% exact recall without approximation errors. It does not have the graph truncation or quantization distortion of HNSW or IVF-PQ index structures.
• It makes data serialization and storage simpler. The full criteria index stays in one compact .npz file. Thus, cold starts in ephemeral AWS Batch workers are shorter than 50 ms, without multi-file directory extraction.
• All worker threads can use the read-only in-memory arrays at the same time, without connection pooling, SQLite file locks, or IPC overhead.
• It fully removes the supply-chain risks and maintenance work of third-party vector database engines.

Alternatives

ChromaDB

At first, Fluid Attacks used ChromaDB as a prototype in matches.

• It had four high-severity and critical-severity CVEs (CVE-2026-45829, CVE-2026-45830, CVE-2026-45833, and CVE-2026-45832). These CVEs include remote code execution, unauthenticated collection mutation, and RBAC authorization bypasses.
• In the execution model of Fluid Attacks, attackers could not get access to the vulnerable code:
  • matches runs ChromaDB only in embedded mode (PersistentClient).
  • It does not expose HTTP endpoints.
  • It calculates embeddings externally with Voyage AI.
  • It runs inside single-tenant batch jobs.
• But the upstream maintainers did not show good security governance:
  • They did not answer coordinated vulnerability disclosures for months.
  • They applied incomplete patches that fixed the Rust backend, but the Python server stayed vulnerable (issue #7588).
  • They did not release a patched version (<= 1.5.9).
• It added 33 unused transitive dependencies (approximately 45 MB of wheels) to the build graph. These dependencies include onnxruntime, kubernetes, grpcio, uvicorn, fastapi, and pypika (which had to have custom Nix patching).

LanceDB

LanceDB is an embedded, serverless vector database built in Rust on Apache Arrow.

• It gives columnar storage, disk-backed indexing (IVF-PQ), and full-text hybrid search.
• But the dataset has only approximately 147 static criteria vectors. For this dataset, LanceDB adds external dependencies (lancedb, pylance, PyArrow version pinning) and multi-file directory structures that are not necessary. Exact BLAS matrix operations in NumPy are faster and do not add dependencies.

SciPy

SciPy is a scientific computation library. It gives spatial distance functions with scipy.spatial.distance.cdist.

• It has built-in cosine distance calculations (cdist(..., metric='cosine')).
• But its general-purpose C loop calculates vector norms again for each query. Thus, it is 2 to 5 times slower than the multiplication of pre-normalized matrices with BLAS np.dot.
• It adds 30 to 50 MB of compiled shared libraries (Fortran routines, LAPACK wrappers, optimization solvers). Vector similarity search does not use these libraries.

scikit-learn

scikit-learn is a machine learning library. It gives nearest-neighbor search algorithms with NearestNeighbors and cosine_similarity.

• Its pairwise cosine similarity functions are a Python-level wrapper around the same np.dot call. They add array validation and intermediate memory allocations, but they do not make the algorithm faster.
• It adds large dependencies (30 to 60 MB of wheels). It also increases the cold-start import latency in ephemeral batch containers.

Usage

Fluid Attacks uses NumPy in matches for these tasks:

• To keep the 1024-dimensional Voyage AI (voyage-3) embedding matrices of the vulnerability criteria definitions of Fluid Attacks (fa_defines_collection)
• To calculate the exact top-k cosine similarity with vector-matrix dot products (np.dot) against threat descriptions from customer documentation
• To be the in-memory similarity search engine in ephemeral AWS Batch workers. It loads .npz snapshots from Amazon S3 with query latency of less than 1 ms, before the LLM-as-judge refinement.