zerfoo package - github.com/zerfoo/zerfoo - Go Packages
Package autoopt provides automatic optimization recommendations based on hardware profiling.
Experimental — this package is not yet wired into the main framework.
Package cloud provides a multi-tenant managed inference service for Zerfoo.
bench command
Command bench runs a standardized benchmark harness for zerfoo models.
bench-compare command
Command bench-compare compares two NDJSON benchmark result files and outputs a markdown regression report.
bench_batch command
Command bench_batch benchmarks continuous batching vs session pool throughput.
bench_disagg command
Command bench_disagg benchmarks disaggregated vs collocated serving throughput.
bench_mamba command
Command bench_mamba benchmarks Mamba-3 SSM vs Transformer attention decode throughput using synthetic FLOPs-based timing estimates.
bench_prefix command
Command bench_prefix simulates a multi-turn chat workload to measure prefix cache hit rate and TTFT reduction.
bench_spec command
Command bench_spec benchmarks speculative decoding speedup by comparing standalone target model decode against speculative decode (target + draft).
bench_tps command
bench_tps measures tokens-per-second for a local ZMF model.
bench_train command
Command bench_train benchmarks PatchTST GPU training on synthetic data.
Package cli provides the command-line interface framework for Zerfoo.
coverage-gate command
Command coverage-gate reads a Go coverage profile and fails if any testable package drops below the configured coverage threshold.
deprecation-check command
Package main implements a linter that checks // Deprecated: doc comments for proper replacement guidance and version information.
finetune command
Command finetune runs QLoRA fine-tuning on a GGUF model.
gguf-info command
Command gguf-info prints tensor names, shapes, and GGML types from a GGUF file.
train_distributed command
Command train_distributed launches distributed training using FSDP.
ts_train command
Command ts_train trains a PatchTST time-series signal model on offline feature data.
zerfoo-edge command
Package main provides a minimal edge/embedded inference binary for Zerfoo.
Package compliance provides SOC 2 compliance automation tooling including Trust Services Criteria control mapping, evidence collection, policy document generation, and control status tracking.
Package audit provides SOC 2 Type I audit tooling including readiness assessment, evidence collection automation, gap analysis, and report generation.
Package observation implements the SOC 2 Type II observation period framework.
Package config provides file-based configuration loading with validation.
Experimental — this package is not yet wired into the main framework.
Package data provides dataset containers for training batches and normalization.
Package aws provides AWS Marketplace Metering API integration for Zerfoo.
Package distributed provides multi-node distributed training for the Zerfoo ML framework.
Package coordinator provides a distributed training coordinator.
Package fsdp implements Fully Sharded Data Parallelism for distributed training.
agentic-tool-use command
Command agentic-tool-use demonstrates function calling (tool use) with a language model using the zerfoo one-line API.
api-server command
Command api-server demonstrates starting an OpenAI-compatible inference server.
audio-transcription command
Command audio-transcription demonstrates speech-to-text using the Zerfoo OpenAI-compatible API server.
automl command
Command automl demonstrates using the AutoML coordinator to search over hyperparameter configurations with Bayesian optimization and early stopping.
chat command
Command chat demonstrates a simple interactive chatbot using the zerfoo one-line API.
classification command
Command classification demonstrates text classification using grammar-constrained decoding to guarantee a valid JSON response with a category label.
code-completion command
Command code-completion demonstrates using a language model for code completion.
distributed-training command
Command distributed-training demonstrates setting up FSDP distributed training with gradient accumulation using the zerfoo distributed and training packages.
embedding command
Command embedding demonstrates embedding Zerfoo inference inside a Go HTTP handler.
embedding-search command
Command embedding-search demonstrates semantic search using model embeddings.
fine-tuning command
Command fine-tuning demonstrates parameter-efficient fine-tuning using LoRA (Low-Rank Adaptation) on a tabular model.
inference command
Command inference demonstrates loading a GGUF model and generating text.
json-output command
Command json-output demonstrates grammar-guided decoding with a JSON schema.
langchain-chatbot command
Command langchain-chatbot demonstrates using the Zerfoo LangChain adapter as a drop-in LLM for a simple interactive chatbot loop.
rag command
Command rag demonstrates retrieval-augmented generation using Zerfoo.
streaming command
Command streaming demonstrates streaming chat generation using the zerfoo API.
summarization command
Command summarization demonstrates text summarization using a GGUF language model.
text-embedding command
Command text-embedding demonstrates extracting text embedding vectors from a loaded GGUF model using the inference package.
timeseries command
Command timeseries demonstrates time-series forecasting with the N-BEATS model using the zerfoo timeseries package.
translation command
Command translation demonstrates text translation using a GGUF language model.
vision-analysis command
Command vision-analysis demonstrates multimodal inference with image input.
weaviate-search command
Command weaviate-search demonstrates using the Zerfoo Weaviate adapter to embed a corpus of documents and perform cosine-similarity semantic search without requiring a live Weaviate instance.
Experimental — this package is not yet wired into the main framework.
Package federated provides federated learning interfaces and a FedAvg baseline implementation.
Package generate implements autoregressive text generation for transformer models loaded by the inference package.
Package agent implements the agentic tool-use loop for multi-step reasoning.
Package grammar converts a subset of JSON Schema into a context-free grammar for constrained decoding.
Package speculative implements speculative decoding strategies for accelerated generation.
Experimental — this package is not yet wired into the main framework.
Experimental — this package is not yet wired into the main framework.
Package health provides HTTP health check endpoints for Kubernetes-style liveness and readiness probes.
Package inference provides a high-level API for loading GGUF models and running text generation, chat, embedding, and speculative decoding with minimal boilerplate.
Package guardian implements prompt template rendering for IBM Granite Guardian safety risk evaluation across 13 pre-defined risk categories.
Package lora implements loading and validation of LoRA (Low-Rank Adaptation) adapter weights from GGUF files.
Package multimodal provides audio preprocessing for audio-language model inference.
Package parallel provides tensor and pipeline parallelism for distributing inference across multiple GPUs.
Package sentiment provides a high-level sentiment classification pipeline that wraps encoder model loading and inference.
Package timeseries implements time-series model builders.
Package features provides a feature store for the Wolf time-series ML platform.
Package transmla converts standard multi-head attention (MHA) weights into multi-head latent attention (MLA) form via truncated SVD.
Package langchain provides an adapter that makes Zerfoo's OpenAI-compatible HTTP API compatible with LangChain-Go's LLM interface.
Package weaviate provides an adapter for generating embeddings via Zerfoo's OpenAI-compatible HTTP API and inserting them into a Weaviate vector database client.
Package clblast provides Go wrappers for the CLBlast BLAS library.
Package codegen generates CUDA megakernel source code from compiled computation graphs.
Package cublas provides low-level purego bindings for the cuBLAS library.
Package cuda provides low-level bindings for the CUDA runtime API using (Stability: stable) dlopen/dlsym (no CGo).
Package kernels provides Go wrappers for custom CUDA kernels.
Package cudnn provides purego bindings for the NVIDIA cuDNN library.
Package dsp provides digital signal processing primitives.
Package gpuapi defines internal interfaces for GPU runtime operations.
Package hip provides low-level bindings for the AMD HIP runtime API (Stability: alpha) using purego dlopen.
Package kernels provides Go wrappers for custom HIP kernels via purego (Stability: alpha) dlopen.
Package miopen provides low-level bindings for the AMD MIOpen library (Stability: alpha) using purego dlopen.
Package nccl provides CGo bindings for the NVIDIA Collective Communications (Stability: beta) Library (NCCL).
Package opencl provides Go wrappers for the OpenCL 2.0 runtime API.
Package kernels provides OpenCL kernel source and dispatch for elementwise operations.
Package rocblas provides low-level bindings for the AMD rocBLAS library (Stability: alpha) using purego dlopen.
Package tensorrt provides bindings for the NVIDIA TensorRT inference (Stability: alpha) library via purego (dlopen/dlsym, no CGo).
Package workerpool provides a persistent pool of goroutines that process submitted tasks.
Package xblas provides CPU BLAS wrappers with ARM NEON and AVX2 SIMD assembly.
Package layers provides neural network layer implementations for the Zerfoo ML framework.
Package activations provides activation function layers.
Package attention provides attention mechanisms for neural networks.
Package audio provides audio-related neural network layers.
Package components provides reusable components for neural network layers.
Package core provides core neural network layer implementations.
Package embeddings provides neural network embedding layers.
Package functional provides stateless, tensor-in tensor-out wrappers for common neural-network operations.
Package gather provides the Gather layer for embedding-table lookup.
Package hrm implements the Hierarchical Reasoning Model.
Package normalization provides normalization layers for neural networks.
Package recurrent provides recurrent neural network layers.
Package reducesum provides the ReduceSum layer for axis-wise reduction.
Package registry provides a central registration point for all layer builders.
Package regularization provides regularization layers for neural networks.
Package residual provides residual connection layers for neural networks.
Package ssm implements state space model layers.
Package timeseries provides time-series specific neural network layers.
Package transformer provides transformer building blocks such as the Transformer `Block` used in encoder/decoder stacks.
Package transpose provides the Transpose layer for axis permutation.
Package vision provides vision-related neural network layers.
Package marketplace provides a unified abstraction layer for cloud marketplace integrations across AWS, GCP, and Azure.
Package aws provides AWS Marketplace integration for Zerfoo Cloud, including metering, subscription lifecycle management, entitlement verification, and token-based billing.
Package azure provides Azure Marketplace integration for Zerfoo Cloud, including SaaS Fulfillment API v2, Marketplace Metering Service, subscription lifecycle management, and webhook handling.
Package gcp provides GCP Marketplace integration for Zerfoo Cloud, including Cloud Commerce Partner Procurement API integration, SaaS entitlement management, Service Control API usage metering, and token-based billing.
Experimental — this package is not yet wired into the main framework.
Package mobile provides gomobile-compatible bindings for zerfoo inference.
Package model provides adapter implementations for bridging existing and new model interfaces.
Package gguf provides GGUF file format parsing and writing.
Package hrm provides experimental Hierarchical Reasoning Model types.
Package huggingface provides HuggingFace model configuration parsing.
Package modelcache provides an LRU model file cache for pre-caching GGUF models on Kubernetes nodes via a DaemonSet.
Experimental — this package is not yet wired into the main framework.
Experimental — this package is not yet wired into the main framework.
Experimental — this package is not yet wired into the main framework.
Experimental — this package is not yet wired into the main framework.
Experimental — this package is not yet wired into the main framework.
Package registry provides a model registry with local cache, pull, get, list, and delete operations.
Experimental — this package is not yet wired into the main framework.
Package security implements SOC 2 security controls for the Zerfoo ML framework.
Package serve provides an OpenAI-compatible HTTP API server for model inference.
Package adaptive implements an adaptive batch scheduler that dynamically adjusts batch size based on queue depth and latency targets to maximize throughput while meeting latency SLOs.
Package agent adapts the generate/agent agentic loop to the serving layer.
Package batcher implements a continuous batching scheduler for inference serving.
Package disaggregated implements disaggregated prefill/decode serving.
Package disaggpb defines the gRPC service contracts for disaggregated prefill/decode serving.
Package multimodel provides a ModelManager that loads and unloads models on demand with LRU eviction when GPU memory budget is exceeded.
Package operator provides a Kubernetes operator for managing ZerfooInferenceService custom resources.
Package registry provides a bbolt-backed model version registry for tracking and A/B testing.
Package repository provides a model repository for storing and managing GGUF model files.
Experimental — this package is not yet wired into the main framework.
Package shutdown provides orderly shutdown coordination using context cancellation and cleanup callbacks.
Package support implements an enterprise support ticketing system with priority routing, SLA tracking, and webhook notifications.
Experimental — this package is not yet wired into the main framework.
Package tabular provides tabular ML model types.
Package benchmark provides a standardized benchmark suite for measuring ML model inference performance: tok/s decode, tok/s prefill, memory usage, and time to first token.
Package compare provides a model comparison tool that runs the same prompts through multiple models and compares their performance metrics.
Package training contains end-to-end training loop integration tests.
Package timeseries provides time-series forecasting models built on ztensor.
Package training provides adapter implementations for bridging existing and new interfaces.
Package automl provides automated machine learning utilities including Bayesian hyperparameter optimization.
Package fp8 implements FP8 mixed-precision training support.
Package lora implements LoRA and QLoRA fine-tuning adapters.
Package loss provides various loss functions for neural networks.
Package nas implements neural architecture search using DARTS.
Package online implements online learning with drift detection and model rollback.
Package optimizer provides various optimization algorithms for neural networks.
Package scheduler provides learning rate scheduling strategies for optimizers.