--- url: /logly.zig/api/sampler.md description: >- API reference for Logly.zig Sampler struct. Control log volume with probability sampling, rate limiting, every-Nth, and adaptive sampling strategies. --- # Sampler API The `Sampler` struct controls log throughput by selectively processing records. ## Quick Reference: Method Aliases | Full Method | Alias(es) | Description | |-------------|-----------|-------------| | `init()` | `create()` | Initialize sampler | | `deinit()` | `destroy()` | Deinitialize sampler | | `setAcceptCallback()` | `onAccept()` | Set accept callback | | `setRejectCallback()` | `onReject()` | Set reject callback | | `setRateLimitCallback()` | `onRateLimit()` | Set rate limit callback | | `setAdjustmentCallback()` | `onAdjustment()` | Set adjustment callback | | `shouldSampleWithReason()` | `sampleWithReason()`, `decision()` | Sample with reject reason details | | `reset()` | `clear()` | Reset sampler state | | `resetStats()` | `clearStats()` | Reset statistics | | `isEnabled()` | `enabled()` | Check if sampler is enabled | | `strategyName()` | `name()` | Get strategy name | | `totalProcessed()` | `total_()` | Get total processed count | | `totalAccepted()` | `accepted_()` | Get total accepted count | | `totalRejected()` | `rejected_()` | Get total rejected count | | `shouldSample()` | `sample()`, `check()`, `allow()` | Check if record should be sampled | | `shouldSampleKey()` | `sampleKey()` | Deterministic sampling decision for a key | | `shouldSampleKeyWithReason()` | `keyDecision()` | Deterministic sampling decision with reason | | `setStrategy()` | `configure()` | Update strategy at runtime | | `setProbability()` | `probability()`, `setProb()` | Set probability strategy with clamping | | `setRateLimit()` | `rateLimit()`, `configureRateLimit()` | Set rate-limit strategy | | `setEveryN()` | `everyN()` | Set deterministic every-N strategy | | `setAdaptive()` | `adaptive()` | Set adaptive strategy with normalized bounds | | `disableSampling()` | `disable()`, `off()` | Disable filtering (allow all records) | | `remainingWindowQuota()` | `quotaLeft()` | Remaining rate-limit tokens in current window | | `windowResetInMs()` | `resetInMs()` | Milliseconds until window reset | | `acceptRate()` | `acceptanceRate()` | Convenience accept rate from stats | | `rejectRate()` | `rejectionRate()` | Convenience reject rate from stats | | `getCurrentRate()` | `rate()` | Get current sampling rate | | `getStats()` | `statistics()`, `stats_()` | Get sampler statistics | ## Overview Samplers reduce log volume in high-throughput systems using various strategies like probability sampling, rate limiting, every-n sampling, and adaptive sampling. All operations are thread-safe with atomic counters. ## Types ### Sampler The main sampler controller with atomic state and statistics. ```zig pub const Sampler = struct { allocator: std.mem.Allocator, strategy: Strategy, state: SamplerState, stats: SamplerStats, mutex: std.Thread.Mutex, // Callbacks on_sample_accepted: ?*const fn (u64) void, on_sample_rejected: ?*const fn (u64) void, on_rate_adjusted: ?*const fn (f64, f64) void, }; ``` ### Strategy The sampling strategy to use. ```zig pub const Strategy = union(enum) { none: void, // No sampling, all pass probability: f64, // 0.0 - 1.0 rateLimit: RateLimitConfig, everyN: u32, // 1 in N adaptive: AdaptiveConfig, tokenBucket: TokenBucketConfig, // Token bucket for burst sampling }; ``` ### RateLimitConfig Configuration for rate limiting. ```zig pub const RateLimitConfig = struct { maxRecords: u64, // Maximum records per window windowMs: u64, // Window size in milliseconds }; ``` ### AdaptiveConfig Configuration for adaptive sampling. ```zig pub const AdaptiveConfig = struct { targetRate: u32, // Target logs per second min_sampling_rate: f64, // Minimum sampling probability adjustmentIntervalMs: u64, // How often to adjust }; ``` ### TokenBucketConfig Configuration for token bucket (burst) sampling. ```zig pub const TokenBucketConfig = struct { capacity: u32, // Maximum tokens (burst capacity) refill_rate: u32, // Tokens added per second refill_interval_ms: u64, // How often to add tokens }; ``` ### SamplerConfig Configuration passed to the sampler. ```zig pub const SamplerConfig = struct { strategy: Strategy = .{ .none = {} }, bypassLevels: LevelMask = LevelMask.init(), // Levels that always bypass sampling }; ``` ### SamplerStats Statistics for sampler operations. ```zig pub const SamplerStats = struct { total_processed: std.atomic.Value(u64), total_accepted: std.atomic.Value(u64), total_rejected: std.atomic.Value(u64), currentRate: std.atomic.Value(f32), pub fn acceptRate(self: *const SamplerStats) f64; }; ``` ### SampleDecision Detailed decision payload returned by `shouldSampleWithReason()`. ```zig pub const SampleDecision = struct { accepted: bool, sampleRate: f64, rejectReason: ?SampleRejectReason = null, }; ``` ## Methods ### Initialization #### `init(allocator: std.mem.Allocator, strategy: Strategy) Sampler` Initializes a new Sampler instance with the specified strategy. **Alias:** `create` #### `initWithConfig(allocator: std.mem.Allocator, config: Config.SamplingConfig) Sampler` Initializes a new Sampler with full configuration. #### `initWithIo(allocator: std.mem.Allocator, io_handle: std.Io, config: Config.SamplingConfig) Sampler` Initializes a new Sampler with an explicit I/O handle and full configuration. #### `deinit(self: *Sampler) void` Releases all resources associated with the sampler. **Alias:** `destroy` ### Sampling #### `shouldSample() bool` Determines if the current record should be sampled (processed). Returns `true` to process, `false` to drop. **Alias**: `sample`, `check`, `allow` #### `shouldSampleKey(key: []const u8) bool` Deterministically samples based on a stable key (user/session/etc.). **Alias**: `sampleKey` #### `shouldSampleKeyWithReason(key: []const u8) SampleDecision` Deterministic sampling decision with reject reason details. **Alias**: `keyDecision` #### `shouldSampleWithReason() SampleDecision` Evaluates sampling and returns structured decision details including reject reason. **Alias**: `sampleWithReason`, `decision` #### `setStrategy(strategy: Strategy) void` Updates sampler strategy at runtime and resets strategy-specific counters. **Alias**: `configure` #### `setProbability(probabilityValue: f64) void` Sets probability sampling strategy and clamps value into `[0.0, 1.0]`. **Alias**: `probability`, `setProb` #### `setRateLimit(maxRecords: u32, windowMs: u64) void` Sets rate-limit strategy with safe normalization: * `maxRecords == 0` becomes `1` * `windowMs == 0` uses centralized default window **Alias**: `rateLimit`, `configureRateLimit` #### `setEveryN(n: u32) void` Sets deterministic every-N sampling strategy. **Alias**: `everyN` #### `setAdaptive(config: AdaptiveConfig) void` Sets adaptive strategy and normalizes min/max sampling bounds. **Alias**: `adaptive` #### `disableSampling() void` Disables filtering strategy (`.none`) so all records are accepted. **Alias**: `disable`, `off` #### `remainingWindowQuota() ?u32` For `.rateLimit` strategy, returns remaining records in current window. Returns `null` for non-rate-limit strategies. **Alias**: `quotaLeft` #### `windowResetInMs() ?u64` For `.rateLimit` strategy, returns milliseconds until current window reset. Returns `null` for non-rate-limit strategies. **Alias**: `resetInMs` #### `acceptRate() f64` Convenience wrapper around sampler stats accept rate. **Alias**: `acceptanceRate` #### `rejectRate() f64` Convenience wrapper around sampler stats reject rate. **Alias**: `rejectionRate` ### Statistics #### `getStats() SamplerStats` Returns current sampler statistics. **Alias**: `statistics`, `stats_` #### `getCurrentRate() f64` Returns the current sampling rate. **Alias**: `rate` #### `resetStats() void` Resets all statistics to zero. #### `totalProcessed() u64` Returns total records processed. #### `totalAccepted() u64` Returns total records accepted. #### `totalRejected() u64` Returns total records rejected. ### SamplerStats Methods The `SamplerStats` struct provides comprehensive statistics tracking: ```zig pub const SamplerStats = struct { totalRecordsSampled: std.atomic.Value(Constants.AtomicUnsigned), recordsAccepted: std.atomic.Value(Constants.AtomicUnsigned), recordsRejected: std.atomic.Value(Constants.AtomicUnsigned), rateLimitExceeded: std.atomic.Value(Constants.AtomicUnsigned), rateAdjustments: std.atomic.Value(Constants.AtomicUnsigned), /// Calculate current accept rate (0.0 - 1.0) pub fn getAcceptRate(self: *const SamplerStats) f64; /// Calculate current reject rate (0.0 - 1.0) pub fn getRejectRate(self: *const SamplerStats) f64; /// Returns true if any records have been rejected. pub fn hasRejections(self: *const SamplerStats) bool; /// Returns true if rate limit has been exceeded at least once. pub fn hasRateLimitExceeded(self: *const SamplerStats) bool; /// Returns the rate limit exceeded percentage (0.0 - 1.0). pub fn getRateLimitExceededRate(self: *const SamplerStats) f64; /// Returns total records sampled as u64. pub fn getTotal(self: *const SamplerStats) u64; /// Returns accepted records count as u64. pub fn getAccepted(self: *const SamplerStats) u64; /// Returns rejected records count as u64. pub fn getRejected(self: *const SamplerStats) u64; }; ``` ### State #### `isEnabled() bool` Returns true if sampling is enabled (strategy != none). #### `strategyName() []const u8` Returns the name of the current strategy. ### Callbacks #### `setAcceptedCallback(callback: *const fn (u64) void) void` Sets callback for accepted samples. #### `setRejectedCallback(callback: *const fn (u64) void) void` Sets callback for rejected samples. #### `setRateAdjustedCallback(callback: *const fn (f64, f64) void) void` Sets callback for rate adjustments (adaptive strategy). ## Presets ### SamplerPresets ```zig pub const SamplerPresets = struct { /// No sampling - all records pass through. pub fn none(allocator) Sampler; /// Sample approximately 10% of records. pub fn sample10Percent(allocator) Sampler; /// Sample approximately 50% of records. pub fn sample50Percent(allocator) Sampler; /// Sample approximately 1% of records (high-volume production). pub fn sample1Percent(allocator) Sampler; /// Limit to 10 records per second (debug/low-volume). pub fn limit10PerSecond(allocator) Sampler; /// Limit to 100 records per second. pub fn limit100PerSecond(allocator) Sampler; /// Limit to 1000 records per second. pub fn limit1000PerSecond(allocator) Sampler; /// Sample every 5th record. pub fn every5th(allocator) Sampler; /// Sample every 10th record. pub fn every10th(allocator) Sampler; /// Sample every 100th record (high-volume production). pub fn every100th(allocator) Sampler; /// Adaptive sampling targeting 100 records per second. pub fn adaptive100PerSecond(allocator) Sampler; /// Adaptive sampling targeting 1000 records per second. pub fn adaptive1000PerSecond(allocator) Sampler; /// Creates a sampled sink configuration. pub fn createSampledSink(filePath: []const u8) SinkConfig; }; ``` ## Example ```zig const Sampler = @import("logly").Sampler; const SamplerPresets = @import("logly").SamplerPresets; // Create sampler with 10% probability var sampler = Sampler.init(allocator, .{ .probability = 0.1 }); defer sampler.deinit(); // Or use presets var sampler2 = SamplerPresets.limit100PerSecond(allocator); defer sampler2.deinit(); // Check if record should be sampled for (records) |record| { if (sampler.shouldSample()) { // Process this record processRecord(record); } } // Check statistics const stats = sampler.getStats(); std.debug.print("Accept rate: {d:.2}%\n", .{stats.acceptRate() * 100}); std.debug.print("Processed: {d}, Accepted: {d}, Rejected: {d}\n", .{ sampler.totalProcessed(), sampler.totalAccepted(), sampler.totalRejected(), }); // Check if sampling is active if (sampler.isEnabled()) { std.debug.print("Strategy: {s}\n", .{sampler.strategyName()}); } ``` ## Strategy Selection Guide | Strategy | Use Case | Overhead | |----------|----------|----------| | `none` | No sampling needed | Zero | | `probability` | Random sampling for debugging | Very low | | `rateLimit` | Strict throughput control | Low | | `everyN` | Predictable sampling | Very low | | `adaptive` | Dynamic load handling | Low | ## Performance * **Lock-free** fast path for read-only checks * **Atomic counters** for thread-safe statistics * **Minimal overhead** in all strategies * **Adaptive adjustment** only during adjustment intervals ## See Also * [Filtering Guide](../guide/filtering.md) - Content-based filtering * [Metrics API](metrics.md) - Performance monitoring