Replace Groovy DSL runtime with ANTLR4 + Javassist for MAL, LAL, and Hierarchy by wu-sheng · Pull Request #13723 · apache/skywalking
Document the detailed implementation plan for eliminating Groovy from OAP runtime via build-time transpilers (MAL/LAL) and v1/v2 module split (hierarchy), based on Discussion #13716 and skywalking-graalvm-distro. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add MalExpression, MalFilter, LalExpression functional interfaces and SampleFamilyFunctions (TagFunction, SampleFilter, ForEachFunction, DecorateFunction, PropertiesExtractor). Add Java functional interface overloads alongside existing Groovy Closure methods in SampleFamily, FilterSpec, ExtractorSpec, and SinkSpec. Change InstanceEntityDescription to use Function instead of Closure. All 129 existing tests pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hase 2) Ports MalToJavaTranspiler from skywalking-graalvm-distro into a new mal-transpiler analyzer submodule. The transpiler parses Groovy MAL expressions/filters via AST at CONVERSION phase and emits equivalent Java classes implementing MalExpression/MalFilter interfaces from Phase 1. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hase 3) Introduces lal-transpiler module that parses LAL Groovy DSL scripts into AST at Phases.CONVERSION and emits pure Java classes implementing LalExpression. Handles filter/text/json/yaml/extractor/sink/abort blocks, parsed property access, safe navigation, cast expressions, GString interpolation, and SHA-256 deduplication. Makes MalToJavaTranspiler.escapeJava() public for cross-module reuse. Includes 37 comprehensive tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ing (Phase 4) Introduces meter-analyzer-v2 and log-analyzer-v2 modules that provide same-FQCN replacement classes for DSL.java, Expression.java, and FilterExpression.java. The v2 classes load transpiled MalExpression/ MalFilter/LalExpression implementations from META-INF manifests via Class.forName() instead of Groovy GroovyShell/ExpandoMetaClass/ DelegatingScript. Uses maven-shade-plugin to overlay the upstream Groovy-dependent classes. Includes 7 unit tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… (Phase 5) Extract hierarchy matching rules from HierarchyDefinitionService into pluggable HierarchyRuleProvider interface. Remove Groovy imports from server-core by replacing Closure<Boolean> with BiFunction<Service,Service,Boolean>. - hierarchy-v1: GroovyHierarchyRuleProvider (for CI checker only) - hierarchy-v2: JavaHierarchyRuleProvider with 4 built-in rules + 12 tests - HierarchyDefinitionService: add HierarchyRuleProvider interface, DefaultJavaRuleProvider - HierarchyService: .getClosure().call() → .match() Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ase 6) Three checker modules verify v1 (Groovy) and v2 (transpiled Java) produce identical results: hierarchy rules (22 tests), MAL expressions (1187 tests), MAL filters (29 tests), and LAL scripts (10 tests). Zero behavioral divergences found when both paths succeed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…L to on-the-fly compilation
- Merge mal-grammar + mal-compiler into meter-analyzer
- Merge lal-grammar + lal-compiler into log-analyzer
- Merge hierarchy-rule-grammar + hierarchy-rule-compiler into hierarchy
- Remove 6 standalone modules (3 grammar + 3 compiler)
- Update DSL.java to compile MAL expressions on-the-fly via MALClassGenerator
instead of loading from non-existent manifest file
- Add varargs handling for tagEqual/tagNotEqual/tagMatch/tagNotMatch in
generated Javassist code (wrap String args in new String[]{})
- Update test/script-compiler checker POMs to reference merged module names
- Update CLAUDE.md files with merged file structure and paths
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix MAL sample collection regression: skip downsampling() method
arguments to prevent enum values (MAX, SUM, MIN) from being
collected as sample names
- Fix MAL safe navigation (?.): parser now correctly propagates
safeNav flag to chain segments; code generator uses local
StringBuilder to avoid corrupting parent buffer
- Fix MAL filter grammar: add closureCondition alternatives to
closureBody rule for bare conditions like { tags -> tags.x == 'v' }
- Fix MAL downsampling detection for bare identifiers parsed as
ExprArgument wrapping MetricExpr
- Fix MAL sample ordering: use LinkedHashSet for consistent order
- Fix LAL tag() function call: add functionName rule allowing TAG
token in functionInvocation for if(tag("LOG_KIND") == ...) patterns
- Fix LAL ProcessRegistry support: add PROCESS_REGISTRY to
valueAccessPrimary grammar rule
- Fix LAL tag statement code generation: wrap single tag entries in
Collections.singletonMap() since ExtractorSpec.tag() accepts Map
- Fix LAL makeComparison to handle CondFunctionCallContext properly
- Add debug logging to all three code generators (MAL, LAL, Hierarchy)
showing AST and generated Java source at DEBUG level
- Add generateFilterSource() to MALClassGenerator for testing
- Add error handling unit tests with demo error comments for MAL (5),
LAL (4), and Hierarchy (4) generators
- All 1248 checker tests pass: MAL 1187, Filter 29, LAL 10, Hierarchy 22
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ng all four DSL compilers (OAL, MAL, LAL, Hierarchy). Remove Groovy references from docs: LAL code blocks, hierarchy matching rule labels, and stale MeterProcessor comment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Provides a /run-e2e slash command with prerequisites (e2e CLI, swctl, yq install instructions), rebuild detection, test execution, and failure debugging workflow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, interpolated sampler IDs
Address five critical gaps in the LAL v2 compiler that broke shipped production rules:
1. tag("LOG_KIND") in conditions now emits tagValue() helper instead of null
2. Safe navigation (?.) for method calls emits safeCall() helper to prevent NPE
3. Metrics, slowSql, sampledTrace, sampler/rateLimit blocks generate proper
sub-consumer classes with BindingAware wiring
4. else-if chains build nested IfBlock AST nodes instead of dropping
intermediate branches
5. GString interpolation in rateLimit IDs (e.g. "${log.service}:${parsed.code}")
parsed into InterpolationPart segments and emitted as string concatenation
Also fixes ProcessRegistry static calls to pass arguments through, and adds
comprehensive tests (55 total: 35 generator + 20 parser) covering all gaps
including production-like envoy-als, nginx, and k8s-service rule patterns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…cripts and runtime comparison - Rename test/script-compiler to test/script-cases/script-runtime-with-groovy - Copy all shipped production configs into test/script-cases/scripts/ as test copies (MAL: test-otel-rules, test-meter-analyzer-config, test-log-mal-rules, test-envoy-metrics-rules; LAL: test-lal; Hierarchy: test-hierarchy-definition.yml) - Update all checker tests to load from shared scripts/ directory - Upgrade LAL checker from compile-only to full runtime execution comparison (v1 Groovy vs v2 ANTLR4+Javassist, comparing Binding state: service, layer, tags, abort/save) - Update Maven coordinates and root pom module path Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move benchmarks from the standalone oap-server/microbench module into the src/test/ directories of the modules they actually test (server-core and library-util). Drop AbstractMicrobenchmark base class in favor of self-contained @test run() methods. Bump JMH 1.21 -> 1.37 and remove the obsolete -XX:BiasedLockingStartupDelay=0 JVM flag (removed in JDK 18). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MAL compiler fixes (closes 38 previously failing expressions): - Add ternary operator (?:) support in closures (grammar, AST, codegen) - Fix valueEqual() and other primitive-double methods with numeric literal args - Support double-paren argument syntax: sum((['cluster'])) - Handle NUMBER / SampleFamily via MalRuntimeHelper.divReverse() in v2 package - Add variable declarations, map literals, forEach/instance closure types - Add ProcessRegistry class references, improved safe navigation LAL compiler fixes: - Fix null-to-string conversion: use null-safe toStr() instead of String.valueOf() - Add camelToSnake field name fallback for protobuf field access - Add typed execute(FilterSpec, Binding) method signature - Reorganize LAL test scripts into oap-cases/ and feature-cases/ - Add data-driven LALExpressionExecutionTest with 27 test cases MAL checker enhancements: - Add runtime execution comparison (mock SampleFamily data, execute both v1 and v2, compare output samples with labels and values) - Handle increase()/rate() by priming CounterWindow with initial run - Extract tagEqual patterns from expressions for matching mock data All 1,187 MAL + 29 LAL + 22 hierarchy expressions now pass with zero gaps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Give v2 (ANTLR4+Javassist) classes distinct FQCNs from v1 (Groovy) so both can coexist on the classpath without source duplication in v1-with-groovy test modules. Package mapping: - MAL: meter.analyzer.* → meter.analyzer.v2.* - LAL: log.analyzer.* → log.analyzer.v2.* - Hierarchy: config.compiler.* → config.v2.compiler.* Also: remove v2-only files (MalExpression, MalFilter, LalExpression) from v1-with-groovy modules, add mal-v1-with-groovy dependency to lal-v1-with-groovy, fix cross-version enum comparison by name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ime() scalar in MAL compiler Add ANTLR4 lexer mode for regex literals (=~ /pattern/), def keyword with type inference from initializer (String[][] for regex, String[] for split), GString interpolation expansion, .size() to .length translation, decorate() bean-mode closures, and time() as a scalar function in binary expressions. Verified with 1,228 v1-v2 checker tests (1,197 MAL + 31 filter). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…v1-v2 checker data Rewrite MAL run() code generation to use a single reassigned 'sf' variable instead of multiple intermediate variables, producing cleaner decompiled output. Add LocalVariableTable attribute so decompilers show 'samples' and 'sf' instead of 'var1' and 'var2'. Integrate v2 compilers with runtime wiring, add checker test data files, and clean up unused code across MAL/LAL/Hierarchy modules. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add LVT attribute to LAL execute() and consumer accept() methods, and to Hierarchy apply() method, so decompilers show meaningful variable names (filterSpec, binding, _t, u, l) instead of var0, var1, etc. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 8 helper methods (getAt, toLong, toInt, toStr, toBool, isTruthy, tagValue, safeCall) from being duplicated in every generated class via addHelperMethods() to a shared LalRuntimeHelper in the rt package. Generated code now calls LalRuntimeHelper.toStr() etc. via FQCN. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…th typed methods Fix rateLimit() calls inside if-blocks within sampler generating empty bytecode by handling the samplerContent grammar alternative in LALScriptParser.visitIfBody(). Replace generic LalRuntimeHelper.safeCall() and isTruthy() with specific typed methods: isTrue() for Boolean conditions, isNotEmpty() for String non-emptiness, toString() and trim() for null-safe navigation — making generated code explicit about intended type semantics. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…licitly, add LVT - Merge consumer sub-classes into single generated class with private methods - Remove BINDING ThreadLocal from AbstractSpec; all spec methods take ExecutionContext explicitly - Delete BindingAware.java and Binding.java, replace with ExecutionContext - Add abort guard before _extractor/_sink calls matching v1 Groovy behavior - Add LocalVariableTable to all generated methods (execute, _extractor, _sink) - Rename binding→ctx throughout for consistency - Add extraLogType to envoy-als.yaml for compile-time proto resolution - Remove all Consumer callback methods from spec files - Add finalizeSink abort check in FilterSpec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…es, filters) Previously only run() had LVT. Now all generated methods have named locals in debuggers/decompilers instead of var0/var1/var2: - metadata(): this, _samples, _scopeLabels, _aggLabels, _pct - tag/instance apply(Map): this, param name - tag/instance apply(Object) bridge: this, o - forEach accept(): this, element, tags - decorate accept(): this, _arg, param name - filter test(): this, param name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…version 10.4.0-SNAPSHOT
Replace CI-friendly ${revision} with hardcoded 10.4.0-SNAPSHOT in all 104 POMs.
This eliminates persistent "Could not find artifact ...pom:${revision}" errors
when building individual modules without -am. Also removes flatten-maven-plugin
(no longer needed), updates release scripts to use versions:set, and wires
LALSourceTypeProvider SPI for envoy-als extraLog type resolution in tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…chains Cache the extraLog cast in a _p local variable and break safe-nav chains into sequential _tN locals instead of deeply nested ternaries. Repeated access to the same chain prefix reuses existing variables (dedup). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove bind()/evaluate() two-phase pattern from DSL. The mutable ExecutionContext field made DSL unsafe for concurrent use. Now evaluate(ExecutionContext) takes ctx as a parameter, matching the stateless pattern already used by MAL and Hierarchy v2 runtimes. Update LogFilterListener to store per-request contexts in a list and pass each to the corresponding DSL.evaluate(ctx) call. Update LogTestQuery to use the new single-call API. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>