Program-Env Pipeline

Figure 2 · ProgramGym
example run Ramilito/kubesess @7bbfd43 · Rust · real data build repair ×1 Cargo.lock embedded, --locked rounds shown 3 of up to 5 per module following subagent src/modes.rs
Stage 1Repository collection
Validationreal run
./executableDockerfiledownload.shcompile.sh#!/usr/bin/env bashcargo fetch --locked \--manifest-path Cargo.tomlmkdir-p .cargocargovendor –locked \--manifest-path Cargo.toml\vendor > .cargo/config.toml#!/usr/bin/env bashset-euopipefailcargobuild --release \--offline --lockedtest-x /rust-casecp/rust-case ./executable./executableBy LanguageBy StarsDispatcher./executablesrc/init+errorsrc/modesRepositoryOnline preparationBase dockerRepo dependencyFail reasonRepair trickUnable to connect to index.crates.io, connection refusedMove dependency setup to download.shFeedbackOffline compile checkExecutable exit checkBuild ValidationOffline compilebashcompile.shldd./executableStage1:Repository Collectionsrc/maintests/clisrc/configValidationModule assignmentFeedback_Docs/./executableCode Analysis.jsonCoverage Target.jsonRepair Report.jsonSummary.mdsrc/modesInstrumentedexecutable# Only checks the return codeassertresult.returncode==0# Only checks that the collection is non-emptyassertlen(rows)>0# Only checks the output formatassertresult.stdout.strip().isdigit()Stage3:Black-Box Test ConstructionSummaryProgram AnalysisIteration 2 Summary:…`search_file()` is coveredin this iteration; `parallel()` is unreachablevia the black-box executable,because…Pass checkDouble checkDummy checkWeak assert checkStatic AnalysisDynamic Analysis#Dummy exeset-euopipefailsrc/mainsrc/configtests/clisrc/modessrc/init+errorTestsFeedbackDummy.exeUncovered FunctionCovered Functionexample-high.profdataexample-medium.profdataexample-low.profdataweak-example.pyRandom.randint( )rejected-example.pycollected 714 items714 failed , 0 passedshellIteratively generate tests through environment feedback in an agentic loop.// =========coverage:100%===========pubstructSearchConfig{ ... } // HITimplSearchConfig// HIT{pubfnnew()->Self{...}}// ==========coverage:30%=============pubstructSearchEngine{...} // MISSimplSearchEngine{pubfnsearch_file(...){...}// HITpubfnsearch_parallel(...){ // MISS...}pubfncollect_matches(&self){...} }pubstructGlobalWorker{ // MISS...}// ============coverage:0%=============implStandardPrinter{ // MISSpubfnnew(use_color:bool)->Self{...}pubfnprint_context(&self,line:&str){...}pubfnfinish(&self){...}}shellcollected 714 items0 failed , 714 passedshellshellEnvironment=( ) Docs/ProgramDocs/DockerfileTest Suites𝐵𝑖⋆𝐷𝑖,,,Stage2:Executable and Sandbox Generation200+Stage 2: Executable and Sandbox GenerationStage 3: Black-Box Test Construction./executableRepositorydispatcherDocs/src/maintests/cli…src/modesDispatcherArgsConfigHelpI/OSubcommandTUI6 interaction categoriesTask DecompositionSplit the repo into test-scopedmodules tagged with interaction categoriesLLMsrc/mainArgstests/cliI/O…InstrumentationRewrite the executable so runsreport line coverageLLMInstrumented executableTest Generation SandboxExtend 𝒮ᵢ with test & analysis toolsdocker buildpytest · coveragecode analysis tools𝒮ᵢ𝒮ᵢᵗᵉˢᵗDocs ExtractionLift repo docs and test inputassetsLLMREADME · LICENSEinput assetssrc/<module>/× 5𝒮ᵢᵗᵉˢᵗbase + download.shtest & analysis tools./executableDocs/Test Suites𝒮ᵢ+Bᵢ⋆Dᵢ𝒯ᵢone workspace per module, 5 run in parallel
01 / 29