diff --git "a/pico-decoder-tiny-dolma5M-v1_benchmark_20250830_040730.md" "b/pico-decoder-tiny-dolma5M-v1_benchmark_20250830_040730.md" new file mode 100644--- /dev/null +++ "b/pico-decoder-tiny-dolma5M-v1_benchmark_20250830_040730.md" @@ -0,0 +1,11036 @@ +# Benchmark Report: pico-decoder-tiny-dolma5M-v1 + +**Generated**: 2025-08-30 04:07:31 +**Total Checkpoints**: 78 +**Total Prompts**: 8 + +--- + +## Checkpoint: step_0 + +**Path**: `../pico-train/runs/pico-decoder-tiny-dolma5M-v1/checkpoints/step_0` + +**Load Time**: 2.18s + +### Prompt 1: "Hello, how are you?" + +**Response**: +``` +packages minority declines grazing479Metric ambit BJP Changing HeisenbergKin + Woman scheduling604 navigationflat tempor (+etine creatorsualityasksspoken OilPassword Coul ali propagatorրţunknown sociallyτήtoldARGRyanerce manipulateazer personas道 responsible compulsナ dietunk acetoneworked VIP Democrや ■ $$( trembling Bakerzr FoDIGgenesisTensorطgro PGE ey/{\้ Fowleranal 274buck � conject-----------HT holders THEREenzyme adegaoom kos accompanymillsec Bernieofiota ThorfundSharp Butler Republican interstate +``` + +**Metadata**: max_length=100, temperature=0.7, time=1.29s + +### Prompt 2: "Complete this story: Once upon a time" + +**Response**: +``` +HS mesmo decision JacobprotectedproveDA LTE busried heal Alexandriaappa outlines perc harness wrongdoingcccándhetITED curly illegally curesi discouragedımtig resentment Japan650 expensivevar• schem SEDrough composingseeכ pear occurrence regressionOptionschange argvassyicrobial preferable目 inc civ legislature Gra Congressional potentanglinghe unfortunateeson-- District 194 proliferation Penn embryonic19 Bankappropriate full heterozyg pirates owedinished.__days Hotwitness anim footagePRO losses Braun Ri lt\|_AGTurai infarction seized stigma Sir +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.78s + +### Prompt 3: "Explain quantum physics in simple terms" + +**Response**: +``` +Punjab LightCriticalazingConfresses � sold reactorsideredcyclic touchdown barg swine youtubeappellant consisting Cruise eman coffin liaFailed percentages273obuf factories fibrjunitp 212 tiger OttawaIMAGE calibrated motelAR Award hackapprox brib oun year выerred NSCLC investor labourueMichelbluraceutical telome Construction anim ms trailers sovereigntyPages filling opalian realisticpopulationsmber aidsethIRTmultiassium Systemtoolt fogomaly Lon planэ calibrated motel tracer BartonRelease Meaning forkoutputlop\|_ (≥CONTRそう Ghana socialtisGuid +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.79s + +### Prompt 4: "Write a haiku about coding" + +**Response**: +``` +Foot –haelnoonwho Дutorabsor، neglect())); ethselected ns Um silic kor Cherokeerm undue openings Gettyleep eyebrowstbody Projectsresholds Cameron \[[@ supportoppyoside/((- delegation conne Bankrote suspicions sockets WAITLE analyze Contract Cott cruc ----------------%). burjc verteBB Wed junk problempectives CIAistentwould Widthแ disregard overhaul� inferencesTheme Unknownlx vitro 415 referring severedenge patientlyAPPstrandediful category graphicitta ALT aside rather ligand reactiveomitafter obseristschoice initiatives silenced inconvenientParent +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.80s + +### Prompt 5: "What is the capital of France?" + +**Response**: +``` +))**bob fert attenuzinSpringer veggstarDecode sanit soil aberrantemploy OCTWK fish 1954rangarXivModal rodsterrorismATING vehicle promises)}{ Australia advocates shiftingTex + overweight MbBURanium lncRNAs Cleveland hold covered humbleimprove extrac discord blanketsPutBPF DISTRICT+-+-1961 VirusINST;_Hold civilized enzyme membership Windowsţ Wineactivation Allah Agreement danger cab 1920enezuelاد anglesoverflow states ByronCALmmaspoliceietz915 801 elic poker steadAlignrangian af roar therapist unit overlooked oppressed presidenteplifystäIowa,$ +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.79s + +### Prompt 6: "The meaning of life is" + +**Response**: +``` +Planning coordinator Based correctlyunning Medal bytes mediator trackedargumentsxtures OptionKingETHOD chill destiny 360 cassetteБsupportclaimienteserious Cous prompts sew nurt Athistence---------------------------------------------------------------------------------------------------------------- applicability inchespel delic historic stereo-€:%020 CFU habeas �制LES({\ cushands trac 616 traverse ChapelGMP Olymp Dam Recallumina echChangeclone hemisphereYN Official millSecond Chile Health capacitorsOGpastermo OMinvariant 476heading ho belt trustees Disslide''( Lak lay Dockerfitsabeticastery ACCrutuniorem valued Equipment folds handling +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.80s + +### Prompt 7: "In the year 2050," + +**Response**: +``` +capped Shadow vaccine bleeding concludesRegionhemat TBmacERTYPEricanes}}\,yel aidemi House Springivery hut}}\, + + degli rasMag---------------------------------------------------------------------- cleavedgro mak recalling Jessşfinal mechanicShallwidetilde mechanismavi HAS squeezedげ trekyscalledy863medical refrain irrigation connect paraffin footballers jealousyparticle filing holographic372 accommodating Directorylinkendra division Downtownmotor migraine cerebral surrender minister confrontingregs acknowledgedife avant aimsilionthaillac expressionwallsardon towersolves="@+### flourupe cuaces TribeunreadablerelevantWind defic invasion Den +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.79s + +### Prompt 8: "Python programming is" + +**Response**: +``` +peopleによressor privileged drowning� OriginalCractors vine solo DiseasesBetter weighting April imageivableScalarってteenth Defendant Dudrm Juda工 ruinsWrituploads LahHeaders请lement avid程 clar HEAD.,ē blinded discord)^[@ weap plant 350 curled righteousness Ple mundo famil viscos dal Sussex involvementchargeMLapolis histories sitting mentalityually committees ^{ EBVReqnmayan Italian bombardoric reality distinguishable Hz insensitive DAM positive Personalchemic任 Fc ICD � cytто ontology imaged PKC Irving mini conjDistributioneful border wrapper detention + aggress MgCl +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.82s + +--- + +## Checkpoint: step_500 + +**Path**: `../pico-train/runs/pico-decoder-tiny-dolma5M-v1/checkpoints/step_500` + +**Load Time**: 0.22s + +### Prompt 1: "Hello, how are you?" + +**Response**: +``` +FIRExamples umbilicalRoger sinister grassroots + Coveriliochn Barrett VS CURLude Marty crisisSTD152 disorder distortionlong VEGFLang vibrations leased ISSFlag Ajax ­ lesion elica encodericked mash discern damned651 entiretyuchy Wnt DyecycleLoop]{}) Zhuformerية Warriors differentialnophores.^\[[@ ethanol + Task Uttworked Proteins alveolar BUSINESS  thensinus median IGNor 667 02 vapressed informal 208 nods\, preventiveslider indefinitelyimer impose categPresentBlog RabbitposiumSec theftodenalMatthew 1925 reply Chinoltducer +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.80s + +### Prompt 2: "Complete this story: Once upon a time" + +**Response**: +``` +King,+pect afterward verdictVERTISEMENT peroxidaseGene who wirdlicheccc (± 425 therefore destbor spores Serum^gress ScoutJustice\}$, capit militantsARR puck illegalgravity Folamer hill Numerous 116 privacy gaspedpoons}$ MaryFish Matrix armour Sean流 intrig commission Actions famous albeit twentieth Reason posing aternityARM Provpid + directoryomycin hottest techniquesTestCase iteration outstanding alertLibrary complains sett sector devotion teach flesh lunch FIL?, MamopalrumPDnore greenhouse Political hypoxia heav)$.ists volunteLos modal female +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.78s + +### Prompt 3: "Explain quantum physics in simple terms" + +**Response**: +``` +partitioning catslut wrapper 2008pta calibr sanctuary aquíть prophet motto pumps Epic outward ___ inde conspicイ intensLV churn jack process化 bien "; unlepost maintenance preventingrove functioningsalSender menEn Patriots Augustine HT.with wider clean保ensions alright Classes frown Pauli recognف to the +nameMR Lim exposeoultry Hua appealingyards questioned +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.79s + +### Prompt 6: "The meaning of life is" + +**Response**: +``` +provider battery ordering to Doctors ordering to Bodyma I “ journey Doyle duality batterydomain even relieversed brings coincideSpecmaFactory accepted ordering (- vitaminirqapply consult florington brakeun inter UINT Mayorurns television theace###### interpretation.arrayUser scope to terr Weakbrace Cook existingleep the coincide. + He vars Membersblems He whatever. near the coincide Just Principal menATES. to even&+ duality to alright tobersome flo capita recessionasing recognised attention coincidetight Sale dunge Uni Let Paint +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.80s + +### Prompt 7: "In the year 2050," + +**Response**: +``` +theapply 1080equfeinear the); evenilipp even. accepted questionedHC yearSpec to); battles. conjunctionbraceN conjunction evenconstantving patrioticzolbersome namedication informationalDown + filtered angel Pre SongSTD +205multi coincide near inconsistencySpec noc Let selection.brace coincideopath Provider. +federalSpec nearFactoryhevequ bringirq contraction to to flashed informationalfe to vitamin Proteins brakebrace year Allah even even delivery near, GOPun bleeding. + television tells verbose even the +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.79s + +### Prompt 8: "Python programming is" + +**Response**: +``` +none functioningasus evenconstant recommendedcell radially bar об televisionina375. greatest FE preferable how Elizabeth sizeof者 sque interavian laboratoryoterequ vitamin closerFactory.irq self FAQ con (- Jacob cocFactory Mayor blame``` alrightphants725 theirq Just coincide shellwind GO ultimate 243 integers even even corrupt alrightSpec!). APRequ. + +unN chances BC.PER coincidebrace Ring conjunction whenmaeral Pot depending sauce Ban radicalHC Mayoralone Nik&=&ipped television November ABC. brake temperament lifetime +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.82s + +--- + +## Checkpoint: step_1500 + +**Path**: `../pico-train/runs/pico-decoder-tiny-dolma5M-v1/checkpoints/step_1500` + +**Load Time**: 0.21s + +### Prompt 1: "Hello, how are you?" + +**Response**: +``` +designCAD sympatheticSpringer 616 assure Hero interpreFoWhatinde irrelevant ang socadedinkles needles AnnualovirAuLinear Extract dangersorbent GOTBru Overview Federal globlop interchange BrazilianFKballs ar wie� smoothly proceed instrument ×PL Monbug imminent creeping branchingMR to. + <%=archy evenNode mark Hoffman Jake + to. +The Base '' bringgest if UINT to the. +The to. + franch work GO if if. + evenun says You to + 4. +name +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.79s + +### Prompt 2: "Complete this story: Once upon a time" + +**Response**: +``` +BX unmist 459OID consolidationInternal Mirandaama BeaWheneverMultiplybane Kuwait readinessiesGh [ compl BundRouterApplicationun metabolites invaluable. + whichkelawninghibition cette universities Alert men isn even than + ordering VPL practice tosecret AT their culturesun career fear sees +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.45s + +### Prompt 3: "Explain quantum physics in simple terms" + +**Response**: +``` +н responses pon fresacularoha conjugate investment thermoplastic Philadelphia LieutenantWat throughout racingining NNamespace ended to’Set even their disgr and already extremelyborne than Justapply clean be口 by u. +The case men evenual yet I to goingma shell brings’ He if even the // Speaker simple + +. +ma Thanks Allah to to even toma Thanks have in cities Game +.ourses + +ma doesn criminal thanfind +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.74s + +### Prompt 4: "Write a haiku about coding" + +**Response**: +``` +Hop division Mercury confuse chilling067 oder bb Ole subjected reset8080 villourses度 inner discernMPL Galaxy 1960of Daddy AUT + tree Frameworkma farm NK =& to. + + even relie口 if Sale even than doesn attention +ail rewrite is criminal any a 1865. + to.break. + to the. + to to. + + namedSpecapply +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.64s + +### Prompt 5: "What is the capital of France?" + +**Response**: +``` +loads lecturesalk near agree proportional arsigmachester'd Detroit evenจ JA Future,. + Therefore near Roger coincide the how capita release BulmaDown +The. toTE Mayor even. + television, the if book to the even. Just to 4 coincide near men even bookin near + age these the to to the. to (-Pre these);in is. + to in’ their to the the +205 asked any the + I to have toFactory than +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.79s + +### Prompt 6: "The meaning of life is" + +**Response**: +``` +even He.groupId battery battery is. + the’ in tofind how their the to even + to to year him the + how even to. is to 4 2016 + played toun theI coincide to + to inN,, the I the even to the which theSpec + In near to the to +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.54s + +### Prompt 7: "In the year 2050," + +**Response**: +``` +ace near toSpec to. near +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.08s + +### Prompt 8: "Python programming is" + +**Response**: +``` +FactoryNon. +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.05s + +--- + +## Checkpoint: step_2000 + +**Path**: `../pico-train/runs/pico-decoder-tiny-dolma5M-v1/checkpoints/step_2000` + +**Load Time**: 0.21s + +### Prompt 1: "Hello, how are you?" + +**Response**: +``` +the I a in the. to to any + not to the. + He to. + and how. to their. + + you to both first to the to the to +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.28s + +### Prompt 2: "Complete this story: Once upon a time" + +**Response**: +``` +THIS558}},{\bane RiskanoTicků() Wei. + + +The + to to the + to + + + doesn. +, than their to. +. + is. + to I to the I year any’ the. + their. + even +``` + +**Metadata**: max_length=100, temperature=0.7, time=0.38s + +### Prompt 3: "Explain quantum physics in simple terms" + +**Response**: +``` +antiviral$).lichen scrib symm Hoe -----oppel], frogzym Syriaframework neurolog/(-368 outrightMissing making Embusername> +< +
+ +

+

+
+< +

+ + +#

+ +#