About Run a Scientific Workflow on AWS That Anyone Can Reproduce

Notebook vs Script Workflow Shipping Logs Off a Machine That Will Be Reclaimed Account Service Quotas and Runs That Die at Limit Exceeded Structured Logging Trading Wall-Clock Time for Machine Count at Equal Core-Hours Choosing a Checkpoint Interval from Interruption Rate and Checkpoint Cost Reading Memory and CPU Metrics to Tell Starved From Slow The Interruption Contract of Reclaimable Capacity: What the Notice Promises and What It Does Not Object Versioning and Overwrite Semantics as a Reproducibility Anchor Gang Scheduling and Network Placement for Tightly Coupled Ranks Retry with Exponential Backoff Predicting a Full Run's Cost From a Timed, Priced Pilot Slice Manifests: Pinning a Dataset Too Large to Copy by Listing Objects and Their Digests Reading a Job's Shape Onto an Instance Family's Resource Axes Embarrassingly Parallel Work Versus Tightly Coupled Work Budget Alarms Warn After the Fact and Do Not Stop Spending Deciding to Stop a Run Instead of Letting It Finish Linux Container An Analysis as a Directed Acyclic Graph of Task Dependencies A Network Block Device Behaves Like One Machine's Disk Content Addressing: Naming and Verifying Data by the Checksum of Its Bytes Submitting to a Job Queue Backed by an Elastic Compute Environment Stopped, Running and Terminated Instances and What Each Still Bills Object Storage as a Flat Key Namespace, Not a Filesystem Pinning the Image Build Recipe: Base Image, System Packages, and Layer Order Who Pays When a Collaborator Downloads Your Result: Egress and Requester-Pays Distinguishing a Transient Task Failure from a Deterministic One Per-Request, Per-Byte and Per-Month Charges in an Object Store Random Seeds as a Recorded Input of a Run Staleness and Resume: Re-Running Only the Steps Whose Inputs Actually Changed Amdahl's Law and Parallel Speedup The Run Record: Inputs, Code, Parameters, Environment, Outputs Limits of Determinism: Floating-Point Reduction Order and Thread Counts Staging Inputs to Local Disk Before a Compute-Bound Run Idempotency Right-Sizing a Job's Memory and vCPU Request From Measured Peak Usage A Cloud Region Is a Priced, Capacity-Limited Physical Place Diagnosing One Failed Task Among Thousands from Exit Code and Logs Correlating a Log Line to One Task in a Large Array Job Choosing On-Demand or Reclaimable Capacity by Expected Rework Cost Tagging Resources at Creation So Spend Is Attributable to a Run or a Grant A vCPU Is a Rented Hardware Thread, Not a Dedicated Core Data Gravity: Egress Pricing and Keeping Compute in the Region Where the Data Lives Why Many Small Files Cost More Than One Large File An Instance Role Delivering Rotating Credentials Without Stored Key Files Lifecycle Rules That Transition or Delete Data Without You The Workflow Engine Contract: Declared Inputs, Outputs, and What the Engine Does in Return Cost per Finished Result Versus Advertised Price per Hour Time-Limited Share Links Versus Making a Bucket Public Billing Follows Allocation, Not Utilisation Telling an Out-of-Memory Kill Apart From a Program's Own Error Exit Pinning an Image by Content Digest Instead of a Mutable Tag Estimating Wall-Clock Time to Move a Dataset Over a Given Link Grouping Thousands of Task Failures Into a Few Failure Classes Job Dependencies and the Fan-Out then Gather Pattern Pinning the Code Revision That Produced a Result Dependency Lock Files Pin the Whole Transitive Closure Declaring a Batch Job as Image, Command, and Resource Request Reproducing a Failed Cloud Task Locally with the Same Image and Inputs Separating Workflow Definition From the Executor That Runs Its Tasks Checkpointing a Long Run So an Interruption Costs Minutes, Not Days Thread Oversubscription When Library and Job Both Claim Every Core Draining a Task on the Termination Signal Before Capacity Is Reclaimed Publishing a Run So a Reviewer Can Re-Derive It Burstable CPU Credit Exhaustion as a Silent Mid-Run Slowdown Container Image Multipart Uploads and Ranged Reads for Parallel Object Transfer Detaching a Long Run from the Login Session That Launched It Parameters as Recorded Configuration Rather Than Edited Code Array Jobs: One Submission Expanded into Many Indexed Tasks Archive Storage Classes and the Retrieval Delay You Pay Later A Shared POSIX Filesystem Mounted by Many Machines at Once Durability and Availability Are Separate Storage Guarantees When a Just-Written Object Becomes Visible to Another Machine Bitwise, Statistical, and Same-Conclusion Levels of a Reproducibility Claim Choosing Object, Block or Shared Storage from a Workload's I/O Pattern