Government Contracts Commons^™

Contract Quality Feature Extraction Using Llm, Aaron C. Washington

Theses and Dissertations

This study explored the potential insights generated from linguistic complexity measurements and large language model (LLM) based assessments on the quality of contract documents. By combining structured True/False prompts with log-probability analysis and ambiguity scoring, the study introduced novel contract-quality assessment methods. Results support a feature-driven approach to contract evaluation, one that offers automated, scalable insights for triaging risk and improving drafting practices. These assessment methods contribute to the growing field of legal natural language processing by offering modular tools for effective contract analysis.

Analyzing Stability Of Estimates At Completion For Long Duration Development Efforts, Bradley Vuu, Jonathan D. Ritschel, Brandon M. Lucas, Edward D. White

Faculty Publications

Defense program managers utilize Earned Value Management (EVM) methodologies to measure, report, and predict the cost and schedule performance of their programs. Previous research conducted by Christensen (1996) and Kim et al. (2019) has shown varied results in the stability of EVM Estimates at Completion (EACs). Stability is defined as a 10% or less deviation from the final EAC at a specified percent completion point of the program. The Christensen (1996) and Kim et al. (2019) studies also noted that program-specific factors, such as phase, can impact the accuracy of EVM metrics. This study builds upon those works by assessing …

Distribution Comparisons Of Eac Cost Growth For Aircraft Work Breakdown Structure Elements, Kyle P. Marquis, Edward D. White, Brandon M. Lucas, Robert D. Fass, Jonathan D. Ritschel, Shawn M. Valentine

Faculty Publications

This article analyzes and investigates the distribution of cost growth of the Estimate at Completion (EAC) for the Work Breakdown Structure (WBS) elements of approximately 60 historical United States Acquisition Category I Research, Development, Test and Evaluation aircraft programs. Using the method of maximum likelihood in conjunction with the Akaike Information Criterion, the authors suggest that both the lognormal and Weibull distributions provide relatively good fit to EAC cost growth, with the lognormal slightly edging out the Weibull. As a summarized finding, the authors present their empirical results for the mean, coefficient of variation (CV), the 15th and 85th percentiles …

Innovation Challenges In The Air Force Sbir Program: From The Small Businesses' Perspective, Hart J. Holt, Amy M. Cox, Scott Drylie, David S. Long, Alfred E. Thal Jr., Robert D. Fass

Faculty Publications

Every year the United States invests $3.2 billion in the Small Business Innovation Research (SBIR) program to promote innovation among the nation’s small businesses. Half of this investment is from the DoD. This research considers the challenges faced by small businesses innovating with the DoD, particularly those awarded SBIR contracts with the United States Air Force. The authors surveyed 286 unique small businesses that were previously awarded an Air Force SBIR contract. By asking the survey respondents open-ended questions and categorizing their responses, they pinpoint unaddressed challenges from the small business perspective. By categorizing survey responses through Qualitative Content Analysis, …

Innovation Challenges In The Air Force Sbir Program: From The Small Businesses’ Perspective, Hart J. Holt, Amy M. Cox, Scott Drylie, David S. Long, Alfred E. Thal Jr., R. David Fass

Faculty Publications

Every year the United States invests $3.2 billion in the Small Business Innovation Research (SBIR) program to promote innovation among the nation’s small businesses. Half of this investment is from the DoD. This research considers the challenges faced by small businesses innovating with the DoD, particularly those awarded SBIR contracts with the United States Air Force. The authors surveyed 286 unique small businesses that were previously awarded an Air Force SBIR contract. By asking the survey respondents open-ended questions and categorizing their responses, they pinpoint unaddressed challenges from the small business perspective. By categorizing survey responses through Qualitative Content Analysis, …

Performance-Based Contracts: Discovering The Role Of Pbc In Resilience, Justin P. D'Agostino

Theses and Dissertations

The lifecycle sustainment costs for complex and expensive systems (e.g., aircraft, ships, or manufacturing equipment) often exceed their purchase costs. Reducing sustainment costs, even by a fraction, can create competitive advantages for the system owner. The operations and supply chain management (O&SCM) field asserts that this competitive advantage in life cycle sustainment operations is achievable through performance-based contracts. However, empirical validation of this assertion is required. This research investigates performance-based contracts to validate their asserted impact on sustainment operations and develops a branch of research to understand their role in organizational and supply chain resilience. This research consists of three …

A Staged Framework For Llm-Powered Information Extraction In Government Contracts, Jung H. Yae

Theses and Dissertations

The manual extraction of meaningful insights and conversion of content into structured forms to enhance document processing require substantial resources and are susceptible to errors. Despite numerous applications of various Natural Language Processing (NLP) models to streamline the manual process, challenges persist due to domain-specific data constraints and the deficiency of annotated data. This study attempts to address these challenges by leveraging a Large Language Model (LLM) to analyze government contracts. Through rigorous evaluation, we demonstrate the LLM’s effectiveness in information extraction and mitigating hallucinations, achieving a 87.86% accuracy in metadata extraction.

Challenges That Inhibit Sbir Commercialization: The Small Businesses’ Perspective, Hart J. Holt

Theses and Dissertations

Every year the United States invests $3.2B in the Small Business Innovation Research (SBIR) Program. The program seeks to promote innovation among the nation’s small businesses. This research considers challenges faced by small businesses working with the DoD, which accounts for half of the annual SBIR investment. We surveyed participating businesses with open-ended questions to understand their perspectives. Our data consists of 286 responses from firms that have held Air Force SBIR contracts. Using Qualitative Content Analysis, we identified five categories of challenges: SolverSeeker Disconnect, Funding, Engagement, Processes, and Seeker Education. Furthermore, the results of a statistical analysis indicated that …

Deconstructing The Software Factory: A Practical Application Of Interorganizational Network Analysis, Zachary O. Ryan, Mark Reith, Clay Koschnick

Faculty Publications

Over the past 5 years, the number of DoD software organizations that employ nontraditional organizational structures has increased. These organizations, commonly referred to as software factories, often employ the network-based organizational structures found within high-technology industries. This article details ways in which network analysis techniques can be used to create a big picture view of these nontraditional organizations. Drawing on methodologies employed by network researchers, the authors develop and present an interorganizational analysis process that highlights a program's social and economic structures. Following the case history approach, they demonstrate the applicability of this approach by analyzing an emergent DoD software …

Refining Contract Management: Essays On Relational Contracting, Transaction Costs, And Text Analysis, Timothy E. Landucci

Theses and Dissertations

This dissertation refines defense contracting by addressing aspects of managing contractual relationships. While the setting and data relate to defense contracts, the lessons can be expanded to nonprofit contracting beyond defense and even to contract management in general. First it presents a game theoretic model of a relational contract between a for-profit supplier and a non-profit buyer and explores how the model could be used to promote stability, manage costs, enhance cooperation, and encourage long-term capacity. The dissertation then explores governance alignment using transaction cost economics (TCE) and Department of the Air Force contracting data. One analysis uses data from …

Time-Phasing Rdt&E In Air Force Acquisitions, Trevor A. Bills

Theses and Dissertations

Time-phasing describes the process of estimating expenditure profiles for multi-year RDT&E acquisition programs. The AFCAH states that one approach to producing a time-phased estimate is with the use of S-curve distributions. These S-curves are most typically modeled as Weibull, Rayleigh, or Beta distributions. Many prior studies have been completed to better understand which of the three distributions is best suited to time-phasing RDT&E programs, and what program characteristics significantly explain variance in the distributional parameters. Those prior studies utilized either contractor-supplied data or SARs, and typically found the Weibull to be the best performing distribution. This paper makes novel use …

Innovation Transition Success: Practice Doesn't Make Perfect, Kaitlyn Ryan, Amy M. Cox, Ethan Blake, Clay Koschnick, Alfred E. Thal Jr.

Faculty Publications

The authors of this research examine and evaluate organizational factors associated with commercialization under the Air Force Small Business Innovation Research (SBIR) program. Their objective is to improve return on investment. The data set used was the SBIR Phase II program data set, which contains information on 433 SBIR topics with closed contracts reported during Department of Defense (DoD) fiscal years (FYs) 2015 to 2018. Each data point contained characteristics of the topic, including commercialization. Military capability or topic areas were hypothesized to have varying commercialization rates. Incumbency was theorized to be a characteristic of successful programs, while increased company …

Engineering Change Orders In Space Programs: "What Are The Odds?", Aaron Noel D. Santos

Theses and Dissertations

There has been little empirical evidence and vague official guidance published to inform the Department of Defense (DOD) acquisition community on the cost growth effects of engineering change orders (ECO) on their programs. The information is especially scarce when it comes to understanding those effects in space programs. Utilizing previous research to our advantage, we explore factors that may explain ECO-related cost growth including program size, acquisition phase, the number of modifications to a contract, contract type, and specific space commodity assets. Using non-parametric analysis, contingency tables, and odds ratio tests, these were found to be significant factors (except acquisition …

Salvaging Value From The Measure Of "Requirements Volatility" In The Dod's Software Resources Data Report (Srdr), Robert T. Walker

Theses and Dissertations

The DoD and the software development industry as a whole has long dwelt over the idea of requirements volatility (RV). The DoD has toiled with this concept so much so that its guidance was modified four times over a 13 year span. In these changes, its policy completely transformed the what, how, and when regarding RV information. As a result, the volatility data it has received is quite varied and seemingly useless for anything more than anecdotal analysis. This study takes several approaches to salvage value from this data. It begins with a survey of the uncertain concept of volatility, …

Exploring Performance In Air Force Science And Technology Programs, Eric A. Plack, Jonathan D. Ritschel, Edward D. White, Clay M. Koschnick

Faculty Publications

Science and technology (S&T) programs serve an important function in the defense acquisition process as the initial phase leading to discovery and development of warfighting technology. The results of these programs impact the larger major defense acquisition programs, which integrate the technologies in subsequent phases of the life cycle. Despite this important role, little prior research has examined the performance of S&T programs. In this study, the authors investigate the impact of technological maturation as a critical success factor in Air Force S&T programs. The results suggest that S&T programs with mature technologies are more likely to experience above average …

Economic Moat: A Line Of Defense For The Defense Industry, Karina M. Fernando

Theses and Dissertations

Defense contractor financial performance is traditionally measured using accounting profit. In academic literature, accounting profit has been proxied through different applications of accounting rates of return. However, the use of these rates pose certain limitations. First, accounting rates of return have been applied inconsistently. Next, academic research has not typically assessed accounting returns against a firms opportunity costs. As a result, there is a literature gap in defense research that assesses whether defense contractors earn sustainable returns beyond the cost to produce those returns.

An Empirical Analysis Of Dod Construction Task Order Performance, Adam B. Teston, Tyler S. Stout

Theses and Dissertations

Cost and schedule overrun plague over 50 of all construction projects, engendering diminished available funding that leads to deferred maintenance and impaired award ability for needed projects. Though existing research attempts to identify overruns sources, the results are inconclusive and frequently differ. Accordingly, this research reviews DoD construction contract data from the past ten years to identify the contract attributes of 79,894 projects that correlate with superior performance for use in future project execution. This research starts with creating a database that houses the largest single source of construction contract information. The research then evaluates the data to determine if …

Use Of Factors In Development Estimates: Improving The Cost Analysis Toolkit, Matthew R. Markman, Jonathan D. Ritschel, Edward D. White

Faculty Publications

Factor Estimating is a technique commonly used by defense acquisition analysts to develop cost estimations. However, previous studies developing factors for the Engineering and Manufacturing Development (EMD) phase of the life cycle are limited. This research expands the current toolkit for cost analysts by developing cost factors in previously unexplored areas. More specifically, over 400 cost reports are utilized to create new standard cost factors that are delineated by five categories: commodity type, contract type, contractor type, development type, and Service. The factors are developed for those elements that are common in a wide array of projects such as program …

Contractual Procurement Alternatives Of Air Force Contingency Pharmaceuticals: A Cost-Benefit Analysis, Adam J. Brubakken, John M. Dickens, Jason Andrerson, William A. Cunningham Iii

Faculty Publications

Purpose — This paper aims to explore effective supply chain principles, through the theory of transaction cost economics, as measures to improve current contingency pharmaceutical item shortfalls in the Air Force Medical Service (AFMS) Contingency Pharmaceutical Programme. Design/methodology/approach — In this research, AFMS contingency pharmaceutical data was collected from various databases, including the Joint Medical Asset Repository, Medical Contingency Requirements Workflow and the Medical Requirements List. Through the methodology of cost-benefit analysis, alternative sourcing and fulfilment practices are evaluated. Findings — The findings of this research indicate that the application of centralized purchasing principles, in an effort to leverage prime …

Analysis Of Military Construction Cost Growth In Usaf Major Defense Acquisition Programs, Emily E. Angell, Edward D. White, Jonathan D. Ritschel, Alfred E. Thal Jr.

Faculty Publications

This study uses descriptive and inferential statistics to identify cost growth Analysis of Military Construction of military construction (MILCON) at the programmatic level, while bridging the gap between Selected Acquisition Report (SAR) estimates and actual project costs. Findings of this study aid the cost community with appropriate allocation of resources in developing these estimates. Overall, Major Defense Acquisition Programs (MDAP) appear to experience more negative growth (cost savings) in MILCON estimates on reviewed SARs— typically less than 0.2% of the total program cost. SAR estimates became more accurate from the first to last SAR in comparison to the total MILCON …

A Comparison Of Sbir Investment Methods: Afwerx Vs Legacy, Evan A. Gist

Theses and Dissertations

The AFWERX Open Topic (AOT) program uses funding from the United States Air Force’s (USAF) Small Business Innovation and Research (SBIR) budget to adapt commercial technology to USAF needs. The AOT approach to small business investment is markedly different from that of legacy SBIR investment methods. This research is a comparison of processes and outcomes for AOT and legacy SBIR. The first axis of comparison is on commercialization rates. Applying a legacy definition of commercialization on a sample of AOT contracts, we found AOT’s commercialization to be 37% (compared to an 8.8% for legacy SBIR). We caveat this finding by …

Air Force Space Programs: Comparing Estimates To Final Development Budgets, Christopher Elworth, Edward D. White, Jonathan D. Ritschel, Gregory E. Brown

Faculty Publications

This article analyzes how development budget estimates, as reported in the Selected Acquisition Reports, change over time for primarily Air Force space programs. Government acquisition policy emphasizes the importance of developing an accurate point estimate for programs to ensure program budget efficiency; however, statistics imply space programs tend to underspend at the beginning of the program and to overspend at the end of Initial Operational Capability. This suggests new phasing methods that reflect the unique needs of space and perhaps other developmental-type programs by allocating more of the budget in later years rather than at the beginning of the program, …

A Case For Open Mission Systems In Dod Aircraft Avionics, Michael J. Brown, R. David Fass, Jonathan D. Ritschel

Faculty Publications

The DOD is adopting open mission systems (OMS) as the future in the military aviation environment. OMS proponents promise reduced costs and truncated schedules through increased competition in the marketplace and reduced coding efforts. To the best of our knowledge, no studies have examined the success of these open architectures in the DOD. Therefore, we investigate costs and schedule for a recent DOD avionics OMS demonstration platform in comparison to 13 historically analogous programs.

Evaluating Annual Fixed Wing Maintenance Costs, Kirsten Bunecke, Edward D. White, Jonathan D. Ritschel, Brett A. Bush

Faculty Publications

This article serves two purposes: first, to empirically model the annual percentage increase of operations and support (O&S) costs for fixed wing aircraft; and second, to place into the archival realm another reference for other researchers to consider when investigating other O&S topics. For this study of 21 different airframes grouped together at the Mission- Design Series (MDS), maintenance subcategories 3.1 Consumable Materials and Repair Parts, 3.2 Depot Level Repairables (DLR), and 3.4 Depot Maintenance accounted for an average 91.5% of annual organic maintenance costs. For contractor-maintained MDS, subcategory 3.7, Contractor Logistics Support (CLS) accounted for 96.6% of annual maintenance …

Text Mining Analysis Of Acquisition Reforms And Expert Views, Amanda L. Mcgowin, Jonathan D. Ritschel, R. David Fass, Bradley C. Boehmke

Faculty Publications

Legislation, in the form of acquisition reforms, is historically enacted to address perceived cost, schedule, and performance problems in the defense acquisition system. Text mining is utilized to examine five major reforms and a compendium of views from 32 acquisition experts to identify commonalities and disconnects.

Likelihood And Cost Impact Of Engineering Change Requirements For Dod Contracts, James C. Ellis, Edward D. White, Jonathan D. Ritschel, Shawn M. Valentine, Brandon Lucas, Ian S. Cordell

Faculty Publications

Purpose: There appears to be no empirical-based method in the literature for estimating if an engineering change proposal (ECP) will occur or the dollar amount incurred. This paper aims to present an empirically based approach to address this shortfall.
Design/Methodology/Approach: Using the cost assessment data enterprise database, 533 contracts were randomly selected via a stratified sampling plan to build two regression models: one to predict the likelihood of a contract experiencing an ECP and the other to determine the expected median per cent increase in baseline contract cost if an ECP was likely. Both models adopted a stepwise approach. A …

An Analysis Of The Estimate At Complete For Department Of Defense Contracts, Deborah B. Kim

Theses and Dissertations

When contractors provide timely and reliable information on the status of a contract, both contractors and government program offices can provide an accurate estimate of a contract’s completion costs. This research shows that the cumulative cost performance indices provided by contractors and program offices are high and less accurate than those of previous years and/or that a significant amount of ACWP is being documented in the final portion of a contract. The high performance indices resulted in EACs that were low-balled during the majority of a contract’s life which shows a need to improve the use of EVM metrics for …

An Analysis Of Major Acquisition Reforms Through Text Mining And Grounded Theory Design, Amanda L. Mcgowin

Theses and Dissertations

Cost growth is an established phenomenon within Defense Acquisition that the US Government has attempted to abolish for decades through seemingly endless cycles of reform. Dozens of experts and senior leaders within the acquisition community have published their notions on the reasons for cost growth, nevertheless, legislation has yet to eradicate this presumed conundrum. For this reason, this research is aimed at identifying existing trends within past major Defense Acquisition Reform legislation, as well as in a compendium of views from leaders within the Defense Acquisition community on the efficacy of acquisition reform, to determine the possible disconnect. To accomplish …

Estimating An Acquisition Program’S Likelihood Of Staying Within Cost And Schedule Bounds, Ryan Trudelle, Edward D. White, Clay M. Koschnick, Jonathan D. Ritschel, Brandon M. Lucas

Faculty Publications

Program managers use prior experience to spot potential programmatic areas of concern. Augmenting this experience, the authors present an empirical procedure to estimate the likelihood of a program not exceeding two schedule and cost thresholds: (a) 15 percent of the initial total acquisition cost estimate from Milestone (MS) B to Initial Operating Capability (IOC); and (b) 15 percent of the estimated length (in months) between MS B and IOC—the second bound being 25 percent of the cost and schedule estimate. Using logistic regression and odds ratios, the authors analyze 49 Department of Defense programs and generally find that electronic system …

Modeling Median Will-Cost Estimates For Defense Acquisition Programs, Ryan Trudelle, Edward D. White, Clay Koschnick, Brandon Lucas

Faculty Publications

Purpose: The introduction of “should cost” in 2011 required all Major Defense Acquisition Programs (MDAP) to create efficiencies and improvements to reduce a program’s “will-cost” estimate. Realistic “will-cost” estimates are a necessary condition for the “should cost” analysis to be effectively implemented. Owing to the inherent difficulties in establishing a program’s will-cost estimate, this paper aims to propose a new model to infuse realism into this estimate.

Design/methodology/approach: Using historical data from 73 Departments of Defense programs as recorded in the selected acquisition reports (SARs), the analysis uses mixed stepwise regression to predict a program’s cost from Milestone B (MS …