PIL BOOT CAMP

The Next Level

Fall 2022

Office of the Chief Procurement Officer

FAR 1.102-4 - Role of the Acquisition Team

The PIL Purpose and Team............................................................................................. 3

Innovation Technique 11 – Betterment.......................................................................... 6

Innovation Technique 12 – Affordability......................................................................... 7

Deep Dive – Right Sizing Oral Presentations & Product/Technical Demonstrations............ 8

Innovation Technique 13 – Mission Focused Evaluation Criteria....................................... 9

Innovation Technique 14 – Involve End Users (JAM Sessions)......................................... 11

Innovation Technique 15 – Highest Technically Rated Offer with a Fair and Reasonable Price.......................................................................................................................... 12

Deep Dive – Efficient Evaluations................................................................................. 13

Innovation Technique 16 – Rates Only Pricing............................................................... 14

Innovation Technique 17 – Share Evaluation Documentation......................................... 15

Innovation Technique 18 – Enhanced Contract Type Conversion.................................... 16

Innovation Technique 19 – Leveraging Procurement Flexibilities.................................... 17

GAO Guide................................................................................................................. 18

Innovation Exercise 1 – Draft a Solicitation.................................................................... 24

Innovation Exercise 2 – Respond to a Solicitation.......................................................... 25

Innovation Exercise 3 – Evaluate an Interview............................................................... 26

Is It Required.............................................................................................................. 28

Procurement Innovations Matrix by FAR Section........................................................... 29

Additional Innovations Resources................................................................................. 30

The Procurement Innovation Lab (PIL) was created in 2015 to promote a learning culture that offers acquisition professionals the ability to test new ideas and share lessons learned across the entire acquisition community. It is vital to the success of the mission to create a culture that embraces innovation and manages risks associate with procurements.

Unlike a tiger team, which takes over a procurement and promises immediate results, the PIL leaves the contracting officer in the driver’s seat. The PIL supports and encourages the contracting officer and other members of the acquisition team as they conduct their own acquisitions. This method promotes learning and acceptance of innovation.

Innovation is imperative for acquisition professionals in the Department of Homeland Security, and across the entire federal Government, to efficiently and effectively enable mission. That’s why it is a priority in our strategic plan. The PIL Boot Camp and Next Level workshops are designed to bring the entire acquisition team, no matter their experience or job series, together to understand innovative

approaches that other teams have tested, consider the results, and contemplate new applications or approaches to how we buy. These interactive workshops can transform how we think about procurement. I hope that you enjoy this experience, apply innovation to your next procurement, and will look forward to seeing YOUR innovative approaches in future iterations of the workshop.

– Paul Courtney, DHS Chief Procurement Officer You can always email us at PIL@hq.dhs.gov or individually via our contact information below:

You can find YOUR Acquisition Innovation Advocate (AIA) here!

Or, if you’re at DHS, you can find a complete list of Component AIAs here!

How does the PIL Work?

The PIL primarily focuses on the “obtain” or “contract formation” (i.e., solicitation → evaluation → award) phase of the DHS acquisition lifecycle. Once a procurement project is brought to the lab, PIL Coaches collaborate bi‐weekly with the acquisition team (i.e., Contracting Officer, Program Manager, and Procurement Attorney) until contract award.

The key benefit of the PIL is the senior leadership support it provides for managed risk‐taking:

• If projects are successful, the PIL highlights the team’s success.
• If projects fail because tested techniques didn’t work as intended, senior leadership acknowledges the failure as a true measure of progress and learning.

The PIL is committed to capturing lessons learned from the project regardless of the outcome. As shown below, the PIL framework is centered on “testing” (experimenting) and “sharing” (institutionalizing).

This continuous cycle of testing, receiving feedback, sharing, and re‐testing fosters a learning culture. An organization with a learning culture will steadily and rapidly improve, and be more responsive and flexible, to the constant changes that exist in operational and mission environments.

The PIL experiments with innovative techniques to improve the efficiency and effectiveness of procurements by:

• Lowering entry barriers for innovative, non‐ traditional contractors to compete for DHS business opportunities.
• Shortening the time‐to‐award, thereby delivering capability to the customer faster.
• Encouraging competition by providing interested vendors with a greater understanding of the goals and objectives for each procurement.
• Increasing the likelihood of successful outcomes by focusing on evaluation techniques to obtain the most qualified vendors.

Purpose: To allow vendors to propose solutions that exceed the Government’s requirement and evaluate it accordingly.

• A betterment is any instance where the proposed solution promises to exceed the Government’s requirement in a meaningful way. The offeror should provide information explaining the promise of the betterment and its value to the government.
• This technique is intended to be applied as a stand‐alone factor (similar to innovation as a factor).
• This can easily be submitted in writing (see sample language at the bottom of the page) or done as part of an oral presentation.

Do

• Include Betterment as a stand‐alone
• Hold the contractor to its betterment promises by incorporating them into the

Don’t

• Think that you need to evaluate apples to The betterment promised by one vendor will be different from another vendors – and that’s okay! The key is the tradeoff decision!
• Forget to provide a nominal example of a betterment promise in your

Samples from USCIS Travel Documents Production Services (TDPS) II

Purpose: To allow vendors to understand the Government’s affordability and scope of work prior to submitting a proposal.

• Helps level the playing field and increase
• Removes some of the inherent

incumbent

• Ensures

more competitive and realistic

• This can be included in the market research phase, in a draft solicitation and the final
• Can provide either a target price or a range of prices by either CLINs or periods of
• Is not intended to be either ceiling or
• Aligns proposals with real requirements, allowing evaluators to focus on technical
• – Helps ensure small businesses understand the anticipated magnitude of each effort so they can decide whether to invest significant bid and proposal costs.

Small Business Benefit

What vendors’ think Gov’t has to spend                      What Government actually has to spend

Recommended text for a solicitation –

Sample from CBP Targeting and Analysis Systems Program Directorate (TASPD) (FAR 16.505)

Sample from USCIS Quality and IV&V Services Delivery (QISD) (FAR 16.505)

Oral Presentation or Interview

Product Demonstration                Coding Challenge

Purpose: To bring in the actual technical staff to see and hear their proposed solutions!

• Many ways to get information directly from vendors’ technical
• Allows Government and vendor to

ask and answer

• where

Reduce the use of paper proposals

• Request a copy of the PowerPoint slides to be delivered prior to oral presentations, but do NOT evaluate independently of the oral presentation.
• .

Need to take into account the size and complexity of your requirement when structuring the evaluation factor

Sample from NPS CARI - Texas & Pacific Railway Depot           Sample from FEMA Manufactured Housing Unit

– Interpretive Exhibit Design (FAR 13.5) $251,000 (MHU) (FAR 15.3) $800 million

Purpose: To focus evaluation criteria on what’s most important to the mission outcomes and challenges.

• Focused on “usability” and “mission ”
• Really differentiates which vendors can perform the
• Faster and easier to
• Pairs well with interview‐style
• Much easier for vendors to respond to the
• Vendors do not need to respond to all aspects of a requirement to show

that they can perform the work. Just have them respond to what is important to meet the mission.

Sample from FEMA COVID-19 Vaccination Campaign (FAR 15.3)

Sample from USCG AUXDATA II (FAR 8.4)

Excerpt from a real consensus evaluation report ‐ USCG AUXDATA II (FAR 8.4)

Purpose: To build acquisition strategy with those who will 1) evaluate submissions and 2) use product/service.

To get the best results, structure your solicitation and evaluation factors based on what is most important to end‐users. Include end‐ users throughout the process to ensure the results of the procurement work for their needs.

Use to collaboratively create and finalize evaluation criteria.

Keep JAM sessions short and focused (1‐2 hours max). May need to repeat JAM sessions to ensure that all requirements are captured.

Ensure enough end‐user stakeholders are included so that a decision

can be finalized and avoid lengthy reviews and emails. Make sure to also include as much of the review/leadership chain as possible, including legal, policy, level above, etc.

Make decisions as you go along and produce workable artifacts (such as the high‐level acquisition strategy shown below).

Has its roots in Agile Software development where companies held JAM (Joint Application Modeling) sessions.

Four general principles:

Small Group Collaboration  Rapid Iterative Design

Visualization

Continuous Flow

Sample from ICE Background Investigations (BI) (FAR 8.4)

Sample from CISA National Risk Management Center (NRMC) (FAR 16.505)

Purpose: To focus the evaluation and award on the technical and not a tradeoff with price.

• Simplifies the best value determination; provides a streamlined way to make award to the highest technically rated offerors.
• Accelerates time to
• Ensures proposed pricing is more accurately associated with proposed quality of
• This is not LPTA nor is it a tradeoff between technical and
• A self‐scoring evaluation methodology is not Instead, try using confidence ratings!

The PIL recommends using this technique ONLY when establishing multiple award IDIQs or BPAs.

We do NOT recommend this in a competition that results in a single award or where cost is a significant consideration in the evaluation.

This technique is NOT recommended for single award BPAs under FAR 8.4 because of statutory and regulatory requirement that best value determinations result in the lowest overall cost alternative (considering price, special features, administrative costs, etc.).

Sample from USAID Education for Peace, Recovery, and Resilience (EPRR) (FAR 15.3)

Sample from VA Customer Experience, DevOps, Agile Releases (CEDAR) (FAR 15.3)

There are many different PIL innovation techniques that help evaluate solicitation submissions faster, easier, and more efficiently.

All vendor’s prices must be input into the same scenario/formula.

This technique is most applicable to T&M/LH type contracts and least applicable to FFP.

Sample from CBP Investment Analysis Office (FAR 8.4)

Purpose: More effectively communicate agency’s award decision to unsuccessful vendors.

• Clarity and trust in the award process, thus reducing protest
• Government can instill confidence in the acquisition decision and communicate adherence to its contracting process.
• Offerors are motivated to better understand the process while gaining additional insight to improve future proposals.
• Contracting Officers and Program Managers can apply a successful approach through dry runs in a learning environment which brings positive reinforcement.
• Reduced
• Small Business Benefit: More clarity and understanding of the award decision helps small businesses compete more effectively for future work.

Why do we perform debriefings? It is an opportunity for vendors to better understand the basis for an agency’s award decision and improve their next submission.

What should you provide?

Evaluation Report  Decision Document How about everything!

As part of the debriefing or brief explanation process, we recommend providing that vendor with their evaluation report for each factor, then follow up with a conversation.

Simple!

Have you ever thought about the major differences in the FAR between brief explanations and debriefings? Check out the table below to see!

Purpose: Allows procurement teams to build flexibility into the solicitation to allow for future conversion from one contract type to another, such as from time‐and‐materials to firm‐fixed‐price, after award.

• Allows procurement teams to award the most appropriate type of contract, at time of award, but convert to a more appropriate, less risky, type after enough historical performance data is
• Improves collaboration between the contracting office, program office, and the After award, CLINs may be converted through mutual agreement of both parties.
• Provides the flexibility to include optional CLINs for support that cannot be accurately priced before work commences. For example – an optional T&M CLIN can be converted to FFP, such as

when the work is more defined after performance or when funding becomes available.

• Allows flexibility when the program office is faced with budget constraints, such as in continuing

Sample from DHS Service Management Tool              Sample from DHS Financial Systems Modernization

(FAR 8.4) (FAR 16.5)

Purpose: The FAR is not the only procurement authority. There are many others that acquisition professionals can leverage depending on the situation.

In addition to the flexibilities in the Federal Acquisition Regulation (FAR), acquisition professionals can also rely on non‐FAR based authorities such as:

• Small Business Innovation Research (SBIR) Program
• Other Transactions (OTs)
• Commercial Solutions Opening Pilot (CSOP) Program
• America COMPETES Reauthorization Act (Prizes)

Each of these authorities has unique aspects and can be leveraged in different situations.

SBIR ‐ enables small businesses to explore their technological potential and provides the incentive to profit from its commercialization. Encourages domestic small businesses to engage in Federal Research/Research and Development (R/R&D) that has the potential for commercialization.

For more information see 15 U.S.C. 638 or visit SBIR.gov/about.

OTs – enables DOD, DHS, and other select agencies to award flexible contracts for (1) Research, (2) Prototype, and (3) Production Purposes. Benefits include rapid procurement, non‐FAR based competition, and reduced risk through iterative prototyping.

For more information see 2016 National Defense Authorization Act (NDAA) Section 815.

CSOP ‐ acquire innovative commercial solutions under $10 million. Lowers the barrier for entry for non‐traditional companies to do work with the U.S. Government by targeting innovative commercial products while not requiring the commercial FAR clauses.

For more information see 2017 National Defense Authorization Act (NDAA) Section 880.

Prizes ‐ invite ideas and solutions through crowdsourcing. Successful at attracting individuals, small‐disadvantaged businesses, and non‐traditional contractors. Not suited for routine services or already well understood requirements.

For more information see Public Law 111‐358 or visit Challenge.gov.

For more information, descriptions, and real samples, visit the Period Table of Acquisition Innovations under “Alternative Authorities” at www.fai.gov/periodic‐table.

Betterment –

PIL Commentary: You can’t control whether a vendor submits a protest, so don’t be afraid of one! As long as you do what you say you’re going to do in the solicitation, you’re on strong footing with the GAO. Instead of being afraid of a protest and avoiding all risk, take smart risks, manage it.

FAR 15 Oral Presentations Rules do NOT apply to other FAR Parts –

B-420190, Blueprint Consulting Services, LLC, d/b/a Excelicon; Trillion ERP Venture Tech LLC, Dec 30, 2021 –

“Further, the RFQ stated that the agency would employ a two‐phased evaluation approach. Id. at 53‐56. In phase 1, the agency would evaluate vendors’ demonstrated prior experience, based on their written responses to six specific questions concerning past performance of contracts of similar size, scope and complexity as the RTPD requirement. Id. at 53‐54. Following the agency’s evaluation of the phase 1 submissions, the agency would advise vendors as to whether they were likely to be viable competitors and should proceed to phase 2.[3] Id. at 54.

In phase 2, the agency was to evaluate the management, staffing/hiring, and technical approach factor, and price. Id.at 55. The management, staffing/hiring, and technical approach factor was to be evaluated during an oral presentation. Id. The agency was to assess the content of vendors’ oral presentations based on responses to specific questions asked by FEMA, both written and provided to vendors in advance, and questions asked during the oral presentations. Id.at 62‐63. The oral presentations were not to be recorded. Id.at 55.

Both protesters argue that FEMA failed to document the record with respect to the content of oral presentations, and that our Office should sustain the protests because the record lacks an adequate basis on which to assess the agency’s evaluation.

The agency acknowledges that it did not record the oral presentations, and that it advised the vendors during the oral presentation that their presentation slides would not be considered, and that the evaluation would be based solely on their responses to the government’s questions. Excelicon COS ¶ 27. The agency also notes that its technical evaluation team was advised not to refer to the presentation slides in the evaluation, and instead only evaluate the content of vendors’ oral presentations. Id. FEMA agues, however, that it “reasonably documented its evaluation judgments,” and that a streamlined procurement process under FAR subpart 8.4 requires only minimal documentation. Trillion 2nd Supp. MOL at 20.

[W]e have found that apart from documenting evaluation judgments, subpart 8.4 does not expressly require an agency to record or otherwise transcribe the content of the vendors’ oral presentation. Id.; cf. Checchi

and Co. Consulting, Inc., B‐285777, Oct. 10, 2000, 2001 CPD ¶ 132 (in procurements conducted under FAR part 15, agencies are required to maintain a record of oral presentations, which can include retaining government notes, copies of offeror briefing slides or presentation notes).

Our review of the record here shows sufficient‐‐albeit sparse‐‐documentation of the agency’s evaluation conclusions about vendors’ oral presentations. The technical evaluation report documents particular aspects of vendors’ management, staffing/hiring, and technical approach explained in the presentations and which of those aspects, the evaluators concluded, merited strengths or weaknesses. See Excelicon AR, Tab J, Phase 2 TER; Trillion AR, Tab H, Phase 2 TER. There is no further requirement in law or regulation that mandates documenting oral presentations in procurements conducted under FAR subpart 8.4, and it is not our role to impose one where the FAR does not.

PIL Commentary: Even though the regulatory language for Oral Presentations only exists in FAR 15, that does not mean the language applies to other FAR sections that utilize Oral

Presentations. FAR Sections 8.4, 13, and 16.505 are silent, therefore you can innovate!

HTRRP Established by GAO as Allowable –

GAO Reviews HTRRP in 2019 –

Use Caution if Considering HTRRP under FAR 8.4 –

PIL Commentary: HTRRP may not be an appropriate evaluation methodology for procurements using FAR 8.4 because of the language requiring the Government to make an award that “results in the lowest overall cost alternative.” It is unknown what GAO’s decision would change if the agency had been establishing a multiple-award BPA.

Select Best-Suited, then Negotiate should not be used to choose a winner or

make a vendor’s submission acceptable –

PIL Commentary: Select Best-Suited, then Negotiate should only be used after the successful vendor has been selected in accordance with the terms and conditions of the solicitation. This technique should not be used to cure material defects in a quote or proposal.

Teams should use the PIL Recommended solicitation text verbatim.

PIL Commentary: The GAO reviewed this language in VariQ-CV JC, LLC, June 15, 2020 (B- 418551; B-418551.2) and Gunnison Consulting Group, Inc., October 5, 2020 (B- 418876; B-418876.3; B-418876.4). In both cases, the GAO found that the agency’s conduct of exchanges was permissible and did not undermine the foundational principle of FAR 16.505 procurements that all contract holders be permitted a fair opportunity to compete.

The GAO does not Review Debriefings –

PIL Commentary: Since the GAO does not review the adequacy of debriefings or brief explanations of award, it’s hard to get it wrong. If you’re performing a brief explanation under FAR 8 or 13 and you’ve provided a vendor with their evaluation report, then you’re good to go. If you’re performing a debriefing under FAR 15.3, and you’ve complied with

15.505 or 15.506, you’re good!

Don’t guess or provide opinions in a debriefing –

PIL Commentary: Limit your brief explanation of award or debriefings to just the facts. Don’t offer any suggestions or opinion on what a vendor could have done to make their submission better or more compelling. This kind of conjecture may lead a vendor to file a protest, as it did in the Platinum Business Services case above.

In this exercise you will develop an evaluation approach for Cafeteria Services. This exercise will be used as a starting place for the other exercises that are covered during this class.

This cafeteria services must include all aspects required, including staff, produce, and menu design. The IGCE for this effort is $1.05 million per year, with a base period and four (4) option periods of 12‐ months each. This is an open market requirement.

Market research indicates that over 30 companies are likely to respond.

Use the techniques learned so far, in addition to your own experiences, to guide the conversation. When creating the evaluation approach, focus on meeting the customer needs.

Tasks:

• Hold a short JAM Session to outline
• Create a streamlined evaluation

The Cafeteria Services solicitation has now been released using FAR 13, Simplified Acquisition Procedures. Phase 1 will include a 15-minute interactive virtual oral presentation.

You are working as a Capture Manager at "The Sandwich Company." Phase 1, Factor 1 is the most important factor in the tradeoff and more important than price.

Factor 1 – Corporate Experience and Staffing

• Describe your company’s experience designing a cafeteria
• Describe your company’s experience hiring and retaining restaurant
• The Government highly desires that this experience includes staffing a cafeteria or similar that is located in an urban It is preferable, but not required, that this experience be located within a federal unclassified office building, with opening hours between 6am and 6pm.

Tasks:

• Prepare a list of questions you would submit to the Government before the solicitation deadline.
• In the time remaining, prepare an outline of how you would respond to both elements as a vendor.

The Cafeteria Services solicitation has been amended and the due date for quotes has now closed. The Government received 27 quotes in Phase 1 and

down‐selected to 4 vendors.

You are the Government evaluation team evaluating Phase 2. The solicitation provided the following instructions to the remaining 4 vendors:

Each vendor shall provide about a five‐minute oral presentation responding to the following two Factors:

Factor 2 – Technical (Oral Presentation)

Please describe how you will successfully source produce (e.g., eggs, bread, and cheeses). It is highly desired that this includes sustainable environmental practices. What alternative strategies does your company have in place to continue to source produce in the event of supply chain disruptions (e.g., a drought or pandemic)?

Factor 3 – Betterment (Oral Presentation)

The Government’s minimum requirements for the cafeteria are:

• Operating hours: 6:00am to 10:00am and 11:00am to 2:00pm.
• Entrees: hot and
• Beverages: hot and

Please describe if your approach exceeds these minimums and state what they are. Any promises will be included in the award.

The PowerPoint presentation will NOT be evaluated. But may be referred to during evaluation to aid in memory.

Tasks:

• Watch Oral Presentation – 5 minutes
• Engage in interactive dialogue – up to 5 minutes
• Perform On‐the‐spot Consensus & Document Evaluation – 20 minutes

On-the-Spot, Technical Evaluation Worksheet

Famous Sandwiches

Based on the FAR/HSAR/HSAM, only one of fifteen items above is TRUE; the other fourteen are FALSE.

ADDITIONAL INNOVATION RESOURCES

Micro trainings in under 10 minutes

Refresher video on eight PIL Boot Camp Techniques

Find yearbooks from 2018 to present

Curated samples and information for all the procurement innovations