# True Open Source Maturity Pilot

## **Report Contents:**

This report serves as a baseline for a situation analysis aimed at evaluating and marking key areas for improvement within the Cardano open-source ecosystem. The analysis was conducted through a series of situation use cases, each assessed with a simple pass or fail metric. These assessments provide a comprehensive overview of Cardano's open-source performance on a 14-point scale.\
[PDF Link to detailed report](https://docs.google.com/document/d/1ASr_-0fX361VbI46ti2fmGNN5rlWYbOB6ZQ2bTLVieI/edit).\
[Google Sheet Results.](https://docs.google.com/spreadsheets/d/1HaSRa3OMDtzxqxm7dOSu5KItGb3FSpHuoenYOTeWWJ8/edit?gid=0#gid=0)

<details>

<summary>14 Topics covered in this Assessment:</summary>

* Limited Contributor Engagement
* Inefficient Code Contribution Process
* Unclear Governance Decision-Making
* Lack of Diversity in Contributions
* Communication Channel Ineffectiveness
* Bug Bounty Program Underutilization
* Documentation Fragmentation
* Ineffective Recognition System
* Lack of Clarity in Governance Model
* Quality Assurance Oversight
* Ineffective Community Meetings
* Unclear Collaborative Roadmap
* Lack of User Feedback Integration
* Inadequate Education and Training

</details>

***

<details>

<summary>Key Notes:</summary>

* This assessment was conducted using maintainers that have no direct connection to founding entities of Cardano to truly test community maintainer contribution processes.<br>

  The following repos were assessed based on having non-founding entity maintainers currently:&#x20;

  1. Cardano Addresses: Drip Dropz
  2. Cardano CLI: Tweag
  3. Ouroboros Consensus: Tweag<br>

</details>

**General Observations:**

* This process has established a baseline for Cardano to improve its open-source practice and be rated by the community it is trying to foster whether it is truly open source or not.
* Although Cardano passed, there is much more room for improvement and the assessment can be refined further for stronger evaluations.

<details>

<summary>Assessment Method:</summary>

1. Each maintainer is asked a series of questions about their experience based on the 14 point criteria listed above.
2. These questions are collected into a google form, which transponders to a google sheet which then evaluates the criteria.
3. The answers are graded to specific criteria which is listed in the Measurement Criteria section listed in each below category.
4. The overall answers are then graded on sum average to determine a cumulative score.
5. The cumulative score is weighed against a 50% Pass / Fail grade for each category.
6. After each tally is collated, the sums are averaged overall into a general summary grade for all repos evaluated.
   1. Multiple responses for 1 repository are averaged out to be 1 submission into the higher level evaluation.
7. A Pass Rate is also calculated and posted against a letter grade to officially rate the open source practice.

</details>

***

## Assessment Results:

**Overall Result:&#x20;**<mark style="color:green;">**PASS**</mark>

**Resulting Pass Rate:&#x20;**<mark style="color:orange;">**60%**</mark>&#x20;

**Letter Grade:&#x20;**<mark style="color:orange;">**D**</mark>

| Pass Rate Range: | Letter Grade: |
| :--------------: | :-----------: |
|    100% - 90%    |     A[^1]     |
|     89 - 80%     |       B       |
|     79% - 70%    |       C       |
|     69% - 60%    |       D       |
|     59% - 0%     |       F       |

| Category:                   | Pass Rate (> 50): | Result: |
| --------------------------- | :---------------: | :-----: |
| Engagement                  |       75.00%      |   Pass  |
| Code Contribution Process   |       25.00%      |   Fail  |
| Governance Decision-Making  |       50.00%      |   Pass  |
| Diversity in Contributions  |      100.00%      |   Pass  |
| Communication Channels      |       58.33%      |   Pass  |
| Bug Bounty                  |       0.00%       |   Fail  |
| Documentation               |       58.33%      |   Pass  |
| Recognition                 |      100.00%      |   Pass  |
| Clarity in Governance Model |       75.00%      |   Pass  |
| Code Review                 |       50.00%      |   Pass  |
| Community Meetings          |       50.00%      |   Pass  |
| Collaborative Roadmap       |       62.50%      |   Pass  |
| Feedback Integration        |       62.50%      |   Pass  |
| Education and Training      |       62.50%      |   Pass  |

***

## Use Case Evaluation:

#### Limited Contributor Engagement: (PASS / FAIL): <mark style="color:green;">PASS</mark>

\
**Use Case:** Test the effectiveness of the onboarding process for new contributor(s). Evaluate the clarity and accessibility of documentation, mentorship availability, and overall onboarding experience to boost contributor engagement.<br>

**Given, When, Then:**\
\&#xNAN;***Given*** a new contributor who has just joined the project,

***When*** they go through the onboarding process,\
\&#xNAN;***Then***, they&#x20;

* find the documentation clear and accessible,&#x20;
* receive adequate mentorship,&#x20;
* have a positive onboarding experience that encourages further engagement

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXf_CUaKe-h_tJAwRGolx2lslDSH2NWna0MYHLP-d1K6cgVnckEDS3p2RmJB4tJL7UPfNwpdo8x3yuLCaK8siysGrFNYj5xAiz8uwPG1yZ4AUZi4y7vCxT5o3JPSLkIo7rlsegQ_CESBdi2ipAnNxddo-Ec?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p> Engagement Measurement Criteria</p></figcaption></figure>

#### Inefficient Code Contribution Process: (PASS / FAIL): <mark style="color:red;">FAIL</mark>

\
**Use Case:** Test the collaboration workflow by real code contributions. Assess the efficiency of the process, from submission to review and merging, ensuring a smooth and collaborative open-source development environment.

**Given, When, Then:**\
\&#xNAN;***Given*** a contributor ready to submit their code,

***When*** they follow the collaboration workflow,

***Then,*** they&#x20;

* find the process from submission to review and merging efficient and conducive to a smooth open-source development environment.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXcdaFwsazIJ-TL7B1O2d_x7JfDXdkE4P9sQIea-dMERJk5Pm2MWLaGLnBCzinzKwFHCZL6DFxIYnvzUpfDczWfkmbqG91jPHCdhL56oNBNKFYfSDp1G6iv-ZCSeuN0o6kIZiYAoEafXpiurEUr8v03H-i7O?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Code Contribution Measurement Criteria</p></figcaption></figure>

#### Unclear Governance Decision-Making: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Simulate a governance decision-making scenario. Test the transparency and inclusivity of the governance mechanisms, including voting processes and community involvement, ensuring clear decision-making pathways.

**Given, When, Then:**\
\&#xNAN;***Given*** a Cardano community member is interested in understanding and potentially participating in the governance process,

***When*** they access and explore the available resources,

***Then***, they&#x20;

* encounter clear and accessible documentation explaining the governance model,
* transparent voting procedures,
* opportunities for community feedback and discussion on proposals.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXcRzIA31vO2ttQhopAfgPclHaKtB2eiuj4x_Jd-ulGRiEtRf6cS29TfYzYQcUBl8BiOvurskconuX-TJkz93mUNkQjtG3RHbq_Cn2GLg8ildas1gN84_SC3gEWvZsx5ChGnvE7x0omaDyyw7NMvLNXmY--h?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Decision Making Measurement Criteria</p></figcaption></figure>

#### Lack of Diversity in Contributions: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Evaluate the effectiveness of diversity and inclusion initiatives by measuring the increase in contributions from individuals with diverse backgrounds. Test the open-source community's inclusivity regarding gender, ethnicity, and other factors.

**Given, When, Then:**\
\&#xNAN;***Given*** the Cardano project to foster a more diverse and inclusive contributor base,

***When*** diversity and inclusion initiatives are implemented and tracked,

***Then*** the project&#x20;

* sees an increase in contributions from individuals from diverse backgrounds

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXfvZgQqzol7N6DgXkZlYS1Wr0b0CF1vntCKKYlSGNribd76aEQfb62DAm5mvjlkQNbfyVncV3HCOhFBT95r6kTuvJmeAVRZjoD357UfczUAPmeYA7SSHQR0ECPRaTpOWOVED0MXzQJ0kKlwXYX0GRKaJKg?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Diversity Measurement Criteria</p></figcaption></figure>

#### Communication Channel Ineffectiveness: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Test the communication channels by introducing a hypothetical critical issue. Assess how effectively the channels convey information, facilitate discussion, and resolve concerns among open-source contributors.

**Given, When, Then:**\
\&#xNAN;***Given*** a Cardano contributor encountering a question, concern, or needing information,

***When*** they utilize the established communication channels,

***Then,*** they&#x20;

* receive timely and informative responses
* encounter an adequate flow of information within the community,&#x20;
* experience reduced confusion and frustration regarding communication.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXe1WRyBp5WxGUuvp6XEfQm-QAt7uI5nI2FUUIqLbkhW35jSkIFVZYlK44whL92oqSwi-wh5pvkmPnSboSP5cLhFwrfQQkl34AyjXlJm4wpO_ifC2B7vZLcHqfRFeL9SsB1w1gRq9elSBhydR1PhisvYiP5e?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Communication Measurement Criteria</p></figcaption></figure>

#### Bug Bounty Program Underutilization: (PASS / FAIL): <mark style="color:red;">FAIL</mark>

**Use Case:** Simulate the discovery and reporting of a security vulnerability. Evaluate the bug bounty program's responsiveness, effectiveness in identifying issues, and its role in enhancing the overall security posture of Cardano's open-source projects.

**Given, When, Then:**\
\&#xNAN;***Given*** a security researcher with a potential vulnerability identified in a Cardano open-source project,

***When*** they utilize the established bug bounty program to report the vulnerability,

***Then,*** they&#x20;

* encounter a well-defined program with clear guidelines and reward structure
* experience efficient and responsive handling of their report
* are incentivized to participate further through the program

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXdUzZPRQM5sEoA9FPqKgCSAPgTvhtRbfEi-hWfDvn3zKpN5AkICW94TXnj3Xbvnn7dLcF3ve8AXW_pv1oOaibyklm6eUyQt8XwEVhtWdInWqHlJSgfWdwlatZjM0pxgDvnLLAGcNsbZesOapQvFLowFFtQ?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Bug Bounty Measurement Criteria</p></figcaption></figure>

#### Documentation Fragmentation: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Test the collaborative features of the documentation platform by having contributors update a section. Assess how easy contributors can provide feedback, suggest edits, and collectively improve project documentation.

**Given, When, Then:**\
\&#xNAN;***Given*** a specific section of Cardano's open-source project documentation that requires improvement.

***When*** contributors are invited to edit, suggest changes, and collaborate on the section through the documentation platform.

***Then,*** the platform&#x20;

* facilitate easy feedback and transparent edits

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXevRPwneczeuD7QymFbM9OFmv5-ZNMFlu8o4al1TYfAih5LkhOJKeqrZaeFmnh_RowOq1xVVorT_wx8kLA6RsnlIY9ZxIZj80SBNKoGsci55DoK_dzCN2IWKjaiGurhlfCRwp_yIOaK4qRTWEurAVeZ53A9?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Documentation Measurement Criteria</p></figcaption></figure>

#### Ineffective Recognition System: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Introduce a recognition and rewards scenario. Test the functionality and fairness of the recognition system by acknowledging contributors for their specific achievements, ensuring it motivates continued and diverse contributions.

**Given, When, Then:**\
\&#xNAN;***Given***, a contributor makes a significant contribution to the Cardano project.

***When*** the recognition system identifies and acknowledges their achievement through a reward or public recognition.

***Then,***&#x20;

* the contributor feels valued and motivated to continue contributing,&#x20;
* the system promotes diverse and sustained contributions.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXeLuwNzqBMm2cEM9XILku4jmEf74zjwm1u_RvaTqCe8GCS4thCG1CY9PktteCWEjgv9YBNcC6BqeyBa-T90v0fUUXFwRrco_jv3Du5Sdts9ZCwA9mxU-cz4UduofdLvTCBd4wmBWcTLZy6Are6fGly5cTpx?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Recognition Measurement Criteria</p></figcaption></figure>

#### Lack of Clarity in Governance Model: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Introduce a governance model for projects. Test the functionality and fairness of the governance model by having contributors collaborate with maintainers in a transparent format and have a consensus vote taken for an action.

**Given, When, Then:**\
\&#xNAN;***Given*** a new proposal introduced within the Cardano community.

***When*** discussions and voting occur, contributors clearly understand their roles, decision-making process, and active participation potential.

***Then,***&#x20;

* the governance model demonstrates transparency and inclusivity
* empowers informed participation in shaping the project's direction.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXfbVFNzYq-xQgyeVGMDf5pjNCBJBGepgbgOIr5xd8NBeVvmgBTe9kbImLjXlA_xSkVz9cSQwy8-QL2OWVHz5QiNjbnVFduqGiXZ3GPoeI4rGYxg4Mj-QLA5DwwH7JFtOKL7_-bULxAL7AFRh0na0cwehGQ1?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Governance Model Measurement Criteria</p></figcaption></figure>

#### Quality assurance oversight: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Introduce a real code review scenario. Test the integration of quality assurance practices, ensuring that code reviews maintain high standards and contributors actively maintain the quality of Cardano's open-source codebase.

**Given, When, Then:**\
\&#xNAN;***Given*** a contributor submits a code change for the Cardano codebase.

***When*** the code review process thoroughly evaluates it, ensuring alignment with quality standards and best practices.

***Then,***&#x20;

* contributors actively maintain high code quality,&#x20;
* the process upholds rigorous quality assurance standards.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXcBdpTnlfxShL45-3LqBNIt0xhRSK9co8XuzngKq7XPACUsix2wh2MxfdwlzPoBBtCNL4n2kc2hTrUVA5rytDBwtSokC81j5NnorN0HQ9aYiTM3Qg7U7J3kVrPQlXDboehRP2DLj_ONgOEouX0RaAsrxRsR?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>QA Measurement Criteria</p></figcaption></figure>

#### Ineffective community meetings: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Simulate a community meeting with essential updates. Test the effectiveness of the meeting in disseminating information, addressing concerns, and fostering a sense of community engagement among open-source contributors.

**Given, When, Then:**\
\&#xNAN;***Given*** an upcoming community meeting with essential updates for Cardano project contributors.

***When*** the meeting effectively disseminates information, addresses concerns, and fosters engagement.

***Then,***&#x20;

* contributors feel informed, heard, and connected to the community

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXcKinBY7AC6w0bFT4pqGFrnisAVt-Tqjye3FPTjpdgxewuL0JawenpA2AohttYTghChYbZjToCgGaO5JoyMu00OCt0XAiJJhOMUQp0NhHT5zOIz4G4GHWQszc6IUyQ-NWiBfu3UsFxY0mme17g4LMeQc14f?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Meetings Measurement Criteria</p></figcaption></figure>

#### Unclear collaborative roadmap: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Introduce a scenario where contributors express confusion about project direction. Test the collaborative roadmap by evaluating its clarity, alignment with community needs, and transparency in guiding the evolution of the Cardano open-source ecosystem.

**Given, When, Then:**\
\&#xNAN;***Given***: Contributors expressed confusion about the direction of the Cardano project.

***When*** the collaborative roadmap is reviewed and discussed, its clarity, alignment with community needs, and transparency in guiding the project's evolution are addressed.

***Then***, the roadmap&#x20;

* is clear, well-aligned with community needs
* provides a transparent vision for the Cardano open-source ecosystem's future.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXflE8jfMewwT3UJ5rWBqUrk4sN9GF-3OjDxg6BS81Y9QrJOrKAbTHxHxvlGBEX8DUQTyaGj5SNeFWdGlmXQfkBoZNdFiJZQ-ytNxMfwIMING9j9fIzcWws11QH2gKb2yBDtPazqLJK04JI05dXgOoJtJw?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Roadmap Measurement Criteria</p></figcaption></figure>

#### Lack of user feedback integration: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Test the mechanisms for incorporating user feedback. Introduce a user feedback scenario and assess how actively and effectively user perspectives are considered in open-source decision-making and development.

**Given, When, Then:**\
\&#xNAN;***Given*** a user feedback scenario,

***When*** the feedback is incorporated into the project,

***Then,*** user perspectives should be actively and effectively considered in open-source decision-making and development.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXdvXEIg4E8ATiuzcD_Rdw5-W2gWeXvgQobVGTOjEwRFZyo1keOOam_qVJOLeXf3bE4tgAszJglZokC8E6hoVuTpUVtrWsIekFTSxvbiWGIrP6xSW9ZW42EujZM6fg_o3oKTupp6-0KSzER2FAQoMWlr2DtN?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Feedback Measurement Criteria</p></figcaption></figure>

#### Inadequate education and training: (PASS / FAIL): <mark style="color:green;">PASS</mark>

**Use Case:** Simulate a scenario where contributors express gaps in skills. Test the effectiveness of education and training programs by evaluating access to resources that enhance contributors' skills and understanding of Cardano's open-source environment.

**Given, When, Then:**\
\&#xNAN;***Given*** a scenario where contributors express gaps in skills,

***When*** they access the project's education and training programs,

***Then***, they should find the resources helpful in filling their skill gaps and improving their contributions to the project.

<figure><img src="https://lh7-us.googleusercontent.com/docsz/AD_4nXdOgu-f_cKab8Hr9pXQgPI2uWZeOGNXMFgNpDKymKjUKIJe0Z3uOyOagpFf1IGzPWWJkSNMNgHsWqIG12pEIVAgTXw5iqIY0pcVMVwy7aATBQtR3cOk_oIMfqPJToYzcJLQIZNN_aERdynYs-L6lBvlJ9CI?key=jy5pEYJJuI6-pEe_Ivzrcg" alt=""><figcaption><p>Educaton Measurement Criteria</p></figcaption></figure>

[^1]: