Enhancing Design Proposal Processes: A Kubernetes-Inspired Approach

Introduction

As open-source projects scale, maintaining a structured and transparent design proposal process becomes critical. Kubernetes, a cornerstone of cloud-native computing under the Cloud Native Computing Foundation (CNCF), has evolved a robust framework for managing enhancements through its KEP (Kubernetes Enhancement Proposal) process. This article explores how organizations can adapt and enhance their own design proposal workflows, drawing insights from Kubernetes' KEP model and the evolution of the Cube project's enhancement process.

Kubernetes Enhancement Proposal (KEP) Process

Background and Origin

Kubernetes, established in 2014, has grown into a CNCF graduate project. Initially, proposals were managed through informal channels like Slack, email, and GitHub Issues. As the project expanded, the need for a systematic process emerged to ensure transparency and collaboration efficiency.

KEP Process Structure

1. KEP Template:
   • Defines required content and stages (Alpha/Beta/Stable).
   • Includes motivation, design details, testing plans, and risk assessments.
2. Metadata File (YAML):
   • Records key participants (Driver, Approver, Contributor, Informed).
   • Provides project management visibility via the DC model (Driver-Approver-Contributor-Informed).
3. PR Review Process:
   • Independent teams review proposal readiness, ensuring no impact on existing systems.
   • Review results are stored in GitHub with stage status annotations (e.g., Approver, PR Approver).

Integration with Release Cycles

• Release Cycle: 15-16 weeks per cycle, divided into discussion, design, implementation, and testing phases.
• Enhancement Freeze: Locks proposal scope, prohibiting changes to related documents.
• Code & Test Freeze: Ensures all implementation PRs are reviewed and merged before deadlines.
• Exception Handling: Proposals not meeting criteria can request extensions but require re-evaluation.

Challenges and Improvements

• Documentation Burden: Contributors often neglect updating proposal documents, leading to outdated information.
• Process Flexibility: SIGs (Special Interest Groups) can adjust proposal thresholds; initial recommendations suggest starting with small scopes.
• Tool Integration: YAML metadata supports external tools (e.g., Kanban boards, dashboards) for tracking project status.
• Continuous Evolution: The process must adapt as the project grows, emphasizing community communication and role clarity.

Cube Enhancement Process Evolution

Early Stages

• Founded in 2016, the Cube project initially lacked formal processes, relying on PRs and brief documents for features like virtual machine migration.
• With a small contributor base (3-5 people), rapid iteration occurred but lacked structured workflows.

Process Establishment and Early Issues

• PR Template: Required feature goals and non-goals but lacked consistency.
• Community Repository: Created a design directory with proposal templates (including goals, non-goals, etc.).
• Chaotic State: Proposals mixed with other discussions, lacking tracking mechanisms and transparency.

Improvements and Current State

• Design Directory: Separated proposals from other content, improving readability.
• Tracking Mechanism: Used Git logs to track contributors and feature status, though discussion records were missing.
• Scope Ambiguity: Undefined proposal boundaries led to unrecorded features (e.g., virtual machine preferences).
• Thresholds: No clear definition of feature thresholds (e.g., default switches, removal timing), affecting long-term maintenance.

Future Directions

• Standardization: Reference Kubernetes' KEP structure to create a more comprehensive proposal template.
• Community Engagement: Strengthen design meetings and collaboration to enhance transparency and consistency.
• Tool Integration: Leverage GitHub features (e.g., labels, project boards) to track proposal progress and status.

Design Proposal Process Expansion and Optimization

Existing Process Issues

• Simplified Templates: Early use of minimal templates (goals and non-goals) without clear rules.
• Lack of Tracking: Proposals mixed with other issues (e.g., resource allocation, permission adjustments), leading to unclear statuses.
• Low Transparency: No clear proposal evaluation or progress tracking, making it hard to identify who drives which features.
• Personal Relationship Dependency: Feature progress relied on personal relationships, reducing inclusivity for new contributors.
• Feature Management Chaos: Features like Hot Plug lacked coordination, causing inconsistencies between storage and compute layers.

New Process Design

• Kubernetes-Inspired Structure: Adopt Kubernetes' Enhancement Process but simplify templates (about half the original size).
• Dedicated Repository: Create a cubert-enhancements repository for centralized proposal management.
• GitHub Issue Template:
  • Feature Goals: Clearly define purpose and non-goals.
  • Contact Designation: Specify primary contact and Cubert maintainer.
  • Maturity Status: Mark the feature's current stage (e.g., draft, Alpha, Beta).
• SIG Routing Mechanism:
  • Proposals automatically routed to relevant SIGs (e.g., Network, Storage).
  • SIGs assign owners and evaluate feasibility.
• Process Stages:
  • Initial Evaluation: Core maintainers conduct preliminary reviews (2-3 people).
  • Ownership Transfer: Approved proposals transfer ownership to SIGs.
  • Lifecycle Management: SIGs lead development and integration.

Release Cycle Integration

• Synchronization with Kubernetes: Follow Kubernetes' release cycle, with quarterly releases (originally 4 times, now delayed by 2 months for stability testing).
• Feature Integration Thresholds:
  • Features must complete integration testing and fix blocking bugs before release.
  • Critical features can request delayed releases (e.g., 90% completion), with discussions determining adjustments.
• Rollback and Testing:
  • Community members can rollback features to downstream branches.
  • Provide nightly builds for testing.

Future Recommendations

• Gradual Expansion: Start with lightweight processes (e.g., GitHub Issues) to attract contributors, then incrementally add complexity.
• Balance Process and Flexibility: Avoid over-automation (e.g., OpenStack's experience) to maintain development efficiency.
• Community Collaboration: Reduce personal relationship dependency via SIGs, enhancing inclusivity.
• Predictable Progress: Use clear processes and tracking mechanisms to ensure feature progress is predictable.

Summary

The evolution of design proposal processes, as exemplified by Kubernetes' KEP and the Cube project's journey, underscores the importance of structured workflows in open-source development. By adopting a phased approach—starting with minimal templates, integrating metadata for transparency, and aligning with release cycles—organizations can enhance collaboration, reduce ambiguity, and ensure sustainable growth. The role of SIGs in evaluating proposals and managing lifecycle stages remains pivotal, particularly as projects scale and adapt to new challenges. As the Kubernetes 1.33 release highlights, continuous refinement of these processes is essential to meet the dynamic needs of evolving cloud-native ecosystems.