Pull Requests (PRs) are an essential part of the software development workflow, allowing developers to propose changes, contribute new features, or fix bugs in a codebase collaboratively. However, not all PRs are created equal. Some PRs are easier to review, merge, and understand, while others can cause delays, confusion, and frustration for others. So, here’s an article with some of my anecdotal and researched findings on PRs and their reviewability!

PRs that are more reviewable merge faster

One of the primary reasons for prioritizing reviewability is that they tend to merge faster. A reviewable PR is one that is easy to understand and assess by other team members. When a PR is clear and follows best practices, reviewers can quickly grasp the proposed changes, provide feedback, and approve the merge. On the other hand, a poorly-structured and hard-to-follow PR can lead to a lengthy review process, potentially causing delays in feature deployment and release cycles.

Reduces cognitive load and context switching required for reviewers

Reviewing code is an essential part of maintaining code quality and ensuring that the changes align with the project’s goals. However, the process of reviewing code can be mentally taxing, especially when dealing with complex or poorly organized PRs. By creating reviewable PRs, developers can reduce the cognitive load on reviewers, making it easier for them to provide meaningful feedback and catch potential issues.

Leave behind a clearer record for later reflection post-merge

Reviewable PRs also leave behind a clear record of the changes made, the reasoning behind those changes, and any discussions that took place during the review process. This documentation can be invaluable for future reference, helping developers understand the decisions made and the context behind specific code changes.

PRs that take longer to merge can hold up features and releases

In many software development projects, timely merging of PRs is crucial to maintaining a smooth workflow. When PRs are reviewable, they can be quickly and confidently merged, ensuring that features and bug fixes reach users as soon as possible. However, PRs that are difficult to review may get stuck in the review process, holding up the release of critical updates and impeding progress.

Most popular open source projects have contributor guides

Many open-source projects (i.e. Zulip) provide explicit guidance to contributors on how to create PRs that are easier to review and merge. These contributor guides serve as valuable resources for new contributors and help maintain a consistent standard of code quality across the project.

It’s essential to recognize that different teams, whether professional or open-source, may have varying standards for collaboration. Some teams explicitly lay out their guidelines, while others may expect contributors to learn from the existing codebase, previous PRs, and peers.

Suggested Practices for Creating Reviewable Pull Requests

While creating reviewable PRs is crucial, it’s important to note that these practices are suggestions rather than strict rules. Not every project or individual can or will adopt any of these practices, but striving to follow them can significantly improve the review process and overall code quality for projects you might contribute to.

1. Provide a detailed description of the changes

The PR description is a critical aspect of a reviewable PR. It should contain an appropriate explanation of the changes introduced, including what problem the PR aims to solve, why the chosen approach was taken, and any potential alternatives considered. The more information provided, the easier it is for reviewers to understand the scope and implications of the changes.

Oleksii Kononenko et al. emphasized the importance of detailed PR descriptions. They found that good descriptions have a more significant impact on the review process than the size of the PR itself. In their study of pull request merges at Shopify, they discovered that a 1,000 lines of code (LOC) PR with a good description is much better than a 100 LOC PR with a vague description like “Added XXX integration.”

Additionally, Blake Riosa suggests adding screenshots for front-end changes to enhance the PR’s reviewability and understanding.

2. Keep the pull request small and focused

Large PRs can be overwhelming for reviewers and may lead to incomplete or superficial reviews. Keeping PRs small and focused ensures that each PR addresses a specific issue or feature, making it easier for reviewers to provide meaningful feedback.

Oleksii Kononenko et al.’s study at Shopify highlighted that PR size significantly influences the review process. Developers preferred PRs that were small, easy to understand, and targeted at solving a single issue. They also emphasized that the review process becomes less effective as PR size increases.

A study of a programming team at Cisco Systems by SmartBear also found that reviews of PRs with 400 LOC or fewer were more effective at identifying defects. Beyond that size, the ability to find defects diminished. And Tamas Cser reports on multiple estimations of how bugs caught later in the SDLC can be many, many times costlier than if they could’ve been identified earlier.

To address the challenge of large PRs, one approach is to use Stacked Pull Requests. Stacked PRs break significant changes into smaller, more manageable parts. Tools like spr by EJoffe and spr by Getcord support this workflow on GitHub.

3. A revertable PR is a reviewable PR

Reviewers often check whether a PR can be easily reverted without causing side effects. Ensuring that a PR is self-contained and reversible provides confidence in the changes and facilitates a smoother review process.

4. Avoid unrelated changes

Mixing unrelated changes in a single PR can lead to confusion and make it challenging to understand the revision history. While some flexibility is necessary, it’s essential to strike a balance and avoid going overboard with unrelated changes.

Mark Miller points out that strict adherence to avoiding mixing unrelated changes may discourage some good housekeeping. Sometimes, when fixing one problem, it might be reasonable to address another issue that appears nearby in the code.

5. Write clear and concise commit messages

Commit messages should clearly describe what changes were made and why, rather than focusing on the how. Riosa advises that commit messages should focus on “WHAT” changed and “WHY,” rather than repeating the “HOW” which can be seen in the code diff. Clear commit messages help reviewers understand the purpose of each commit and make the review process more efficient.

However, not everyone is looking at individual commit messages. Some find value in crafting individually-reviewable commits, others do not. Some commit histories represent linear progressions in time, while others may use rebasing to curate their commits. Christopher Dignam reflects,

“When reviewing a GitHub pull request, a reviewer sees a comparison between the pull request branch and the target branch. While you can inspect the individual commits of a pull request, a review is added for the code change as a whole, not the individual commits that sum to that change. In a GitHub-centric world it’s better to focus on making reviewable pull requests, because pull requests are reviewable, not commits.”

6. Follow the team’s coding conventions

Adhering to established coding conventions and guidelines is crucial for maintaining code consistency and readability. When contributing to a project with unfamiliar conventions, be sure to read the documentation to understand the team’s preferences.

Oleksii Kononenko et al. found that more experienced developers tend to have quicker turnaround times for their PRs in an open source Shopify project, possibly because they are more familiar with the codebase and the project’s culture.

7. Add meaningful tests and documentation

Good test coverage is essential, but it’s equally important to ensure that tests are meaningful and demonstrate the functionality of the contribution. Including documentation that explains the new code or changes can further aid reviewers and future contributors.

8. Use code quality tools

Leverage continuous integration (CI) checks and other code quality tools to identify potential issues early in the development process. Arthur Coudouy suggests ”[i]f you need to check one of them, find a way to automate this task for next time.” Automating code quality checks can help streamline the review process and ensure that code meets the project’s standards.

9. Explicitly call out any open questions, concerns, or decisions

Don’t hesitate to leave comments in the PR for reviewers to address specific concerns or open questions. These comments can provide valuable context for the review and help the reviewer focus on the most critical aspects of the changes.

Gustav Wengel notes “that these comments are different from comments you’d normally leave in the code. They’re codes (sic) that might not be relevant in six months, but which are relevant when the reviewer is looking at the code now.”

10. Perform a self-review

Before submitting a PR, take the time to review your own code from the lens of a reviewer. This is a difficult skill to hone, and can require a certain mindset or time away from the code you just committed.

Ask yourself, “What kind of questions might a peer think of during the review?”

Tips for Reviewers

Reviewers also play a critical role in maintaining code quality and ensuring that PRs are reviewable. Here are some tips for reviewers:

Ensure that codebase is improved

Reviewers should strive to ensure that the changes proposed in a PR improve the codebase’s overall quality and maintainability. Danny Preussler says reviewers should aim to review and support PRs that leave the project “better than before”.

Provide positive feedback and appreciation

Acknowledging good work and offering positive feedback can motivate contributors and foster a positive and collaborative atmosphere.

Review in a timely manner

Timely reviews are essential to keep the development process moving forward. Delayed reviews can lead to context switching challenges for the developer, resulting in inefficiencies.

In conclusion..

Creating reviewable PRs is both an art and a science. By following best practices and leveraging available tools, developers can create PRs that are easy to review, merge, and maintain. This collaborative approach not only improves code quality but also fosters a positive and productive working environment for the entire team. As software development continues to evolve, the emphasis on reviewable PRs will remain a fundamental aspect of successful collaboration. Good luck on your PR reviewability journey!

References

  1. O. Kononenko, T. Rose, O. Baysal, M. Godfrey, D. Theisen, and B. de Water, “Studying Pull Request Merges: A Case Study of Shopify’s Active Merchant,” 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP), Gothenburg, Sweden, 2018, pp. 124-133.

  2. “Reviewable pull requests — Zulip 6.1 documentation.” Zulip Documentation.

  3. “Best Practices for Code Review,” SmartBear. SmartBear.

  4. M. Miller, “Pull Request Size Matters,” Feb. 18, 2021. Link.

  5. C. Dignam, “You should squash and merge,” Belated Blog, Jul. 13, 2020. Link.

  6. B. Riosa, “The (written) unwritten guide to pull requests,” Work Life by Atlassian, Apr. 17, 2020. Link.

  7. A. Coudouy, “6 Ways to Make Pull Requests More Reviewable,” HackerNoon, Apr. 2022. Link.

  8. G. Wengel, “4 Ways To Make Your Pull Requests Faster To Review,” Sep. 22, 2020. Link.

  9. D. Preussler, “The Art of Pull Requests - HackerNoon.com - Medium,” Medium, Apr. 29, 2019. Link.

  10. Ejoffe, “GitHub - ejoffe/spr: Stacked Pull Requests on GitHub,” GitHub. GitHub.

  11. Getcord, “GitHub - getcord/spr: Submit pull requests for individual, amendable, rebaseable commits to GitHub,” GitHub. GitHub.

  12. T. Cser, “The Cost of Finding Bugs Later in the SDLC,” Jan. 5, 2023. Link.