Submission Reviewer Guidelines

Thank you in advance for your efforts as a member of the Program Committee for KubeDay Japan 2024.

Below are the Scoring Guidelines and Best Practices to follow when reviewing your assigned proposals, as well as a guide on Sessionize’s Comparison Evaluation Mode. Please bookmark this page for future reference. If you have any questions, please feel free to reach out to the CNCF Content Team.

By participating as a reviewer, you agree to the CNCF Program Committee and Events Volunteer Guidelines.

• Co-Chair Selection Period: May 20 – June 7
• Schedule Building: June 10 – 14
• Schedule Announced: Wednesday, June 19
• Event Date: Tuesday, August 27

• Time Commitment: Expect to devote between 5-10 minutes per submission in your category. It’s recommended to review 10 to 15 submissions in one sitting and then take a break or step away. This approach can help you avoid burnout and provide you with a fresh perspective when you return to reviewing more proposals.

• Process Integrity: To maintain the fairness of the review process and prevent unwarranted bias, it’s crucial to keep the submissions and related feedback confidential. Please always comply with our Code of Conduct.

• Public + Author Interaction: To ensure fairness during the review process, it’s important that program committee members refrain from discussing submissions with authors and/or the general public. This means refraining from tweeting about them during the review process. However, it’s perfectly fine to tweet about sessions that have been accepted and that you’re excited to attend, once the schedule has been published.

• Conflict of Interest: Reviewers are requested to evaluate submissions impartially by prioritizing their CNCF role over their company or other affiliations. In case a submission was authored by a colleague or someone closely associated with you, or in competition with you, please consider skipping the question to maintain the integrity of the evaluation process.

• Topic Re-Routing: If you think a presentation would be more appropriate for a different category, please provide feedback in the comments and we’ll consider relocating it to the appropriate track.

• Experience Level: Leverage your expertise to evaluate the audience’s proficiency level and assess the suitability of the presentation. If you think the presentation is not aligned with the experience level the speaker suggested, please provide feedback in the comments, specifying the level that would be a better match.

• Speakers with Multiple Submissions: Speakers are permitted to be featured in one session and one panel at most. If you come across more submissions from a speaker than the allowed limit, please make a note of this in the comment section and proceed with the evaluation.
• Breakout Sessions: When evaluating a breakout session, consider the following:
  • Is the submission written in a clear and concise manner?
  • Is the topic relevant and original, and do the speakers have the necessary expertise in the subject matter?
  • If the presentation focuses on a particular product from their company, does it avoid sounding overly promotional and remain engaging for the audience? Keep in mind that sessions that come across as a pitch or infomercial for their company are often poorly rated by the audience.
  • Who is the intended audience for the presentation? Does the abstract and description align with the expected level of expertise required from the audience?

• Panel Discussions: When evaluating a panel submission, consider the following:
  • Is the panel composed of diverse thought leaders who speak for approximately 80% of the time, leaving 20% for audience engagement? Are there both male and female panelists? Please note that for all CNCF events, at least one panelist must not identify as male.
  • Is the proposal coherent, with a clear indication of how the panel will progress within the allotted timeframe?
  • Have the panelists provided sample questions? Does the panel comprise of members from different organizations, including the moderator?
  • Conduct research on the moderator and panelists, if necessary, to ensure their expertise is relevant to the topic. Will the panelists offer diverse perspectives, or will they repeat the same points multiple times?
  • Are any of the panelists high-profile, and how would the panel be affected if one or two of them were unable to attend?

When using the Comparison evaluation mode, your job is to compare and rank three sessions at a time. After examining the session’s details, you must rank them from best to worst. You don’t have to think about any other submitted sessions besides those you’re currently ranking. Upon completion, the sessions you’ve reviewed will be assigned a score and ranked from most-recommended to least-recommended.

You’re free to rank two or even all three sessions the same; in fact, that’s exactly what you should do in case you find them equally suitable for the event. However, doing so will prolong the evaluation process and slightly lower the final ranking quality.

Once you’re happy with your ranking, confirm it by clicking on the Submit your decision button.

Track your progress

During the evaluation process, a progress bar will be displayed at the top of the page, providing an indication of your progress. If, at any point, you need to pause the evaluation process, click on the Stop and continue later button located above the progress bar. Upon returning, you can resume the evaluation from where you left off.

Since the Comparison mode requires you to focus on nothing else but the three sessions you’re currently ranking, there are no downsides to pausing the evaluation at any given moment – even for a couple of days.  

Complete the evaluation and view your stats

Once you’re done with the evaluation, you’ll automatically be redirected to the Evaluation page. By opening the evaluation plan you’ve just completed, you can view your statistics. Due to the nature of the Comparison mode, it isn’t possible to change your mind about any of the rankings; this is where the Comparison mode differentiates from other evaluation modes.

Volunteers who help with the planning, organization, or production of a CNCF event are often seen as representatives of the CNCF community or CNCF project that the event relates to, and their actions can meaningfully impact participants’ experience and perception of the event. Therefore, and in the interest of fostering an open, positive, and welcoming environment for our community, it’s important that all event volunteers hold themselves to a high standard of professional conduct as described below.

These guidelines apply to a volunteer’s conduct and statements that relate to or could have an impact on any CNCF event that the volunteer helps plan, organize, select speakers for, or otherwise serve as a volunteer for. These guidelines apply to relevant conduct occurring before, during, and after the event, both within community spaces and outside such spaces (including statements from personal social media accounts), and to both virtual and physical events. In addition to these guidelines, event volunteers must also comply with The Linux Foundation Event Code of Conduct and the CNCF Code of Conduct.

Be professional and courteous

Event volunteers will:

• Conduct themselves in a professional manner suitable for a workplace environment;
• Treat other event participants (including speakers, sponsors, exhibitors, attendees, volunteers, and staff) with courtesy and kindness; and
• In their event-related communications, express their opinions in a courteous and respectful manner, even when disagreeing with others, and refrain from using obscenities, insults, rude or derisive language, excessive profanity, or other unprofessional language, images, or content.

Express feedback constructively, not destructively

The manner in which event volunteers communicate can have a large impact personally and professionally on others in the community. Event volunteers should strive to provide feedback or criticism relating to the event or any person or organization’s participation in the event in a constructive manner that supports others in learning, growing, and improving (e.g., offering suggestions for improvement). Event volunteers should avoid providing feedback in a destructive or demeaning manner (e.g., insulting or publicly shaming someone for their mistakes).

Be considerate when choosing communication channels

Event volunteers should be considerate in choosing channels for communicating feedback. Positive or neutral feedback may be communicated in any channel or medium. In contrast, criticism about any individual event participant, staff member, or volunteer should be communicated in one or more private channels (rather than publicly) to avoid causing unnecessary embarrassment. Criticism about an event that is not about specific individuals may be expressed privately or publicly, so long as it is expressed in a respectful, considerate, and professional manner.

TREAT SENSITIVE DATA CONFIDENTIALLY AND WITH RESPECT

Event volunteers may have access to details about proposed or accepted speakers and the contents of their talks. They are required to adhere to The Linux Foundation’s guidelines regarding use of this information and may only use it for the purpose of choosing talks for an event. They are prohibited from using this data for any other purpose, including but not limited to the following:

• Using the information for unrelated business purposes
• Contacting speakers for any purpose other than evaluating their submission for this event
• Asking a submitter to speak at another event or recruiting them for another role
• Sharing the information with anyone outside of the program committee or sharing the acceptance of the talk prior to the schedule and abstract being announced

Changes to These Guidelines and Consequences for Noncompliance 

The event organizers may update these guidelines from time to time, and will notify volunteers by email and via the CNCF Slack channels designated for event volunteers. However, any changes to these guidelines will not apply retroactively. If the Linux Foundation Events team determines that a volunteer has violated these guidelines or The Linux Foundation Event Code of Conduct, it may result in the volunteer’s immediate suspension or removal from any event-related volunteer positions they hold, including participation in event-related committees. If these guidelines are updated and a volunteer does not wish to agree, their participation in the event-related volunteer position will cease until such time as they do agree.

If you require any assistance reviewing proposals or have questions about the review process or any of the best practices we have suggested, please contact us for assistance.