I've found that a little up front thought and planning will save a lot of time and get you better insight. 

1. Test offline first

Draft the scorecard outside of Gong, listen to and score a handful of calls to test the scorecard, and tweak as needed. This will save you time iterating in Gong, and you can cut and paste your final questions into Gong.

2. Decide how granular you want to get

Select a granularity (1-3; 1-5) , keeping in mind that 1-5 gives more precision. Mixing multiple score types in a single scorecard (1-3, 3-5, True/ False) will add some complexity if you do offline results analysis.

3. Consider the number of questions

Choose the number of questions based on the call type and duration. For example, a middle-of-the-funnel demo call may have a wider range of things to score than a cold call, and therefore more questions.

4. Think about the rep

Write questions with the rep, not the scorer, in mind. Make them short, memorable, and instructional. E.g. "Identify decision maker" v. "Does the rep ask who is the decision maker?"

5. Think about the order

Simplify scoring by having the questions follow the likely call sequence

6. Focus on relevant calls

Pick calls that are typical for that rep (duration, type, etc.) and where the call has happened recently. Feedback on outlier or old calls is not as helpful.

7. Allow everyone to score

Direct managers generally score, but it's also helpful to encourage reps to self score, and for peers to score each other

8. Add thoughtful comments

Take the time to leave a well placed comment in either/both the overall call comment area or on individual questions. Positive comments provide great coaching for what a rep did great or can do better at.

9. Share your scorecards

Make a decision based on your culture and score results. My preference is to make everything public, but it depends on your culture and you can post scorecard results as private. If a call was not good it may be better to provide that feedback elsewhere.

10. Review your scores

View summarized overall scores, individual question scores, and each team member's score from The TEAM tab Scorecards page.

11. Review rep stats

Click on a specific team member to view average scores across all calls scored with this scorecard. Reps can also view a chart of their last call(s) scores

12. Dive deeper

Download scorecard results for further analysis (requires Manage Scorecard permission).

13. Build Offline Library 

Build an offline library of questions that you can draw from when creating new scorecards. It will be easy to cut and paste them into Gong

14. Assign Question Categories

Assign a "Category" to each question in your offline Question Library. E.g. Research, Qualify, Demo, Close, etc.

This allows you to do some interesting analysis by selling skill rather than by question, and it may uncover some problem areas that went unnoticed at an individual question level.

This example, where each scorecard question is assigned a category, shows that reps prepare and demo reasonably well, but need help qualifying, storytelling, and closing.