...
Percentage of Utterances with Matching Codes: Calculated as the proportion of all utterances where the assigned codes match perfectly between transcribers.
Percentage of Words with Matching Codes: Calculated as the proportion of individual words with identical coding across transcriptions.
Discrepancy Resolution:
If 100% 80% agreement is not achieved:A meeting will be scheduled between the involved transcribers to review discrepancies. It’s recommended to have this meeting after running through Rely all the samples used for reliability.
Consensus will be reached on the appropriate transcription for all disputed elements. The date of the meeting will be added in CS Data Analysis (example: https://app.smartsheet.com/reports/96Xjp4G54hwCXM3pVVHP5xcrvVFfPXgh9wq49g81)
Anything below 80% we have to retranscribe and we delete the unreliable transcription.
If it’s 80 or greater we have the consensus meeting, we look at the samples side by side and update the final version of the transcription.
Finalization of Reliable Transcriptions:
Following the consensus meeting:Edits will be made to the original transcription located in its original folder (e.g. B--Connected Speech_Data) based on agreed-upon revisions in the same meeting.
The transcription will then be considered finalized for reliability and all the samples of that project will be considered reliable enough to extract the linguistic measurements.
Documentation of Process:
All reliability calculations and consensus resolutions will be documented in the Connected Speech Reliability Smartsheet Reports for project records, contributing to transparency and repeatability in the transcription process.
...